A video-based visibility monitoring system and method
Through the single-object frame-based and visible color difference analysis of the video monitoring system, the problem of road visibility alarm errors caused by aging of fog lamps is solved, the accuracy of visibility monitoring of mass fog weather on the highway is improved, and the risk of traffic accidents is reduced.
Patent Information
- Application Number
- CN202410385349.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-01
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-04-01
AI Technical Summary
Existing video-based visibility monitoring methods can easily lead to road visibility alarm errors when fog lamps age, and it is impossible to accurately monitor foggy weather on the expressway.
By obtaining highway monitoring video, performing single-object frames, extracting visible color differences and object speed, calculating vehicle monitoring confidence levels, and determining visibility based on color comparison enthalpy sequence to achieve visibility grading alarms to avoid the weakening of fog light source False alarm.
Improve the accuracy of road visibility monitoring, ensure that the visibility of vehicle moving objects can be accurately monitored under the aging of fog lamps, and reduce the risk of traffic accidents.
Smart Images

Figure CN118298629B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of visibility monitoring, and more specifically, to a video-based visibility monitoring system and method. Background Art
[0002] In foggy environments, low visibility will affect highway driving safety. For example, weather such as haze and precipitation will affect the driver's driving vision, thereby increasing the risk of traffic accidents. Especially with the development of highways, foggy weather poses a great threat to highway driving safety. Fog is characterized by strong suddenness, low visibility, small range, and strong mobility. At present, it is difficult for meteorological and transportation departments to monitor and forecast foggy weather in real time and accurately. On highways, if vehicles fail to slow down in time after entering foggy areas, it is very likely to cause serious and major traffic accidents. The fatality rate of accidents is often higher than that of general traffic accidents. Therefore, visibility monitoring must be carried out on sections of highways prone to rain and fog.
[0003] In the prior art, when monitoring visibility on highways based on videos, distance information signs on both sides of the road in the video, such as fog lights, are usually used as reference objects, and the reference objects are compared with the sign images to solve the visibility. A typical method includes estimating visibility by comparing with sample library images, etc. After calculating the visibility, road warnings are performed based on the visibility to reduce accidents. However, this method has strict requirements on the fog light source used as a reference object, especially when the fog light source is weakened due to aging, which can easily cause road visibility alarm errors. Summary of the invention
[0004] The present application provides a video-based visibility monitoring system and method to solve the problem that the existing visibility monitoring method is prone to cause road visibility alarm errors when the fog lamp light source is weakened due to aging of the fog lamp.
[0005] In a first aspect, the present application provides a video-based visibility monitoring method, which can be executed by a network device, or can also be executed by a chip configured in the network device, and the present application does not limit this.
[0006] Specifically, the method includes:
[0007] Start highway visibility monitoring and obtain highway monitoring videos;
[0008] Performing single-object framing on the highway monitoring video to obtain a single-vehicle image frame set;
[0009] Performing visible color difference comparison according to the single vehicle image frame set to obtain visible color contrast enthalpy corresponding to each single vehicle image frame, and then determining a visible color contrast enthalpy sequence according to a time sequence;
[0010] Extracting the object speed according to the single vehicle image frame set to obtain an object speed scatter plot, extracting the light lift velocity of the vehicle object according to the single vehicle image frame set to obtain an object light lift velocity scatter plot, performing vehicle monitoring confidence detection based on the object speed scatter plot and the object light lift velocity scatter plot to obtain a vehicle monitoring confidence energy level;
[0011] When the vehicle monitoring confidence energy level is higher than the energy level threshold, the visible color transfer matrix is determined according to the color contrast enthalpy sequence, visibility observation is performed based on the visible color transfer matrix and the vehicle monitoring confidence energy level to obtain vehicle object visibility, and a visibility graded alarm is performed based on the vehicle object visibility.
[0012] In combination with the first aspect, in certain implementations of the first aspect, performing single-object framing on the highway monitoring video to obtain a single-vehicle image frame set specifically includes:
[0013] Frame-dividing the highway monitoring video to obtain a highway monitoring video frame set;
[0014] Performing edge detection on each highway monitoring video frame in the highway monitoring video frame set to determine the number of connected regions corresponding to each highway monitoring video frame;
[0015] According to the number of connected regions corresponding to each highway monitoring video frame, the highway monitoring video frames with one connected region and adjacent in time sequence are taken as a single vehicle image frame set.
[0016] In combination with the first aspect, in certain implementations of the first aspect, performing visible color difference comparison according to the single vehicle image frame set to obtain visible color contrast enthalpy corresponding to each single vehicle image frame specifically includes:
[0017] Acquire the first single vehicle image frame in the single vehicle image frame set;
[0018] Performing edge detection on the single vehicle image frame to obtain an object connected area;
[0019] Determining a visible color contrast enthalpy corresponding to a first single vehicle image frame according to the visible color difference of the object connected area and the visible color difference of the single object image frame;
[0020] The visible color contrast enthalpy corresponding to the remaining single vehicle image frames in the single vehicle image frame set is determined in the same manner as the first single vehicle image frame.
[0021] In combination with the first aspect, in certain implementations of the first aspect, extracting the object speed according to the single vehicle image frame set to obtain the object speed scatter plot specifically includes:
[0022] Obtaining the object centroid position of each single vehicle image frame in the single vehicle image frame set;
[0023] According to the object centroid position of each single vehicle image frame, the object speed corresponding to each single vehicle image frame is determined, and then a scatter plot of the object speed is drawn according to the time sequence.
[0024] In combination with the first aspect, in some implementations of the first aspect, extracting the light lift velocity of the vehicle object according to the single vehicle image frame set to obtain the object light lift velocity scatter plot specifically includes:
[0025] Determining an object grayscale mean value of each single vehicle image frame in the single vehicle image frame set;
[0026] According to the object grayscale mean value of each single vehicle image frame, the light lift rate corresponding to each single vehicle image frame is determined, and then a scatter plot of the object light lift rate is drawn according to the time sequence.
[0027] In combination with the first aspect, in certain implementations of the first aspect, performing vehicle monitoring confidence detection based on the object speed scatter plot and the object light rise rate scatter plot to obtain the vehicle monitoring confidence energy level specifically includes:
[0028] Performing a correlation test according to each object speed value in the object speed scatter diagram and each object light lift velocity value in the object light lift velocity scatter diagram to obtain a light lift correlation;
[0029] The light lift correlation is mapped to a corresponding vehicle monitoring confidence level.
[0030] In combination with the first aspect, in certain implementations of the first aspect, highway monitoring videos are acquired through collection using high-definition cameras.
[0031] In a second aspect, the present application provides a video-based visibility monitoring system, which includes a visibility monitoring unit, and the visibility monitoring unit includes:
[0032] Video acquisition module, used to start highway visibility monitoring and obtain highway monitoring videos;
[0033] A visibility monitoring module, used for performing single-object framing on the highway monitoring video to obtain a single-vehicle image frame set;
[0034] The visibility monitoring module is further used to compare visible color differences according to the single vehicle image frame set, obtain visible color contrast enthalpy corresponding to each single vehicle image frame, and then determine the visible color contrast enthalpy sequence according to the time sequence;
[0035] The visibility monitoring module is further used to extract the object speed according to the single vehicle image frame set to obtain an object speed scatter plot, extract the light lift rate of the vehicle object according to the single vehicle image frame set to obtain an object light lift rate scatter plot, perform vehicle monitoring confidence detection based on the object speed scatter plot and the object light lift rate scatter plot to obtain a vehicle monitoring confidence level;
[0036] An alarm module is used to determine a visible color transfer matrix according to the color contrast enthalpy sequence when the vehicle monitoring confidence energy level is higher than the energy level threshold, perform visibility observation based on the visible color transfer matrix and the vehicle monitoring confidence energy level to obtain vehicle object visibility, and perform visibility graded alarm according to the vehicle object visibility.
[0037] In a third aspect, the present application provides a computer terminal device, comprising a memory and a processor, wherein the memory stores codes, and the processor is configured to obtain the codes and execute the above-mentioned video-based visibility monitoring method.
[0038] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one computer program, and the computer program is loaded and executed by a processor to implement the operations performed by the above-mentioned video-based visibility monitoring method.
[0039] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects:
[0040] In a video-based visibility monitoring system and method provided by the present application, a highway monitoring video is first acquired, and the highway monitoring video is framed by single objects to obtain a set of single vehicle image frames. Visible color difference comparison is performed based on the set of single vehicle image frames to obtain visible color contrast enthalpy corresponding to each single vehicle image frame, and then a visible color contrast enthalpy sequence is determined based on a time sequence. The object speed is extracted based on the set of single vehicle image frames to obtain an object speed scatter plot. The light rise rate is extracted based on the set of single vehicle image frames to obtain an object light rise rate scatter plot. Vehicle monitoring confidence detection is performed based on the object speed scatter plot and the object light rise rate scatter plot to obtain a vehicle monitoring confidence Energy level, when the vehicle monitoring confidence energy level is higher than a preset threshold, the visible color transfer matrix is determined according to the color contrast enthalpy sequence, the vehicle object visibility is determined according to the visible color transfer matrix and the vehicle monitoring confidence energy level, and a visibility graded alarm is performed according to the vehicle object visibility. The visible color contrast enthalpy in the present application reflects the color visibility of the vehicle moving object in the video, and then the vehicle object visibility is determined according to the change in the image color visibility of the vehicle moving object. An alarm is performed according to the vehicle object visibility, which avoids the problem of road visibility alarm errors caused by the weakening of the fog lamp light source due to the aging of the fog lamp, thereby improving the accuracy of road visibility monitoring. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] Figure 1 is an exemplary flow chart of a video-based visibility monitoring method according to some embodiments of the present application;
[0042] Figure 2 is an exemplary flow chart of performing single object framing on the highway monitoring video in some embodiments of the present application;
[0043] Figure 3 is a schematic diagram of exemplary hardware and / or software of a visibility monitoring unit according to some embodiments of the present application;
[0044] Figure 4 It is a structural diagram of a computer terminal device for implementing a video-based visibility monitoring method according to some embodiments of the present application. DETAILED DESCRIPTION
[0045] The present application obtains a highway monitoring video, performs single-object frame segmentation on the highway monitoring video, obtains a single-vehicle image frame set, performs visible color difference comparison based on the single-vehicle image frame set, obtains the visible color contrast enthalpy corresponding to each single-vehicle image frame, and then determines the visible color contrast enthalpy sequence according to the time sequence, extracts the object speed based on the single-vehicle image frame set to obtain an object speed scatter plot, extracts the light rise rate based on the single-vehicle image frame set to obtain an object light rise rate scatter plot, performs vehicle monitoring confidence detection based on the object speed scatter plot and the object light rise rate scatter plot, obtains the vehicle monitoring confidence level, and when the vehicle monitoring confidence level is When the level is higher than a preset threshold, the visible color transfer matrix is determined according to the color contrast enthalpy sequence, the vehicle object visibility is determined according to the visible color transfer matrix and the vehicle monitoring confidence level, and a visibility graded alarm is performed according to the vehicle object visibility. The visible color contrast enthalpy in the present application reflects the color visibility of the vehicle moving object in the video, and then the vehicle object visibility is determined according to the change in the image color visibility of the vehicle moving object. An alarm is performed according to the vehicle object visibility, thereby avoiding the problem of road visibility alarm errors caused by the weakening of the fog lamp light source due to the aging of the fog lamp, thereby improving the accuracy of road visibility monitoring.
[0046] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1 , which is an exemplary flow chart of a visibility monitoring method based on video according to some embodiments of the present application. The visibility monitoring method based on video 100 mainly includes the following steps:
[0047] In step S101, highway visibility monitoring is started to obtain highway monitoring video.
[0048] It should be noted that the highway monitoring video refers to a video taken by a camera or other monitoring equipment for monitoring highway conditions. Highway monitoring video plays an important role in traffic management, road maintenance and safety monitoring. In this embodiment, the highway monitoring video is mainly used to monitor the road due to reduced visibility caused by extreme weather such as fog, rain and snow.
[0049] Optionally, in some embodiments, highway monitoring videos may be acquired through a high-definition camera, and in some other embodiments, other devices or equipment capable of video acquisition may be used, which is not limited here.
[0050] In step S102, the highway monitoring video is framed by single object to obtain a set of single vehicle image frames.
[0051] It should be noted that the single vehicle image frame set is a video image frame set that only contains one vehicle moving object, so the road visibility on the highway can be monitored by monitoring the image frame visibility changes of the vehicle moving object in the video.
[0052] Optionally, in some embodiments, reference Figure 2 As shown, this figure is an exemplary flow chart of performing single object framing on the highway monitoring video in some embodiments of the present application. Performing single object framing on the highway monitoring video to obtain a single vehicle image frame set specifically includes:
[0053] In step S1021, the highway monitoring video is framed to obtain a highway monitoring video frame set. Frame division is a process of decomposing a continuous video stream into separate image frames. In specific implementation, a corresponding decoder may be used to frame the highway monitoring video.
[0054] In step S1022, edge detection is performed on each highway monitoring video frame in the highway monitoring video frame set to determine the number of connected regions corresponding to each highway monitoring video frame;
[0055] In step S1023, according to the number of connected regions corresponding to each highway monitoring video frame, the highway monitoring video frames having one connected region and being adjacent in time sequence are taken as a single vehicle image frame set.
[0056] It should be noted that edge detection on highway monitoring video frames refers to finding places in the video frames where the grayscale changes significantly, which usually represent the boundaries of objects or important features in the image. In some embodiments, the Sobel operator commonly used in edge detection of images can be used to perform edge detection on the highway monitoring video frames. The Sobel operator is a commonly used edge detection operator, which utilizes the first-order derivative of the pixel grayscale value in the image. By performing Sobel filtering on the image in the horizontal and vertical directions, the gradient image of the image in these two directions can be obtained, and then the edge is determined based on the size of the gradient image. The edge area is then connected to obtain one or more connected areas, and the number of connected areas is counted. The highway monitoring video frames with a connected area number of 1 and adjacent and uninterrupted in time sequence are taken as a group of single-vehicle image frame sets, and the single-vehicle image frame set with the largest number of video frame images is taken as the final single-vehicle image frame set for visibility monitoring.
[0057] Optionally, in some embodiments, when the brightness of the video scene is too low, a grayscale threshold may be set to perform edge detection on the highway monitoring video frame according to a preset grayscale threshold.
[0058] In step S103, visible color difference comparison is performed according to the single vehicle image frame set to obtain visible color contrast enthalpy corresponding to each single vehicle image frame, and then a visible color contrast enthalpy sequence is determined according to the time sequence.
[0059] It should be noted that the visible color contrast enthalpy is a quantified value of the color visibility of the vehicle object in a single vehicle image frame, which is determined based on the uniformity and brightness value of the object connected area in the image frame, and can reflect the ability of the human eye to detect objects in the image frame. Specifically, the greater the brightness difference within the object connected area and the greater the brightness difference between the object connected area and the surrounding environment, the easier it is for the human eye to capture the movement of the vehicle object.
[0060] Preferably, in some embodiments, performing visible color difference comparison according to the single vehicle image frame set to obtain visible color contrast enthalpy corresponding to each single vehicle image frame specifically includes:
[0061] Acquire the first single vehicle image frame in the single vehicle image frame set;
[0062] Performing edge detection on the single vehicle image frame to obtain an object connected area;
[0063] Determining a visible color contrast enthalpy corresponding to a first single vehicle image frame according to the visible color difference of the object connected area and the visible color difference of the single object image frame;
[0064] The visible color contrast enthalpy corresponding to the remaining single vehicle image frames in the single vehicle image frame set is determined in the same manner as the first single vehicle image frame.
[0065] It should be noted that the object connected region can detect edge feature points of the vehicle object in the image through a Sobel edge detection operator, and then connect the edge feature points, and use the connected region as the object connected region.
[0066] Optionally, in some embodiments, the visible color difference of the object connected area = the standard deviation of the brightness of the edge area of the object connected area - the standard deviation of the brightness of the center area, the visible color difference of the single object image frame = the brightness of the edge area of the object connected area - the brightness of the single object image frame, and the visible color contrast enthalpy = the visible color difference of the single object image frame × the visible color difference of the object connected area.
[0067] In step S104, the object speed is extracted according to the single vehicle image frame set to obtain an object speed scatter plot, the light lift velocity of the vehicle object is extracted according to the single vehicle image frame set to obtain an object light lift velocity scatter plot, and vehicle monitoring confidence detection is performed based on the object speed scatter plot and the object light lift velocity scatter plot to obtain a vehicle monitoring confidence energy level.
[0068] It should be noted that the vehicle monitoring confidence level reflects the credibility of visibility monitoring based on the vehicle object in the video. The vehicle monitoring confidence level is determined based on the correlation between the vehicle speed in the video image frame and the brightness change speed of the vehicle light source. The visibility monitoring process is then adjusted according to the vehicle monitoring confidence level to avoid interference from other light sources such as lightning in thunderstorms and affect road visibility monitoring.
[0069] It should be noted that the object speed is the driving speed of the vehicle object. In some embodiments, extracting the object speed according to the single vehicle image frame set to obtain the object speed scatter plot specifically includes:
[0070] Obtaining the object centroid position of each single vehicle image frame in the single vehicle image frame set;
[0071] According to the object centroid position of each single vehicle image frame, the object speed corresponding to each single vehicle image frame is determined, and then a scatter plot of the object speed is drawn according to the time sequence.
[0072] In some embodiments, the object center of mass position can be obtained by obtaining the connected area of the object, taking the coordinate average of all pixel points in the connected area as the object center of mass position, and then determining the object speed corresponding to different image frames according to the change of the object center of mass position in the image, and then taking the object speed corresponding to different image frames as data points to draw an object speed scatter plot. In specific implementation, the object speed = (difference in object center of mass positions between adjacent single-vehicle image frames × video scale) / video frame time interval.
[0073] In some embodiments, the object speed may also be determined by the relative displacement rate between the object's centroid position and the video marker point, which is not limited here.
[0074] It should be noted that the light lift rate is the grayscale change rate of the vehicle object at the corresponding moments of different image frames in the process of changing the image frames of the video, reflecting the increase in the light brightness of the vehicle object at the corresponding moments of different image frames of the video. In some embodiments, the light lift rate of the vehicle object is extracted according to the single vehicle image frame set to obtain the object light lift rate scatter plot specifically including:
[0075] Determining an object grayscale mean value of each single vehicle image frame in the single vehicle image frame set;
[0076] According to the object grayscale mean of each single vehicle image frame, the light lift rate corresponding to each single vehicle image frame is determined, and then a scatter plot of the object light lift rate is drawn according to the time series. In specific implementation, the object grayscale mean is the grayscale mean in the connected area, and the light lift rate of the vehicle object = the difference between the object grayscale means of adjacent single vehicle image frames / video frame time interval.
[0077] In some embodiments, performing vehicle monitoring confidence detection based on the object speed scatter plot and the object light rise rate scatter plot to obtain the vehicle monitoring confidence level specifically includes:
[0078] According to each object speed value in the object speed scatter diagram and each object light lift velocity value in the object light lift velocity scatter diagram, a correlation test is performed to obtain a light lift correlation degree;
[0079] The light lift correlation is mapped to a corresponding vehicle monitoring confidence level.
[0080] It should be noted that when a vehicle is traveling from a distance toward the camera, its overall brightness ratio in the image frame will have a certain regularity with its driving speed. By detecting the correlation between the actual speed change of the vehicle object and the grayscale change of the vehicle object, the degree of interference of the video by random light sources can be detected and used as the confidence level of the video, thereby avoiding interference of the visibility monitoring process with external random light sources in the video. For example, when there are multiple lightning strikes or the driver of the vehicle changes the state of the headlights and other random situations, the vehicle monitoring confidence level will not be used for visibility decision-making because it is too low. In specific implementation, the Pearson correlation coefficient between each object speed value in the object speed scatter plot and each object light lift rate value in the object light lift rate scatter plot can be used as the light lift correlation, and then the light lift correlation can be mapped to the corresponding vehicle monitoring confidence level through a preset mapping table. For example, in some embodiments, the vehicle monitoring confidence level can be divided into five levels: 1, 2, 3, 4, and 5. The larger the light lift correlation, the higher the corresponding vehicle monitoring confidence level.
[0081] In step S105, when the vehicle monitoring confidence energy level is higher than the energy level threshold, a visible color transfer matrix is determined according to the color contrast enthalpy sequence, visibility observation is performed based on the visible color transfer matrix and the vehicle monitoring confidence energy level to obtain vehicle object visibility, and a visibility graded alarm is performed based on the vehicle object visibility.
[0082] Optionally, in some embodiments, the energy level threshold is preset to a fixed value based on a safety factor set during visibility monitoring, which is not elaborated here.
[0083] Optionally, in some embodiments, when the vehicle monitoring confidence level is lower than a level threshold, the single vehicle image frame is re-determined for visibility monitoring.
[0084] It should be noted that the vehicle object visibility is determined based on the color change rate of the vehicle object in the highway monitoring video. Specifically, the lower the visibility in foggy weather, the lower the degree of video color change caused by the vehicle object approaching the camera. Therefore, the visibility of the vehicle in foggy weather can be determined based on the degree of color change of the moving object in each image frame of the video, thereby realizing the monitoring of the overall visibility of the highway in foggy weather, thereby avoiding the problem of road visibility alarm errors caused by the weakening of the fog light source due to the aging of the fog lamp, thereby improving the accuracy of road visibility monitoring.
[0085] It should be noted that the visible color transfer matrix includes the color feature change information of the vehicle object during the video monitoring process, which is used to identify the visibility of the vehicle object. Preferably, in some embodiments, determining the visible color transfer matrix according to the color contrast enthalpy sequence specifically includes:
[0086] Acquire object connected regions corresponding to each single vehicle image frame in the single vehicle image frame set;
[0087] The color eigenvalues corresponding to the connected regions of a single object are used as matrix columns to obtain an initial visible color transfer matrix;
[0088] The initial visible color transfer matrix is corrected according to each visible color contrast enthalpy in the visible color contrast enthalpy sequence to obtain a visible color transfer matrix.
[0089] In some embodiments, the color eigenvalues in the initial visible color transfer matrix include: the average red brightness, the average green brightness and the average blue brightness of the object connected area. In some embodiments, each visible color contrast enthalpy in the visible color contrast enthalpy sequence can be used as a corresponding correction coefficient to correct a column of matrix elements in the initial visible color transfer matrix. For example, the visible color contrast enthalpy of each single-object image frame is multiplied by the corresponding color eigenvalues of the object connected area in each single-object image frame, so as to achieve correction of the initial visible color transfer matrix and obtain the visible color transfer matrix.
[0090] It should be noted that the visible color contrast enthalpy can reflect the actual degree of observation clarity of the human eye. Therefore, using the visible color contrast enthalpy to correct the visible color transfer matrix can be more consistent with the results of observation captured by the human eye, improve the accuracy of the vehicle object visibility determination process, and reduce the accident rate caused by low road visibility in foggy weather.
[0091] It should be noted that the vehicle object visibility is the visibility of the vehicle object during the video monitoring time period, which is dimensionless. Preferably, in some embodiments, the visibility observation is performed based on the visible color transfer matrix and the vehicle monitoring confidence level, and the vehicle object visibility is obtained specifically including:
[0092] Performing inter-column difference processing on the visible color transfer matrix to obtain a visible color difference matrix;
[0093] Determining a color change sequence according to the visible color difference matrix;
[0094] Selecting color change values in the color change sequence to form a visibility decision sequence according to the vehicle monitoring confidence level;
[0095] The visibility of the vehicle object is determined according to the visibility decision sequence. In a specific implementation, the visibility of the vehicle object can be obtained by multiplying the sequence mean of the visibility decision sequence with a preset proportional coefficient, so that when the color change speed of the vehicle is low, the visibility alarm is activated to reduce the traffic accident rate caused by low road visibility.
[0096] In some embodiments, performing inter-column difference processing on the visible color transfer matrix to obtain a visible color difference matrix specifically includes: replacing original matrix columns with difference values between adjacent matrix columns to obtain the visible color difference matrix; determining a color change sequence based on the visible color difference matrix specifically includes: taking the mean values of the matrix columns as color change values, and sorting the color change values from small to large according to their sizes to obtain the color change sequence.
[0097] Optionally, in some embodiments, according to the vehicle monitoring confidence level, different proportions of color change values in the color change sequence are selected to form the visibility decision sequence. Specifically, the lower the vehicle monitoring confidence level, the more color change values are in the visibility decision sequence. The specific selected proportion value is not limited here. Thus, when the vehicle monitoring confidence level is low, more color change values can be obtained for visibility decision-making, thereby improving the credibility and accuracy of visibility monitoring.
[0098] Optionally, in some embodiments, performing visibility graded alarm according to the vehicle object visibility specifically includes: mapping the vehicle object visibility to a corresponding alarm level according to a preset alarm mapping table, and sending the alarm level to an alarm center for alarm, thereby improving the stability of highway visibility monitoring and reducing the accident rate on highways in foggy weather.
[0099] In addition, in another aspect of the present application, in some embodiments, the present application provides a video-based visibility monitoring system, the system comprising a visibility monitoring unit, referring to Figure 3 , which is a schematic diagram of exemplary hardware and / or software of a visibility monitoring unit according to some embodiments of the present application, wherein the visibility monitoring unit 200 includes: a video acquisition module 201, a visibility monitoring module 202 and an alarm module 203, which are described as follows:
[0100] Video acquisition module 201, in some specific embodiments of the present application, video acquisition module 201 is mainly used to start highway visibility monitoring and obtain highway monitoring video;
[0101] The visibility monitoring module 202, in some specific embodiments of the present application, is mainly used to perform single-object framing on the highway monitoring video to obtain a single-vehicle image frame set;
[0102] In some embodiments, the visibility monitoring module 202 is further used to perform visible color difference comparison according to the single vehicle image frame set, obtain visible color contrast enthalpy corresponding to each single vehicle image frame, and then determine the visible color contrast enthalpy sequence according to the time sequence;
[0103] In some embodiments, the visibility monitoring module 202 is further used to extract the object speed according to the single vehicle image frame set to obtain an object speed scatter plot, extract the light lift rate of the vehicle object according to the single vehicle image frame set to obtain an object light lift rate scatter plot, perform vehicle monitoring confidence detection based on the object speed scatter plot and the object light lift rate scatter plot to obtain a vehicle monitoring confidence level;
[0104] Alarm module 203. In some specific embodiments of the present application, the alarm module 203 is mainly used to determine the visible color transfer matrix according to the color contrast enthalpy sequence when the vehicle monitoring confidence energy level is higher than the energy level threshold, perform visibility observation based on the visible color transfer matrix and the vehicle monitoring confidence energy level to obtain vehicle object visibility, and perform visibility graded alarm based on the vehicle object visibility.
[0105] The above describes in detail an example of a video-based visibility monitoring system and method provided in an embodiment of the present application. It can be understood that in order to achieve the above functions, the corresponding device includes a hardware structure and / or software module corresponding to executing each function.
[0106] Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed in this document, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function in the application is executed in hardware or in a computer software-driven hardware manner depends on the specific application and design constraints of the technical solution. Therefore, professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0107] In addition, the present application also provides a computer terminal device, which includes a memory and a processor, wherein the memory stores a code, and the processor is configured to obtain the code and execute the above-mentioned video-based visibility monitoring method.
[0108] In some embodiments, reference Figure 4 , which is a schematic diagram of the structure of a computer terminal device using a video-based visibility monitoring method according to some embodiments of the present application. A video-based visibility monitoring method in the above embodiment can be performed by Figure 4The computer terminal device 300 shown in the figure is implemented, and the computer terminal device 300 includes at least one communication bus 301, a communication interface 302, a processor 303 and a memory 304.
[0109] The processor 303 may be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more processors for controlling the execution of a video-based visibility monitoring method in the present application.
[0110] The communication bus 301 may include a pathway for transmitting information between the above-mentioned components.
[0111] The memory 304 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, an optical disc storage (including a compressed optical disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a disk or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 304 may exist independently and be connected to the processor 303 via the communication bus 301. The memory 304 may also be integrated with the processor 303.
[0112] The memory 304 is used to store the program code for executing the solution of the present application, and the execution is controlled by the processor 303. The processor 303 is used to execute the program code stored in the memory 304. The program code may include one or more software modules. The determination of the visible color contrast enthalpy in the above embodiment can be implemented by the processor 303 and one or more software modules in the program code in the memory 304.
[0113] The communication interface 302 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.
[0114] Optionally, the computer terminal device 300 may further include a power supply 305 for providing power to various devices or circuits in the real-time computer terminal device.
[0115] In a specific implementation, as an embodiment, a computer terminal device may include multiple processors, each of which may be a single-CPU processor or a multi-CPU processor. The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).
[0116] The computer terminal device may be a general-purpose computer terminal device or a dedicated computer terminal device. In a specific implementation, the computer terminal device may be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device or an embedded device. The embodiment of the present application does not limit the type of computer terminal device.
[0117] In addition, in other aspects of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores at least one computer program, and the computer program is loaded and executed by a processor to implement the operations performed by the above-mentioned video-based visibility monitoring method.
[0118] In summary, in a video-based visibility monitoring system and method disclosed in an embodiment of the present application, a highway monitoring video is first acquired, and the highway monitoring video is framed by single objects to obtain a set of single vehicle image frames. Visible color difference comparison is performed based on the set of single vehicle image frames to obtain visible color contrast enthalpy corresponding to each single vehicle image frame, and then a visible color contrast enthalpy sequence is determined based on a time sequence. The object speed is extracted based on the set of single vehicle image frames to obtain an object speed scatter plot. The light rise rate is extracted based on the set of single vehicle image frames to obtain an object light rise rate scatter plot. Vehicle monitoring confidence detection is performed based on the object speed scatter plot and the object light rise rate scatter plot to obtain a vehicle monitoring confidence test. Control confidence level, when the vehicle monitoring confidence level is higher than a preset threshold, the visible color transfer matrix is determined according to the color contrast enthalpy sequence, the vehicle object visibility is determined according to the visible color transfer matrix and the vehicle monitoring confidence level, and a visibility graded alarm is performed according to the vehicle object visibility. The visible color contrast enthalpy in the present application reflects the color visibility of the vehicle moving object in the video, and then the vehicle object visibility is determined according to the change in the image color visibility of the vehicle moving object, and an alarm is performed according to the vehicle object visibility, thereby avoiding the problem of road visibility alarm errors caused by the weakening of the fog lamp light source due to the aging of the fog lamp, thereby improving the accuracy of road visibility monitoring.
[0119] The above is only an embodiment of the present application, and the common knowledge such as the specific technical scheme or characteristics in the scheme is not described in detail here. It should be pointed out that for those skilled in the art, without departing from the technical scheme of the present application, several modifications and improvements can be made, which should also be regarded as the scope of protection of the present application, and these will not affect the effect of the implementation of the present application and the practicality of the patent.
[0120] The scope of protection claimed by this application shall be based on the content of its claims. The specific implementation methods and other records in the specification can be used to interpret the content of the claims. Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include these modifications and variations.
Claims
1. A visibility monitoring method based on video, characterized in that: include: Start highway visibility monitoring and obtain highway monitoring videos; Performing single-object framing on the highway monitoring video to obtain a single-vehicle image frame set; Performing visible color difference comparison according to the single vehicle image frame set to obtain visible color contrast enthalpy corresponding to each single vehicle image frame, and then determining a visible color contrast enthalpy sequence according to a time sequence; Extracting the object speed according to the single vehicle image frame set to obtain an object speed scatter plot, extracting the light lift velocity of the vehicle object according to the single vehicle image frame set to obtain an object light lift velocity scatter plot, performing vehicle monitoring confidence detection based on the object speed scatter plot and the object light lift velocity scatter plot to obtain a vehicle monitoring confidence energy level; When the vehicle monitoring confidence energy level is higher than the energy level threshold, a visible color transfer matrix is determined according to the visible color contrast enthalpy sequence, visibility observation is performed based on the visible color transfer matrix and the vehicle monitoring confidence energy level to obtain vehicle object visibility, and visibility graded alarm is performed based on the vehicle object visibility; Among them, performing visible color difference comparison according to the single vehicle image frame set to obtain the visible color contrast enthalpy corresponding to each single vehicle image frame specifically includes: Acquire the first single vehicle image frame in the single vehicle image frame set; Performing edge detection on the single vehicle image frame to obtain an object connected area; Determining a visible color contrast enthalpy corresponding to a first single vehicle image frame according to the visible color difference of the object connected area and the visible color difference of the single vehicle image frame; Determine the visible color contrast enthalpy corresponding to the remaining single vehicle image frames in the single vehicle image frame set in the same manner as the first single vehicle image frame; The step of extracting the light lift rate of the vehicle object according to the single vehicle image frame set to obtain the object light lift rate scatter plot specifically includes: Determining an object grayscale mean value of each single vehicle image frame in the single vehicle image frame set; According to the object grayscale mean value of each single vehicle image frame, the light lift rate corresponding to each single vehicle image frame is determined, and then a scatter plot of the object light lift rate is drawn according to the time sequence; The light rise rate of the vehicle object=the difference between the object grayscale means of adjacent single vehicle image frames / the video frame time interval.
2. The method according to claim 1, characterized in that The highway monitoring video is framed by a single object to obtain a single vehicle image frame set, which specifically includes: Frame-dividing the highway monitoring video to obtain a highway monitoring video frame set; Performing edge detection on each highway monitoring video frame in the highway monitoring video frame set to determine the number of connected regions corresponding to each highway monitoring video frame; According to the number of connected regions corresponding to each highway monitoring video frame, the highway monitoring video frames with one connected region and adjacent in time sequence are taken as a single vehicle image frame set.
3. The method according to claim 1, characterized in that Extracting the object speed according to the single vehicle image frame set and obtaining the object speed scatter plot specifically includes: Obtaining the object centroid position of each single vehicle image frame in the single vehicle image frame set; According to the object centroid position of each single vehicle image frame, the object speed corresponding to each single vehicle image frame is determined, and then a scatter plot of the object speed is drawn according to the time sequence.
4. The method according to claim 1, characterized in that The vehicle monitoring confidence detection is performed based on the object speed scatter plot and the object light rise rate scatter plot to obtain the vehicle monitoring confidence level, which specifically includes: Perform a correlation test based on each object speed value in the object speed scatter diagram and each object light lift velocity value in the object light lift velocity scatter diagram to obtain a light lift correlation; The light lift correlation is mapped to a corresponding vehicle monitoring confidence level.
5. The method according to claim 1, characterized in that Video is collected through high-definition cameras to obtain highway monitoring videos.
6. A video-based visibility monitoring system, which uses the method of any one of claims 1 to 5 to perform visibility monitoring, and the video-based visibility monitoring system includes a visibility monitoring unit, characterized in that: The visibility monitoring unit comprises: Video acquisition module, used to start highway visibility monitoring and obtain highway monitoring videos; A visibility monitoring module, used for performing single-object framing on the highway monitoring video to obtain a single-vehicle image frame set; The visibility monitoring module is further used to compare visible color differences according to the single vehicle image frame set, obtain visible color contrast enthalpy corresponding to each single vehicle image frame, and then determine the visible color contrast enthalpy sequence according to the time sequence; The visibility monitoring module is further used to extract the object speed according to the single vehicle image frame set to obtain an object speed scatter plot, extract the light lift rate of the vehicle object according to the single vehicle image frame set to obtain an object light lift rate scatter plot, perform vehicle monitoring confidence detection based on the object speed scatter plot and the object light lift rate scatter plot to obtain a vehicle monitoring confidence level; An alarm module is used to determine a visible color transfer matrix according to the visible color contrast enthalpy sequence when the vehicle monitoring confidence energy level is higher than the energy level threshold, perform visibility observation based on the visible color transfer matrix and the vehicle monitoring confidence energy level to obtain vehicle object visibility, and perform visibility graded alarm according to the vehicle object visibility.
7. A computer terminal device, characterized in that: The computer terminal device includes a memory and a processor, the memory stores codes, and the processor is configured to obtain the codes and execute a video-based visibility monitoring method as described in any one of claims 1 to 5.
8. A computer-readable storage medium storing at least one computer program, characterized in that: The computer program is loaded and executed by a processor to implement the operations performed by the video-based visibility monitoring method as described in any one of claims 1 to 5.
Citation Information
Patent Citations
Method and system capable of preventing highway traffic accident caused by agglomerate fog
CN108735000A
Highway visibility detection and intelligent induction method, system and device
CN116798222A
Vehicle parking risk early warning system based on vehicle-mounted video monitoring
CN117315991A
Video transmission device, video transmission method, program, and recording medium
JP2022157267A
