System for video ai to identify congestion state of vehicle

Abstract

The application discloses a system for identifying congestion state of a vehicle by a video AI, comprising a video image conversion module, a vehicle position calculation module, a congestion state identification module, a window calculation unit, a vehicle color calculation unit, a data cache unit and a congestion state calculation unit, wherein the video image conversion module converts a video image into image information; the vehicle position calculation module calculates image position and confidence of a vehicle in the image information; the window calculation unit divides a time period into multiple segments according to image input frequency; the vehicle color calculation unit identifies whether the colors of stacked vehicles in overlapping area positions are consistent; the data cache unit stores vehicle position information in a cached vehicle list; and the congestion state calculation unit compares data information of the cached vehicle list in the cache unit with the image position and confidence of the vehicle in the image information calculated by the vehicle position calculation module, to calculate and obtain the congestion state of the vehicle, thereby realizing intelligent management of vehicle congestion.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence vehicle detection, and more specifically to a video AI system for recognizing vehicle congestion. Background Technology

[0002] In communities, industrial parks, commercial complexes, scenic spots, and other business formats, traffic congestion caused by excessive traffic flow, breakdowns, or road obstacles is a common phenomenon, resulting in poor user travel experience, traffic and safety problems, and even economic losses.

[0003] Traditional solutions for managing traffic congestion typically involve manual reporting and traditional video surveillance of road conditions, which face challenges such as higher project labor costs and reduced processing efficiency.

[0004] Therefore, it is necessary to develop a video AI system for recognizing traffic congestion with high accuracy. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, the present invention aims to provide a video AI system, electronic device, and storage medium for recognizing vehicle congestion. By flexibly setting the number of vehicles in congestion and the duration of congestion triggering, the detection results are output as alarm events to automatically notify relevant management personnel, and the system supports linkage control of corresponding traffic diversion devices to achieve the purpose of automatic vehicle diversion.

[0006] This invention is achieved through the following technical solution: a video AI system for recognizing vehicle congestion status, comprising: a video image conversion module for converting video images into image information; a vehicle position calculation module for calculating the image position and confidence level of a vehicle in the image information; and a congestion status recognition module having a window calculation unit, a vehicle color calculation unit, a data caching unit, and a congestion status calculation unit. The window calculation unit divides a time period into multiple segments according to the image input frequency. The vehicle color calculation unit identifies whether the colors of stacked vehicles in overlapping areas are consistent. The data caching unit stores vehicle position information from a cached vehicle list. The congestion status calculation unit compares the image position and confidence level of the vehicle in the image information calculated by the current vehicle position calculation module with the cached vehicle list data information in the caching unit to calculate the vehicle's congestion status.

[0007] Furthermore, the window calculation unit divides the time period into multiple segments according to the image input frequency, specifically by feeding back the calculation results of multiple time points as the calculation results of the time period.

[0008] Furthermore, the method by which the vehicle color calculation unit identifies whether the colors of stacked vehicles in the overlapping area are consistent specifically includes the following steps: Step S1: Convert the vehicle image color from BGR to HSV; Step S2: Split the HSV skin and obtain the color panel and brightness panel based on the skin; Step S3: Loop through each pixel of the image to obtain the HSV histogram and brightness histogram; Step S4: Calculate the range of each color in the histogram, and the highest range is the vehicle color.

[0009] Furthermore, the method by which the congestion status calculation unit calculates the vehicle's congestion status by comparing the vehicle's image position and confidence level in the image information calculated by the current vehicle position calculation module with the cached data in the cache unit is as follows:

[0010] Step S10: When the confidence level calculated by the vehicle position calculation module is higher than the set reference value Vehicle location information, area set to , , ..., ;

[0011] Step S20: Obtain the list of the last vehicles within the time window from the data cache unit, and set the area to... , , ..., ;

[0012] Step S30: Loop through the vehicle location list in the vehicle location calculation module and find the intersection of each location with the cached vehicle list. Let the intersection area be denoted as . Then the intersection rate is calculated as follows:

[0013]

[0014] If one of the locations in the cached vehicle list is found, the intersection rate is greater than a set reference value. If the colors are consistent, then the cached information is overwritten and the current time point is set to true;

[0015] If not found, add a new vehicle to the cached vehicle list and set the current time point to true;

[0016] If there are no covered locations in the cached vehicle list, set the current time point to false.

[0017] Step S40: Calculate the true / false status of each position at each time point within the current time window. For each position with all true values, increment the total count by 1. When the total count exceeds a set reference value... It can determine when vehicles are in a congested state.

[0018] Furthermore, the video image conversion module includes a video access unit, a timed execution unit, an image acquisition unit, an image storage unit, and a local message queue. The video access unit can acquire output signals from mainstream video devices in the market. The timed execution unit controls the image acquisition unit to acquire video images at certain time intervals. The image acquisition unit acquires images by capturing images from the device or by parsing the video stream. The image storage unit stores the images acquired by the image acquisition unit locally and pushes image acquisition messages to the local message queue.

[0019] Furthermore, the video device can be any video device that supports national standard GB28181, ONVIF, RTMP, RTP, WebRTC and other video protocols. Parsing the video stream to obtain the image specifically involves parsing the video stream, extracting the image from the stream, and pushing the image acquisition message, which includes the local storage address of the image and device information.

[0020] Furthermore, the vehicle location calculation module includes an image receiving unit, an image preprocessing unit, and an artificial intelligence model inference unit. The image receiving unit obtains image information from a local message queue. The image preprocessing unit crops the target inference region from the image obtained by the image receiving unit and scales it to the pixel combination and channel number input to the artificial intelligence model. The artificial intelligence model inference unit receives the preprocessed image information and infers the vehicle's location and confidence level in the image.

[0021] Furthermore, the artificial intelligence model is based on the MobileNet object_detection_v2 deep learning network, and the model is optimized and trained by collecting a large number of vehicle images. The target inference region is the rectangular region configured when the system starts. The pixel combination and channel number input to the artificial intelligence model are the image pixel value and channel number, the image position is the rectangular region relative to the image pixel value, and the confidence level is the degree of confidence that the image is a vehicle and appears in the image position, which fluctuates between 0 and 1.

[0022] Furthermore, it also includes an Internet of Things (IoT) module and an intelligent linkage module. The IoT module receives the input results from the congestion status identification module and uploads them to the intelligent linkage module. The intelligent linkage module, based on the changes in the received vehicle congestion status information, coordinates with surrounding vehicle diversion devices in real time.

[0023] Compared with existing technologies, the beneficial effects achieved by this invention are as follows:

[0024] Since the present invention flexibly sets the number of congested vehicles to be detected, sets the trigger duration for congestion, outputs the detection result in the form of an alarm event to automatically notify relevant management personnel, and supports linkage control of the corresponding diversion device to achieve automatic vehicle diversion. Compared with the traditional management scheme, this technology reduces the labor cost of management, improves the response time for fault handling, and makes vehicle passage smoother. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 The figure shows a schematic flowchart of a system for the video AI to identify the congestion status of vehicles according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] Next, in combination with the drawings and the specific embodiments, the present invention will be further described. It should be noted that, on the premise of no conflict, the following-described embodiments or technical features can be arbitrarily combined to form new embodiments.

[0027] As Figure 1 shown, an embodiment of the present invention discloses a flowchart of a system for the video AI to identify the congestion status of vehicles. The method for the video AI to identify the congestion status of vehicles provided in this embodiment is applicable to efficient vehicle congestion management to achieve automatic vehicle diversion, and is specifically applied to a system for the video AI to identify the congestion status of vehicles composed of a terminal device, a network, and a server. The network is a medium for providing a communication link directly between the terminal device and the server, and it can include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.; the operating system on the terminal device can include a handheld device operating system (iPhone operating system, iOS system), Android system, or other operating systems. The terminal device is connected to the server through the network to achieve interaction, so as to perform operations such as receiving or sending data, and specifically can be various electronic devices with a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, portable computers, and desktop servers, etc. This embodiment is applicable to the situation of artificial intelligence managing vehicle congestion status to achieve big data vehicle traffic scheduling. Typically applicable to community, park, commercial complex, and scenic area management. The system for the video AI to identify the congestion status of vehicles can be implemented in software and / or hardware. Before executing the system for the video AI to identify the congestion status of vehicles, a confidence reference value is preset, and the specific reference value is the credibility of the recognition result, and a decimal value between 0 and 1 is configured according to the actual detection environment and the accuracy requirement of manual needs; the intersection rate reference value is preset, and the specific reference value This indicates the degree of overlap between two recognition results: a value of 1 requires an inclusion relationship between the two results; a value of 0 indicates that no intersection is needed to classify them as the same location; a value between 0 and 1 indicates that intersection is required to classify them as the same location. The larger the value, the greater the area of ​​overlap required to classify them as the same location. This is a critical reference value for vehicles in congested conditions. Specific reference values To specify the minimum number of vehicles required for congestion, for example, if 5 vehicles are set, then 6 or more vehicles on that road segment are a necessary condition for determining congestion.

[0028] See Figure 1 This embodiment provides a system for video AI to identify vehicle congestion status, including:

[0029] The video image conversion module converts video images into image information. Specifically, in this embodiment, the video image conversion module includes a video access unit, a timing execution unit, an image acquisition unit, and an image storage unit. The video access unit can acquire output signals from mainstream video devices in the market. The timing execution unit controls the image acquisition unit to acquire video images at certain time intervals. The image acquisition unit acquires images by capturing images from the device or by parsing the video stream. The image storage unit stores the images acquired by the image acquisition unit locally to form a local message queue and pushes image acquisition messages. Furthermore, to increase the versatility of this system, the video device can be any video device that supports national standard GB28181, ONVIF, RTMP, RTP, WebRTC, and other video protocols. Parsing the video stream to acquire images specifically involves parsing the video stream, extracting images from the stream, and pushing image acquisition messages that include the local image storage address and device information.

[0030] The vehicle location calculation module calculates the vehicle's image location and confidence level within the image information. Specifically, the vehicle location calculation module includes an image receiving unit, an image preprocessing unit, and an artificial intelligence model inference unit. The image receiving unit acquires image information from a local message queue. The image preprocessing unit crops the image acquired by the image receiving unit to extract the target inference region and scales it to the pixel combination and channel number input to the artificial intelligence model. The artificial intelligence model inference unit receives the preprocessed image information and infers the vehicle's image location and confidence level. The artificial intelligence model is based on the MobileNet object_detection_v2 deep learning network and is optimized and trained through extensive collection of vehicle images to achieve better inference performance. The target inference region is a rectangular area configured at system startup to reduce the impact of invalid data in the image. The pixel combination and channel number input to the artificial intelligence model are the image pixel values ​​and channel number. The image location is a rectangular area relative to the image pixel value. The confidence level is the degree to which the image is of a vehicle and appears in the image location. It fluctuates between 0 and 1. The closer the confidence level is to 1, the closer the vehicle is to its actual location in the image location, and the more reliable the result is.

[0031] The congestion status recognition module includes a window calculation unit, a vehicle color calculation unit, a data caching unit, and a congestion status calculation unit. The window calculation unit divides the time period into multiple segments according to the image input frequency. The vehicle color calculation unit identifies whether the colors of stacked vehicles in overlapping areas are consistent. The data caching unit stores vehicle position information from a cached vehicle list. The congestion status calculation unit calculates the vehicle's congestion status by comparing the image position and confidence level of the vehicle in the image information calculated by the current vehicle position calculation module with the cached vehicle list data information in the caching unit.

[0032] Specifically, the window calculation unit divides the time period into multiple segments according to the image input frequency, specifically by feeding back the calculation results of multiple time points as the calculation result of the time period; the vehicle color calculation unit identifies whether the colors of stacked vehicles in overlapping areas are consistent, specifically through steps S1: converting the vehicle image color from BGR to HSV; step S2: splitting the HSV skin to obtain the color panel and brightness panel based on the skin; step S3: looping through each pixel of the image to obtain the HSV histogram and brightness histogram; step S4: calculating the range of each color in the histogram, with the highest value indicating the vehicle color; the congestion state calculation unit calculates the vehicle's congestion state by comparing the image position and confidence score of the vehicle in the image information calculated by the current vehicle position calculation module with the cached data in the cache unit.

[0033] Step S10: When the confidence level calculated by the vehicle position calculation module is higher than the set reference value Vehicle location information, area set to , , ..., ;

[0034] Step S20: Obtain the list of the last vehicles within the time window from the data cache unit, and set the area to... , , ..., ;

[0035] Step S30: Loop through the vehicle location list in the vehicle location calculation module and find the intersection of each location with the cached vehicle list. Let the intersection area be denoted as . Then the intersection rate is calculated as follows:

[0036]

[0037] If one of the locations in the cached vehicle list is found, the intersection rate is greater than a set reference value. If the colors are consistent, then the cached information is overwritten and the current time point is set to true;

[0038] If not found, add a new vehicle to the cached vehicle list and set the current time point to true;

[0039] If there are no covered locations in the cached vehicle list, set the current time point to false.

[0040] Step S40: Calculate the true / false status of each position at each time point within the current time window. For each position with all true values, increment the total count by 1. When the total count exceeds a set reference value... It can determine when vehicles are in a congested state.

[0041] In addition, to enhance the intelligence and functionality of this system, it also includes an Internet of Things (IoT) module and an intelligent linkage module. The IoT module receives the input results from the congestion status identification module and uploads them to the intelligent linkage module. The intelligent linkage module, based on the received vehicle congestion status information, dynamically coordinates with surrounding vehicle guidance devices in real time to achieve intelligent vehicle volume scheduling and alleviate congestion. Furthermore, the vehicle guidance devices can be any equipment or system used to guide and release vehicles, such as indicator lights or barriers. The intelligent linkage module receives vehicle congestion status data reported by the IoT module and dynamically schedules and controls the vehicle guidance devices to enter corresponding operating states, achieving intelligent vehicle volume scheduling, alleviating congestion, and improving traffic efficiency.

[0042] The above embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any non-substantial changes and substitutions made by those skilled in the art based on the present invention shall fall within the scope of protection claimed by the present invention.

Claims

1. A system for identifying a congestion state of a vehicle by video AI, characterized by, The method comprises the following steps: a video image conversion module for converting video images into image information; a vehicle position calculation module for calculating image positions and confidence levels of vehicles in the image information, and forming a cache vehicle list; a congestion state identification module having a window calculation unit, a vehicle color calculation unit, a data cache unit, and a congestion state calculation unit, the window calculation unit divides a time period into multiple segments according to image input frequency, the vehicle color calculation unit identifies whether the colors of stacked vehicles at overlapping area positions are consistent, the data cache unit stores vehicle position information in the cache vehicle list, and the congestion state calculation unit calculates congestion states of vehicles by comparing vehicle positions and confidence levels in the image information calculated by the vehicle position calculation module with data information in the cache vehicle list in the cache unit; the window calculation unit divides a time period into multiple segments according to image input frequency, and the method for feeding back calculation results of multiple time points as calculation results of a time period; the vehicle color calculation unit identifies whether the colors of stacked vehicles at overlapping area positions are consistent, and the method comprises the following steps: S1: converting vehicle image colors from BGR to HSV; Step S10: When the confidence level calculated by the vehicle position calculation module is higher than a set reference value of the vehicle position information, the area is set as , , ; Step S20: obtaining the last vehicle list in the time window from the data cache unit, and setting the area as , ,… ; Step S30: Loop through the list of vehicle positions in the vehicle position calculation module, intersect each position with the list of cached vehicle positions, and set the intersection area as Then the intersection rate is calculated: If one of the locations in the cached vehicle list is found, the intersection rate is greater than the set reference value , and the colors are consistent, the cached information is overwritten, and the current time point is set to true. S2: splitting HSV skins to obtain color panels and brightness panels according to the skins; S3: looping through each pixel of an image to obtain HSV histograms and brightness histograms; S4: counting each color range in the histograms, and the highest one is the vehicle color; When the total number is greater than a set reference value the vehicle is determined to be in a congested state. 2.The video AI identification vehicle congestion state system of claim 1, wherein, the congestion state calculation unit calculates congestion states of vehicles by comparing vehicle positions and confidence levels in the image information calculated by the vehicle position calculation module with data information in the cache unit, and the method comprises the following steps: 3.The video AI identification vehicle congestion state system of claim 2, wherein, if not found, adding a vehicle in the cache vehicle list, and setting the current time point as true; if there is no covered position in the cache vehicle list, setting the current time point as false; S40: calculating true and false states of each position at each time point in the current time window, and setting the total number as 1 each time a position with all true states appears; the video image conversion module comprises a video access unit, a timing execution unit, an image acquisition unit, and an image storage unit, the video access unit can obtain output signals of mainstream video devices in the market, the timing execution unit controls the image acquisition unit to obtain video images at certain time intervals, the image acquisition unit obtains images by device screenshot or analyzes video streams to obtain images, and the image storage unit stores images obtained by the image acquisition unit into a local message queue to push image acquisition messages. The video device is any video device supporting national standard GB28181, ONVIF, RTMP, RTP, and WebRTC video protocols, the method for analyzing video streams to obtain images comprises the following steps: after analyzing a video stream, images are intercepted from the stream, and the image acquisition message includes an image local storage address and device information. 4.The video AI identification vehicle congestion state system of claim 2, wherein, The vehicle position calculation module comprises an image receiving unit, an image preprocessing unit and an artificial intelligence model inference unit, the image receiving unit obtains image information from a local message queue, the image preprocessing unit crops a target inference area from the image obtained by the image receiving unit and scales the target inference area to a combination of pixel groups and a number of channels for input of an artificial intelligence model, and the artificial intelligence model inference unit receives the preprocessed image information and infers a vehicle position in an image and a confidence level. 5.The video AI identification vehicle congestion state system of claim 4, wherein, The artificial intelligence model is a deep learning network MobileNet object_detection_v2 model, and a large number of vehicle pictures are collected for model optimization training, the target inference region is a rectangular region configured when the system starts, and the pixel combination and channel number input by the artificial intelligence model are image pixel values and channel numbers , the image position is a rectangular region relative to the image pixel value, the confidence is the credibility of the image being a vehicle and appearing in the image position, and the confidence is floating between 0 and 1. 6.The video AI identification vehicle congestion state system of claim 1, wherein, Further comprising: an Internet of Things module and an intelligent linkage module, the Internet of Things module receives an input result of the congestion state recognition module and uploads the input result to the intelligent linkage module, and the intelligent linkage module links surrounding vehicle dredging devices in real time according to a situation change of the received vehicle congestion state information.

Images