Vehicle abnormality recognition method and system based on highway dynamic data

Abstract

The present invention discloses a vehicle abnormality recognition method and system based on highway dynamic data, which relates to the field of image processing. A visual device and a server are provided, and the visual device is connected to the server signal; the server obtains image features of a first working area as a reference image to create a dynamic image library corresponding to a vehicle ID; the image features of a second working area are obtained to obtain a plurality of posture images and are integrated into a deceleration picture set; the front features of the deceleration picture set are obtained and a deceleration contour is obtained through an edge algorithm, the front features of the reference image are obtained to obtain a reference contour and an extended contour is obtained according to a deviation coefficient; the deceleration contour is compared with the extended contour, it is determined whether the deceleration contour exceeds the coverage of the extended contour and a warning message containing the vehicle ID is issued, so as to improve the risk of vehicle abnormalities that may be difficult for the driver to detect when a transport vehicle is traveling long distances on the road due to the limitations of the vehicle's own conditions and the influence of the field of vision.

Description

Technical Field

[0001] The invention relates to image data processing, and in particular to a vehicle abnormality recognition method and system based on highway dynamic data. Background Art

[0002] With the diversification of traffic management and the expansion of transportation channels, road transportation has become the main artery of cargo transportation. With the increase in the number of trucks and vehicle mileage, there are many types of trucks, and there are huge differences in the cargo transported by different trucks; this makes the current road maintenance and management have many difficulties. On the one hand, the flatness and safety of the road are mainly affected by the use environment. Generally, the more trucks there are on a road section, the shorter the road damage and maintenance cycle; this makes the management of overloaded trucks (overload control) particularly important. At present, in addition to setting up weighbridges on the road section to detect trucks, the road management system also uses networked signboards and environmental monitoring equipment to actively remind and guide the road traffic conditions and vehicle driving requirements, hoping to achieve higher road safety and coordination. On the other hand, while strict management is in place, some trucks take advantage of the limitations of the weighbridge and use some means to carry out overloaded transportation during the gaps in the weighbridge's measurements. This is highly irregular and creates certain safety hazards. More importantly, due to the age of trucks and the uncertainty of the cargo and condition of the vehicles during long-distance transportation, trucks are frequently in danger or have accidents on some roads. Therefore, in addition to the management of overloaded trucks, it is also necessary to improve the technology for identifying abnormal trucks.

[0003] The current mainstream abnormality recognition is to use high-definition camera equipment to identify vehicle violations and monitor the vehicle driving area and driver driving status in real time through the Beidou system; however, although this model can restrain and reduce the risk of accidents caused by human factors to a certain extent, for long-distance trucks, there are still many risks that drivers fail to notice in time, resulting in accidents. Therefore, it is worth studying how to better help vehicles prevent some difficult-to-detect accident risks and ensure the safety of all road users while strengthening the abnormal behavior of vehicles on long-distance sections (such as speeding, sudden braking, and frequent lane changes). Summary of the invention

[0004] The purpose of the present invention is to provide a vehicle abnormality identification method and system based on highway dynamic data, in order to improve the risk of vehicle abnormalities that may be difficult for the driver to detect when a transport vehicle is traveling long distances on the road due to the vehicle's own conditions and field of vision.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions: a vehicle abnormality recognition method based on highway dynamic data, first, determine the working section of the monitored section, set up a visual device and a server, connect the visual device with the server signal, and use the visual device to obtain the image features of the working section and upload them to the server; the above image features include the front features and side features of the vehicle; and the image features include a timestamp; the above working section includes a first working area and a second working area. The above server performs the following steps: S100, obtain the image features of the first working area as a reference image; identify the vehicle ID in the reference image, and create a dynamic image library corresponding to the vehicle ID.

[0006] S200, obtaining image features of the second working area to obtain a plurality of posture images; the server traverses the front features in the posture images, and integrates the posture images of the same vehicle ID into a deceleration picture set; wherein the deceleration picture set at least includes the front features of the same vehicle entering and exiting the second working area; and the deceleration picture set is associated with the reference image through the vehicle ID.

[0007] S300, obtain the front features of the deceleration picture set and obtain multiple deceleration contours through the edge algorithm, obtain the front features of the baseline image through the vehicle ID and obtain the baseline contour through the edge algorithm; the server presets the deviation coefficient, and performs feature stacking on the baseline contour according to the deviation coefficient to obtain the extended contour; the deceleration contour is compared with the extended contour to determine whether the deceleration contour exceeds the coverage of the extended contour; if all the deceleration contours are within the coverage of the extended contour, it indicates that the vehicle's deceleration posture is normal; if a part of the deceleration contour exceeds the coverage of the extended contour, it indicates that the vehicle's deceleration posture is abnormal; the server sends a warning message containing the vehicle ID to the highway management system.

[0008] Preferably, before comparing the deceleration profile with the extended profile, the following steps are performed: S301, setting a sensitive area of ​​the baseline profile and obtaining a boundary of the sensitive area; identifying the same sensitive area under the deceleration profile and the baseline profile; wherein the sensitive area is an area including a vehicle ID.

[0009] S302, scaling the deceleration profile geometrically until the boundary of the sensitive area of ​​the deceleration profile coincides with the boundary of the sensitive area of ​​the reference profile. S303, determining whether the sensitive area of ​​the deceleration profile is consistent with the sensitive area of ​​the reference profile; if the sensitive area of ​​the deceleration profile is consistent with the sensitive area of ​​the reference profile, the deceleration profile is compared with the extended profile; if the sensitive area of ​​the deceleration profile is inconsistent with the sensitive area of ​​the reference profile, the sensitive area of ​​the reference profile is reset or the geometric scaling ratio is corrected.

[0010] A further technical solution is that when the above-mentioned deceleration contour is geometrically scaled, the deceleration contour is adjusted to a figure of the same size as the reference contour, and the same coordinate system is established; the width and height of the sensitive area on the reference contour are obtained through the coordinate system; the width and height of the sensitive area on the deceleration contour are obtained, and the scaling factor S of the deceleration contour is obtained; and the deceleration contour is geometrically scaled according to the scaling factor S.

[0011] The calculation formula of the above scaling factor S is:

[0012] ;

[0013] In the formula, is the deceleration profile after geometric scaling, is the deceleration profile without proportional scaling; is the width of the sensitive area on the reference contour, is the height of the sensitive area on the reference contour, is the width of the sensitive area on the deceleration profile, The height of the sensitive area on the deceleration profile, r is the correction coefficient of the scaling factor.

[0014] A further technical solution is to perform the following steps when determining whether the sensitive area of ​​the deceleration contour is consistent with the sensitive area of ​​the reference contour: obtain the coordinates of the center points of the sensitive areas of the deceleration contour and the reference contour after geometric scaling, and determine the distance between the center points of the two; calculate the center point deviation value K, preset the identification deviation number E, compare the center point deviation value K with the identification deviation number E, and determine whether the sensitive area of ​​the deceleration contour is consistent with the sensitive area of ​​the reference contour.

[0015] Among them, the calculation formula of the center point deviation value K is:

[0016] ;

[0017] In the formula, and is the coordinate value of the center point of the sensitive area of ​​the reference contour, and is the coordinate value of the center point of the sensitive area of ​​the deceleration contour after geometric scaling; if K≤E, the sensitive area of ​​the deceleration contour is consistent with the sensitive area of ​​the reference contour; if K>E, the sensitive area of ​​the deceleration contour is inconsistent with the sensitive area of ​​the reference contour.

[0018] Preferably, the first working area is placed in a slow-moving or waiting area of ​​a road section, and the reference image is a clear image feature; the second working area is placed in a turning area or an uphill or downhill area of ​​a road section, and the deceleration picture set includes blurred image features; when executing S300, the server presets a similarity threshold PS; the server compares all posture images in the deceleration picture set with the associated reference image and calculates the similarity P.

[0019] Among them, when P<PS, the corresponding posture image in the deceleration picture set is eliminated.

[0020] Among them, when P ≥ PS, the corresponding posture image in the deceleration picture set is converted into a deceleration contour.

[0021] A further technical solution is that the visual equipment of the second work area is at least arranged at the entrance and exit of the second work area. Before executing S300, the server captures the driving time of the vehicle entering and leaving the second work area through the timestamp of the deceleration picture set, and calculates the vehicle speed corresponding to the posture image obtained by the visual equipment in the second work area through the driving time and the distance of the second work area; if the vehicle speed is greater than the vehicle speed specified in the second work area, the server abandons the processing of the deceleration picture set.

[0022] Preferably, the above-mentioned working section also includes a third working area, acquiring image features of the third working area to obtain a parking image, and obtaining a static image according to the front features and side features of the parking image according to the vehicle ID and timestamp; the server associates the static image with the reference image through the vehicle ID; the server extracts associated side features from the static image and the reference image respectively and forms an image pair.

[0023] A further technical solution is to extract the tire contours on the image pair, and set more than 3 corresponding points for each tire contour, calculate the Euclidean distance between the corresponding points, aggregate the Euclidean distances of all corresponding points, and obtain the overall deformation value. The server sets reasonable parameters; it is determined whether the deformation value is within the range of reasonable parameters. If the deformation value is within the range of reasonable parameters, it is normal. If the deformation value exceeds the range of reasonable parameters, it is abnormal, and the server sends a warning message containing the vehicle ID to the highway management system.

[0024] Preferably, the server sets up image packages of several vehicle IDs in the database, and each image package has vehicle features corresponding to the vehicle ID; when executing S100, the server matches the vehicle ID in the reference image with the database; if the vehicle ID is recognized in the database, the match is successful, and the server stores the image features in the image package corresponding to the vehicle ID; if the vehicle ID is not recognized in the database, the match fails, and the server creates an image package of the vehicle ID in the database.

[0025] The present invention also discloses a vehicle abnormality recognition system based on highway dynamic data, the system includes a server and a visual device, the visual device includes a high-definition camera and an infrared sensor, the high-definition camera obtains the image features of the vehicle, and the infrared sensor obtains the infrared thermal map of the vehicle; the visual device is connected to the server through an edge device; the edge device is used to adjust the image features of the vehicle obtained by the high-definition camera to the same size and send them to the server; the server executes the above-mentioned vehicle abnormality recognition method.

[0026] Compared with the prior art, the beneficial effects of the present invention are at least one of the following:

[0027] The present invention obtains a reference image by monitoring the image features of the vehicle in the first working area, obtains the image features of the vehicle in the second area, and integrates the images of the same vehicle ID into a deceleration picture set; obtains multiple deceleration contours through an edge algorithm, compares the deceleration contours with the preset extended contours, and checks whether there are any abnormalities. If the deceleration contour exceeds the coverage of the extended contour, it may be that the vehicle has an abnormal posture when decelerating, and the server automatically sends a warning message containing the vehicle ID to the highway management system; this method mainly uses the phenomenon of the center of gravity moving forward when the vehicle brakes during the vehicle's deceleration process, and uses multiple deceleration contours to judge vehicles with abnormal deceleration, thereby preliminarily preventing possible traffic accidents. In addition, the dynamic image library and analysis data that can be accumulated by the server can provide data support for future traffic safety management and provide support data for the subsequent formulation of more effective traffic management strategies.

[0028] The present invention compares the deceleration profile with the reference profile in advance, and realizes the positioning of the image comparison by setting sensitive areas and boundaries. The positioning is conducive to better acquisition of abnormal changes in the deceleration profile, especially when the center of gravity moves forward. At the same time, the deceleration profile can reduce the influence of the vehicle position in the image data by geometric scaling; and by establishing a coordinate system and calculating the scaling factor, the size of the deceleration profile is determined and adjusted, so that the deceleration profile has a high comparability with the preset extended profile. The present invention also uses the coordinates of the center point of the sensitive area and obtains the center point deviation value between the images, and uses the center point deviation value and the identification deviation number to determine the consistency. The sensitive area of ​​the deceleration profile is consistent with the sensitive area of ​​the reference profile, so that the extended profile can be compared with the deceleration profile, and a reasonable difference area can be obtained, so as to facilitate the subsequent analysis of the vehicle situation with abnormal deceleration.

[0029] The present invention also obtains a static image of the vehicle in the third working area, associates the static image with a reference image to form an image pair, obtains the tire contour on the image pair, sets three corresponding points, calculates the Euclidean distance between the corresponding points and aggregates them, thereby obtaining the overall deformation value, and analyzes whether the deformation value is within a reasonable parameter range to determine whether there is any abnormality in the vehicle state. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the process of the present invention.

[0031] Figure 2 This is a schematic diagram of the layout of the visual device of the present invention in the second working area.

[0032] Figure 3 It is a schematic diagram of the comparison process of the deceleration profile and the expansion profile of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. It should be noted that all directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. in a certain specific working state. If the specific posture changes, the directional indication also changes accordingly. In the present invention, unless otherwise clearly specified and limited, the term "connection" and the like should be understood in a broad sense. For example, "connection" can be an electrical signal connection or a signal connection; it can also be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0034] If there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the meaning of "and / or" appearing in the full text includes three parallel schemes. Taking "A and / or B" as an example, it includes scheme A, or scheme B, or a scheme that satisfies both A and B. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0035] The existing road management system has been equipped with many visual devices, but the existing visual devices are mainly used to cooperate with the management system to record traffic flow and monitor traffic violations. The image data collected by the visual devices mainly analyzes the vehicle's motion trajectory and sensitive elements on the image to determine whether the vehicle has violated the rules. The identification of this abnormal behavior mainly comes from the abnormal behavior of the driver itself. For abnormal behaviors that are not easy to detect, such as abnormal wear of the brake pads of long-distance vehicles (brake failure may occur on downhill sections), tilt of the cargo inside the truck (the cargo may be at risk of spilling), etc., there are certain limitations in the dynamic identification of the abnormality of the vehicle itself. In many cases, pedestrians or passing vehicles are warned only when an accident is approaching or a significant phenomenon has already occurred. Therefore, the current research is to use real-time data for dynamic response and identify potential risks, so as to combine the road management system to issue immediate warnings and prevent potential traffic accidents.

[0036] Based on this, reference Figure 1 As shown, one embodiment of the present invention is a method for identifying vehicle anomalies based on highway dynamic data. First, determine the working section of the monitored section, set up a visual device and a server, connect the visual device to the server signal, and use the visual device to obtain the image features of the working section and upload them to the server; collect relevant image features to the server through the visual device, and the server analyzes and processes the image features. It is worth noting that if there are many image features, the server can usually allocate an edge generation device, and the edge generation device performs collaborative calculations to ensure stable operation of the server. Among them, the image features include the front features and side features of the vehicle; and the image features include a timestamp; the front features usually refer to the front image of the vehicle, and its front features usually include license plate information representing the identity. Its side features mainly include the side view and the rear view of the vehicle.

[0037] The working section includes a first working area and a second working area, and the server performs the following steps:

[0038] S100, obtaining the image features of the first working area as a reference image; identifying the vehicle ID in the reference image, and creating a dynamic image library corresponding to the vehicle ID. By identifying the front features in the image features, the license plate information in the front features is obtained and the vehicle ID is determined; since the first working area is usually located in the road entry area, the vehicle is usually in a stopped state or slow-moving state in the first working area, so the image features of the first working area usually have high clarity and complete image details, and therefore, the image features of the first working area are used as the reference image.

[0039] For reference, when the vehicle ID in the reference image is identified, the server can create a dynamic image library, which is used to update and record all dynamic information related to the vehicle ID in real time. In principle, it is ensured that each vehicle entering the monitored section has a dynamic image library established to facilitate timely and accurate tracking of each vehicle, laying a solid foundation for subsequent vehicle management and dispatching work.

[0040] S200, acquiring image features of the second working area to obtain a plurality of posture images; the server traverses the front features in the posture images, and integrates the posture images of the same vehicle ID into a deceleration picture set.

[0041] In order to better reflect the difference between the same vehicle entering and exiting the second working area, the deceleration picture set at least includes the front features of the same vehicle entering and exiting the second working area; the deceleration picture set is associated with the reference image through the vehicle ID.

[0042] For reference, vehicles in the second working area are usually dynamic, and the image features of vehicles in the second working area are usually obtained in the form of continuous snapshots. Therefore, when the visual device collects image features in the second working area, several posture images are obtained. There may be multiple vehicles driving on the monitored road section, so it is necessary to identify the vehicle ID and obtain the corresponding posture image to form a deceleration picture set. The deceleration picture set includes at least posture images of the same vehicle entering and exiting the deceleration zone. The front features of the posture image can better identify the vehicle ID and the downward movement of the vehicle head during deceleration. Therefore, the deceleration picture set requires at least the front features of the same vehicle entering and exiting the second working area.

[0043] S300, obtain the front features of the deceleration picture set and obtain multiple deceleration contours through the edge algorithm, obtain the front features of the reference image through the vehicle ID and obtain the reference contour through the edge algorithm; the server presets the deviation coefficient, and performs feature stacking on the reference contour according to the deviation coefficient to obtain the extended contour. Among them, the coverage of the extended contour is mainly to take into account the normal contour changes that may occur in the vehicle during the deceleration process, such as slight deformation and position movement. The extended contour is mainly used to provide a tolerance range, which contains all possible changes in the vehicle contour under normal deceleration posture.

[0044] The deceleration profile is compared with the extended profile to determine whether the deceleration profile exceeds the coverage of the extended profile.

[0045] If all deceleration contours are within the coverage of the extended contours, it means that the vehicle's deceleration posture is normal; if a certain deceleration contour exceeds the coverage of the extended contour, it means that the vehicle's deceleration posture is abnormal; the server sends a warning message containing the vehicle ID to the highway management system.

[0046] Among them, all the deceleration contours are within the coverage of the extended contour, which means that the deceleration contour points are all within the coverage of the extended contour, indicating that the deceleration posture of the vehicle is within the preset normal range, that is, the deceleration process is stable and there is no abnormality.

[0047] Among them, a part of the deceleration profile exceeds the coverage of the extended profile. Generally, any point of the deceleration profile exceeds the coverage of the extended profile, indicating that the vehicle has undergone a posture change beyond the normal range during the deceleration process, such as excessive forward movement of the center of gravity. This abnormality may indicate that the vehicle has potential mechanical failures or unstable driving behavior, and a warning needs to be issued in time.

[0048] For reference, before the edge algorithm is performed, the server determines the image to be processed, converts the image into grayscale to form a grayscale image; uses a Gaussian filter to remove noise from the grayscale image and smooth the image. Although the existing Canny edge detection algorithm tool is used to perform edge detection, contour data is obtained; the contour data is extracted to generate a contour point set; the largest contour is extracted from the contour point set as a result.

[0049] For reference, the baseline profile mainly represents the contour features of the vehicle under normal deceleration conditions. Taking into account that when the vehicle decelerates, its contour may change to a certain extent due to factors such as the forward shift of the center of gravity, in order to better apply the baseline profile, a deviation coefficient is preset. The deviation coefficient is used to expand the baseline profile to accommodate changes within the normal range represented by the forward shift of the center of gravity. When the baseline profile obtained by the edge algorithm generates an extended profile, the contour point set of the baseline profile is extracted, the largest bounding box is selected, and then the bounding box is expanded according to obtain the extended profile. The extended profile is used to define the contour changes under the normal deceleration range. By applying the deviation coefficient, the system can tolerate slight changes in the contour of the vehicle during deceleration. This avoids misjudging as abnormalities due to small contour changes caused by different deceleration amplitudes of the vehicle.

[0050] It should be noted that the present invention aims to discover potential risks in the vehicle driving process as much as possible through highway dynamic data. Although this solution does not mean that the technical solution of the present invention can provide absolute safety, this solution can cooperate with the existing road management system to issue corresponding warnings to vehicles along the way, such as using remote road notice boards, or notifying checkpoints in advance to pay attention to vehicles that may be abnormal and promote highway management efficiency.

[0051] It should also be noted that, in general, the deviation coefficient is set to N, where the value of N is usually between 1 and 1.3, which means that based on the baseline profile, the range of (1-N)*100% is expanded in each direction. The specific parameters of the deviation coefficient are generally set according to the vehicle type, which can be obtained through the vehicle ID.

[0052] Based on the above embodiments, reference Figure 3 As shown, another embodiment of the present invention is that when the deceleration profile is compared with the extended profile, the following steps are performed: S301, setting a sensitive area of ​​the reference profile and obtaining a boundary of the sensitive area; identifying the same sensitive area under the deceleration profile and the reference profile. The sensitive area is an area including a vehicle ID.

[0053] For example, by selecting sensitive areas as specific areas in the image that contain vehicle IDs, the sensitive areas utilize the specific location and unique characteristics of the vehicle to ensure accurate positioning of the same vehicle in different images and improve the accuracy of contour comparison. Generally speaking, sensitive areas are concentrated near the location of the vehicle ID (license plate), which can minimize the waste of computing resources caused by full-image comparison. Since the size represented by the license plate itself is constant, the risk of interference with the overall contour of the vehicle by the sensitive area under different lighting, weather or camera angle conditions is lower.

[0054] S302, geometrically scale the deceleration profile until the sensitive area of ​​the deceleration profile coincides with the boundary of the sensitive area of ​​the reference profile. The geometric scaling ratio is usually expressed as a scaling factor. When geometrically scaling, the size and position of the vehicle may be different in images of the same vehicle in different time periods. Therefore, when geometrically scaling, the scaling factor for controlling the geometric scaling needs to be adjusted. The scaling factor is used to determine the size ratio of the geometric scaling, so that the deceleration profile is adjusted to coincide with the boundary of the sensitive area of ​​the reference profile.

[0055] S303, determine whether the sensitive area of ​​the deceleration profile is consistent with the sensitive area of ​​the baseline profile; wherein, if the sensitive area of ​​the deceleration profile is consistent with the sensitive area of ​​the baseline profile, the deceleration profile is compared with the extended profile. wherein, if the sensitive area of ​​the deceleration profile is inconsistent with the sensitive area of ​​the baseline profile, the sensitive area of ​​the baseline profile is reset or the geometric scaling ratio is corrected. when the sensitive areas are inconsistent, the sensitive area of ​​the baseline profile is reset or the geometric scaling ratio is corrected to ensure consistency in the comparison process between the deceleration profile and the baseline profile until the sensitive areas are consistent. only when the sensitive areas are consistent can the deceleration profile be compared with the extended profile in the hope of reducing or eliminating the impact caused by changes in vehicle size or shooting angle.

[0056] Based on the above embodiment, another embodiment of the present invention is that when the above deceleration profile is geometrically scaled, the deceleration profile is adjusted to a figure of the same size as the reference profile, and the same coordinate system is established; the width and height of the sensitive area on the reference profile are obtained through the coordinate system; the width and height of the sensitive area on the deceleration profile are obtained, and the scaling factor S of the deceleration profile is obtained; and the deceleration profile is geometrically scaled according to the scaling factor S. The calculation formula of the above scaling factor S is:

[0057] ;

[0058] In the formula, is the deceleration profile after geometric scaling, is the deceleration profile without proportional scaling; is the width of the sensitive area on the reference contour, is the height of the sensitive area on the reference contour, is the width of the sensitive area on the deceleration profile, The height of the sensitive area on the deceleration profile, r is the correction coefficient of the scaling factor.

[0059] For reference, the calculation formula of the scaling factor S is based on the ratio of the width and height of the sensitive area of ​​the baseline contour and the deceleration contour. The geometric scaling ensures that the two contours are compared at the same scale, and the establishment of the coordinate system can better obtain image pixels in the same coordinate dimension. In practical applications, due to reasons such as camera angle, lighting conditions or vehicle dynamic changes, contour extraction may have systematic deviations. The introduction of the correction coefficient r of the scaling factor can finely control the scaling process, so that the scaling factor can more accurately reflect the actual contour change requirements.

[0060] For reference, all vehicles are placed at the same scale when performing profile comparison, and the relevant positions, sensitive area sizes of the baseline profile and the deceleration profile are obtained through the coordinate system; it is assumed that the width of the sensitive area on the baseline profile is 100, the height of the sensitive area on the baseline profile is 50, the width of the sensitive area on the deceleration profile is 95, and the height of the sensitive area on the deceleration profile is 48; the correction coefficient of the scaling factor is set to 1.02; then the value of the scaling factor S is calculated as (100 / 95+50 / 48)×0.5×1.02≈1.0692.

[0061] Furthermore, when determining whether the sensitive area of ​​the deceleration profile is consistent with the sensitive area of ​​the reference profile, the following steps are performed: obtaining the coordinates of the center points of the sensitive areas of the deceleration profile and the reference profile after geometric scaling, and determining the distance between the two center points; calculating the center point deviation value K, presetting the identification deviation number E, and comparing the center point deviation value K with the identification deviation number E to determine whether the sensitive area of ​​the deceleration profile is consistent with the sensitive area of ​​the reference profile. Among them, the calculation formula of the center point deviation value K is:

[0062] ;

[0063] In the formula, and is the coordinate value of the center point of the sensitive area of ​​the reference contour, and is the coordinate value of the center point of the sensitive area of ​​the deceleration contour after geometric scaling; if K≤E, the sensitive area of ​​the deceleration contour is consistent with the sensitive area of ​​the reference contour; if K>E, the sensitive area of ​​the deceleration contour is inconsistent with the sensitive area of ​​the reference contour.

[0064] Based on the above embodiments, another embodiment of the present invention is that the above-mentioned first working area is placed in a slow-moving or waiting area of ​​a road section, and the above-mentioned reference image is a clear image feature; the above-mentioned second working area is placed in a turning area or an uphill or downhill area of ​​a road section, and the deceleration picture set includes blurred image features; when executing S300, the server presets a similarity threshold PS; the server compares all posture images in the deceleration picture set with the associated reference image and calculates the similarity P.

[0065] Among them, when P<PS, the corresponding posture image in the deceleration picture set is eliminated.

[0066] Among them, when P ≥ PS, the corresponding posture image in the deceleration picture set is converted into a deceleration contour.

[0067] For reference, the calculated similarity P between the pose image and the associated reference image can be represented by obtaining the structural similarity index; the difference in the pixel grayscale values ​​of the image is used for calculation, where the value range of PS is 0.8 to 1, and the value range of the similarity P is usually between -1 and 1. Generally, 1 means the same, 0 means irrelevant, and -1 means completely opposite.

[0068] Among them, the calculation formula of the similarity P between the posture image and the associated reference image is:

[0069] ;

[0070] In the formula, is the mean of the local window of the pose image, is the mean value of the local window of the reference image; where, is the variance of the local window of the pose image, is the variance of the local window of the reference image, where is the covariance between the pose image and the reference image, and It is a stability constant, usually taken as 1% to 3% of the dynamic range of the image (the difference between the darkest and brightest pixels).

[0071] Among them, the local window refers to a small area (contour area) selected in the image, the mean of the local window refers to the average value of the pixel grayscale values ​​of all pixels in the local window; the variance of the local window refers to the distribution discreteness of the pixel grayscale values ​​in the local window; the covariance of the posture image and the reference image is used to measure the linear similarity of the pixel values ​​of the two images in the local window.

[0072] For further reference, Figure 2 As shown, the visual device of the second work area is at least arranged at the entrance and exit of the second work area, that is, the second work area is arranged with at least two placement areas, and the visual device is placed on the placement area. Before executing S300, the server captures the driving time of the vehicle entering and leaving the second work area through the timestamp of the deceleration picture set, and calculates the vehicle speed corresponding to the posture image obtained by the visual device in the second work area through the driving time and the distance of the second work area; if the vehicle speed is greater than the speed specified in the second work area, the server abandons the processing of the deceleration picture set.

[0073] In order to achieve effective monitoring and management of vehicles, the visual device acquires image data at the entrance and exit of the second work area. Since the deceleration image set contains the timestamps of the vehicle at the entrance and exit, the driving time of the vehicle entering and leaving the second work area can be captured by analyzing the deceleration image set. Since the distance between the entrance and exit of the second work area is constant, the server can use the algorithm to calculate the vehicle speed corresponding to the posture image acquired by the visual device in the second work area through the driving time and the known distance of the second work area.

[0074] For reference, if the sampled vehicle speed of the deceleration picture set exceeds the regulations, it usually means that the vehicle itself is in abnormal driving. Regardless of whether the vehicle has deceleration action, it should be regarded as abnormal, so the server does not need to process the deceleration picture set. It should be noted that when speeding, the server's safety warning mechanism should also be triggered to avoid safety risks that may be caused by speeding. And after obtaining the vehicle's speed information, if the vehicle speed is within the prescribed range, the preset deviation coefficient setting can also be adjusted according to the speed regulations when necessary. This allows the server to provide a preliminary warning of the vehicle's driving status through the baseline image and deceleration profile.

[0075] Based on the above embodiment, another embodiment of the present invention is that the above working section also includes a third working area, the image features of the third working area are obtained to obtain a parking image, and the front features and side features of the parking image are obtained according to the vehicle ID and timestamp to obtain a static image; the server associates the static image with the reference image through the vehicle ID; the server extracts the associated side features from the static image and the reference image respectively and forms an image pair. This solution mainly associates the static image with the side features of the reference image, so as to judge the changes of the vehicle in the first working area and the third working area through the side features.

[0076] The image pair is a relative image composed of two interrelated side features extracted from the static image and the reference image respectively.

[0077] The third working area can usually be a parking area or a waiting exit of a monitored road section; generally speaking, the image features obtained in the third working area are usually static. The server associates the static image with the reference image through the vehicle ID. In theory, the third working area provides stable static features (such as vehicle body contour, license plate position, etc.), which can provide a certain image data reference for the server.

[0078] It is worth noting that, if necessary, after analyzing the parking image and finding that everything is normal, the third workspace can be used as a node, and the parking image can be used as a new reference image for comparison with subsequent deceleration image sets.

[0079] Furthermore, in order to reduce the risk of subsequent vehicle driving, the tire contours of the image pair are extracted, and more than 3 corresponding points are set for the tire contours, the Euclidean distances between the corresponding points are calculated, and the Euclidean distances of all corresponding points are aggregated to obtain the overall deformation value. The server sets reasonable parameters; it is determined whether the deformation value is within the range of reasonable parameters. Among them, if the deformation value is within the range of reasonable parameters, it is normal. Among them, if the deformation value exceeds the range of reasonable parameters, it is abnormal, and the server sends a warning message containing the vehicle ID to the highway management system.

[0080] Among them, in the two-dimensional space of the tire contour, more than three corresponding points are set for the tire contour, and more than three corresponding points (non-collinear points) are distributed at different positions of the tire, so that deformation information can be captured from multiple angles and positions; by calculating the Euclidean distance between the corresponding points of the tire contour on the image pair, the Euclidean distances of all corresponding points are aggregated, and the aggregation is usually the average value, which can quantify the deformation degree of the two tires on the image pair. If necessary, if the deformation degree exceeds the reasonable parameters, a warning message containing the vehicle ID and deformation value is sent to the highway management system.

[0081] For reference, the parameters whose deformation values ​​exceed reasonable parameters are divided into mild anomalies and severe anomalies. For deformation values ​​that only exceed a certain value within the range of reasonable parameters, it is defined as mild anomalies. For mild anomalies, after the server sends a warning message containing the vehicle ID to the highway management system, it calls the existing highway management system to record and continuously monitor the driving status of the vehicle, and prompts the driver to slow down and drive to the next parking area for self-inspection through the existing electronic display board. If the deformation value seriously exceeds the range of reasonable parameters, it is defined as severe anomalies. After the server sends a warning message containing the vehicle ID to the highway management system, it will immediately trigger the emergency warning mechanism, which will not only send warning information through the highway management system, but also directly notify the driver through the vehicle terminal or other communication means, suggesting that he immediately pull over for inspection or take other safety measures. Through this multi-level exception handling mechanism, the safety of road driving can be effectively improved and the risk of accidents caused by vehicle failures can be reduced.

[0082] Based on the above embodiment, another embodiment of the present invention is that the above server sets several image packages of vehicle IDs in the database, and each image package has vehicle features corresponding to the vehicle ID; when executing S100, the server matches the vehicle ID in the reference image with the database; if the vehicle ID is recognized in the database, the match is successful, and the server stores the image features in the image package corresponding to the vehicle ID; if the vehicle ID is not recognized in the database, the match fails, and the server creates an image package of the vehicle ID in the database. The image package records the situation of the vehicle during the entire driving process. The server matches the vehicle ID in the reference image with the database by creating a table containing the vehicle ID and the image features. If the match is successful, the image features are stored in the image package corresponding to the vehicle ID. If the match fails, a new vehicle ID image package is created in the database.

[0083] The image package can use the existing SQLAlchemy to create a database link to store the unique vehicle ID. When matching, use SQLAlchemy Session to query whether the image package ID exists in the database. If the match is successful, it is still necessary to create a dynamic image library according to the process of the present invention and associate it with the existing image package; otherwise, a new image package is created and the corresponding vehicle ID of the image package is written. Finally, it is still necessary to create a dynamic image library according to the process of the present invention to associate the image package. It should be noted that considering the server storage capacity, the storage time of the image package usually does not exceed the set period.

[0084] Based on the above embodiments, another embodiment of the present invention is a vehicle abnormality recognition system based on highway dynamic data, the system including a server and a visual device, the visual device including a high-definition camera and an infrared sensor, the high-definition camera obtains image features of the vehicle, and the infrared sensor obtains an infrared thermal map of the vehicle; the visual device is connected to the server via an edge device; the edge device is used to adjust the image features of the vehicle obtained by the high-definition camera to the same size and send them to the server; the server executes the above-mentioned vehicle abnormality recognition method.

[0085] Among them, the edge device can be an existing device. Usually, the edge device can be set between the visual device and the server, responsible for preprocessing the image data (such as adjusting the image size, format conversion), realizing primary data filtering and accelerating transmission to reduce the image processing amount of the server.

[0086] Among them, when the second working area of ​​the visual device is a curve, the curve is divided into an entrance straight zone, a curve zone and an exit straight zone. Considering that the vehicle generally needs to slow down before entering the curve, the visual device usually needs to be distributed in the entrance straight zone and the exit straight zone to obtain the posture characteristics of the vehicle before and after entering the curve.

[0087] Among them, multiple visual devices can also be set in the curve area. Although the visual devices in the curve area may not participate in identifying abnormalities, when it is determined that an abnormality occurs when the vehicle passes through the second working area, relevant image data of the curve area can also be obtained for abnormality analysis.

[0088] The "one embodiment", "another embodiment", "embodiment", "preferred embodiment" and the like mentioned in this specification refer to the specific features, structures or characteristics described in conjunction with the embodiment as included in at least one embodiment generally described in this application. The same expression appearing in multiple places in the specification does not necessarily refer to the same embodiment. Further, when a specific feature, structure or characteristic is described in conjunction with any embodiment, it is claimed that the realization of such feature, structure or characteristic in conjunction with other embodiments also falls within the scope of the present invention.

[0089] Although the present invention is described herein with reference to a number of illustrative embodiments of the present invention, it will be appreciated that those skilled in the art may devise many other modifications and implementations that fall within the scope and spirit of the principles disclosed herein. More specifically, within the scope of the present disclosure, drawings, and claims, a variety of variations and modifications may be made to the components and / or layout of the subject combination layout. In addition to the variations and modifications made to the components and / or layout, other uses will also be apparent to those skilled in the art.

Claims

1. A vehicle abnormality identification method based on highway dynamic data, characterized in that: Determine a working section of the monitored road section, set up a visual device and a server, connect the visual device to the server signal, and use the visual device to obtain image features of the working section and upload them to the server; the image features include front features and side features of the vehicle; and the image features include a timestamp; the working section includes a first working area and a second working area; The server executes the following steps: S100, obtaining the image features of the first working area as a reference image; identifying the vehicle ID in the reference image, and creating a dynamic image library corresponding to the vehicle ID; S200, obtaining image features of the second working area to obtain a plurality of posture images; the server traverses the front features in the posture images, and integrates the posture images of the same vehicle ID into a deceleration picture set; wherein the deceleration picture set at least includes the front features of the same vehicle entering and exiting the second working area; and the deceleration picture set is associated with the reference image through the vehicle ID; S300, obtaining the front features of the deceleration picture set and obtaining multiple deceleration contours through an edge algorithm, obtaining the front features of the reference image through the vehicle ID and obtaining the reference contour through an edge algorithm; the server presets a deviation coefficient, and performs feature stacking on the reference contour according to the deviation coefficient to obtain an extended contour; the deceleration contour is compared with the extended contour to determine whether the deceleration contour exceeds the coverage range of the extended contour; If all deceleration contours are within the coverage of the extended contour, it means that the vehicle deceleration posture is normal; If a certain deceleration contour part exceeds the coverage of the extended contour, it means that the vehicle's deceleration posture is abnormal; the server sends a warning message containing the vehicle ID to the highway management system; Before comparing the deceleration profile with the extended profile, the following steps are performed: S301, setting a sensitive area of ​​the reference profile and obtaining a boundary of the sensitive area; identifying the same sensitive area under the deceleration profile and the reference profile; wherein the sensitive area is an area including a vehicle ID; S302, scaling the deceleration profile geometrically until the boundary of the sensitive area of ​​the deceleration profile coincides with the boundary of the sensitive area of ​​the reference profile; S303, determining whether the sensitive area of ​​the deceleration profile is consistent with the sensitive area of ​​the reference profile; wherein, if the sensitive area of ​​the deceleration profile is consistent with the sensitive area of ​​the reference profile, the deceleration profile is compared with the extended profile; If the sensitive area of ​​the deceleration profile is inconsistent with the sensitive area of ​​the reference profile, the sensitive area of ​​the reference profile is reset or the ratio of geometric scaling is corrected.

2. The vehicle abnormality identification method based on highway dynamic data according to claim 1 is characterized in that: When the deceleration profile is geometrically scaled, the deceleration profile is adjusted to a figure of the same size as the reference profile, and the same coordinate system is established; the width and height of the sensitive area on the reference profile are obtained through the coordinate system; the width and height of the sensitive area on the deceleration profile are obtained, and the scaling factor S of the deceleration profile is obtained; And the deceleration profile is scaled proportionally according to the scaling factor S; The calculation formula of the scaling factor S is: ； In the formula, is the deceleration profile after geometric scaling, is the deceleration profile without proportional scaling; is the width of the sensitive area on the reference contour, is the height of the sensitive area on the reference contour, is the width of the sensitive area on the deceleration profile, The height of the sensitive area on the deceleration profile, r is the correction coefficient of the scaling factor.

3. The vehicle abnormality identification method based on highway dynamic data according to claim 2 is characterized in that: To determine whether the sensitive area of ​​the deceleration profile is consistent with the sensitive area of ​​the reference profile, perform the following steps: Obtain the coordinates of the center points of the sensitive areas of the deceleration profile and the reference profile after geometric scaling, and determine the distance between the two center points; calculate the center point deviation value K, preset the identification deviation number E, compare the center point deviation value K with the identification deviation number E, and determine whether the sensitive area of ​​the deceleration profile is consistent with the sensitive area of ​​the reference profile; Among them, the calculation formula of the center point deviation value K is: ； In the formula, and is the coordinate value of the center point of the sensitive area of ​​the reference contour, and The coordinate value of the center point of the sensitive area of ​​the deceleration contour after geometric scaling; If K≤E, the sensitive area of ​​the deceleration profile is consistent with the sensitive area of ​​the reference profile; If K>E, the sensitive area of ​​the deceleration profile is inconsistent with the sensitive area of ​​the reference profile.

4. The vehicle abnormality identification method based on highway dynamic data according to claim 1 is characterized in that: The first working area is placed in a slow-moving or waiting area of ​​the road section, and the reference image is a clear image feature; the second working area is placed in a turning area or an uphill or downhill area of ​​the road section, and the deceleration picture set includes blurred image features; when executing S300, the server presets a similarity threshold PS; The server compares all posture images in the deceleration image set with the associated reference image and calculates the similarity P; When P < PS, the corresponding posture image in the deceleration image set is removed; When P ≥ PS, the corresponding posture image in the deceleration picture set is converted into a deceleration contour.

5. The vehicle abnormality identification method based on highway dynamic data according to claim 4 is characterized in that: The visual equipment of the second work area is at least arranged at the entrance and exit of the second work area. Before executing S300, the server captures the driving time of the vehicle entering and leaving the second work area through the timestamp of the deceleration picture set, and calculates the vehicle speed corresponding to the posture image obtained by the visual equipment in the second work area through the driving time and the distance of the second work area; if the vehicle speed is greater than the vehicle speed specified in the second work area, the server abandons the processing of the deceleration picture set.

6. The vehicle abnormality identification method based on highway dynamic data according to claim 1 is characterized in that: The working section also includes a third working area, obtaining image features of the third working area to obtain a parking image, obtaining a static image based on the front features and side features of the parking image according to the vehicle ID and the timestamp; the server associates the static image with the reference image through the vehicle ID; The server extracts associated side features from the static image and the reference image respectively and forms an image pair.

7. The vehicle abnormality identification method based on highway dynamic data according to claim 6 is characterized in that: Extract the tire contours on the image pair, and set more than 3 corresponding points for each tire contour, calculate the Euclidean distance between the corresponding points, aggregate the Euclidean distances of all corresponding points, obtain the overall deformation value, and set reasonable parameters on the server; determine whether the deformation value is within the range of reasonable parameters; Among them, if the deformation value is within the range of reasonable parameters, it means it is normal; Among them, if the deformation value exceeds the range of reasonable parameters, it indicates an abnormality, and the server sends a warning message containing the vehicle ID to the highway management system.

8. The vehicle abnormality identification method based on highway dynamic data according to claim 1 is characterized by: The server sets a plurality of image packages of vehicle IDs in a database, each image package and a vehicle feature of a corresponding vehicle ID; when executing S100, the server matches the vehicle ID in the reference image with the database; If the vehicle ID is recognized in the database, the match is successful, and the server stores the image features in the image package corresponding to the vehicle ID; If the vehicle ID is not recognized in the database, the match fails and the server creates an image package for the vehicle ID in the database.

9. A vehicle abnormality recognition system based on highway dynamic data, characterized in that: The system includes a server and a visual device, wherein the visual device includes a high-definition camera and an infrared sensor, wherein the high-definition camera obtains image features of a vehicle, and the infrared sensor obtains an infrared thermal image of the vehicle; the visual device is connected to the server via an edge device; the edge device is used to adjust the image features of the vehicle obtained by the high-definition camera to the same size and send them to the server; the server executes the vehicle abnormality recognition method described in any one of claims 1 to 8.

Images