Vehicle positioning and state monitoring method based on visual technology

Abstract

The invention discloses a vehicle positioning and state monitoring method based on a visual technology, and the method comprises the following steps: 1), collecting a vehicle image through front-end camera equipment when a vehicle enters a parking lot; 2) extracting image features; 3) comparing the currently input vehicle image features with the pre-stored template image features, and identifying the category of the vehicle through the similarity between the features; 4) according to an identification result, guiding the vehicle to park in a designated area; 5) vehicle positioning; and (6) the moving state of the parked vehicle is monitored in real time, and an alarm is triggered once abnormal movement including position sliding or abnormal driving direction is found. According to the method, rapid positioning, accurate detection and identification of the target are realized through image identification and computer vision technologies. Therefore, in the fields of safety monitoring and intelligent amphibious ship monitoring, abnormal conditions can be found in time, the safety and early warning capability are improved, and safe and stable operation in the vehicle and the cabin is guaranteed.

Description

Technical Field

[0001] The present invention relates to computer vision technology, and in particular to a vehicle positioning and state monitoring method based on vision technology. Background Art

[0002] Combined with the existing vehicle management mode in the dock, in the limited environment of the cabin, the management of equipment vehicles and the maintenance of warehouse space usually adopt traditional methods such as manual counting, manual ledger records, and personnel patrolling in shifts. There are problems such as management lag, untimely information transmission, and difficulty in information sharing. It has been found that there is room for improvement and improvement in the efficiency of cabin space utilization, equipment vehicle entry and exit management, and maintenance and repair timeliness of problem equipment vehicles. It is urgent to introduce scientific and technological capabilities to automate operation management, timely and accurately grasp cabin information, parking space information, equipment vehicle positioning and status information, monitor them in real time, conduct data analysis on the operating rules in the cabin, detect and guide the rational deployment of the cabin, and automated management of equipment vehicles. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a vehicle positioning and status monitoring method based on vision technology in view of the defects in the prior art.

[0004] The technical solution adopted by the present invention to solve the technical problem is: a vehicle positioning and state monitoring method based on vision technology, comprising the following steps:

[0005] 1) When a vehicle enters the site, the front-end camera device is used to collect vehicle images;

[0006] 2) Image feature extraction;

[0007] 3) Compare the current input vehicle image features with the previously saved template image features, and identify the vehicle category based on the degree of similarity between them;

[0008] 4) Based on the recognition results, guide the vehicle to park in the designated area;

[0009] By analyzing the vehicle's movement trajectory and behavior pattern, the vehicle's behavior is identified, and by tracking the vehicle's speed and direction changes in real time, it is determined whether the vehicle has received instructions to park in the designated area.

[0010] 5) Vehicle positioning;

[0011] Use a combination of panoramic cameras to ensure full coverage of the equipment vehicles in the cabin, search the monitoring images of the equipment vehicles in the cabin based on the pictures of the vehicles, and locate the vehicles through image analysis and feature comparison;

[0012] 6) Monitor the movement status of parked vehicles in real time and trigger an alarm if any abnormal movement is detected, including position slippage or abnormal driving direction.

[0013] According to the above scheme, in step 2), the feature extraction is as follows:

[0014] 2.1) Image preprocessing;

[0015] 2.1.1) Calculation of Directional Pattern

[0016] The method to extract the directional map is:

[0017] a) Divide the image into sufficiently small sub-blocks. For example, divide the image into 16X16 non-overlapping blocks;

[0018] b) For each point of each sub-block, use the Sobel operator to calculate its x-direction gradient and y-direction gradient respectively;

[0019]

[0020] Among them, S x ,S y represents the corresponding Sobel operator, and f(u, v) represents the gray value of the pixel (u, v);

[0021] c) Calculate the direction of each sub-block according to the directional gradient;

[0022]

[0023] Where w represents the width of the image block, which is 16 here;

[0024] d) After obtaining the direction θ of each sub-block, quantize it into 8 directions to obtain the direction of the image;

[0025] 2.1.2) Image Segmentation

[0026] For an image, it is divided into four types of image areas: background area, irrecoverable area, clear area, and recoverable area.

[0027] A three-level segmentation is adopted: the first level segments the background area; the second level segments the blurred area from the foreground; the third level segments the irrecoverable part from the blurred area.

[0028] The purpose of image segmentation is to keep the clear area and the recoverable area, and remove the first two areas.

[0029] 2.1.3) Image Enhancement

[0030] The image is enhanced by using the enhancement method based on Gabor filtering and the low-quality image enhancement algorithm based on Fourier filtering;

[0031] Gabor filter enhancement is based on the mathematical model of the image. It comprehensively considers the directional characteristics and frequency characteristics of the image, uses the improved Gabor filter for image processing, and changes the even Gabor function into a digital filter. As shown in the following formula, the filter obtained by using the real part of the Gabor function as a template is more realistic and close to the characteristics of the image.

[0032]

[0033] Among them, δx and δy represent the standard deviation of the Gabor filter in the x-axis and y-axis directions respectively, and fx represents the frequency in the x-axis direction;

[0034] 2.1.4) Binarization and thinning of images;

[0035] In terms of image binarization, a dynamic threshold image binarization method based on directional map is adopted. This method directly obtains the dynamic threshold from the image grayscale image to binarize the image, and completes the process of separating invalid areas, filtering, enhancing, and binarization in general image preprocessing at one time.

[0036] The 0PTA algorithm is used for refinement;

[0037] 2.2) Feature extraction of images;

[0038] Vehicle image feature extraction includes global features and local features;

[0039] Global features are obtained by extracting and analyzing the directional map. Global features mainly reflect the overall characteristics of the vehicle and have good classification effects on tasks such as vehicle color recognition and vehicle type recognition (communication vehicles, transport vehicles, tanks, etc.) with large differences between classes. Global features include color, texture, and shape features;

[0040] Local features;

[0041] The algorithm for extracting local detail features is as follows:

[0042] A 3×3 feature extraction template is used to extract the features of endpoints and divergence points of the refined image. Let m be the point to be detected, P1, P2, P3, ... P8 arranged clockwise are its 8 neighborhood points, R(1), R(2), R(3) ... R(8) are the grayscale values ​​of P1, P2, P3, ... P8 respectively. If m is an endpoint, then its neighborhood point G N Satisfy the following formula:

[0043]

[0044] If m is a branch point, then its neighboring point G N Satisfy the following formula:

[0045]

[0046] By traversing the image, the feature points of the image are found;

[0047] 2.3) Combine the global feature and local feature comparison for vehicle image matching;

[0048] First, perform the global feature matching according to the recognition task, and then use the local feature recognition to supplement and refine, reducing the recognition error;

[0049] The recognition process based on local features is as follows:

[0050] First, extract local features (including two methods: interest point detection and dense extraction) for description;

[0051] Then, use the feature transformation algorithm to encode the local features, including vector quantization, sparse coding, etc.;

[0052] Perform the feature aggregation operation to obtain a more discriminative and robust feature representation, and get a fused feature vector;

[0053] Finally, select an appropriate deep learning model structure to learn and obtain a classifier to complete the classification and recognition of the vehicle.

[0054] According to the above solution, in step 3), in the image comparison, the detail matching method is adopted. The set of detail point feature vectors P in the template detail point pattern and the set of detail point feature vectors Q in the input detail point pattern are respectively expressed as:

[0055]

[0056] Among them, the template detail point feature set P includes M detail points, and the input detail point feature set Q includes N detail points. Based on the above representation method, the detail point matching problem is described as: search for the best correspondence relationship between the points in P and Q, obtain the matching score MS according to the number of corresponding detail points under this correspondence relationship, and compare it with the threshold T. If MS≥T, the two detail point patterns match; if MS<T, the two detail point patterns do not match.

[0057] The beneficial effects produced by the present invention are:

[0058] 1. The present invention provides timely and accurate positioning and status information by performing real-time tracking and monitoring on the vehicle target. At the same time, by guiding the parking position to optimize the garage deployment, the in-cabin usage efficiency is improved.

[0059] 2. The present invention uses image recognition and computer vision technology to achieve rapid positioning, accurate detection and identification of targets. This helps to detect abnormal situations in a timely manner in the field of security monitoring and intelligent amphibious ship monitoring, improve safety and early warning capabilities, and ensure safe and stable operation of vehicles and cabins. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0061] Figure 1 It is a flow chart of a method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0063] like Figure 1 As shown, a vehicle positioning and status monitoring method based on vision technology includes the following steps:

[0064] 1) When a vehicle enters the site, the front-end camera device collects vehicle images; the camera positions meet the requirements, such as entrances and exits, straight roads, curves, viewing angles, heights, etc.

[0065] 2) Feature extraction;

[0066] 2.1) Image preprocessing;

[0067] 2.1.1) Calculation of Directional Pattern

[0068] The method to extract the directional map is:

[0069] a) Divide the image into sufficiently small sub-blocks. For example, divide the image into 16X16 non-overlapping blocks;

[0070] b) For each point of each sub-block, use the Sobel operator to calculate its x-direction gradient and y-direction gradient respectively;

[0071]

[0072] Among them, S x ,S y represents the corresponding Sobel operator, and f(u, v) represents the gray value of the pixel (u, v);

[0073] c) Calculate the direction of each sub-block according to the directional gradient;

[0074]

[0075] Where w represents the width of the image block, which is 16 here;

[0076] d) After obtaining the direction θ of each sub-block, quantize it into 8 directions to obtain the direction of the image;

[0077] 2.1.2) Image Segmentation

[0078] For an image, it is divided into four types of image areas: background area, irrecoverable area, clear area, and recoverable area.

[0079] A three-level segmentation is adopted: the first level segments the background area; the second level segments the blurred area from the foreground; the third level segments the irrecoverable part from the blurred area.

[0080] The purpose of image segmentation is to keep the clear area and the recoverable area, and remove the first two areas.

[0081] 2.1.3) Image Enhancement

[0082] The image is enhanced by using the enhancement method based on Gabor filtering and the low-quality image enhancement algorithm based on Fourier filtering;

[0083] Gabor filter enhancement is based on a mathematical model of the image, and is defined as follows:

[0084]

[0085] Taking into account the directional characteristics and frequency characteristics of the image, the improved Gabor filter is used for image processing in this embodiment, and the even Gabor function is changed into a digital filter, as shown in the following formula. The filter obtained by taking the real part of the Gabor function as a template is more realistic and close to the characteristics of the image;

[0086]

[0087] δx and δy represent the standard deviation of the Gabor filter in the x-axis and y-axis directions, respectively. x represents the angular frequency of the Gabor filter, fx represents the frequency in the x direction, which is related to the wavelength (λ);

[0088] 2.1.4) Binarization and thinning of images;

[0089] According to the grayscale distribution of the image, the core of the research on the binarization algorithm is to calculate the optimal threshold. Traditional binarization algorithms include static algorithms and dynamic algorithms. The dynamic algorithm has a better effect than the static algorithm. The static algorithm generates a large number of pseudo feature points for image areas with poor quality. The dynamic algorithm can make up for the defects of the static algorithm to a certain extent, but it will also introduce more noise. In terms of image binarization, the dynamic threshold image binarization method based on the directional map is adopted in this embodiment. This method directly obtains the dynamic threshold from the image grayscale image to binarize the image, and completes the process of separating invalid areas, filtering, enhancing, and binarization in general image preprocessing at one time.

[0090] For the thinning problem, there are two image thinning algorithms that are currently used more frequently: the fast thinning algorithm based on morphological processing and the 0PTA algorithm and its improved form. The fast thinning algorithm is a 4-connected parallel thinning algorithm. The principle is to determine the boundary points of the image lines and gradually delete them. The algorithm is very fast, but the thinning is not thorough, and the thinned lines are not single-pixel wide. The 0PTA algorithm is used in this embodiment;

[0091] 2.2) Feature extraction of images;

[0092] Vehicle image feature extraction includes global features and local features;

[0093] Global features are obtained by extracting and analyzing the directional map. Global features mainly reflect the overall characteristics of the vehicle and have good classification effects on tasks such as vehicle color recognition and vehicle type recognition (communication vehicles, transport vehicles, tanks, etc.) with large differences between classes. Global features include color, texture, and shape features;

[0094] Local features:

[0095] The algorithm for extracting local detail features is as follows:

[0096] A 3×3 feature extraction template is used to extract the features of endpoints and divergence points of the refined image. Let m be the point to be detected, P1, P2, P3, ... P8 arranged clockwise are its 8 neighborhood points, R(1), R(2), R(3) ... R(8) are the grayscale values ​​of P1, P2, P3, ... P8 respectively. If m is an endpoint, then its neighborhood point G N Satisfy the following formula:

[0097]

[0098] If m is a branch point, then its neighboring point G N Satisfy the following formula:

[0099]

[0100] By traversing the image, the feature points of the image are found;

[0101] 2.3) Combine the global feature and local feature comparison for vehicle image matching;

[0102] First, perform the matching of global features according to the recognition task, and then use local feature recognition to supplement and refine, reducing the recognition error;

[0103] The recognition process based on local features is as follows:

[0104] First, extract local features (including two methods: interest point detection and dense extraction) for description;

[0105] Then, use feature transformation algorithms to encode the local features, including vector quantization, sparse coding, etc.;

[0106] Perform feature aggregation operations to obtain a more discriminative and robust feature representation, and get a fused feature vector;

[0107] Finally, select an appropriate deep learning model structure to learn and obtain a classifier to complete the classification and recognition of the vehicle.

[0108] 3) Compare the features of the currently input vehicle image with the features of the template image saved in advance, and identify the classification category of the vehicle according to their similarity;

[0109] In terms of image comparison and matching, the detail matching method is adopted. The set of detail point feature vectors P in the template detail point pattern and the set of detail point feature vectors Q in the input detail point pattern can be respectively expressed as:

[0110]

[0111] Among them, the template detail point feature set P includes M detail points, and the input detail point feature set Q includes N detail points. Based on the above representation method, the detail point matching problem is described as: Search for the best correspondence between the points in P and Q, obtain the matching score MS according to the number of corresponding detail points under this correspondence, and compare it with the threshold T. If MS≥T, the two detail point patterns match; if MS<T, the two detail point patterns do not match.

[0112] 4) According to the recognition result, guide the vehicle to park in the designated area;

[0113] By analyzing the movement trajectory and behavior pattern of the vehicle, identify the behavior of the vehicle, and judge whether the vehicle receives the instruction to park in the designated area by real-time tracking of the speed and direction changes of the vehicle.

[0114] 5) Vehicle positioning;

[0115] Use a combination of panoramic cameras to ensure full coverage of the equipment vehicles in the cabin, search the monitoring images of the equipment vehicles in the cabin based on the pictures of the vehicles, and locate the vehicles through image analysis and feature comparison;

[0116] 6) Monitor the movement status of parked vehicles in real time and trigger an alarm if any abnormal movement is detected, including position slippage or abnormal driving direction.

[0117] The method of the present invention mainly monitors vehicle entry, guides lanes, designates parking spaces, and warns of abnormal behavior in a warehouse environment.

[0118] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A vehicle positioning and status monitoring method based on vision technology, characterized in that: The following steps are involved: 1) When a vehicle enters the site, the front-end camera device is used to collect vehicle images; 2) Image feature extraction; 3) Compare the current input vehicle image features with the previously saved template image features, and identify the vehicle category based on the degree of similarity between them; 4) Based on the recognition results, guide the vehicle to park in the designated area; identify the vehicle's behavior by analyzing the vehicle's movement trajectory and behavior pattern, and determine whether the vehicle has received the instruction to park in the designated area by tracking the vehicle's speed and direction changes in real time; 5) Vehicle positioning; Use a combination of panoramic cameras to ensure full coverage of the equipment vehicles in the cabin, search the monitoring images of the equipment vehicles in the cabin based on the pictures of the vehicles, and locate the vehicles through image analysis and feature comparison; 6) Monitor the movement status of parked vehicles in real time and trigger an alarm if any abnormal movement is detected, including position slippage or abnormal driving direction.

2. The vehicle positioning and state monitoring method based on vision technology according to claim 1 is characterized in that in the step 2), feature extraction is as follows: 2.1) Image preprocessing; 2.1.1) Calculation of Directional Pattern 2.1.2) Image Segmentation For an image, it is divided into four types of image areas: background area, irrecoverable area, clear area, and recoverable area; Keep clear and recoverable areas; 2.1.3) Image Enhancement 2.1.4) Binarization and thinning of images; 2.2) Feature extraction of images; Vehicle image feature extraction includes global features and local features; The global features are obtained by extracting and analyzing the direction map, and the global features include color, texture, and shape features; The method of local feature extraction is as follows: A 3×3 feature extraction template is used to extract the features of endpoints and divergence points of the refined image. Let m be the point to be detected, P1, P2, P3, ... P8 arranged clockwise are its 8 neighborhood points, R(1), R(2), R(3) ... R(8) are the grayscale values ​​of P1, P2, P3, ... P8 respectively. If m is an endpoint, then its neighborhood point G N Satisfy the following formula: If m is a branch point, then its neighboring point G N Satisfy the following formula: By traversing the image, local feature points of the image are found for feature extraction; 2.3) Combine global features and local features to perform vehicle image matching; According to the recognition task, global features are matched first, and then local feature recognition is used to supplement and refine to reduce recognition errors.

3. The vehicle positioning and state monitoring method based on vision technology according to claim 2 is characterized in that: In the step 2.1.1), the method for extracting the directional pattern is: a) Divide the image into sufficiently small sub-blocks; b) For each point of each sub-block, use the Sobel operator to calculate its x-direction gradient and y-direction gradient respectively; c) Calculate the direction of each sub-block according to the directional gradient; d) After obtaining the direction θ of each sub-block, it is quantized into 8 directions to obtain the direction of the image.

4. The vehicle positioning and status monitoring method based on vision technology according to claim 2 is characterized in that: In the step 2.1.3), the image is enhanced by using an enhancement method based on Gabor filtering and a low-quality image enhancement algorithm based on Fourier filtering; Taking into account the directional and frequency characteristics of the image, an improved Gabor filter is used for image processing. The even Gabor function is changed into a digital filter. The filter obtained by taking the real part of the Gabor function as a template is more realistic and close to the characteristics of the image. Digital filter, as shown below: Where δx and δy represent the standard deviation of the Gabor filter in the x-axis and y-axis directions, respectively, and fx represents the frequency in the x-axis direction.

5. The vehicle positioning and status monitoring method based on vision technology according to claim 2 is characterized in that: In the step 2.1.4), a dynamic threshold image binarization method based on a directional map is used to directly obtain a dynamic threshold from the image grayscale image to binarize the image; and an 0PTA algorithm is used to refine the image.

6. The vehicle positioning and status monitoring method based on vision technology according to claim 2 is characterized in that: In the step 2.3), the recognition process based on local features is as follows: Firstly, local features are extracted by using two methods: interest point detection and dense extraction; Then, the local features are encoded using feature transformation algorithms, including vector quantization and sparse coding; Perform feature aggregation operation to obtain a fused feature vector; Finally, a deep learning model structure is selected to learn a classifier to complete vehicle classification and recognition based on local features.

7. An electronic device, characterized in that: include: one or more processors; as well as a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are caused to execute the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images