Police station-based image recognition method and device, electronic device, and storage medium

Abstract

The application provides a police station-based image recognition method and device, electronic equipment and storage medium, and belongs to the technical field of police safety, and the method comprises the following steps: acquiring first thermal imaging data and second thermal imaging data of a target area; analyzing each data, determining a target vehicle through redundancy verification, and acquiring a visible light image of the target vehicle; performing segmentation processing and feature extraction on the visible light image to obtain a vehicle body feature vector, a license plate number and a face feature vector; thereby generating a comprehensive image file of the target vehicle; and matching the comprehensive image file with a police database, and identifying an abnormal vehicle according to a matching result. The police station-based image recognition method and device, electronic equipment and storage medium provided by the application can improve the accuracy and anti-interference capability of vehicle recognition.

Description

Technical Field

[0001] This application belongs to the field of police security technology, and more specifically, relates to an image recognition method and device, electronic device, and storage medium based on a police station. Background Technology

[0002] With the increasing demands for public security, police stations, as core nodes in grassroots security and prevention, bear the important responsibility of dynamically monitoring vehicles and personnel within their jurisdiction and identifying abnormal behavior. In the field of public safety, vehicles, as mobile carriers, require identification, trajectory tracking, and early warning of abnormal behavior as key links in crime prevention and maintaining order.

[0003] Currently, image recognition technology is widely used in police surveillance, but due to limitations such as the performance of single sensors and insufficient feature extraction dimensions, the accuracy of vehicle identification is low and the anti-interference ability is weak, making it difficult to meet the needs of precise monitoring in complex scenarios. Summary of the Invention

[0004] This application provides an image recognition method and device, electronic device, and storage medium based on a police station to improve the accuracy and anti-interference capability of vehicle recognition.

[0005] According to one aspect of the embodiments of this application, an image recognition method based on a police station is provided, comprising: First thermal imaging data and second thermal imaging data of the target area are acquired. The first thermal imaging data and second thermal imaging data are obtained by first and second sensors deployed at different locations, respectively. The first thermal imaging data and the second thermal imaging data are analyzed, the target vehicle is identified through redundancy verification, and the visible light image of the target vehicle is acquired. The visible light image is segmented to obtain the segmented vehicle body image, license plate image, and person face image; features are extracted from the vehicle body image, license plate image, and person face image to obtain vehicle body feature vector, license plate number, and face feature vector; the vehicle body feature vector includes vehicle body shape feature vector and vehicle body texture feature vector. A comprehensive image file of the target vehicle is generated based on the vehicle body feature vector, license plate number, and facial feature vector; the comprehensive image file is used to represent the comprehensive feature identifier of the target vehicle. The system matches comprehensive image archives with police databases to identify abnormal vehicles based on the matching results.

[0006] According to one aspect of the embodiments of this application, an image recognition device based on a police station is provided, comprising: The data acquisition module is used to acquire first thermal imaging data and second thermal imaging data of the target area. The first thermal imaging data and second thermal imaging data are obtained by sensors deployed at different locations. The image acquisition module is used to analyze the first thermal imaging data and the second thermal imaging data, determine the target vehicle through redundancy verification, and acquire the visible light image of the target vehicle. The image processing module is used to segment the visible light image to obtain the segmented vehicle body image, license plate image, and human face image; it performs feature extraction on the vehicle body image, license plate image, and human face image respectively to obtain the vehicle body feature vector, license plate number, and human face feature vector; the vehicle body feature vector includes the vehicle body shape feature vector and the vehicle body texture feature vector. The fusion encapsulation module is used to generate a comprehensive image file of the target vehicle based on the vehicle body feature vector, license plate number, and facial feature vector; the comprehensive image file is used to represent the comprehensive feature identifier of the target vehicle. The vehicle recognition module is used to match comprehensive image files with the police database and identify abnormal vehicles based on the matching results.

[0007] According to one aspect of the embodiments of this application, an electronic device is provided, the electronic device including a processor and a memory, the memory storing a computer program, the computer program being loaded and executed by the processor to implement the above-described image recognition method based on a police station.

[0008] According to one aspect of the embodiments of this application, the computer program product includes a computer program stored in a computer-readable storage medium. A processor of an electronic device reads the computer program from the computer-readable storage medium and executes the computer program, causing the electronic device to perform the aforementioned police station-based image recognition method.

[0009] The technical solution provided in this application embodiment can include the following beneficial effects: This embodiment performs redundancy verification on the first thermal imaging data and the second thermal imaging data, effectively eliminating misjudgments caused by environmental interference (such as occlusion, haze) of a single sensor, and improving the accuracy of recognition; after the verification is passed, four dimensions of features are extracted respectively: vehicle body shape feature vector, vehicle body texture feature vector, license plate number, and face feature vector, and they are fused to generate a comprehensive image file of the target vehicle. This multi-dimensional feature fusion mechanism significantly improves the anti-interference ability in complex scenes (e.g., at night, in occluded or blurry situations), and has uniqueness and anti-tampering properties; This embodiment also performs multi-dimensional matching between the comprehensive image file and the police database, which significantly improves the efficiency and accuracy of abnormal vehicle recognition. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A flowchart illustrating the image recognition method based on a police station provided in this application embodiment; Figure 2 A structural block diagram of an image recognition device based on a police station provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0013] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0014] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0015] It should be understood that although this application may use terms such as first, second, etc., to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, a first parameter may also be referred to as a second parameter, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0016] It should be noted that this application may display a prompt interface, pop-up window, or output voice prompt information before and during the collection of users' facial images. This prompt interface, pop-up window, or voice prompt information is used to inform the user that their relevant data is being collected. This ensures that the application only begins the steps related to acquiring the user's facial image after receiving confirmation from the user regarding the prompt interface or pop-up window; otherwise, if no confirmation is received from the user, the steps related to acquiring the user's relevant data are terminated, i.e., the user's facial image is not acquired. In other words, all user facial images collected in this application are collected with the user's consent and authorization, and the collection, use, and processing of facial images must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0017] Figure 1 This is a flowchart of an image recognition method based on a police station, provided in an embodiment of this application. The method is executed by an electronic device and may include: S101: Acquire the first thermal imaging data and the second thermal imaging data of the target area.

[0018] In this embodiment, the target area can refer to a specific area within the jurisdiction of the police station that requires vehicle monitoring, such as an intersection, parking lot, or key control area. The first thermal imaging data and the second thermal imaging data are obtained by a first sensor and a second sensor deployed at different locations, respectively. Both the first and second sensors can be infrared thermal imaging devices used to collect the infrared radiation energy of objects within the target area and convert it into a temperature distribution image. By deploying sensors at different locations, thermal radiation information of the target can be obtained from multiple angles, facilitating subsequent redundancy verification and improving the accuracy and robustness of target identification.

[0019] In this embodiment, the first and second sensors can be installed at two different monitoring points in the target area. For example, the first sensor can be installed on the roof of the police station, and the second sensor can be installed on a roadside lamppost. The fields of view of the two sensors partially overlap to ensure that the same target vehicle can be captured by both sensors simultaneously or sequentially. The two sensors are connected to the control system of the police station via wired or wireless means to transmit the collected thermal imaging data in real time.

[0020] In this embodiment, thermal imaging data is typically acquired in frames. Each frame consists of multiple pixels, and the grayscale or pseudo-color value of each pixel corresponds to a temperature value. To facilitate subsequent processing, this embodiment adds a timestamp to each frame of data and records the corresponding sensor identifier and location information.

[0021] S102: Analyze the first thermal imaging data and the second thermal imaging data, determine the target vehicle through redundancy verification, and acquire the visible light image of the target vehicle.

[0022] In one embodiment of this application, the analysis of first thermal imaging data and second thermal imaging data, and the determination of the target vehicle through redundancy verification, includes: Within a preset time window, vehicle identification and redundancy verification operations are performed on the first thermal imaging data and the second thermal imaging data to determine the target vehicle. The vehicle identification and redundancy verification operations include: Extract pixels whose pixel temperature is higher than the preset ambient temperature from the first thermal imaging data, generate a first binary image based on each pixel, perform connected component analysis on the first binary image to obtain a first target contour region, and determine the first area and first average temperature of the region based on the first target contour region. Based on the temperature distribution within the first target contour region, a first temperature gradient feature of the target vehicle engine region is obtained; the first temperature gradient feature is used to characterize the thermal distribution characteristics of the target vehicle engine region, and the first target contour region includes the target vehicle engine region. If the first area is higher than a preset area threshold, the first average temperature is higher than a preset temperature threshold, and there is a first temperature gradient feature within the first target contour area, then the vehicle is determined to be a candidate vehicle. Extract pixels whose pixel temperature is higher than the preset ambient temperature from the second thermal imaging data, generate a second binary image based on each pixel, perform connected component analysis on the second binary image to obtain the second target contour region, and determine the second area and second average temperature of the region based on the second target contour region. Based on the temperature distribution within the second target contour region, the second temperature gradient feature of the vehicle engine region is obtained; the second temperature gradient feature is used to characterize the thermal distribution characteristics of the target vehicle engine region. If, within a preset time window, the second area is higher than a preset area threshold, the second average temperature is higher than a preset temperature threshold, and a second temperature gradient feature exists within the second target contour area, then the candidate vehicle is determined to be the target vehicle.

[0023] In this embodiment, a preset time window is set. This preset time window refers to the maximum allowable time difference between when the same vehicle may be captured sequentially by the first sensor and the second sensor. The window length can be set based on the average speed of vehicles within the target area, the deployment distance between the two sensors, and the sensor's frame rate. For example, in a typical urban road scenario, if the two sensors are 50 meters apart and the average vehicle speed is 30 km / h, it would take approximately 6 seconds to traverse that distance. Therefore, the preset time window can be set to 10 seconds to ensure that the same vehicle can be detected and matched in both sensor data. In this embodiment, the starting point of the preset time window is the moment when the first sensor first detects the candidate vehicle, and it slides backward until the window ends.

[0024] In this embodiment, the preset ambient temperature refers to the ambient background temperature of the target area at the current moment. It can be obtained in real time by an ambient temperature sensor deployed in the monitoring area, or a typical value can be set according to historical meteorological data (for example, 30°C during the day in summer and 3°C at night in winter).

[0025] In this embodiment, the temperature value corresponding to each pixel is read from the first thermal imaging data collected by the first sensor, and the temperature of each pixel is compared with a preset ambient temperature. If the pixel temperature is higher than the preset ambient temperature, the pixel is marked as 1 (foreground); otherwise, it is marked as 0 (background), thereby generating a first binary image. This binary image highlights objects with temperatures higher than the background, typically corresponding to heat-generating parts such as vehicle engines and exhaust pipes.

[0026] Connectivity analysis is performed on the first binary image to find connected regions formed by adjacent foreground pixels. In this embodiment, the connectivity analysis can use four-neighbor or eight-neighbor methods, treating interconnected pixels as the same region, with each connected region corresponding to a potential heat-generating object. Based on actual monitoring needs (e.g., excluding interference areas that are too small), this embodiment filters out the region including the engine from all connected regions as the first target contour region. The first target contour region is a closed contour region delineated by boundary pixels, representing the projection range of the candidate vehicle engine in thermal imaging.

[0027] This embodiment focuses on the first target contour region, counting the total number of pixels contained in the region and the actual area corresponding to each pixel in thermal imaging (which can be determined according to the spatial resolution of the sensor). The first area of ​​the region is calculated based on the total number of pixels and the actual area represented by a single pixel (e.g., 0.01 square meters per pixel). Simultaneously, the average temperature of all pixels within the region is calculated to obtain the first average temperature.

[0028] In this embodiment, the spatial distribution of temperature of each pixel within the first target contour region is further analyzed to obtain the first temperature gradient feature of the engine region. This temperature gradient feature describes the local temperature variation pattern of the engine region. Vehicle engines typically exhibit distinct thermal distribution characteristics during operation, with the highest temperature at the center, gradually decreasing outwards to form a temperature gradient from the inside out. This embodiment uses a local extremum detection method to determine the pixel with the highest temperature within the region. Based on this pixel, the rate and direction of temperature decrease are analyzed to generate the first temperature gradient feature reflecting the direction and intensity of temperature change. This feature is used to distinguish the engine heating area from other high-temperature interference sources (e.g., ground reflections, pedestrians, etc.).

[0029] In this embodiment, after determining the first area, the first average temperature, and the first temperature gradient feature, candidate vehicles are determined based on preset area thresholds, preset temperature thresholds, and the existence conditions of gradient features. Specifically, if the first area is higher than the preset area threshold, the first average temperature is higher than the preset temperature threshold, and the first temperature gradient feature exists within the first target contour region, then the vehicle is determined to be a candidate vehicle, and the timestamp of the candidate vehicle's appearance and the contour region information are recorded.

[0030] In this embodiment, the preset area threshold is used to exclude noise areas that are too small. This preset area threshold can be determined based on the projected area range of a typical vehicle engine; for example, it can be set to 0.8 square meters. The preset temperature threshold should be higher than the ambient temperature but lower than the typical temperature at which the engine operates normally; for example, it can be set to 50°C (when the ambient temperature is 25°C) to exclude non-heat-generating objects.

[0031] In this embodiment, within the same preset time window, the second thermal imaging data acquired by the second sensor undergoes the same processing procedure as the first thermal imaging data described above to obtain the second area, second average temperature, and second temperature gradient features of the second target contour region. Specifically, pixels with a temperature higher than a preset ambient temperature are extracted from the second thermal imaging data to generate a second binary image; connected component analysis is performed to obtain the second target contour region; the second area and second average temperature are calculated; and the second temperature gradient features are extracted.

[0032] After completing the above analysis, if the second area is higher than a preset area threshold, the second average temperature is higher than a preset temperature threshold, and a second temperature gradient feature exists within the second target contour region, then the heat-generating object is determined to be the same target as the candidate vehicle previously obtained from the first data. In this case, the candidate vehicle is confirmed as the target vehicle.

[0033] The core of redundancy verification in this embodiment lies in using two sensors at different locations to independently detect the same target. Only when both sensors detect a heating area that matches vehicle characteristics within a preset time window, and the two are consistent in space and time, is the target vehicle finally identified. This dual confirmation mechanism effectively reduces the probability of false detection by a single sensor (such as misidentifying a high-temperature pipe as a vehicle) and improves the reliability of target recognition.

[0034] In this embodiment, if only a single sensor (the first sensor or the second sensor) detects a candidate vehicle, or if the areas detected by the two sensors cannot match within a time window, the candidate vehicle is abandoned, and the monitoring continues for the next vehicle. In this embodiment, "cannot match" means that both sensors detect a heat-generating area matching the vehicle's characteristics within a preset time window (i.e., both detect the candidate vehicle), but after spatiotemporal consistency analysis, it is found that these two heat-generating areas do not originate from the same physical vehicle.

[0035] As can be seen from the above, this embodiment significantly improves the accuracy and robustness of target vehicle identification by introducing a redundancy verification mechanism. Through dual-sensor redundancy verification, both independent data sources (first thermal imaging data and second thermal imaging data) meet the vehicle determination criteria, effectively reducing the false detection rate. This embodiment further improves the targeting of detection by distinguishing the unique heat distribution of the engine from other heat sources (e.g., direct sunlight on the ground, high-temperature exhaust gas, etc.) using temperature gradient features. This embodiment sets a preset time window to ensure the time correspondence of the same vehicle in the two sensor data, improving time matching accuracy and avoiding time misalignment caused by different vehicle speeds, providing a reliable foundation for subsequent feature fusion and profile generation.

[0036] In this embodiment, after acquiring the target vehicle, a capture command is sent to the visible light camera deployed in the target area. During capture, the pan-tilt angle or zoom is adjusted based on the target contour position information provided by thermal imaging to ensure the target vehicle is centered in the frame and to acquire a high-resolution visible light image. In this embodiment, the visible light camera and thermal imaging sensor are typically installed in conjunction, with the viewing angle aligned with the target area.

[0037] In one embodiment of this application, the method for determining the existence of temperature gradient features within the target contour includes: For each target contour, the pixel with the highest temperature within the target contour is determined by the local extremum localization detection method. Using this pixel as the seed point, the region growing method is used to determine the third target contour region. Based on the third target contour region, the third area and the third average temperature of the region are determined. Obtain the ratio of the third area to the total area of ​​the target contour; the total area of ​​the target contour is either the first area or the second area. Obtain the absolute value of the difference between the third average temperature and the average temperature of the target profile; the average temperature of the target profile is either the first average temperature or the second average temperature. If the ratio is higher than the preset ratio threshold and the absolute value of the difference is higher than the preset temperature difference threshold, then it is determined that there is a temperature gradient feature in the target contour.

[0038] In this embodiment, a preset ratio threshold is used to determine whether the area ratio of the high-temperature core region reaches the minimum ratio of typical engine characteristics. If the ratio is higher than this threshold, the high-temperature region is considered sufficiently significant. The preset ratio threshold can be set to 0.3. A preset temperature difference threshold is used to determine whether the difference between the average temperature of the high-temperature core region and the average temperature of the overall profile is sufficiently large. If the absolute value of the difference is higher than this threshold, a significant temperature gradient is considered to exist. The preset temperature difference threshold can be set to 15℃.

[0039] This embodiment can be executed after obtaining the first or second target contour region to verify whether the contour region possesses the temperature gradient characteristics unique to the engine. This embodiment uses the first target contour region as an example for explanation; the processing method for the second target contour region is exactly the same.

[0040] For the first target contour region, the temperature values ​​of all pixels within this region are traversed, and the pixel with the highest temperature is identified using a local extremum localization detection method. Specifically, a sliding window comparison method can be used. For each pixel within the contour, its temperature is compared with the temperatures of its eight neighboring pixels. If the temperature of this pixel is higher than the temperatures of all its neighboring pixels, it is marked as a local extremum point. After traversal, the highest temperature among all local extremum points is selected as the global highest point, i.e., the seed point. If multiple highest points with the same temperature exist, any one can be chosen, or the geometric center point can be selected. This seed point represents the location of the strongest thermal radiation within the engine region.

[0041] In this embodiment, after determining the seed point, a region growth operation is performed starting from the seed point, continuously merging pixels whose surrounding temperatures meet the growth conditions to form the third target contour region. The region growth criterion in this embodiment can be set as follows: pixels in the eight neighboring pixels around the seed point are considered candidate pixels. If the temperature difference between the candidate pixel and the seed point is less than a preset growth temperature difference threshold, the candidate pixel is merged with the seed point and used as a new growth point. This neighbor check is repeated for newly added pixels until no new pixels meet the conditions, thus obtaining the third target contour region. The preset growth temperature difference threshold can be set according to the engine's heat dissipation characteristics, for example, 15℃~25℃. If the ambient temperature is low (e.g., in winter), and engine heat dissipation is accelerated, the threshold can be adjusted to 10℃. The optimal value in this embodiment can be determined through experimental calibration based on actual application scenarios.

[0042] In this embodiment, the total number of pixels contained in the third target contour region is counted, and the third area of ​​the region is determined based on the actual physical area represented by each pixel (determined by the sensor resolution). Simultaneously, the average temperature of all pixels within this region is calculated to obtain the third average temperature.

[0043] Obtain the ratio of the third area to the first area (total area of ​​the target contour), which characterizes the proportion of the high-temperature core region in the entire engine heat-generating area. Obtain the absolute value of the difference between the third average temperature and the first average temperature (average temperature of the target contour), which characterizes the average temperature difference between the high-temperature core region and the entire heat-generating area.

[0044] In this embodiment, the calculated ratio is compared with a preset ratio threshold, and the absolute value of the temperature difference is compared with a preset temperature difference threshold. If the ratio is greater than the preset ratio threshold and the absolute value of the temperature difference is greater than the preset temperature difference threshold, it is determined that there is a temperature gradient feature within the first target contour region. If neither condition is met, it is determined that there is no temperature gradient feature, that is, the target contour may not be generated by the vehicle engine (for example, it may be a uniformly heated object or a false detection area).

[0045] As can be seen from the above, this embodiment uses a combination of local extremum localization and region growing to accurately extract the high-temperature core area of ​​the engine region, and uses two quantitative indicators, area ratio and temperature difference, to effectively determine whether there are typical temperature gradient features within the target contour.

[0046] This embodiment improves detection accuracy by utilizing temperature gradient features; the region growing method can adaptively extract high-temperature core areas, unaffected by irregular target contour shapes, and has good adaptability to different vehicle models and angles; by using the absolute values ​​of area ratios and temperature differences, qualitative features are transformed into quantitative judgments, facilitating threshold adjustment and optimization, and making the detection results more objective and reliable.

[0047] In one embodiment of this application, obtaining a first temperature gradient feature of the engine region of the target vehicle includes: The temperature distribution within the first target contour region is gradient calculated to obtain the temperature change rate corresponding to each direction, including the horizontal direction, vertical direction, and diagonal direction. Based on the temperature change rate corresponding to each direction, a first temperature gradient direction histogram is generated as the first temperature gradient feature.

[0048] In this embodiment, for each pixel within the first target contour region, the rate of temperature change in the horizontal, vertical, and diagonal directions is calculated. This embodiment can employ classical difference operators for calculation, such as the Sobel operator or a simple central difference method.

[0049] For each pixel, the gradient magnitude and gradient direction are determined based on the horizontal and vertical rates of change. In this embodiment, the gradient magnitude is used to represent the intensity of the temperature change, and the formula for calculating the gradient magnitude is as follows: ,in, Indicates the gradient magnitude. Indicates the rate of change in the horizontal direction. The gradient direction represents the rate of change in the vertical direction. It is used to represent the direction of the fastest change in dimension, and the formula for calculating the gradient direction is: ,in, Indicates the gradient direction. In this embodiment, the gradient direction... ∈[0°,180°).

[0050] In this embodiment, the temperature change rate in the diagonal direction can be used to assist in determining the reliability of the gradient direction. This embodiment can... or As an auxiliary feature, and combined with the rate of change in the horizontal direction and vertical rate of change Generate a gradient histogram.

[0051] In this embodiment, the gradient directions of all pixels within the first target contour region are statistically analyzed to generate a gradient direction histogram. The specific operation is as follows: The angular range from 0° to 180° is divided into N equal intervals, each interval serving as a "direction post". In this embodiment, the value of N can be set according to actual needs; for example, N=9 indicates that each interval covers 20°. The larger N is, the higher the directional resolution and the higher the feature dimension. Generally, the value of N can be between 9 and 18.

[0052] For each pixel within the first target contour region, based on its gradient direction Determine the direction range to which it belongs, and then calculate the gradient magnitude of that pixel. The counts are accumulated in the corresponding intervals to obtain the first temperature gradient direction histogram. In this embodiment, the histogram not only reflects the distribution of gradient directions but also the intensity of temperature changes in each direction.

[0053] To eliminate the influence of the first area size, this embodiment normalizes the histogram. In this embodiment, the cumulative value of each direction bar is divided by the sum of the cumulative values ​​of all direction bars, making the total histogram sum of 1, resulting in a normalized gradient direction histogram. The normalized histogram has scale invariance, facilitating subsequent feature matching. This embodiment uses the first temperature gradient direction histogram as the first temperature gradient feature. This feature is an N-dimensional vector, where each component represents the cumulative intensity of temperature change within the corresponding direction interval.

[0054] In this embodiment, the same processing procedure is applied to the second target contour region to obtain the second temperature gradient feature of the target vehicle engine region.

[0055] As can be seen from the above, this embodiment effectively captures the directional pattern of heat diffusion from the center to the surrounding areas of the engine region through the gradient direction histogram; it also introduces the first temperature gradient feature and the second temperature gradient feature, which have better invariance to global disturbances such as light and ambient temperature changes, and enhance anti-interference ability.

[0056] S103: Segment the visible light image to obtain the segmented vehicle body image, license plate image, and person face image; extract features from the vehicle body image, license plate image, and person face image to obtain the vehicle body feature vector, license plate number, and person face feature vector.

[0057] In this embodiment, the vehicle body feature vector includes the vehicle body shape feature vector and the vehicle body texture feature vector.

[0058] In one embodiment of this application, a visible light image is segmented to obtain a segmented vehicle body image, a license plate image, and a person's facial image, including: The sharpness of visible light images is evaluated based on the gray-level variance method, and pixels with gray-level variance values ​​higher than the preset gray-level threshold are selected as images to be processed. The image to be processed is subjected to grayscale normalization to obtain a normalized image; The normalized image is segmented to extract the vehicle body image, license plate image, and facial region of the person, resulting in segmented vehicle body image, license plate image, and facial image of the person.

[0059] In this embodiment, the preset grayscale threshold can be used as a lower limit for the grayscale variance of images to filter for clear images. If the grayscale variance of an image is higher than this threshold, the image is considered sufficiently clear; if the grayscale variance of an image is not higher than this threshold, the image is considered blurry (possibly due to motion, misalignment, or insufficient lighting), and needs to be re-acquired. This preset grayscale threshold can be determined using empirical methods, adaptive methods, or fixed value settings. For example, the preset grayscale threshold can be set to 80.

[0060] In this embodiment, the vehicle body image refers to the segmented image block containing the main body of the vehicle (excluding detailed areas such as license plates and windows), used for subsequent extraction of vehicle body shape and texture features. The license plate image refers to the segmented image block containing the vehicle license plate area, used for subsequent license plate character recognition. The person's face image refers to the segmented image block containing the driver's or passenger's face area, used for subsequent facial feature extraction.

[0061] This embodiment converts the original color visible light image into a grayscale image (if the acquired image is grayscale, it is used directly), and calculates the grayscale variance of the entire image. If the variance is higher than a preset grayscale threshold, the image is determined to be clear and is retained as the image to be processed; if the variance is lower than or equal to the preset grayscale threshold, the image is determined to be blurry, the image is discarded, and a re-acquisition command is triggered until an image that meets the clarity requirements is obtained.

[0062] For the images that pass the sharpness screening, grayscale normalization is performed to eliminate the impact of illumination differences on subsequent segmentation and feature extraction. The specific steps are as follows: Iterate through all pixels of the image to be processed, find the minimum and maximum gray values ​​in the image; normalize the gray value of each pixel by using a linear transformation, obtain the normalized gray value, and use the normalized gray value as the new gray value of the pixel to obtain the normalized image.

[0063] This embodiment performs target detection and segmentation operations on the normalized image for vehicle body, license plate, and human face, respectively. A feature-based detection method can be used in this embodiment. Specifically, the vehicle body image segmentation is as follows: The vehicle body region is located based on its edge information and geometric features. First, edge detection (e.g., using the Canny operator) is performed on the normalized image to obtain the vehicle's edge map. Morphological closing operations are then used to connect adjacent edges, forming connected regions. Given that vehicles typically occupy a large area of ​​the image and are rectangular (with an aspect ratio of approximately 2:1), the candidate connected region with the largest area is selected as the vehicle body region. If multiple candidates exist, further confirmation can be made by combining the vehicle's bottom shadow (usually with lower grayscale) or the vehicle's symmetry, obtaining the bounding rectangle of this region, and cropping the vehicle body image from the image. In this embodiment, the vehicle body image should include the main body of the vehicle, typically from the front bumper to the rear bumper, but excluding details such as license plates and windows to avoid interfering with subsequent shape and texture feature extraction.

[0064] The specific steps for segmenting the license plate image are as follows: A license plate localization method based on edge and texture analysis is adopted. First, the normalized image is converted to grayscale, and the horizontal gradient is calculated using the Sobel operator to obtain a horizontal edge map. The horizontal edge map is then binarized, and morphological dilation is used to connect dense edges in the horizontal direction to form candidate connected regions. Possible license plate regions are selected based on the aspect ratio (usually 3:1 to 5:1) and area range of the license plate. For each candidate region, it is further verified using vertical projection or character jump count (the license plate region should have frequent grayscale jumps in the vertical direction). After successful verification, the license plate image is cropped from the image for subsequent character recognition.

[0065] The specific steps for segmenting facial images are as follows: A method combining skin color detection and geometric feature analysis is used to locate facial regions. First, the normalized image is converted to the YCbCr or HSV color space. Skin color is segmented based on its clustering characteristics in the color space (e.g., blue chromaticity component Cb and red chromaticity component Cr are within a certain range), resulting in a binarized skin color mask. Morphological opening operations are performed on the skin color mask to remove noise, and connected regions are labeled. Candidate regions are selected based on the geometric features of the face (e.g., aspect ratio close to 1:1, moderate area, and location typically in the upper half of the image). Considering the possibility of multiple occupants inside the vehicle, all connected regions meeting the criteria are cropped to obtain a facial region encompassing the area from the forehead to the chin and between the left and right ears. If multiple faces are detected, they are cropped and saved separately for subsequent extraction of facial features.

[0066] In this embodiment, the three segmentation operations can be performed in any order or in parallel. The detector parameters used in the segmentation process (such as detection window size, stride, threshold, etc.) are determined during the offline training phase and fixed in actual application.

[0067] As can be seen from the above, this embodiment eliminates blurry images through sharpness evaluation, avoiding invalid calculations; grayscale normalization eliminates illumination differences, ensuring consistent performance of the segmentation and feature extraction algorithms under different illumination conditions; this embodiment ensures the purity of the vehicle body, license plate, and face regions through the above segmentation method, reducing background interference and thus improving the accuracy of feature extraction; the three independent image blocks obtained by this embodiment correspond to three types of key vehicle information (overall appearance, identification, and personnel characteristics), facilitating the subsequent targeted design of feature extraction methods and providing a structured data source for the construction of comprehensive image archives.

[0068] In one embodiment of this application, feature extraction is performed on the vehicle body image, license plate image, and facial image to obtain vehicle body feature vector, license plate number, and facial feature vector, including: Extract the Fourier descriptor of the vehicle body outline from the vehicle body image, and use the Fourier descriptor as the vehicle body shape feature vector. Extract the texture gradient change in the vehicle body image, and use the texture gradient change as the vehicle body texture feature vector. The characters in the license plate image are tilted and corrected. The geometric features of each character are extracted and matched with standard characters to obtain a character sequence. The license plate number is then determined based on the character sequence. Based on the facial image of the person, key parts are located, local binary pattern features of key parts are extracted, and facial feature vectors are formed.

[0069] In this embodiment, the key parts refer to representative local areas in the face image, such as the forehead, eyes, nose, mouth, and eyebrows.

[0070] For the vehicle body shape feature vector, this embodiment can use the Canny operator to perform edge detection on the vehicle body image to obtain the external contour point set of the vehicle body. In this embodiment, the external contour of the vehicle body is composed of M contour points arranged in a counterclockwise or clockwise order, and the coordinates of each contour point are... ,in, The coordinates of each contour point are represented in complex form to obtain a complex number sequence. .

[0071] Perform a discrete Fourier transform on the complex sequence to obtain the Fourier coefficients. ,in, .in, Represents the DC component, reflecting the centroid position of the profile. This represents the corresponding basic elliptical shape (low-frequency component), reflecting the approximate shape of the contour. (u≥2) represents the detailed changes in the contour (high-frequency components). In this embodiment, to obtain invariance to translation, rotation, and scaling, the Fourier coefficients are normalized: typically ignored. In this embodiment, | Normalize the subsequent coefficients to obtain the normalized descriptor. Take the first... ( The amplitude (or real and imaginary parts) of the normalized coefficients constitute the vehicle body shape feature vector. The value of determines the dimension and descriptive precision of the feature vector, and is generally between 10 and 30. For example, taking ... =20, resulting in a 20-dimensional shape feature vector.

[0072] For the vehicle body texture feature vector, this embodiment divides the vehicle body image into several small cell units, for example, each cell unit can be 8×8 pixels in size. Within each cell unit, a histogram of gradient directions is calculated: 0°~180° is divided into N intervals (N is 9, representing each interval as 20°), and the gradient magnitude is weighted to obtain the 9-dimensional feature vector of that cell unit. Several adjacent cell units (e.g., 2×2) are combined into a block, and the feature vectors of the cell units within this block are concatenated and normalized to eliminate the influence of lighting and contrast variations. In this embodiment, the feature vectors of all blocks are concatenated to obtain the HOG feature vector of the entire vehicle body image, which serves as the vehicle body texture feature vector.

[0073] This embodiment performs binarization processing on the license plate image (e.g., using the Otsu method) to obtain a black-and-white binary image. Hough transform is used to detect straight lines in the image, and the longest straight line is selected as candidate lines for the top and bottom borders of the license plate. Based on the tilt angles of these candidate lines, the overall tilt angle of the license plate is determined.

[0074] If the absolute value of the tilt angle is greater than a preset angle threshold (which can be 5°), the image is rotated to correct the character rows to a horizontal orientation. If the license plate has perspective distortion (e.g., due to an incorrect shooting angle), perspective transformation can be used to correct the license plate to a front view. After correction, the characters should be basically horizontal and of uniform size. The corrected binary image is then horizontally projected, the number of black dots in each row is counted, and the upper and lower boundaries of the character rows are determined based on the trough positions.

[0075] Vertical projection is performed within the character row, and the number of black dots in each column of pixels is counted. Individual characters are segmented based on the trough positions. This embodiment considers character spacing and adhesion during segmentation, and can be corrected by combining the aspect ratio constraint of the characters (the aspect ratio of standard license plate characters is approximately 0.4 to 0.6).

[0076] This embodiment normalizes each segmented character image to a fixed size (e.g., 24×48 pixels) and extracts geometric features to form a feature vector for the character to be recognized. For each feature vector of the character to be recognized, its similarity (e.g., Euclidean distance, correlation coefficient, etc.) with the feature vector of each standard character in the standard character template library is obtained, and the character with the highest similarity is selected as the recognition result. All characters are combined sequentially to obtain a character sequence. For example, for Chinese license plates, the format is usually a combination of "province Chinese characters + letters + numbers", ultimately forming the license plate number string.

[0077] In this embodiment, the standard character template library includes geometric feature vectors of provincial characters, letters, and numbers. This standard character template library can be obtained by statistically analyzing a large number of license plate character images.

[0078] This embodiment can locate facial key points using a cascaded regression method, including the center or contour points of areas such as the eyes, eyebrows, nose, and mouth. For each key point, a fixed-size neighborhood window (e.g., 32×32 pixels) is taken centered on it. Within each neighborhood window, the LBP code for each pixel is calculated. A circular neighborhood LBP operator is used, for example, with a radius of 1 and 8 sampling points (i.e., LBP8,18,1), to obtain an 8-bit binary number (0~255) for each pixel. The LBP code histogram of all pixels within the neighborhood window is statistically analyzed, and the histogram is normalized to obtain the feature vector of that area.

[0079] The feature vectors of all parts are concatenated to form the LBP feature vector of the entire face image. For example, if 5 key parts are selected, and each part uses a 59-dimensional uniform LBP, then the total feature dimension is 5 × 59 = 295 dimensions.

[0080] As can be seen from the above, this embodiment uses Fourier descriptors to represent the vehicle body contour. By normalizing the Fourier coefficients (ignoring the DC component and normalizing with the fundamental frequency amplitude), the extracted shape features are made to have translation, rotation and scaling invariance, which improves the accuracy of vehicle body recognition.

[0081] This embodiment extracts texture features based on histogram of oriented gradients (HBP). By statistically analyzing gradient directions within local cell units and normalizing within blocks, it effectively captures texture gradient changes on the vehicle surface, while exhibiting strong robustness to lighting variations and shadows. This embodiment uses Otsu's binarization method for tilt correction and combines horizontal and vertical projection to achieve accurate character segmentation. Then, based on geometric features and matching with a standard template library, it reliably identifies license plate characters, providing a core basis for vehicle identification. By extracting texture features from key facial features (eyes, nose, mouth, etc.) using Local Binary Pattern (LBP), it effectively describes the local micro-textures of the face. By concatenating the feature vectors of multiple key features, a comprehensive description of the face is formed, providing a reliable feature representation for subsequent face comparison and identification.

[0082] S104: Generate a comprehensive image file of the target vehicle based on the vehicle body feature vector, license plate number, and facial feature vector.

[0083] In this embodiment, the integrated image archive is used to characterize the comprehensive features of the target vehicle.

[0084] In one embodiment of this application, a comprehensive image file of the target vehicle is generated based on the vehicle body feature vector, license plate number, and facial feature vector, including: The license plate number is character-encoded to obtain a numerical vector of a preset length, which serves as the license plate feature vector; Acquire the timestamp, ambient light intensity, and image sharpness evaluation value when the visible light image is acquired; the image sharpness evaluation value is the grayscale variance value. Based on the ambient light intensity and image sharpness evaluation value, the first confidence level of the vehicle body feature vector, the second confidence level of the license plate feature vector, and the third confidence level of the face feature vector are determined respectively. The first confidence level, the second confidence level, and the third confidence level are positively correlated with the ambient light intensity and the image sharpness evaluation value. Based on the first confidence level, the second confidence level, and the third confidence level, the vehicle body feature vector, license plate feature vector, and face feature vector are weighted and fused to generate a comprehensive feature vector. Encapsulating the comprehensive feature vector, the timestamp, and the position information of the target vehicle to form a comprehensive image file.

[0085] In this embodiment, a mapping table from the license plate character set to numerical indices is pre-established, each character in the license plate number string is converted into a corresponding ID to obtain an integer sequence. For example, the integer sequence corresponding to the license plate "Jing A12345" is [0, 50, 74, 75, 76, 77, 78] (the ID of the Chinese character "Jing" is 0, the ID of the letter "A" is 50, and the ID of the number "1" is 74).

[0086] In this embodiment, the integer sequence can be converted into a fixed-length numerical vector by means of padding or truncation. For example, if the fixed length is 8 and the length of the integer sequence is less than 8, a special marker is padded at the end (for example, padding with ID=0, representing a padding marker); if the length of the integer sequence is not less than 8, the redundant characters at the end are truncated.

[0087] When acquiring the visible light image of the target vehicle in this embodiment, the timestamp, the ambient light intensity, and the image sharpness evaluation value (the grayscale variance value obtained above) are acquired synchronously. The ambient light intensity and the image sharpness evaluation value directly affect the quality of feature extraction. Generally, the more sufficient the illumination and the clearer the image, the more reliable the extracted features. Therefore, in this embodiment, three confidence degrees respectively correspond to the vehicle body feature vector, the license plate feature vector, and the face feature vector, all of which are positively correlated with the illumination intensity and the sharpness evaluation value.

[0088] In this embodiment, the ambient light intensity L and the image sharpness evaluation value S are respectively normalized to obtain the normalized ambient light intensity and the normalized image sharpness evaluation value. The normalization processing of the ambient light intensity is as follows: , where, represents the normalized ambient light intensity, represents the minimum value of the ambient light intensity, represents the maximum value of the ambient light intensity.

[0089] The normalization processing of the image sharpness evaluation value is as follows: , where, represents the normalized image sharpness evaluation value, represents the minimum value of the image sharpness evaluation value, represents the maximum value of the image sharpness evaluation value.

[0090] The function in this embodiment limits X to the interval of [0,1], if < 0, the function takes 0, and if > 1, the function takes 1.

[0091] In this embodiment, the vehicle body feature vector mainly relies on the vehicle body outline and texture, and has moderate sensitivity to lighting and sharpness. The formula for calculating the first confidence level of the vehicle body feature vector can be set as: ,in, This indicates the first confidence level.

[0092] In this embodiment, license plate character recognition has high requirements for lighting and clarity. Therefore, a higher sensitivity can be set, and the formula for calculating the second confidence level can be: ,in, This indicates the second confidence level.

[0093] In this embodiment, facial feature extraction is sensitive to illumination direction, illumination intensity, and image sharpness. The best facial feature extraction results are achieved under frontal illumination, while the extraction effect decreases under side illumination or low light. Therefore, the formula for calculating the third confidence level can be set as follows: ,in, This indicates the third confidence level.

[0094] In this embodiment, after obtaining three confidence levels, the vehicle body feature vector (including shape and texture, which can be concatenated into a long vector), license plate feature vector, and face feature vector are weighted, and the three weighted feature vectors are concatenated and fused in a preset order to form a comprehensive feature vector.

[0095] This embodiment encapsulates a comprehensive image archive by structurally combining feature vectors, timestamps, target vehicle location information (e.g., latitude and longitude coordinates or monitoring point number), and other metadata (e.g., sensor ID, image quality markers, etc.). The encapsulation format can be JSON, XML, or binary serialization for easy storage and transmission. In this embodiment, the comprehensive image archive is a multimodal digital description of the target vehicle, used for subsequent retrieval and matching.

[0096] As can be seen from the above, this embodiment fuses vehicle body, license plate, and facial features into a single comprehensive feature vector, fully utilizing the complementarity of multimodal information and exhibiting stronger discriminative ability compared to single features. Simultaneously, the confidence weighting mechanism dynamically adjusts the contribution of each feature based on the collection conditions, avoiding the negative impact of low-quality features on the fusion result, thereby improving the accuracy of matching with the police database.

[0097] S105: Match the integrated image files with the police database and identify abnormal vehicles based on the matching results.

[0098] In this embodiment, the police database refers to a vehicle information database established and maintained by public security organs, which contains feature files of known abnormal vehicles. Abnormal vehicles include, but are not limited to, stolen vehicles, vehicles with counterfeit license plates, vehicles suspected of being involved in cases, vehicles with expired annual inspections, and scrapped vehicles. Each record in the database typically includes the vehicle's license plate number, vehicle body feature vector (or comprehensive feature vector), vehicle type, color, owner information, deployment time, and case-related information.

[0099] This embodiment retrieves a comprehensive image file of the current target vehicle from local storage or a real-time processing module. Simultaneously, a connection is established with a police database. This database can be a centralized database deployed on the public security intranet or a distributed storage system, containing pre-recorded abnormal vehicle feature files. Each abnormal vehicle file typically includes: a unique vehicle identifier (e.g., license plate number), a comprehensive feature vector (or feature vectors of various modalities); vehicle attributes (e.g., brand, model, color, etc.), deployment information (e.g., deployment start time, end time, case number, and warning level, etc.), and other remarks.

[0100] This embodiment calculates the similarity between the current vehicle's comprehensive feature vector and the feature vector of each abnormal vehicle file in the police database using the cosine similarity method.

[0101] This embodiment sets a preset similarity threshold to determine whether the current vehicle matches a record in the database. The preset similarity threshold can be set based on empirical values, business requirements, or adaptive thresholding.

[0102] If the calculated maximum similarity value is greater than or equal to the preset similarity threshold, the current vehicle is determined to be the same vehicle as the vehicle corresponding to the record, i.e., the match is successful; if the maximum value is less than the preset similarity threshold, the current vehicle is determined to be not in the abnormal database and belongs to the normal vehicle.

[0103] This embodiment executes corresponding operations based on the matching results. If the match is successful (an abnormal vehicle is detected), an alarm is generated based on the detailed information in the matched abnormal vehicle file (including license plate number, vehicle type, reason for deployment, warning level, etc.). The alarm information can be pushed to relevant police officers via police terminals, command center screens, or SMS. At the same time, surrounding monitoring equipment can be linked to continuously track the vehicle, and on-the-ground police officers can be notified to intercept and check it.

[0104] If a match fails (no anomaly is found), the current vehicle's comprehensive image file will be stored in a temporary cache or historical database for subsequent trajectory analysis and data mining.

[0105] As can be seen from the above, the integrated image archive in this embodiment combines multimodal features of the vehicle body, license plate, and face, exhibiting stronger robustness compared to single license plate recognition and effectively addressing issues such as cloned license plates and obscured license plates. Furthermore, the similarity threshold is adjustable, allowing for flexible configuration based on different task requirements. This embodiment supports real-time early warning and interception.

[0106] Corresponding to the image recognition method based on the police station in the above embodiment, Figure 2 This is a structural block diagram of an image recognition device based on a police station, provided as an embodiment of this application. For ease of explanation, only the parts relevant to the embodiment of this application are shown. References Figure 2 The image recognition device 20 based on the police station includes: a data acquisition module 21, an image acquisition module 22, an image processing module 23, a fusion and encapsulation module 24, and a vehicle recognition module 25.

[0107] The data acquisition module 21 is used to acquire first thermal imaging data and second thermal imaging data of the target area. The first thermal imaging data and second thermal imaging data are obtained by sensors deployed at different locations. The image acquisition module 22 is used to analyze the first thermal imaging data and the second thermal imaging data, determine the target vehicle through redundancy verification, and acquire the visible light image of the target vehicle. Image processing module 23 is used to segment the visible light image to obtain the segmented vehicle body image, license plate image and human face image; and to extract features from the vehicle body image, license plate image and human face image respectively to obtain vehicle body feature vector, license plate number and human face feature vector; the vehicle body feature vector includes vehicle body shape feature vector and vehicle body texture feature vector. The fusion encapsulation module 24 is used to generate a comprehensive image file of the target vehicle based on the vehicle body feature vector, license plate number, and facial feature vector; the comprehensive image file is used to represent the comprehensive feature identifier of the target vehicle. The vehicle recognition module 25 is used to match the comprehensive image files with the police database and identify abnormal vehicles based on the matching results.

[0108] In one embodiment of this application, when analyzing the first thermal imaging data and the second thermal imaging data to determine the target vehicle through redundancy verification, the image acquisition module 22 is specifically used for: Within a preset time window, vehicle identification and redundancy verification operations are performed on the first thermal imaging data and the second thermal imaging data to determine the target vehicle. The vehicle identification and redundancy verification operations include: Extract pixels whose pixel temperature is higher than the preset ambient temperature from the first thermal imaging data, generate a first binary image based on each pixel, perform connected component analysis on the first binary image to obtain a first target contour region, and determine the first area and first average temperature of the region based on the first target contour region. Based on the temperature distribution within the first target contour region, a first temperature gradient feature of the target vehicle engine region is obtained; the first temperature gradient feature is used to characterize the thermal distribution characteristics of the target vehicle engine region, and the first target contour region includes the target vehicle engine region. If the first area is higher than a preset area threshold, the first average temperature is higher than a preset temperature threshold, and there is a first temperature gradient feature within the first target contour area, then the vehicle is determined to be a candidate vehicle. Extract pixels whose pixel temperature is higher than the preset ambient temperature from the second thermal imaging data, generate a second binary image based on each pixel, perform connected component analysis on the second binary image to obtain the second target contour region, and determine the second area and second average temperature of the region based on the second target contour region. Based on the temperature distribution within the second target contour region, the second temperature gradient feature of the vehicle engine region is obtained; the second temperature gradient feature is used to characterize the thermal distribution characteristics of the target vehicle engine region. If, within a preset time window, the second area is higher than a preset area threshold, the second average temperature is higher than a preset temperature threshold, and a second temperature gradient feature exists within the second target contour area, then the candidate vehicle is determined to be the target vehicle.

[0109] In one embodiment of this application, when determining the presence of temperature gradient features within the target contour, the image acquisition module 22 is specifically used for: For each target contour, the pixel with the highest temperature within the target contour is determined by the local extremum localization detection method. Using this pixel as the seed point, the region growing method is used to determine the third target contour region. Based on the third target contour region, the third area and the third average temperature of the region are determined. Obtain the ratio of the third area to the total area of ​​the target contour; the total area of ​​the target contour is either the first area or the second area. Obtain the absolute value of the difference between the third average temperature and the average temperature of the target profile; the average temperature of the target profile is either the first average temperature or the second average temperature. If the ratio is higher than the preset ratio threshold and the absolute value of the difference is higher than the preset temperature difference threshold, then it is determined that there is a temperature gradient feature in the target contour.

[0110] In one embodiment of this application, when obtaining the first temperature gradient features of the engine region of the target vehicle, the image acquisition module 22 is specifically used for: The gradient of the temperature distribution within the first target contour is calculated to obtain the temperature change rate corresponding to each direction, including the horizontal direction, the vertical direction, and the diagonal direction. Based on the temperature change rate corresponding to each direction, a first temperature gradient direction histogram is generated as the first temperature gradient feature.

[0111] In one embodiment of this application, when segmenting a visible light image to obtain a segmented vehicle body image, license plate image, and facial image of a person, the image processing module 23 is specifically used for: The sharpness of visible light images is evaluated based on the gray-level variance method, and pixels with gray-level variance values ​​higher than the preset gray-level threshold are selected as images to be processed. The image to be processed is subjected to grayscale normalization to obtain a normalized image; The normalized image is segmented to extract the vehicle body image, license plate image, and facial region of the person, resulting in segmented vehicle body image, license plate image, and facial image of the person.

[0112] In one embodiment of this application, when extracting features from the vehicle body image, license plate image, and facial image to obtain the vehicle body feature vector, license plate number, and facial feature vector, the image processing module 23 is specifically used for: Extract the Fourier descriptor of the vehicle body outline from the vehicle body image, and use the Fourier descriptor as the vehicle body shape feature vector. Extract the texture gradient change in the vehicle body image, and use the texture gradient change as the vehicle body texture feature vector. The characters in the license plate image are tilted and corrected. The geometric features of each character are extracted and matched with standard characters to obtain a character sequence. The license plate number is then determined based on the character sequence. Based on the facial image of the person, key parts are located, local binary pattern features of key parts are extracted, and facial feature vectors are formed.

[0113] In one embodiment of this application, when fusing vehicle body feature vectors, license plate numbers, and facial feature vectors to form a comprehensive image file of the target vehicle, the fusion encapsulation module 24 is specifically used for: The license plate number is character-encoded to obtain a numerical vector of a preset length, which serves as the license plate feature vector; Acquire the timestamp, ambient light intensity, and image sharpness evaluation value when the visible light image is acquired; the image sharpness evaluation value is the grayscale variance value. Based on the ambient light intensity and image sharpness evaluation value, the first confidence level of the vehicle body feature vector, the second confidence level of the license plate feature vector, and the third confidence level of the face feature vector are determined respectively. The first confidence level, the second confidence level, and the third confidence level are positively correlated with the ambient light intensity and the image sharpness evaluation value. Based on the first confidence level, the second confidence level, and the third confidence level, the vehicle body feature vector, license plate feature vector, and face feature vector are weighted and fused to generate a comprehensive feature vector. The integrated feature vector, timestamp, and target vehicle location information are encapsulated to form a comprehensive image archive.

[0114] It should be noted that the specific limitations of the above-described embodiment of the image recognition device 20 based on a police station can be found in the limitations of the image recognition method based on a police station described above, and will not be repeated here. Each module of the above device can be implemented entirely or partially through software, hardware, or a combination thereof. Each module can be embedded in the processor of the computer device in hardware form or independent of it, or it can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0115] Figure 3 A schematic diagram of the structure of an electronic device to which this application embodiment applies is shown, such as... Figure 3 As shown, the electronic device can be used to implement the methods provided in any embodiment of this application.

[0116] like Figure 3 As shown, the electronic device 300 may primarily include at least one processor 301. Figure 3 The diagram shows components such as a memory 302, a communication module 303, and an input / output interface 304. Optionally, these components can be connected and communicate with each other via a bus 305. It should be noted that... Figure 3 The structure of the electronic device 300 shown is merely illustrative and does not constitute a limitation on the electronic devices to which the methods provided in the embodiments of this application are applicable.

[0117] The memory 302 can be used to store operating systems and applications, etc. The applications may include computer programs that implement the methods shown in the embodiments of this application when invoked by the processor 301, and may also include programs for implementing other functions or services. The memory 302 may be ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices that can store information and computer programs, or it may be EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disk storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited thereto.

[0118] Processor 301 is connected to memory 302 via bus 305 and implements corresponding functions by calling the application programs stored in memory 302. Processor 301 can be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 301 can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0119] Electronic device 300 can connect to a network via communication module 303 (which may include, but is not limited to, components such as a network interface) to communicate with other devices (such as user terminals or servers) through the network and achieve data interaction, such as sending data to or receiving data from other devices. Communication module 303 may include wired network interfaces and / or wireless network interfaces, meaning the communication module may include at least one of wired or wireless communication modules.

[0120] The electronic device 300 can connect to necessary input / output devices, such as a keyboard and display device, via the input / output interface 304. The electronic device 300 itself may have a display device, and other display devices can also be connected externally via the interface 304. Optionally, a storage device, such as a hard drive, can also be connected via the interface 304 to store data from the electronic device 300, retrieve data from the storage device, or store data from the storage device in the memory 302. It is understood that the input / output interface 304 can be a wired interface or a wireless interface. Depending on the actual application scenario, the device connected to the input / output interface 304 can be a component of the electronic device 300 or an external device connected to the electronic device 300 when needed.

[0121] The bus 305 used to connect the components may include a path for transmitting information between the components. The bus 305 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Depending on its function, the bus 305 may be divided into an address bus, a data bus, a control bus, etc.

[0122] Optionally, for the solution provided in the embodiments of this application, the memory 302 can be used to store a computer program that executes the solution of this application, and the processor 301 runs the computer program. When the processor 301 runs the computer program, it implements the operation of the method or apparatus provided in the embodiments of this application.

[0123] Based on the same principle as the method provided in the embodiments of this application, the embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the corresponding content of the aforementioned method embodiments.

[0124] It should be noted that the terms "first," "second," "third," "fourth," "1," "2," etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in a sequence other than that shown in the figures or text.

[0125] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0126] It should be understood that although arrows indicate various operation steps in the flowcharts of this application's embodiments, the order in which these steps are implemented is not limited to the order indicated by the arrows. Unless explicitly stated herein, in some implementation scenarios of this application's embodiments, the implementation steps in each flowchart can be executed in other orders as required. Furthermore, some or all steps in each flowchart, based on the actual implementation scenario, may include multiple sub-steps or multiple stages. Some or all of these sub-steps or stages can be executed at the same time, and each sub-step or stage can also be executed at different times. In scenarios where execution times differ, the execution order of these sub-steps or stages can be flexibly configured according to requirements, and this application's embodiments do not limit this.

[0127] The above are only optional implementation methods for some implementation scenarios of this application. It should be noted that for those skilled in the art, other similar implementation methods based on the technical concept of this application without departing from the technical concept of this application also fall within the protection scope of the embodiments of this application.

Claims

1. An image recognition method based on a police station, characterized in that, include: Acquire first thermal imaging data and second thermal imaging data of the target area, wherein the first thermal imaging data and the second thermal imaging data are obtained by a first sensor and a second sensor deployed at different locations, respectively. The first thermal imaging data and the second thermal imaging data are analyzed, the target vehicle is determined through redundancy verification, and the visible light image of the target vehicle is acquired. The visible light image is segmented to obtain a segmented vehicle body image, license plate image, and person face image; features are extracted from the vehicle body image, license plate image, and person face image respectively to obtain vehicle body feature vector, license plate number, and face feature vector; the vehicle body feature vector includes vehicle body shape feature vector and vehicle body texture feature vector; A comprehensive image file of the target vehicle is generated based on the vehicle body feature vector, license plate number, and facial feature vector; the comprehensive image file is used to characterize the comprehensive feature identifier of the target vehicle. The integrated image archives are matched with the police database, and abnormal vehicles are identified based on the matching results.

2. The method as described in claim 1, characterized in that, The step of analyzing the first thermal imaging data and the second thermal imaging data to determine the target vehicle through redundancy verification includes: Within a preset time window, vehicle determination and redundancy verification operations are performed on the first thermal imaging data and the second thermal imaging data to determine the target vehicle. The vehicle determination and redundancy verification operations include: Extract each pixel whose pixel temperature is higher than the preset ambient temperature from the first thermal imaging data, generate a first binary image based on each pixel, perform connected component analysis on the first binary image to obtain a first target contour region, and determine the first area and first average temperature of the region based on the first target contour region. Based on the temperature distribution within the first target contour region, a first temperature gradient feature of the target vehicle engine region is obtained; the first temperature gradient feature is used to characterize the thermal distribution characteristics of the target vehicle engine region, and the first target contour region includes the target vehicle engine region. If the first area is higher than a preset area threshold, the first average temperature is higher than a preset temperature threshold, and the first temperature gradient feature exists within the first target contour area, then the vehicle is determined to be a candidate vehicle. Extract each pixel whose pixel temperature is higher than the preset ambient temperature from the second thermal imaging data, generate a second binary image based on each pixel, perform connected component analysis on the second binary image to obtain a second target contour region, and determine the second area and second average temperature of the region based on the second target contour region. Based on the temperature distribution within the second target contour region, a second temperature gradient feature of the vehicle engine region is obtained; the second temperature gradient feature is used to characterize the thermal distribution characteristics of the target vehicle engine region. If, within a preset time window, the second area is higher than a preset area threshold, the second average temperature is higher than a preset temperature threshold, and the second temperature gradient feature exists within the second target contour area, then the candidate vehicle is determined to be the target vehicle.

3. The method as described in claim 2, characterized in that, Methods for determining the presence of temperature gradient features within the target contour include: For each target contour, the pixel with the highest temperature within the target contour is determined by the local extremum localization detection method. Using this pixel as a seed point, the region growing method is used to determine the third target contour region. Based on the third target contour region, the third area and the third average temperature of the region are determined. Obtain the ratio of the third area to the total area of ​​the target contour; the total area of ​​the target contour is either the first area or the second area. Obtain the absolute value of the difference between the third average temperature and the average temperature of the target profile; the average temperature of the target profile is either the first average temperature or the second average temperature. If the ratio is higher than a preset ratio threshold and the absolute value of the difference is higher than a preset temperature difference threshold, then it is determined that there is a temperature gradient feature in the target contour.

4. The method as described in claim 2, characterized in that, The acquisition of the first temperature gradient characteristics of the engine region of the target vehicle includes: The temperature distribution within the first target contour is gradient calculated to obtain the temperature change rate corresponding to each direction, including the horizontal direction, the vertical direction, and the diagonal direction. Based on the temperature change rate corresponding to each direction, a first temperature gradient direction histogram is generated as the first temperature gradient feature.

5. The method as described in claim 1, characterized in that, The visible light image is segmented to obtain segmented vehicle body images, license plate images, and facial images of people, including: The sharpness of the visible light image is evaluated according to the gray-level variance method, and pixels with gray-level variance values ​​higher than a preset gray-level threshold are selected as images to be processed. The image to be processed is subjected to grayscale normalization to obtain a normalized image; The normalized image is segmented to extract the vehicle body image, license plate image, and facial region of the person, respectively, to obtain the segmented vehicle body image, license plate image, and facial image of the person.

6. The method as described in claim 1, characterized in that, The vehicle body feature vector includes a vehicle body shape feature vector and a vehicle body texture feature vector; the step of extracting features from the vehicle body image, the license plate image, and the person's facial image to obtain the vehicle body feature vector, license plate number, and facial feature vector includes: Extract the Fourier descriptor of the vehicle body outline from the vehicle body image, and use the Fourier descriptor as the vehicle body shape feature vector. Extract the texture gradient change in the vehicle body image, and use the texture gradient change as the vehicle body texture feature vector. The characters in the license plate image are tilted and corrected. The geometric features of each character are extracted and matched with standard characters to obtain a character sequence. The license plate number is then determined based on the character sequence. Based on the facial image of the person, key parts are located, and local binary pattern features of the key parts are extracted to form a facial feature vector.

7. The method as described in claim 1, characterized in that, The step of generating a comprehensive image file of the target vehicle based on the vehicle body feature vector, license plate number, and facial feature vector includes: The license plate number is character-encoded to obtain a numerical vector of a preset length, which is used as the license plate feature vector; The timestamp, ambient light intensity, and image sharpness evaluation value of the visible light image acquisition are obtained; the image sharpness evaluation value is the grayscale variance value. Based on the ambient light intensity and image sharpness evaluation value, the first confidence level of the vehicle body feature vector, the second confidence level of the license plate feature vector, and the third confidence level of the face feature vector are determined respectively; the first confidence level, the second confidence level, and the third confidence level are positively correlated with the ambient light intensity and the image sharpness evaluation value. Based on the first confidence level, the second confidence level, and the third confidence level, the vehicle body feature vector, license plate feature vector, and face feature vector are weighted and fused to generate a comprehensive feature vector; The integrated feature vector, the timestamp, and the location information of the target vehicle are encapsulated to form the integrated image archive.

8. An image recognition device based on a police station, characterized in that, include: The data acquisition module is used to acquire first thermal imaging data and second thermal imaging data of the target area. The first thermal imaging data and second thermal imaging data are obtained by sensors deployed at different locations. The image acquisition module is used to analyze the first thermal imaging data and the second thermal imaging data, determine the target vehicle through redundancy verification, and acquire the visible light image of the target vehicle. The image processing module is used to segment the visible light image to obtain a segmented vehicle body image, license plate image, and person face image; and to extract features from the vehicle body image, license plate image, and person face image to obtain a vehicle body feature vector, a license plate number, and a face feature vector; the vehicle body feature vector includes a vehicle body shape feature vector and a vehicle body texture feature vector. The fusion encapsulation module is used to generate a comprehensive image file of the target vehicle based on the vehicle body feature vector, license plate number, and facial feature vector; the comprehensive image file is used to characterize the comprehensive feature identifier of the target vehicle. The vehicle recognition module is used to match the comprehensive image archive with the police database and identify abnormal vehicles based on the matching results.

9. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing a computer program, which is loaded and executed by the processor to implement the image recognition method based on a police station as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the image recognition method based on a police station as described in any one of claims 1 to 7.

Images