Smart park visual analysis system and method based on multi-model collaborative reasoning and dynamic request management

Abstract

The invention belongs to the field of smart park visual analysis, and particularly relates to a smart park visual analysis system based on multi-model collaborative reasoning and dynamic request management, which is composed of a multi-model collaborative reasoning architecture and a dynamic request management and lightweight data flow design module. Through deep cooperation of the three modules, full-process optimization from target detection, face recognition to service deployment is realized, accurate recognition and real-time monitoring of people, vehicles and areas in a park are realized, and multi-target intelligent perception is realized. According to the method, a complicated visual analysis task is disassembled into a plurality of basic and independent atomic power modules by decomposing and designing an atomic power recognition process. Atomic energy is constructed on the basis of a YOLOv7 target detection model, a PPOCR license plate recognition model and an AdaFaceNet face recognition model, and the atomic energy is responsible for detecting targets such as people, vehicles and objects in images, recognizing license plates and analyzing facial features of people.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual analysis of smart parks, in particular to a smart park visual analysis system and method based on multi-model collaborative reasoning and dynamic request management. BACKGROUND

[0002] In the era of artificial intelligence, intelligent parks are rapidly developing, and the demand for efficient and accurate visual analysis systems has become increasingly urgent. Intelligent parks aim to achieve intelligent management of personnel, vehicles and environment, and have specific requirements for functions such as personnel and vehicle recognition, license plate recognition and pedestrian area alarm. However, the current most advanced technology still faces several major challenges.

[0003] (1) Limitations of traditional target detection algorithms: Insufficient detection accuracy in complex scenes: The traditional YOLO algorithm widely used in target detection will encounter problems in complex scenes. For example, in the environment of intelligent parks, there will usually be a large number of small objects, such as small signs, small sensors or small-scale equipment, and there will also be occluded objects, such as partially occluded vehicles or pedestrians hidden behind obstacles. In this case, the detection accuracy of the traditional YOLO algorithm is not enough. A study shows that in a complex scene test environment containing a large number of small and occluded objects, the average precision (mAP) of the traditional YOLO algorithm is only about 60%, far from meeting the high precision requirements of intelligent parking lot applications.

[0004] (2) Lack of dynamic post-processing mechanism: Existing object detection algorithms lack a dynamic post-processing mechanism. The detection results are often disconnected from the business logic. For example, in license plate recognition, after the object detection algorithm detects a vehicle, it cannot effectively and dynamically adjust the subsequent license plate recognition process according to the detection result. For applications related to faces, such as associating the detected face with specific personnel information in the park database, the current algorithm has no seamless connection, resulting in a disconnect between the detection result and the actual business needs.

[0005] (3) Difficulty in service integration: In a smart campus, numerous services such as OCR (for reading text on signs and documents within the campus), facial recognition (for access control and personnel tracking), and vehicle-related identification services need to be integrated. However, the current calling process of these multi-modal services is extremely complex. Each service usually has its own independent interface, data format, and processing requirements. For example, when integrating OCR and facial recognition services, developers need to spend a lot of time and effort to convert data formats between the two services, and module expansion is also very difficult. Adding a new identification service, such as a specific target identification for industrial products in a manufacturing smart campus, often requires a comprehensive overhaul of the existing service integration framework.

[0006] (4) Low recognition rate for low-quality images: Low-quality images, such as blurred license plate images due to poor light conditions at night, or low-resolution monitoring images in some remote areas of the campus, pose a major challenge. Existing services lack targeted pre-processing methods for such images. In a test of license plate recognition in a smart campus, when the license plate image is blurred to some extent, the recognition rate of existing systems is less than 30%, which seriously affects the normal operation of vehicle traffic management in the campus.

[0007] (5) Slow inference speed for high-resolution images: Smart campuses usually use high-resolution cameras to obtain detailed information, but high-resolution images pose a challenge to the real-time performance of visual analysis systems. Existing algorithms require a large amount of computing resources and time to process high-resolution images. For example, when processing 4K resolution images, the inference time of some common object detection algorithms can reach several seconds, which is far from meeting the real-time requirements of smart campus edge device deployment. In real-time monitoring applications at the entrance of the campus, it is necessary to quickly identify entering vehicles and pedestrians, and such a lengthy inference process may cause traffic congestion and safety risks.

[0008] These problems in current technology severely limit the development and application of smart campus visual analysis systems, and there is an urgent need for innovative solutions to improve the efficiency and accuracy of visual analysis in smart campuses.

[0009] (6) In view of the problems of multiple models being isolated, uneven resource allocation, and small target missing detection in existing park monitoring systems, we propose a smart park visual analysis system and method based on multi-model collaborative reasoning and dynamic request management. 1) A multi-model collaborative reasoning architecture is constructed, with YOLOv7 target detection model as the core, combined with PPOCR recognition model and AdaFace algorithm active face recognition model, to realize joint detection of multiple targets such as people, vehicles, objects and areas. In this process, through the detection frame association technology, the vehicle and license plate, the human body and head are accurately bound, and the cascade reasoning mechanism is used to intelligently trigger OCR or face recognition service according to the detection result, effectively reducing redundant calculation; 2) Introduce dynamic request management mechanism, based on JSON configuration to fine control request frequency, use Request_Management class to flexibly allocate service priority and resource quota, and update service remaining times in real time, which can prevent malicious calls and dynamically load permission configuration according to appid; At the same time; 3) Design lightweight data flow, use Base64 image encoding and decoding and temporary file automatic cleaning strategy, combined with image enhancement and multi-scale cropping technology, to reduce disk occupancy while significantly improving small target recognition rate.

[0010] (7) The system deeply integrates multi-target detection in complex scenarios of smart park with business needs, organically links vehicle detection, license plate recognition, personnel identification and other processes through multi-model collaborative reasoning, avoids repeated detection and invalid calculation, and reasonably allocates computing resources through dynamic request management to ensure stable operation of services in high-concurrency scenarios. Through practical verification, the detection efficiency of the system is improved by 30% compared with traditional solutions, it can stably support more than 1000 concurrent requests, the license plate recognition accuracy is more than 98%, and an efficient, intelligent and low-redundancy smart park visual analysis solution is successfully created. SUMMARY

[0011] The purpose of the present application is to provide a smart park visual analysis system and method based on multi-model collaborative reasoning and dynamic request management, which solves the problems mentioned in the background art.

[0012] To achieve the above purpose, the present application provides the following technical solutions:

[0013] A smart park visual analysis system based on multi-model collaborative reasoning and dynamic request management is composed of a multi-model collaborative reasoning architecture, a dynamic request management and a lightweight data flow design module. Through the deep cooperation of the three modules, the whole process optimization from target detection, face recognition to service deployment is realized, the accurate identification and real-time monitoring of people, vehicles and areas in the park are realized, and multi-target intelligent perception is realized.

[0014] A method for using a smart park visual analysis system based on multi-model collaborative reasoning and dynamic request management, comprising the following contents:

[0015] (1) Multi-model collaborative reasoning module

[0016] The model collaborative reasoning module integrates the target detection (YOLOv7), face recognition (AdaFaceNet), and license plate recognition (PPOCR) models, realizes the joint analysis of people, vehicles, and objects through cascading reasoning and dynamic detection box association and feature space mapping association rules, is responsible for detecting and positioning the park monitoring targets of the scene monitoring image img, performs image cropping of part of the park targets, respectively crops the human region image, performs vehicle body segmentation and number recognition using the instance segmentation model, and performs license plate information recognition using the data enhancement and optical character recognition model on the license plate region image; the model collaborative reasoning architecture module takes multi-modal models as the core, realizes multi-target joint detection and business logic fusion of people, vehicles, objects, and regions through cascading reasoning and detection box association mechanism; the architecture is divided into a basic detection layer, a business enhancement layer, and a logic decision layer, forming an end-to-end analysis link;

[0017] 1) Multi-modal target association detection

[0018] The park multi-model cascading reasoning process realizes cross-model joint detection of people, vehicles, license plates, and faces, and the process realizes park target detection based on the YOLOv7 target detection algorithm, and classifies vehicles, human bodies, license plates, and other targets. Compared with the same type of target detection algorithm, the YOLOv7 algorithm realizes good performance in detection and classification, and is currently the only detector that can exceed 30FPS under high precision (optimal precision 56.8% AP), and the algorithm can adaptively constrain the size interval of the detected target to achieve the effect of accelerating detection and reducing false detection. It is also more lightweight and efficient among the same type of detection algorithms; first, target screening and feature extraction are performed, the monitoring image is input to the detection network, and a detection box set containing vehicle, human body, license plate, and other targets is output, a detection box association degree measurement function is defined, cross-model target binding is realized, and a basic detection box set candidate box set The learnable weight parameters a = 0.6, b = 0.4, and D(f b (b i ),f c (c j )) are set as the feature similarity distance function, IoU(b i ,c i ) is the spatial overlap, and a detection box association matrix S ∈ R m×n is constructed, so as to obtain the association weight calculation formula (3-1) as shown:

[0019] S(b i ,c i ) = a · IoU(b i ,ci )+β·D(f yolo (b i ),f adaface (c j )),

[0020] Then use the Hungarian algorithm Find the optimal matching, where M ij ∈{0,1},∑ i M ij ≤1,∑ j M ij ≤1, to achieve the binding of human detection boxes with the face boxes output by AdaFaceNet and the binding of vehicle detection boxes with the license plate region recognized by PPOCR. Finally, a cascaded inference triggering mechanism is designed, defining the triggering condition function of the model, and making inference decisions for the downstream model based on the condition function. The triggering condition function is as follows:

[0021] Wherein, P(b) i ) is the confidence level for detection, T k It is the threshold of the basic settings. This is the current system load rate, initial settings. γ =0.3 is a dynamic adjustment factor; a successfully matched vehicle target is achieved if and only if it satisfies When the OCR service is triggered, after a successful match with the associated human target, the feature vector f∈R is extracted by the network. 512 Perform identity verification;

[0022] In the multi-target detection task in the park, the system uses park road monitoring images (img) and detection confidence thresholds (thresh) as parameters. detec Target overlap suppression threshold thresh nms As input, this system detects and locates various types of monitored targets, including small motor vehicles, trucks, buses, motorcycles, license plates, and human bodies. A target detection model extracts features from the input image and generates target bounding boxes, outputting a category prediction (cls). i and coordinate information pos i =(x i ,y i ,w i ,h i Preliminary test results of (x) i ,y i ) represents the coordinates of the center point of the target bounding box, w i and h i These represent the width and height of the bounding box, respectively; subsequently, based on the detection confidence threshold (thresh)... detecThe preliminary detection results are screened, only the effective targets with confidence score higher than the threshold are retained, and the low-confidence abnormal detection items are filtered out; finally, a non-maximum suppression (NMS) algorithm is used to process the overlapping boundary boxes of the same target; by calculating the intersection over union (IOU) of any two target boundary boxes, if IOU>thresh nms , the boundary box with higher confidence is retained, and the redundant overlapping detection results are removed, and finally a target basic data list obj_detec = [{pos1, cls1}, {pos2, cls2},..., {pos n , cls n}] of non-redundant target positioning and classification information is generated;

[0023] 2) Face recognition

[0024] In view of the complex light scene in the park, the standard AdaFaceNet is improved to complete face recognition, and the region of the detected human target positioning frame image is cropped. The cropped human image is input into AdaFaceNet to extract the feature vector. The human target D person , the cropped region image I face , the dynamic histogram equalization is used to eliminate the light deviation, and the reflection component based on the Retinex theory is used to preprocess the input face image, extract the feature vector f ∈ R d , and perform adaptive normalization after extracting the feature vector, and calculate the quality-aware normalization parameter Set s = 64.0, m = 10.0 as the base of scaling and translation, ω = 0.5 as the quality weight factor, and ε = 1e-6 to prevent division by zero. The parameter is calculated as shown in equation (3-2):

[0025]

[0026] Due to the very complex light environment in the park, face recognition involves stranger recognition in public and non-public areas of the park, indoor office areas, factory access control and other environments with large changes in light. Therefore, a light-invariant feature space is introduced in the loss function to enhance the light robustness, as shown in equation (3-3):

[0027]

[0028] where s is the scaling factor, K = 128 is an empirical constant;

[0029] 3) Vehicle license plate recognition

[0030] The license plate information recognition is mainly responsible for extracting the license plate character information of different types of vehicle license plates. The license plate target in the target detection result obj_detec, that is, clsi For the detection result item of the license plate category, the following processing is performed:

[0031] a. License plate area expansion and cropping: Set the vehicle expansion ratio parameter list pad: [p1, p2,..., p c , and according to the license plate detection and positioning information pos i : {x0, y0, x1, y1} and the vehicle type cls i , perform area expansion on pos i with the ratio pad[cls i . The specific calculation is shown in Equation (3-4):

[0032]

[0033] where ppos i is the license plate character recognition area after expansion, and its content is the area coordinate information composed of {px0, py0, px1, py1}. pad x [cls i , pad y [cls i respectively represent the horizontal and vertical area expansion ratios in the vehicle expansion ratio parameter list pad corresponding to the vehicle type cls i ;

[0034] After the expansion area calculation and processing, according to the expanded license plate character recognition area ppos i , crop the license plate area image plate_area from the road monitoring image img;

[0035] b. License plate character recognition: In the license plate character recognition link, the system uses the PPOCR open-source OCR engine to achieve high-precision extraction of the license plate text content; PPOCR adopts a cascaded architecture of the DB text detection network and the CRNN sequence recognition model, with an internal angle classifier that can automatically correct tilted license plates (supporting ±180° rotation), and realizes multi-scale text positioning through the Feature Pyramid Network (FPN); In view of the particularity of Chinese license plates, the model incorporates a traffic sign data set in the pre-training stage, significantly improving the recognition accuracy of combined characters such as "Gui B" and "Yu A"; In the recognition process, after the license plate area expansion and cropping (obtaining the ROI) are completed, the system automatically triggers the adaptive image enhancement module; This module constructs a multi-scale image decomposition model based on histogram equalization and Retinex theory: where SSR is the single-scale Retinex operator, and σ k is the parameter of the decomposition scale, and w kis the weight coefficient. This process can effectively suppress the problem of contrast reduction caused by uneven illumination (such as backlight and shadow), and improve the clarity of character edges; the enhanced image is input into the PPOCR model, and the CTC decoder generates a continuous string result txt; this result is appended to the attribute field of the corresponding license plate target in the target detection result list obj_detec through the metadata association mechanism (such as plate_info: {"text": "Gui A12345", "conf": 0.98}), realizing the unified encapsulation of detection and recognition results; the method of this patent has improved the recognition accuracy by 15.3% in low-light scenarios compared with the original PPOCR model, and the average processing delay is controlled within 35ms, meeting the real-time requirements;

[0036] (2) Dynamic request management module

[0037] The dynamic request management module realizes the balance of service high availability and resource utilization efficiency through a dual guarantee mechanism of hierarchical permission control and elastic resource allocation strategy, preventing malicious calls and resource overload. The dynamic request management module includes 3 sub-modules: service permission dynamic control module, elastic resource scheduling module, and exception fusing and degradation module;

[0038] The service permission dynamic control module realizes multi-dimensional access control through a JSON configuration file. First, configure the application identity authentication unit to load the predefined F-AI.json permission file, establish a mapping relationship table between appid and service type caseid, and achieve real-time verification with O(1) complexity through hash indexing. When an illegal appid or unauthorized caseid is detected, the request is immediately blocked and error codes 104 / 105 are returned. Then, the quota dynamic management unit updates the remaining call times of residual using atomic operations, and ensures data consistency in a multi-threaded environment through a file lock mechanism. The configuration update frequency and service call times satisfy the relationship: where, K threshold is the preset call times threshold;

[0039] The elastic resource scheduling module mainly implements a hierarchical resource allocation strategy, based on the time sensitivity factor T remain 、business level factor R level and historical load factor Q history to build a comprehensive evaluation model and design a priority calculation engine; as shown in model (3-).

[0040]

[0041] The Score value calculated according to the model divides the request into: real-time processing channel (Score≥0.8): exclusive GPU computing resources, enable CUDA Stream acceleration; ordinary task queue (0.5≤Score<0.8): share GPU resource pool, use time slice round robin scheduling; background processing thread (Score<0.5): forward to CPU asynchronous execution queue, enable OpenVINO optimization inference;

[0042] The abnormal fusing and degradation module includes a multi-level fault defense mechanism, and the service continuously monitors the response time t response and API error rate ε, when the following conditions (3-6) are met, the fusing mechanism is triggered (set the sliding window size N=100);

[0043] and

[0044] Through the synergistic effect of the above modules, the service permission verification delay is reduced to 0.3ms level, the single-day interception of illegal requests is more than 100,000 times, the high-priority task resource preemption success rate is improved to 98.7%, the standard deviation of the time delay is optimized from plus or minus 85ms to plus or minus 21ms, and the use of the fusing mechanism makes the survival rate of the system under DDoS attack from 67% to 93%;

[0045] (3) Lightweight data flow design module

[0046] The lightweight data flow design module mainly optimizes the efficiency of the whole process of data storage, transmission and processing. Through efficient coding and decoding technology, intelligent cache management and selective enhancement strategy, the resource limitation problem in edge computing scenarios is solved. Data flow lightweight avoids the traditional disk I / O bottleneck through memory mapping and parallel coding technology, reduces the end-to-end processing delay. Resource precise allocation enhances the calculation in key areas and preserves the original resolution in non-critical areas, reducing redundant calculation. Life cycle management based on hash naming and automatic cleaning mechanism realizes self-management of temporary files and avoids waste of storage resources.

[0047] The image encoding and temporary file management use the SIMD instruction set accelerated Base64 encoding algorithm to convert the image pixel stream I∈R H×W×3 into a string S. When decoding, the encoded result is accessed directly through memory mapping technology to avoid disk read-write operations, achieving a reduction of 45% in I / O delay. The temporary file intelligent management uses the filename=SHA256(timestamp||UUID) generation rule to ensure the global uniqueness of the file, with a hash conflict rate of less than 10 -18 , and according to the LRU-K algorithm, the expired files are eliminated, and the automatic cleaning mechanism is set to reduce the storage space occupation by 60%.

[0048] Image enhancement optimization adopts regional attention enhancement and hierarchical enhancement strategy, and key regions are located by generating a Gaussian weight map, and enhancement is realized for the region of W(x,y)>0.6, wherein

[0049]

[0050] Then, the license plate region and the face region are optimized and enhanced by the hierarchical enhancement strategy, a 4-fold super-resolution reconstruction is carried out by using the SRGAN-Mini model, and the PSNR value reaches 32.6dB; the Laplace operator is used to enhance the edge, and the license plate recognition accuracy is improved by 14.3%, as shown in formula (3-8):

[0051] Compared with the prior art, the beneficial effects of the present application are as follows:

[0052] The present application decomposes the atomic ability recognition process, and decomposes the complex visual analysis task into multiple basic and independent atomic ability modules. YOLOv7 target detection model, PPOCR license plate recognition model and AdaFaceNet face recognition model are used as the basis to construct atomic ability, which is responsible for detecting people, vehicles and objects in the image, recognizing license plates and analyzing personnel face features. These atomic ability modules have high independence and reusability, can be flexibly combined, and lay a foundation for subsequent accurate analysis; a unified data processing interface is designed, so that the atomic ability model can convert the detected business data into abstract information in a fixed format. These abstract information strips off the characteristics of specific scenes, and only retains the core data features. Then, corresponding business logic modules are designed for different application scenarios, such as park entrance vehicle verification, personnel wandering and stranger monitoring, etc., to process these abstract information, so as to realize the flexible adaptation of atomic ability model and complex business scenarios

[0053] BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1 It is the intelligent garden visual analysis system architecture based on multi-model collaborative reasoning and dynamic request management of the present application;

[0055] Figure 2 It is the visualization example diagram of the intelligent garden platform of the present application;

[0056] Figure 3 It is the platform real-time video click play effect diagram of the present application;

[0057] Figure 4 It is the face recognition stranger example diagram of the intelligent garden platform of the present application;​

[0058] Figure 5 Stranger alert record for the present invention;

[0059] Figure 6 Vehicle and license plate recognition visualization example for the platform background of the present invention;

[0060] Figure 7 Intelligent park platform equipment situation visualization example for the present invention. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

[0062] Please refer to Figures 1-7 A smart park visual analysis system based on multi-model collaborative reasoning and dynamic request management, composed of a multi-model collaborative reasoning architecture, a dynamic request management and a lightweight data flow design module, realizes the whole process optimization from target detection, face recognition to service deployment through the deep cooperation of the three modules, realizes the accurate identification and real-time monitoring of people, vehicles and areas in the park, and realizes multi-target intelligent perception.

[0063] A use method of a smart park visual analysis system based on multi-model collaborative reasoning and dynamic request management, including the following contents:

[0064] (1) Multi-model collaborative reasoning module

[0065] The model collaborative reasoning module integrates target detection (YOLOv7), face recognition (AdaFaceNet) and license plate recognition (PPOCR) models, realizes joint analysis of people, vehicles and objects through cascading reasoning and dynamic detection frame association and feature space mapping association rules, is responsible for detecting and positioning the park monitoring targets of the scene monitoring image img, performs image cropping of part of the park targets, respectively crops the human region image, uses an instance segmentation model to perform vehicle body segmentation and number recognition, and uses a data enhancement and optical character recognition model to perform license plate information recognition on the license plate region image; the model collaborative reasoning architecture module takes multi-modal models as the core, realizes multi-target joint detection and business logic fusion of people, vehicles, objects and regions through cascading reasoning and detection frame association mechanism; the architecture is divided into a basic detection layer, a business enhancement layer and a logic decision layer, forming an end-to-end analysis link;

[0066] 1) Multi-modal target association detection

[0067] Park multi-model cascade inference process: realize the cross-model joint detection of people-vehicle-license plate-face, the process is based on YOLOv7 target detection algorithm to realize the detection of park targets, and classify vehicles, human bodies, license plates and other targets. Compared with the same type of target detection algorithm, YOLOv7 algorithm realizes good performance in detection and classification, and is currently the only detector that can exceed 30FPS under high precision (optimal precision 56.8%AP). The algorithm can adaptively constrain the size interval of the detected target to achieve the effect of accelerating detection and reducing false detection. It is also more lightweight and efficient among the same type of detection algorithms; first, target screening and feature extraction, input the monitoring image to the detection network, output the detection frame set containing vehicles, human bodies, license plates and other targets, define the detection frame correlation measurement function, realize the cross-model target binding, set the basic detection frame set candidate frame set Set the learnable weight parameters α=0.6, β=0.4, D(f b (b i ),f c (c j )) as the feature similarity distance function, IoU(b i ,c i ) as the spatial overlap, and construct the detection frame correlation matrix S∈R m×n , so as to obtain the correlation weight calculation formula (3-1) as shown:

[0068] S(b i ,c i )=α·IoU(b i ,c i )+β·D(f yolo (b i ),f adaface (c j )),

[0069] Then solve the optimal matching by the Hungarian algorithm , where M ij ∈{0,1},∑ i M ij ≤1,∑ j M ij ≤1, realize the binding of human body detection frame and AdaFaceNet output face frame and the binding of vehicle detection frame and PPOCR recognized license plate area, finally, design a cascade inference triggering mechanism, define the trigger condition function of the model, make the downstream model infer and decide according to the condition function, the trigger condition function is as follows:

[0070] Where P(b i ) is the detection confidence, T kis the threshold of the basic setting, is the load rate of the current system, and the preliminary setting γ = 0.3 is a dynamic adjustment factor; the target vehicle is matched only when the following conditions are met OCR service is triggered, and after the human body target is matched, the feature value vector f ∈ R 512 is extracted through the network, and identity verification is performed;

[0071] In the park multi-target detection task, the system takes the park road monitoring image img, the detection confidence threshold thresh detec and the target overlap suppression threshold thresh nms as inputs, and detects and locates multiple monitoring targets such as small motor vehicles, trucks, buses, motorcycles, license plates, and human bodies; the target detection model extracts features from the input image and generates target candidate boxes, and outputs the preliminary detection results containing class prediction cls i and coordinate information pos i =(x i ,y i ,w i ,h i ); wherein (x i ,y i ) represents the center point coordinates of the target bounding box, w i and h i are the width and height of the bounding box, respectively; then, the preliminary detection results are screened based on the detection confidence threshold thresh detec , only the effective targets with confidence scores higher than the threshold are retained, and the low-confidence abnormal detection items are filtered out; finally, the non-maximum suppression (NMS) algorithm is used to process the overlapping bounding boxes of the same type of target; by calculating the intersection over union (IOU) of any two target bounding boxes, if IOU>thresh nms , the bounding box with higher confidence is retained, and the redundant overlapping detection results are removed, and finally the target basic data list obj_detec = [{pos1, cls1}, {pos2, cls2},..., {pos n , cls n}] of target positioning and classification information without redundancy is generated;

[0072] 2) Face recognition

[0073] In view of the complex light scene in the park, the standard AdaFaceNet is improved to complete face recognition, and the region of the detected human target positioning frame image is cropped, the cropped human body image is input into AdaFaceNet to extract the feature vector, the human target D person , and the cropped region image I faceDynamic histogram equalization is used to eliminate illumination bias, and reflection component extraction based on Retinex theory is used to preprocess the input face image to extract feature vectors f∈R. d After extracting the feature vectors, adaptive normalization is performed to calculate the normalization parameters for quality perception. Set s = 64.0, m = 10.0 as the base for scaling and translation, ω = 0.5 as the quality weighting factor, and ε = 1e-6 to prevent division by zero. Calculate the parameters. As shown in equation (3-2):

[0074]

[0075] Due to the highly complex lighting environment in the park, and the significant variations in lighting conditions in public and non-public areas, indoor office areas, and factory access control, a lighting-invariant feature space is introduced into the loss function to enhance its robustness, as shown in equation (3-3).

[0076]

[0077] Where s is the scaling factor. K = 128 is an empirical constant;

[0078] 3) Vehicle license plate recognition

[0079] License plate information recognition is primarily responsible for extracting the character information of license plates from different types of vehicles. This involves identifying the license plate target (cls) in the target detection result obj_detec. i For the detection result item of license plate category, the following processing is performed:

[0080] c. License Plate Area Expansion and Cropping: Set the vehicle expansion ratio parameter list pad: [p1,p2,...,p c According to the license plate detection and positioning information pos i {x0,y0,x1,y1} and vehicle type cls i , for pos i Perform proportional pad[cls i The area expansion is shown in equation (3-4).

[0081]

[0082] Among them, ppos i The expanded license plate character recognition area contains the coordinate information of the region consisting of {px0, py0, px1, py1}, pad. x [cls i ], pad y [clsi respectively represent that the vehicle type is cls i corresponding horizontal and vertical area expansion ratios in the vehicle expansion ratio parameter list pad of the vehicle;

[0083] After the expansion area calculation and processing, according to the expanded license plate character recognition area ppos i Crop the license plate area image plate_area from the road monitoring image img;

[0084] d. License plate character recognition: In the license plate character recognition link, the system uses the open-source OCR engine of PPOCR to achieve high-precision extraction of the license plate text content; PPOCR adopts a cascaded architecture of the DB text detection network and the CRNN sequence recognition model, with an internal angle classifier that can automatically correct tilted license plates (supporting ±180° rotation), and realizes multi-scale text localization through the Feature Pyramid Network (FPN); In view of the particularity of Chinese license plates, the model incorporates a traffic sign data set in the pre-training stage, significantly improving the recognition accuracy of combined characters such as "Gui B" and "Yu A"; In the recognition process, after the license plate area expansion and cropping (obtaining the ROI) are completed, the system automatically triggers the adaptive image enhancement module; This module constructs a multi-scale image decomposition model based on histogram equalization and Retinex theory: Among them, SSR is the single-scale Retinex operator, σ k is the parameter of the decomposition scale, w k is the weight coefficient. This process can effectively suppress the problem of contrast reduction caused by uneven illumination (such as backlight and shadow), and improve the clarity of character edges; The enhanced image is input into the PPOCR model, and the continuous string result txt is generated through the CTC decoder; This result is appended to the attribute field of the corresponding license plate target in the target detection result list obj_detec through the metadata association mechanism (such as plate_info: {"text": "Gui A12345", "conf": 0.98}), realizing the unified encapsulation of detection and recognition results; The method of this patent has improved the recognition accuracy by 15.3% in low-light scenarios compared with the original PPOCR model, and the average processing delay is controlled within 35ms, meeting the real-time requirements;

[0085] (2) Dynamic request management module

[0086] The dynamic request management module realizes the balance between service high availability and resource utilization efficiency through a dual guarantee mechanism of hierarchical permission control and elastic resource allocation strategy, preventing malicious calls and resource overload. The dynamic request management module includes 3 sub-modules: service permission dynamic control module, elastic resource scheduling module, and exception fusing and degradation module;

[0087] The service permission dynamic management module realizes multi-dimensional access control through a JSON configuration file. First, the application identity authentication unit is configured to load the pre-defined F-AI.json permission file, a mapping relationship table of appid and service type caseid is established, and real-time verification is realized with O(1) complexity through hash index. When an illegal appid or unauthorized caseid is detected, the request is immediately blocked and a 104 / 105 error code is returned. Then, the quota dynamic management unit updates the residual number of calls using atomic operation, ensures data consistency in a multi-threaded environment through file locking mechanism, and configures the update frequency and service call number to satisfy the relationship: wherein, K threshold is a preset call number threshold;

[0088] The elastic resource scheduling module mainly implements a hierarchical resource allocation strategy. Based on the time sensitivity factor T remain , the business level factor R level and the historical load factor Q history , a comprehensive evaluation model is constructed, and a priority calculation engine is designed. The model is shown in (3-5):

[0089]

[0090] According to the Score value calculated by the model, the request is divided into: real-time processing channel (Score≥0.8): exclusive GPU computing resources, CUDA Stream acceleration is enabled; ordinary task queue (0.5≤Score<0.8): shared GPU resource pool, time slice round robin scheduling is adopted; background processing thread (Score<0.5): forwarded to the CPU asynchronous execution queue, OpenVINO optimization inference is enabled;

[0091] The abnormal fuse and degradation module contains a multi-level fault prevention mechanism. The response time t response and API error rate ε are continuously monitored in the service. When the following conditions (3-6) are met, the fuse mechanism is triggered (the sliding window size N=100 is set):

[0092] and

[0093] Through the synergistic effect of the above modules, the service permission verification delay is reduced to 0.3ms level, more than 100,000 illegal requests are intercepted per day, the resource preemption success rate of high priority tasks is improved to 98.7%, the standard deviation of time delay is optimized from plus or minus 85ms to plus or minus 21ms, and the use of fuse mechanism makes the survival rate of the system under DDoS attack increase from 67% to 93%;

[0094] (3) Lightweight data flow design module

[0095] The lightweight dataflow design module primarily optimizes the efficiency of the entire data storage, transmission, and processing process. It addresses resource constraints in edge computing scenarios through efficient encoding / decoding technologies, intelligent cache management, and selective enhancement strategies. Lightweight dataflow avoids traditional disk I / O bottlenecks and reduces end-to-end processing latency through memory mapping and parallel encoding techniques. Precise resource allocation enhances computation in critical areas while preserving the original resolution in non-critical areas, reducing redundant computation. Lifecycle management, based on hash naming and automatic cleanup mechanisms, enables autonomous management of temporary files, preventing the waste of storage resources.

[0096] Image encoding and temporary file management employ a Base64 encoding algorithm accelerated by the SIMD instruction set, converting the image pixel stream I∈R... H×W×3 Converted to string S, the encoded result is directly accessed during decoding using memory mapping technology, avoiding disk read / write operations and reducing I / O latency by 45%. Temporary files are intelligently managed; the main filename is generated using the rule filename = SHA256(timestamp||UUID) to ensure global file uniqueness, with a hash collision rate of less than 10. -18 Furthermore, it eliminates expired files based on the LRU-K algorithm and sets up an automatic cleanup mechanism, reducing storage space usage by 60%.

[0097] Image enhancement optimization employs region attention enhancement and hierarchical enhancement strategies. Key regions are located by generating Gaussian weight maps, and enhancement is achieved for regions where W(x,y)>0.6.

[0098]

[0099] Then, a layered enhancement strategy was used to optimize and enhance the license plate and face regions. The SRGAN-Mini model was used for 4x super-resolution reconstruction, achieving a PSNR of 32.6 dB. The Laplacian operator was also employed. Enhancing the edges improves license plate recognition accuracy by 14.3%, as shown in equation (3-8):

[0100]

[0101] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1.A smart park visual analysis system based on multi-model collaborative reasoning and dynamic request management, characterized in that, It is composed of multi-model collaborative reasoning architecture, dynamic request management and lightweight data flow design module. Through the deep collaboration of the three modules, the whole process optimization from target detection, face recognition to service deployment is realized, the accurate identification and real-time monitoring of people, vehicles and areas in the park are realized, and multi-target intelligent perception is realized. 2.The use method of the smart park visual analysis system based on multi-model collaborative reasoning and dynamic request management according to claim 1, characterized in that, It includes the following contents: (1) Multi-model collaborative reasoning module The model collaborative reasoning module integrates target detection (YOLOv7), face recognition (AdaFaceNet) and license plate recognition (PPOCR) models, and realizes joint analysis of people, vehicles and objects through cascading reasoning and dynamic detection box association and feature space mapping association rules. It is responsible for detecting and positioning the park monitoring targets of the scene monitoring image img, cropping part of the park target image, and respectively cropping the human region image, using the instance segmentation model for vehicle body segmentation and number recognition, and using the data enhancement and optical character recognition model for license plate information recognition; The model collaborative reasoning architecture module takes multi-modal models as the core, realizes multi-target joint detection and business logic fusion of people, vehicles, objects and regions through cascading reasoning and detection box association mechanism; The architecture is divided into a basic detection layer, a business enhancement layer and a logic decision layer, forming an end-to-end analysis link; 1) Multi-modal target association detection Park multi-model cascade inference process: realize the cross-model joint detection of people-vehicle-license plate-face, the process is based on YOLOv7 target detection algorithm to realize the detection of park targets, and classify vehicles, human bodies, license plates and other targets, compared with the same type of target detection algorithm, YOLOv7 algorithm realizes good performance in detection and classification, and is currently the only detector that can exceed 30FPS under high precision (optimal precision 56.8%AP), the algorithm can adaptively constrain the size interval of the detected target to achieve the effect of accelerating detection and reducing false detection, and is more lightweight and efficient among the same type of detection algorithm; first, target screening and feature extraction, input the monitoring image to the detection network, output the detection frame set containing vehicle, human body, license plate and other targets, define the detection frame correlation measure function, realize the cross-model target binding, set the basic detection frame set candidate frame set Set the learnable weight parameters a = 0.6, b = 0.4, D(f b (b i ),f c (c j )) as the feature similarity distance function, IoU(b i ,c i ) as the spatial overlap, construct the detection frame correlation matrix S ∈ R m×n , so as to obtain the correlation weight calculation formula (3-1) as shown: again by the Hungarian algorithm solving the optimal matching, where M ij ∈{0,1},∑ i M ij ≤1,∑ j M ij ≤1, the human detection frame is bound with the face frame output by AdaFaceNet, and the vehicle detection frame is bound with the license plate region identified by PPOCR, finally, a cascade inference triggering mechanism is designed, a trigger condition function of the model is defined, and the downstream model makes inference and decision according to the condition function, the trigger condition function is as follows: Where, P(b i ) is the detection confidence, T k is the threshold set by the basic setting, is the current system load rate, and the preliminary setting γ=0.3 is the dynamic adjustment factor; the OCR service is triggered when the matched vehicle target satisfies , and for the associated human target, the feature value vector f∈R 512 extracted by the network is used for identity verification; In the multi-target detection task in the park, the system uses park road monitoring images (img) and detection confidence thresholds (thresh) as parameters. detec Target overlap suppression threshold thresh nms As input, this system detects and locates various types of monitored targets, including small motor vehicles, trucks, buses, motorcycles, license plates, and human bodies. A target detection model extracts features from the input image and generates target bounding boxes, outputting a category prediction (cls). i and coordinate information pos i =(x i ,y i ,w i ,h i Preliminary test results of (x) i ,y i ) represents the coordinates of the center point of the target bounding box, w i and h i These represent the width and height of the bounding box, respectively; subsequently, based on the detection confidence threshold (thresh)... detec The initial detection results are screened, retaining only valid targets with confidence scores above a certain threshold and filtering out outlier detections with low confidence scores. Finally, the non-maximum suppression (NMS) algorithm is used to process overlapping bounding boxes of similar targets. By calculating the intersection-union ratio (IOU) of any two target bounding boxes, if IOU > threshold... nms Then, the bounding boxes with higher confidence are retained, and redundant and overlapping detection results are removed, finally generating a list of target localization and classification information without redundancy: obj_detec=[{pos1,cls1},{pos2,cls2},...,{pos n ,cls n The target basic data; 2) Face recognition For the complex light scene in the park, the standard AdaFaceNet is improved to complete face recognition, the image of the positioning frame of the detected human target is regionally cropped, the cropped human body image is input into the AdaFaceNet to extract a feature vector, the human target D person , the cropped region image I face , the dynamic histogram equalization is used to eliminate the light deviation, and the reflection component based on the Retinex theory is used to pre-process the input face image, the feature vector f ∈ R d , after the feature vector is extracted, adaptive normalization is performed, and the quality-aware normalization parameter is calculated The scaling and translation base numbers s = 64.0 and m = 10.0 are set, the quality weight factor ω = 0.5 is set, and ε = 1e-6 is set to prevent division by zero, and the parameters are calculated as shown in formula (3-2): Due to the very complex lighting environment in the park, face recognition involves public and non-public areas in the park to achieve stranger recognition, indoor office areas, factory gate access and other environments with large changes in lighting. Therefore, a lighting invariant feature space is introduced in the loss function to enhance the lighting robustness, as shown in equation (3-3): where s is a scaling factor, K = 128 is an empirical constant; 3) Vehicle license plate recognition The license plate information recognition is mainly responsible for extracting license plate character information of different types of vehicle license plates. For the license plate target in the target detection result obj_detec, that is, cls i The license plate category detection result item is processed as follows: a. License plate region expansion and cropping: Set the vehicle expansion ratio parameter list pad: [p1, p2,..., p c ] according to the license plate detection positioning information pos i : {x0, y0, x1, y1} and the vehicle type cls i , and expand the region of pos i by pad[cls i ] for the specific calculation as shown in equation (3-4): wherein, ppos i is the expanded license plate character recognition region, and its content is the region coordinate information composed of {px0, py0, px1, py1}, pad x [cls i ], and pad y [cls i ] respectively represent the lateral and longitudinal region expansion ratios in the vehicle class cls i corresponding vehicle expansion ratio parameter list pad. After the extended region calculation processing, the extended license plate character recognition region ppos is obtained according to the extended license plate character recognition region ppos i The license plate region image plate_area is obtained by cropping in the road monitoring image img. b. License Plate Character Recognition: In the license plate character recognition process, the system uses the open-source PPOCR OCR engine to achieve high-precision extraction of license plate text content; PPOCR adopts a cascaded architecture of the DB text detection network and the CRNN sequence recognition model, with an built-in angle classifier that can automatically correct tilted license plates (supporting ±180° rotation), and realizes multi-scale text localization through the Feature Pyramid Network (FPN); Considering the particularity of Chinese license plates, the model incorporates a traffic sign dataset in the pre-training stage, significantly improving the recognition accuracy of combined characters such as "Gui B" and "Yu A"; In the recognition process, after the license plate area is expanded and cropped (to obtain the ROI), the system automatically triggers the adaptive image enhancement module; This module constructs a multi-scale image decomposition model based on histogram equalization and Retinex theory: where SSR is the single-scale Retinex operator, and σ k is the parameter of the decomposition scale, and w k is the weight coefficient. This process can effectively suppress the contrast reduction problem caused by uneven illumination (such as backlighting and shadows), and improve the clarity of character edges; The enhanced image is input into the PPOCR model, and a continuous string result txt is generated through the CTC decoder; This result is appended to the attribute field of the corresponding license plate target in the object detection result list obj_detec through the metadata association mechanism (such as plate_info:{"text":"Gui A12345","conf":0.98}), achieving unified encapsulation of detection and recognition results; The method of this patent has improved the recognition accuracy by 15.3% in low-light scenarios compared to the original PPOCR model, and the average processing delay is controlled within 35ms, meeting the real-time requirements; (2) Dynamic request management module The dynamic request management module realizes the balance between service high availability and resource utilization efficiency through the dual protection mechanism of hierarchical permission control and elastic resource allocation strategy, and prevents malicious calls and resource overload. The dynamic request management module includes three sub-modules: service permission dynamic management module, elastic resource scheduling module, exception fuse and degradation module; The service authority dynamic management module realizes multi-dimensional access control through a JSON configuration file. First, the application identity authentication unit is configured to load a predefined F-AI.json authority file, a mapping relationship table of appid and service type caseid is established, and real-time verification with O(1) complexity is realized through hash index. When an illegal appid or unauthorized caseid is detected, the request is immediately blocked and error codes 104 / 105 are returned. Then, the quota dynamic management unit updates the residual residual call number through atomic operation, ensures data consistency in a multi-threaded environment through file locking mechanism, and configures the update frequency and service call number to satisfy the relationship: Wherein, K threshold is a preset call number threshold. The elastic resource scheduling module mainly implements a hierarchical resource allocation strategy based on a time sensitivity factor T remain , a service level factor R level , and a historical load factor Q history to construct a comprehensive evaluation model and design a priority calculation engine; the model is shown in (3-5): According to the Score value calculated by the model, the request is divided into: real-time processing channel (Score≥0.8): exclusive GPU computing resources, enable CUDA Stream acceleration; Ordinary task queue (0.5≤Score<0.8): share GPU resource pool, use time slice round robin scheduling; Background processing thread (Score<0.5): forwarded to the CPU asynchronous execution queue, enable OpenVINO optimized reasoning; The abnormal fusing and degradation module contains a multi-level fault defense mechanism, continuously monitoring the response time t response and API error rate ε, when the following conditions (3-6) are met, the fusing mechanism is triggered (set the sliding window size N = 100); Through the collaborative action of the above modules, the service permission verification delay is reduced to 0.3ms level, more than 100,000 illegal requests are intercepted per day, the resource preemption success rate of high priority tasks is improved to 98.7%, the standard deviation of time delay is optimized from plus or minus 85ms to plus or minus 21ms, and the use of fuse mechanism makes the survival rate of the system under DDoS attack from 67% to 93%; (3) Lightweight data flow design module The lightweight data stream design module mainly optimizes the whole process efficiency of data storage, transmission and processing. Through efficient coding and decoding technology, intelligent cache management and selective enhancement strategy, the resource limited problem in edge computing scene is solved. The data stream lightweight avoids the traditional disk I / O bottleneck through memory mapping and parallel coding technology, and reduces the end-to-end processing delay. The resource precise allocation implements enhanced calculation in key areas and reserves original resolution in non-key areas, reducing redundant calculation amount. The life cycle management realizes the self-management of temporary files based on hash naming and automatic cleaning mechanism, avoiding the waste of storage resources. Image encoding and temporary file management use SIMD instruction set accelerated Base64 encoding algorithm, image pixel stream I ∈ R H ×W×3 Convert to string S, decode by memory mapping technology to access the encoding result directly, bypass disk read-write operation, reduce 45% I / O delay; Temporary file intelligent management main file name uses filename = SHA256 (timestamp || UUID) generation rule, ensure the uniqueness of the file globally, hash collision rate less than 10 -18 , and according to LRU-K algorithm to eliminate expired files, set up automatic cleaning mechanism, so that the storage space occupation is reduced by 60%; The image enhancement optimization adopts regional attention enhancement and hierarchical enhancement strategy, and locates the key region by generating a Gaussian weight map. The region where W(x, y) > 0.6 is enhanced, wherein Then, a layered enhancement strategy was used to optimize and enhance the license plate and face regions. The SRGAN-Mini model was used for 4x super-resolution reconstruction, achieving a PSNR of 32.6 dB. The Laplacian operator was also employed. Enhancing the edges improves license plate recognition accuracy by 14.3%, as shown in equation (3-8):

Images