AI visual identification system based on neural network

Abstract

The invention relates to the field of visual processing, and discloses an AI visual identification system based on a neural network, which comprises an image acquisition and preprocessing subsystem, an image identification subsystem, a multi-target tracking subsystem, a target tracking cooperation subsystem, a model optimization subsystem, a cloud server and a user terminal, the multi-target tracking subsystem comprises a self-adaptive sampling module, a sampling area optimization module, a streaming frame tracking processing module and a multi-task collaborative optimization module; the target tracking collaboration subsystem comprises a multi-source data collaboration module, a scene tracking adaptation module, a dynamic-visual fusion tracking module and a system monitoring emergency module; the model optimization subsystem comprises a data optimization expansion module, a model feature optimization module, an algorithm fair management and control module and a system monitoring feedback module. Technical innovation is carried out from multiple levels of data processing, model optimization, multi-target tracking, target tracking and the like, and finally, the balance between real-time performance and accuracy is realized.

Description

Technical Field

[0001] This invention relates to the field of visual recognition technology, and more specifically to an AI visual recognition system based on neural networks. Background Technology

[0002] As artificial intelligence technology moves from theoretical exploration to industrial application, AI vision systems, with their powerful image analysis capabilities, have become a core technological support for fields such as intelligent manufacturing, intelligent security, and autonomous driving. However, existing AI vision recognition systems have many shortcomings in practical applications: (1) In terms of image acquisition environment, poor lighting conditions such as night or strong backlight will cause the pure vision recognition scheme to completely fail, making it impossible to effectively identify targets and severely limiting the applicability of the system. (2) Due to the limited visual imaging area, once the target leaves the visual imaging area, the system loses the ability to track its motion trajectory. Any change in the visual imaging equipment, whether it is environmental interference in a static scene, equipment movement, or the movement of the equipment itself in a dynamic scene, will further aggravate the problem of tracking failure. The lack of equipment scene adaptability directly leads to the system's difficulty in maintaining stable target tracking capability in complex environments. (3) In multi-target tracking scenarios, due to the computational complexity of the model, the computation time delay is long when multiple targets are processed at the same time, which far exceeds the requirements of real-time applications such as industrial inspection.

[0003] To compensate for the deficiencies in scene processing, environmental adaptability, and feature extraction, developers often use complex models to improve accuracy. However, this inevitably leads to a surge in computation and a decrease in real-time performance. While simplified models can speed up processing, they cannot solve underlying problems, resulting in a significant decrease in accuracy.

[0004] Therefore, existing visual recognition technologies cannot simultaneously meet the needs of environmental adaptability, continuous target tracking, and multi-target recognition processing in application scenarios. They also present a contradiction between real-time performance and accuracy. Summary of the Invention

[0005] To address this, the present invention provides an AI visual recognition system based on neural networks, which effectively solves the technical problem that existing visual recognition technologies cannot simultaneously meet the requirements of environmental adaptability, continuous target tracking, and multi-target recognition processing in application scenarios, and also have the contradiction of difficulty in balancing real-time performance and accuracy.

[0006] To solve the above-mentioned technical problems, the present invention specifically provides the following technical solution: an AI visual recognition system based on neural networks, including an image acquisition and preprocessing subsystem, an image recognition subsystem, a multi-target tracking subsystem, a target tracking collaboration subsystem, a model optimization subsystem, and also including a cloud server and a user terminal; The image recognition subsystem includes a feature extraction module, a classification module, and a pipelined parallel processing module; The multi-target tracking subsystem includes an adaptive sampling module, a sampling region optimization module, a streaming frame tracking processing module, and a multi-task collaborative optimization module. The target tracking collaboration subsystem includes a multi-source data collaboration module, a scene tracking adaptation module, a dynamic vision fusion tracking module, and a system monitoring and emergency response module. The model optimization subsystem includes a data optimization and expansion module, a model feature optimization module, an algorithm fairness management module, and a system monitoring and feedback module.

[0007] Furthermore, the image acquisition and preprocessing subsystem includes an image acquisition device and a preprocessing module; The preprocessing module includes a preprocessing acceleration unit, a spectral image fusion unit, an image enhancement unit, a geometric correction unit, and a size normalization unit. The image acquisition device acquires visible light images and NIR images. The preprocessing acceleration unit performs region-of-interest cropping and encoding / decoding on the images. The spectral image fusion unit uses a pixel-level fusion algorithm to fuse the visible light images and NIR images. The image enhancement unit enhances the images. The geometric correction unit corrects the images. The size normalization unit adjusts images of different sizes to a standard size.

[0008] Furthermore, the feature extraction module includes a key point extraction unit, a semantic feature extraction unit, and a texture feature extraction unit; The key point extraction unit locates key feature points in the image and captures local features; the semantic feature extraction unit mines features with semantic information in the image based on the model optimized by the model optimization subsystem; and the texture feature extraction unit extracts the texture information of the image. The classification module includes an adaptive computation scheduling unit and a real-time classification unit; The adaptive computation scheduling unit calls the model optimized by the model optimization subsystem based on the scene complexity of the input image and dynamically switches the inference model or the number of network layers. The real-time classification unit inputs the extracted features into the scheduled model to determine the category to which the target belongs and sends the classification result to the cloud server and user terminal.

[0009] Furthermore, the pipelined parallel processing module breaks down the visual recognition process into multiple stages, constructs a pipelined parallel mechanism, and connects each stage through a buffer, eliminating waiting time.

[0010] Furthermore, the adaptive sampling module includes a region division unit and a dynamic sampling control unit; The region division unit divides each frame of image into several regions to be identified, and the dynamic sampling control unit adjusts the sampling interval according to the number of targets, movement speed and scene complexity. The sampling area optimization module includes an area random sorting unit and an adjacent area avoidance unit. The region random sorting unit labels all the regions to be identified and randomly numbers them before each round of sampling so that undersampling is performed in the order of the numbers. The adjacent region avoidance unit ensures that the regions to be identified in adjacent positions are not in consecutive undersampling sequence numbers. The streaming frame tracking processing module includes an inter-frame tracking execution unit and a streaming data scheduling unit; The inter-frame tracking execution unit performs trajectory association and prediction for multiple targets in the image, and the streaming data scheduling unit splits the video stream into a continuous frame sequence and performs real-time processing in a pipeline mode. The multi-task collaborative optimization module includes a feature sharing processing unit and a task linkage optimization unit; The feature sharing processing unit shares the extracted basic features in the target recognition and tracking task, and the task linkage optimization unit associates the recognition and tracking results to help each other correct deviations.

[0011] Furthermore, the multi-source data collaboration module includes a multi-device acquisition unit, a cloud synchronization unit, a cross-device matching unit, a location correction unit, and a shared access control unit; The multi-device acquisition unit is responsible for accessing the image recognition subsystem and the image acquisition and preprocessing subsystem, and for acquiring image data and target feature data in real time. The cloud synchronization unit uploads the image data and target feature data acquired by all acquisition devices to the cloud server in real time. The cross-device matching unit retrieves the real-time acquisition data of other image acquisition devices through the cloud server, performs cross-device matching based on the core features of the target, and confirms the association relationship of the same target in different image acquisition devices. The position correction unit uses the same target data acquired by multiple devices from different angles, and calculates the actual spatial position of the target by combining the triangulation principle to correct the positioning error. The shared permission management unit manages the permissions and verifies the data of the cross-device shared recognition results. The scene tracking and adaptation module includes a scene recognition unit, a range change detection unit, a static tracking unit, and a dynamic tracking unit; The scene recognition unit analyzes the attributes of the current monitoring scene in real time and distinguishes between static and dynamic scenes. The range change detection unit identifies the type of change in the image acquisition range by comparing the pixel alignment and inter-frame content differences of adjacent multi-frame images. The static tracking unit is adapted to the tracking requirements of static scenes, and the dynamic tracking unit is adapted to the tracking requirements of dynamic scenes.

[0012] Furthermore, the static tracking unit includes a jitter elimination subunit, a large-range change processing subunit, and an early warning subunit; When the image acquisition range is shaken, the jitter elimination subunit starts the jitter elimination algorithm to repair the image offset. When the image acquisition range changes, the large-scale change processing subunit calls multi-device data to complete the target position information or predicts the short-term movement direction and position of the target through a deep learning model. When the early warning subunit detects an abnormal change in the image acquisition range, it sends a device abnormality alarm signal.

[0013] The dynamic tracking unit includes a speed judgment subunit, a slow processing subunit, and a fast processing subunit; The speed judgment subunit distinguishes between "slow change" and "fast change" of the imaging range by analyzing the pixel change amplitude of multiple frames of images. The slow processing subunit simultaneously tracks "targets within the imaging range" and "targets that have just left the imaging range" when the imaging range changes slowly. The fast processing subunit allocates computing power priority based on the target's spatial position and target distance.

[0014] Furthermore, the motion-vision fusion tracking module includes an inertial measurement unit, a spatial simulation unit, a trajectory prediction unit, a three-point verification unit, and an interference alarm unit; The inertial measurement unit monitors the motion direction, acceleration, and spatial position of the moving subject in real time and generates dynamic data. The spatial simulation unit abstracts the moving subject followed by the imaging device into a line segment model composed of "midpoint + reference point" based on the dynamic data of the moving subject obtained by the inertial measurement unit, and abstracts the target into a single point model, so as to construct a spatial relative position model between the moving subject and the target. The trajectory prediction unit predicts the possible trajectory and potential arrival point of the target in the future based on visual data extracted from historical images. The three-point verification unit verifies whether the reference point, the midpoint and the potential arrival point or the actual arrival point of the target meet the three-point alignment condition. If they meet the condition, the tracking state continues. If they do not meet the condition, the tracking is canceled within a predetermined time. The interference alarm unit calculates the distance between the midpoint and the target's real-time potential arrival point or actual arrival point. If the three points are in a straight line and the distance is less than the threshold, it sends a "target interference alarm" to the user terminal and the cloud server. The actual arrival point is obtained by the multi-source data collaboration module.

[0015] Furthermore, the system monitoring and emergency module includes an equipment monitoring unit, a tracking quality assessment unit, an emergency dispatch unit, and an alarm management unit; The equipment monitoring unit monitors the operating status of all equipment in the system in real time and generates equipment health reports periodically. The tracking quality assessment unit evaluates the tracking quality and automatically triggers strategy adjustments based on the tracking quality. The emergency dispatch unit calls corresponding emergency plans for different emergencies. The alarm management unit classifies and manages all alarm signals in the system and stores historical alarm records.

[0016] Furthermore, the data optimization and expansion module includes a multi-scene image acquisition unit, a data enhancement processing unit, and a data balancing processing unit; The multi-scene image acquisition unit collects multi-scene image data for different recognition tasks; the data enhancement processing unit enhances the image data; the data balancing processing unit uses a combination of oversampled minority class images and undersampled majority class images to achieve a relative balance in the number of samples in each category; and the model feature optimization module adjusts the model structure. The model feature optimization module includes a structure-aware convolutional unit, a GNN structure integration unit, and a feature attention adjustment unit. The structure-aware convolutional unit is based on a convolutional neural network integrating a structure-aware convolutional layer. The GNN structure integration unit draws on graph neural networks to abstract the object structure into a graph structure and integrate it into the model design. The feature attention adjustment unit evaluates the model's attention to different features in real time during model training and adjusts the model training parameters accordingly. The algorithm's fairness management module includes a data population detection unit, a fairness assessment unit, and a bias correction and optimization unit. The data group detection unit detects the feature coverage of different groups in the training data and feeds it back to the "multi-scene image acquisition unit" and the "data equalization processing unit". The fairness evaluation unit periodically evaluates the model's recognition accuracy for different groups and determines whether the model has algorithmic bias. The bias correction and optimization unit performs adaptive correction and optimization of the model when the fairness evaluation unit detects that the model has algorithmic bias. The system monitoring and feedback module includes an operation status monitoring unit and a result feedback processing unit. The operation status monitoring unit monitors the operation status of the data optimization and expansion module, the model feature optimization module, and the algorithm fairness management module in real time, and issues early warning signals based on the monitoring results. The result feedback processing unit collects user feedback channels established by the model to receive user objections to the recognition results and classifies and statistically analyzes recognition error cases.

[0017] Compared with the prior art, the present invention has the following advantages: This invention provides an AI visual recognition system based on neural networks, which innovates in multiple aspects such as data processing, model optimization, multi-target tracking, and target tracking, and ultimately achieves a balance between real-time performance and accuracy. Scene adaptability and data sharing linkage: Through a multi-device data sharing mechanism, continuous tracking of the target is achieved across different acquisition devices, effectively overcoming the processing limitations of dynamic and static scenes. For changes in the visual imaging area, the system adopts strategies such as jitter elimination, deep learning prediction, and data sharing between devices in different scenarios to ensure the continuity of tracking. Synergistic Environmental Optimization and Feature Extraction: When dealing with poor imaging environments such as nighttime, multispectral fusion technology combines near-infrared cameras with visible light cameras to enrich visual information. High-quality image processing data from unconventional scenes lays the foundation for reducing the blindness and sample bias in key feature extraction. Complementary real-time processing and data optimization: By adopting streaming video frames and combining optical flow, Kalman filtering technology, adaptive sampling and inference strategies, and multi-task learning, the latency of multi-target tracking is significantly reduced, achieving mutual promotion of real-time performance and data quality.

[0018] Model optimization and performance balancing are mutually reinforcing: By expanding the dataset, balancing data distribution, and optimizing the model, the blindness in key feature extraction and the identification errors caused by sample bias are reduced, thereby improving identification accuracy and fairness. At the same time, the implementation of strategies such as cross-device data sharing achieves a balance between real-time performance and accuracy. The improvement in real-time performance creates conditions for the model to quickly obtain more effective data feedback, further optimizing model performance.

[0019] This system, encompassing scene processing, environmental adaptation, real-time tracking, feature extraction, and performance balancing, collectively constructs a multi-functional collaborative system. Multi-device data sharing and scene processing technologies ensure data acquisition and transmission; real-time processing technology improves data processing efficiency; model optimization enhances data processing accuracy; and the performance balancing strategy coordinates all parties, enabling the system to achieve AI visual recognition efficiently and accurately in different scenarios, meeting diverse application needs. Attached Figure Description

[0020] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0021] Figure 1 A structural block diagram of an AI visual recognition system based on a neural network provided in an embodiment of the present invention; Figure 2 This is a flowchart of the image preprocessing and recognition process in an embodiment of the present invention; Figure 3 This is a structural block diagram of the streaming frame tracking processing module and the adaptive sampling module in an embodiment of the present invention; Figure 4 This is a structural block diagram of the multi-task collaborative optimization module in an embodiment of the present invention; Figure 5 This is a structural block diagram of the multi-source data collaboration module in an embodiment of the present invention; Figure 6 This is a flowchart of the multi-source data collaboration and scene tracking adaptation process in an embodiment of the present invention; Figure 7 This is a flowchart of the motion-vision fusion tracking process in an embodiment of the present invention; Figure 8 This is a flowchart of the model optimization subsystem in an embodiment of the present invention; Figure 9 This is a table showing the test metrics and their effects for this system in some test scenarios. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] like Figure 1 As shown, the present invention provides an AI visual recognition system based on neural networks, including an image acquisition and preprocessing subsystem, an image recognition subsystem, a multi-target tracking subsystem, a target tracking collaboration subsystem, a model optimization subsystem, and also includes a cloud server and a user terminal.

[0024] This system innovates in multiple aspects, including data processing, model optimization, multi-target tracking, and target tracking, ultimately achieving a balance between real-time performance and accuracy. Scene adaptability and data sharing linkage: Through a multi-device data sharing mechanism, continuous tracking of the target is achieved across different acquisition devices, effectively overcoming the processing limitations of dynamic and static scenes. For changes in the visual imaging area, the system adopts strategies such as jitter elimination, deep learning prediction, and data sharing between devices in different scenarios to ensure the continuity of tracking. Synergistic Environmental Optimization and Feature Extraction: When dealing with poor imaging environments such as nighttime, multispectral fusion technology combines near-infrared cameras with visible light cameras to enrich visual information. High-quality image processing data from unconventional scenes lays the foundation for reducing the blindness and sample bias in key feature extraction. Complementary real-time processing and data optimization: By adopting streaming video frames and combining optical flow, Kalman filtering technology, adaptive sampling and inference strategies, and multi-task learning, the latency of multi-target tracking is significantly reduced, achieving mutual promotion of real-time performance and data quality.

[0025] Model optimization and performance balancing are mutually reinforcing: By expanding the dataset, balancing data distribution, and optimizing the model, the blindness in key feature extraction and the identification errors caused by sample bias are reduced, thereby improving identification accuracy and fairness. At the same time, the implementation of strategies such as cross-device data sharing achieves a balance between real-time performance and accuracy. The improvement in real-time performance creates conditions for the model to quickly obtain more effective data feedback, further optimizing model performance.

[0026] This system, encompassing scene processing, environmental adaptation, real-time tracking, feature extraction, and performance balancing, collectively constructs a multi-functional collaborative system. Multi-device data sharing and scene processing technologies ensure data acquisition and transmission, real-time processing technologies improve data processing efficiency, model optimization enhances data processing accuracy, and performance balancing strategies coordinate all parties, enabling the system to achieve AI visual recognition efficiently and accurately in different scenarios, meeting diverse application needs.

[0027] In this system, such as Figure 2 As shown, the image acquisition and preprocessing subsystem is used to acquire images and preprocess them. The image acquisition and preprocessing subsystem includes an image acquisition device and a preprocessing module. The preprocessing module includes a preprocessing acceleration unit, a spectral image fusion unit, an image enhancement unit, a geometric correction unit, and a size normalization unit; The image recognition subsystem includes a feature extraction module, a classification module, and a pipelined parallel processing module; The feature extraction module includes a key point extraction unit, a semantic feature extraction unit, and a texture feature extraction unit; The classification module includes an adaptive computation scheduling unit and a real-time classification unit; like Figure 2 As shown, the modules in the image acquisition and preprocessing subsystem and the image recognition subsystem contain multiple functional units that work together to complete the visual recognition task: The image acquisition equipment can acquire visible light images and NIR images; The preprocessing acceleration unit uses hardware (FPGA, dedicated image processing chip) to achieve region of interest (ROI) cropping and efficient encoding and decoding, reducing the amount of data in subsequent processing. While ensuring image quality, the bit rate is reduced by 50%, accelerating video stream transmission and decoding speed, and the preprocessing time is shortened from 20ms to 6ms. The spectral image fusion unit employs a pixel-level fusion algorithm to fuse the color information of the visible light image with the contour and detail information of the NIR image. For example, in nighttime surveillance, the fused image retains the clear contours of pedestrians and vehicles in the NIR image while also overlaying the ambient colors of the visible light image, avoiding the monochromatic and detail-lacking problems of pure NIR images. During transitional periods such as dawn and dusk, the fusion weights of the two types of images are dynamically adjusted (e.g., 60% NIR weight and 40% visible light weight) to balance image fluctuations caused by changes in light, providing high-quality images for subsequent recognition. The image enhancement unit enhances the image through algorithms such as histogram equalization and filtering, improving image clarity and contrast, highlighting key information, and suppressing noise interference. The geometric correction unit corrects images deformed by factors such as shooting angle and lens distortion, restoring the true shape of the target and laying the foundation for accurate recognition. The size normalization unit adjusts images of different sizes to a standard size, which facilitates efficient operation of subsequent processing units and improves processing efficiency. The key point extraction unit locates key feature points in the image and captures the unique local features of the target.

[0028] The semantic feature extraction unit extracts semantic features from images based on the optimized model of the model optimization subsystem, thereby understanding the deeper meaning of the image content.

[0029] The texture feature extraction unit extracts texture information from the image, such as roughness and directionality, to distinguish targets with different materials and surface properties.

[0030] The adaptive computation scheduling unit calls the model optimized by the model optimization subsystem based on the scene complexity of the input image and dynamically switches the inference model or the number of network layers. For example, in security monitoring, when the scene is a static and open scene (without moving objects), a shallow network is called and the frame processing is completed within 10ms; when complex scenes such as crowds gathering or objects moving frequently are detected, the system automatically switches to a full deep model and completes fine analysis within 50ms, which avoids resource waste and ensures the recognition accuracy of complex scenes. The real-time classification unit inputs the extracted features into the scheduled model to quickly determine the category to which the target belongs, achieving efficient and accurate real-time classification, and sends the classification results to the cloud server and user terminal.

[0031] The pipeline parallel processing module breaks down the visual recognition process (acquisition → preprocessing → feature extraction → classification) into multiple stages and constructs a pipeline parallel mechanism. When the first frame of the image is in the "feature extraction" stage, the second frame of the image is simultaneously "preprocessed" and the third frame of the image begins "acquisition". Each stage is connected through a buffer, with no waiting time. The system throughput is increased by 3-5 times, and the single frame processing cycle is compressed from 105ms to 34ms, meeting the high-speed inspection requirements of the production line.

[0032] The multi-target tracking subsystem includes an adaptive sampling module, a sampling region optimization module, a streaming frame tracking processing module, and a multi-task collaborative optimization module. The adaptive sampling inference module dynamically adjusts the sampling rate based on the target state and scene complexity, concentrating computing power on key targets, reducing invalid calculations, and balancing latency and recognition accuracy.

[0033] like Figure 3 As shown, the adaptive sampling module includes a region division unit and a dynamic sampling control unit; The region segmentation unit divides each frame of image into several regions to be identified according to a specific size; The dynamic sampling control unit adjusts the sampling interval based on the number of targets, their movement speed, and the complexity of the scene. For example, in 12 frames of images, if there are 18 targets and they move frequently (such as during peak traffic at an intersection), sampling is performed in groups of 3 (frames 1, 4, 7, and 10 form one group, frames 2, 5, 8, and 11 form another group, etc.). Within each group, only a specific area to be identified is identified, and the areas to be identified are the same in different frames within the same group. If the scene is simple (such as tracking static objects in a park), the sampling rate can be increased by sampling one frame at a time, ensuring reduced load in complex scenes and maintained accuracy in simple scenes.

[0034] The sampling region optimization module solves the problem of missing moving targets by fixed sampling regions through randomized sampling order and adjacent region avoidance strategy, ensuring the integrity of multi-target tracking and avoiding secondary processing delays caused by missed detections.

[0035] like Figure 3 As shown, the sampling area optimization module includes a region random sorting unit and an adjacent region avoidance unit; The area random sorting unit labels the areas to be identified in the monitoring screen (e.g., 4 areas, numbered 1-4). Before each round of sampling, the numbers are randomly shuffled and undersampled sequentially according to the new order. For example, in the first round, each frame is undersampled in the order of "3→1→2→4", and in the second round, each frame is undersampled in the order of "1→3→4→2". This avoids the continuous omission of moving targets (such as pedestrians crossing the screen) due to the fixed order, and ensures that each round of sampling covers all areas.

[0036] The division of the region to be identified must satisfy the following condition: minimize the number of adjacent regions to be identified.

[0037] In the above embodiments, if four horizontally arranged regions to be identified are obtained, the dynamic sampling control unit can adjust the sampling to be performed in four-frame intervals, with each four frames constituting one round of sampling. That is: in the first round of sampling, the first frame undersamples the third region to be identified (the first, second, and fourth regions to be identified are sampled and identified, but the third does not need to be sampled and identified), the second frame undersamples the first region to be identified, the third frame undersamples the second region to be identified, and the fourth frame undersamples the fourth region to be identified; in the second round of sampling, the fifth frame undersamples the first region to be identified, the sixth frame undersamples the third region to be identified, the seventh frame undersamples the fourth region to be identified, and the eighth frame undersamples the first region to be identified.

[0038] The adjacent region avoidance unit ensures that the sampling areas (areas to be identified) in adjacent locations are not in consecutive undersampling sequence numbers, avoiding the omission of moving targets between two areas. For example, if area 1 and area 2 are adjacent, the sequence number difference between the two is guaranteed to be ≥2 during sorting (e.g., if area 1 is sequence number 2, area 2 must be at least sequence number 4). This avoids the "connection omission" caused by undersampling area 1 first and then undersampling area 2, ensuring that moving targets (such as vehicles moving from area 1 to area 2) can be covered by at least one round of sampling, improving tracking integrity, and greatly reducing data processing time on this basis.

[0039] The streaming frame tracking processing module uses streaming processing as its core and combines inter-frame tracking technology to reduce redundant calculations, enabling low-latency continuous tracking of multiple targets and avoiding latency accumulation caused by full frame recognition.

[0040] The streaming frame tracking processing module includes an inter-frame tracking execution unit and a streaming data scheduling unit; The inter-frame tracking execution unit uses optical flow and Kalman filtering techniques to perform trajectory association and prediction for multiple targets in consecutive video frames. For example, after completing the identification of 18 targets in a certain area of ​​the first frame, the target motion vector is calculated by optical flow, or the target position in the second and third frames is predicted by Kalman filtering. Supplementary detection is triggered only when the target position deviation exceeds the threshold (e.g., pixel difference > 5), without having to perform complete identification for each frame, reducing the amount of repetitive calculation by more than 60% and reducing the processing time per frame.

[0041] The streaming data scheduling unit breaks down the video stream into a continuous sequence of frames and adopts a "receive-process-output" pipeline mode, enabling real-time processing without waiting for the full amount of data. For example, in a 12-frame video stream, while the previous frame is in the "tracking calculation" stage, the next frame is temporarily stored in a buffer and preprocessed (such as size normalization) to avoid data accumulation; the processed frame outputs the tracking results in real time, ensuring an uninterrupted end-to-end process and reducing overall latency.

[0042] The multi-task collaborative optimization module avoids redundant calculations in identifying and tracking tasks by sharing features and linking tasks, thereby improving overall processing efficiency and indirectly reducing latency.

[0043] like Figure 4 As shown, the multi-task collaborative optimization module includes a feature sharing processing unit and a task linkage optimization unit; In target recognition and tracking tasks, the feature sharing processing unit shares the extracted basic features, such as the target contour and texture features obtained by the recognition task, which can be directly used by the tracking task for inter-frame matching. This eliminates the need to extract features separately for the two tasks, saving 30%-50% of the feature extraction time and reducing the latency caused by repeated calculations.

[0044] The task-linked optimization unit associates the recognition and tracking results, mutually assisting each other to correct deviations. For example, the target location predicted by the tracking task can narrow the search range of the recognition task (e.g., from a full-image search to a 200×200 pixel area search), and the target coordinates corrected by the recognition task can update the tracking trajectory to avoid drift. Through task linkage, accuracy is improved, duplicate detections caused by result deviations are reduced, and latency is further reduced.

[0045] The multi-target tracking subsystem addresses the issue of end-to-end latency exceeding 200ms caused by the simultaneous processing of multiple objects in multi-target recognition and tracking scenarios. Through streaming processing, adaptive strategies, multi-task collaboration, and sampling optimization, it significantly reduces latency while ensuring tracking continuity and accuracy.

[0046] The target tracking and collaboration subsystem includes a multi-source data collaboration module, a scene tracking and adaptation module, a motion-visual fusion tracking module, and a system monitoring and emergency response module. The multi-source data collaboration module includes a multi-device acquisition unit, a cloud synchronization unit, a cross-device matching unit, a location correction unit, and a shared access control unit; The multi-source data collaboration module breaks down the data barriers of a single device, enabling cross-device data sharing, synchronization, and correction, and providing multi-dimensional reference data for tracking.

[0047] like Figure 5 As shown, this module contains 5 units, and the functions of each unit are as follows: Multi-device acquisition unit: responsible for connecting to the image recognition subsystem and the image acquisition and preprocessing subsystem, acquiring image data and target feature data in real time, and providing a basic data source for subsequent tracking.

[0048] Cloud synchronization unit: Uploads image data and target feature data acquired by all acquisition devices to the cloud database in real time. During the upload process, it ensures that the timestamps of all data are accurately aligned, avoids deviations caused by data transmission delays from multiple devices, and ensures the time consistency of data across devices.

[0049] Cross-device matching unit: From the moment the target enters the monitoring range of a certain imaging device until it leaves the monitoring range of a certain imaging device, it retrieves real-time data collected by other imaging devices through the cloud server, and performs cross-device matching based on the core features of the target (such as shape outline, movement trajectory, color features, etc.) to confirm the correlation of the same target in different devices, and realize cross-device relay of tracking tasks.

[0050] Position correction unit: Utilizes data of the same target collected from different angles by multiple devices, and combines the principle of triangulation to calculate the actual spatial position of the target, corrects the positioning error caused by the viewing angle deviation of a single device, and improves the accuracy of the target position data.

[0051] The shared access control unit manages access permissions and verifies data for the identification results shared across devices, ensuring data security and accuracy. For example, if the information shared by a device deviates significantly from the information detected by other devices, a re-identification mechanism is triggered to prevent the spread of erroneous results and ensure the reliability of shared data.

[0052] The scene tracking adaptation module adaptively selects the tracking strategy based on the type of image range change in static / dynamic scenes (jitter, large-scale change, rapid change, etc.) to ensure short-term tracking continuity.

[0053] like Figure 6 As shown, this module comprises four core units: scene recognition unit, range change detection unit, static tracking unit, and dynamic tracking unit. The functions of each unit are as follows: Scene recognition unit: Analyzes the attributes of the current monitoring scene in real time, distinguishes between static scenes (such as traffic intersection monitoring, industrial production line monitoring) and dynamic scenes (such as vehicle-mounted mobile cameras, patrol robot inspection), and provides scene basis for subsequent selection of appropriate tracking strategies.

[0054] Range variation detection unit: By comparing the pixel alignment and inter-frame content differences of adjacent multi-frame images, it identifies the type of variation in the image acquisition range, which is specifically divided into three categories: brief jitter (small shaking of the image acquisition range), large range variation (caused by environmental interference or device displacement), and rapid variation (caused by high-speed movement of the image acquisition device in dynamic scenes).

[0055] Static Tracking Unit: Specifically designed for tracking needs in static scenes, it includes three sub-functions: First, a jitter elimination sub-unit, which activates a jitter elimination algorithm to correct image shifts and maintain the continuity of target tracking when the imaging range experiences slight shaking. Second, a large-scale change handling sub-unit, which predicts the short-term movement direction and position of the target through a deep learning model (based on the target's historical trajectory data) or calls data from multiple devices to complete the target's position information to avoid tracking interruption if the imaging range changes significantly. Third, an early warning sub-unit, which does not require the inertial measurement unit to be activated in static scenes, directly sends a device abnormality alarm signal when an abnormal change in the imaging range is detected (such as device obstruction or human displacement).

[0056] The dynamic tracking unit is designed for tracking needs in dynamic scenarios (such as vehicle-mounted and cruise robots). It includes three sub-units: First, a speed judgment sub-unit, which distinguishes between "slow changes" and "fast changes" in the imaging range by analyzing the pixel change amplitude of multiple frames of images; second, a slow processing sub-unit, which simultaneously tracks "targets within the imaging range" and "targets that have just left the imaging range" when the imaging range changes slowly, to avoid missing targets; and third, a fast processing sub-unit, which allocates computing power priority based on the target's spatial location (safe area / unsafe area) and target distance (far / near): if the target is in an unsafe area, it prioritizes tracking targets that have just left the imaging range, and then identifies and tracks targets within the imaging range; if the target is in a safe area, it first identifies and tracks targets within the imaging range, and then tracks targets that have just left the imaging range; if the target is far away, it prioritizes tracking targets within the imaging range, and then tracks targets that have left the range; if the target is close, it tracks both types of targets simultaneously to ensure that close targets are not lost.

[0057] The motion-vision fusion tracking module solves the problems of limited image acquisition range and equipment occlusion. By fusing inertial measurement and visual recognition, it enables short-term tracking and trajectory prediction after the target has separated from the target.

[0058] like Figure 7 As shown, this module contains 5 core units, and the functions of each unit are as follows: Inertial Measurement Unit (IMU): It is only activated in dynamic scenes or when the target leaves the imaging range. It monitors the motion direction, acceleration and spatial position of moving objects (such as vehicle-mounted equipment and cruise robots) in real time, generates dynamic data, and provides a motion reference for target trajectory simulation.

[0059] Space simulation unit: Based on the dynamic data of the moving subject obtained by the inertial measurement unit, the moving subject (such as a vehicle) followed by the equipment is abstracted into a line segment model composed of "midpoint + reference point", and the target is abstracted into a single point model, thus constructing a spatial relative position model between the moving subject and the target.

[0060] Trajectory prediction unit: Based on visual data such as target location and direction of movement extracted from historical images, predict the possible trajectory and potential destination of the target in the future.

[0061] The three-point verification unit verifies whether the "reference point," "midpoint," and the target's "potential arrival point" or actual arrival point meet the three-point alignment condition. If they do, it indicates that the target's trajectory and the moving body may overlap at some point in the future, and the tracking state continues (maintaining this tracking state and the tracking results mainly rely on data sharing from other devices; if no other devices are monitoring, algorithmic prediction can also be used). If they do not meet the condition, the tracking is canceled within a predetermined time (the predetermined time value is adjusted according to the actual situation) to avoid unnecessary computational power consumption. The actual arrival point is obtained by the multi-source data collaboration module.

[0062] Interference alarm unit: Calculates the real-time "potential arrival point" or actual arrival point of the target from the midpoint. If the three points are in a straight line and the distance is less than a certain value, it sends a "target interference alarm" to the user terminal and cloud server.

[0063] The system monitoring and emergency module ensures stable system operation, handles emergencies such as equipment malfunctions and target loss, and provides emergency intervention capabilities.

[0064] This module contains 4 core units, and the functions of each unit are as follows: Equipment monitoring unit: Monitors the operating status of all equipment in the system in real time, including image acquisition equipment, inertial measurement units (IMU), cloud data transmission links, etc., focusing on troubleshooting issues such as equipment offline, data transmission delay, and sensor failure, and regularly generates equipment health reports to provide data support for operation and maintenance.

[0065] Tracking quality assessment unit: Based on two core indicators, tracking quality is evaluated: tracking continuity (duration of time without target tracking interruption) and position error (deviation between corrected target position and actual position). When the quality is below a set threshold (e.g., positioning error exceeds 10% for 3 consecutive frames), strategy adjustment is automatically triggered, such as increasing the number of devices participating in data linkage and optimizing trajectory prediction algorithms.

[0066] Emergency Dispatch Unit: Invokes corresponding emergency plans for different emergencies: When multiple devices fail simultaneously, activates locally cached historical trajectory data of the target to temporarily maintain short-term tracking; when the target loss time exceeds the threshold, sends a "target loss warning" and coordinates with surrounding devices to expand the search range; when the cloud data link is interrupted, switches to local offline mode to prioritize the tracking function of the current device, and synchronizes historical data after the link is restored.

[0067] Alarm Management Unit: Classifies and manages all alarm signals within the system, including equipment malfunction alarms, target interference alarms, target loss alarms, etc.; supports alarm classification (general alarms / emergency alarms), multi-channel information push (SMS notifications / management platform pop-ups), and stores historical alarm records, supporting queries by time and type, providing a basis for operation and maintenance review and system optimization.

[0068] The target tracking collaborative subsystem in this invention achieves the following technical effects through "multi-module collaboration + cross-technology integration": 1. Break through scene limitations and achieve full scene coverage. Traditional tracking systems often suffer from insufficient adaptability due to differences in scenarios (static / dynamic). For example, static monitoring is interrupted when the equipment is moved, and dynamic vehicle tracking loses the target when the vehicle turns rapidly. This system works by linking the "Scene Tracking Adaptation Module" and the "Dynamic Vision Fusion Tracking Module". In static scenarios, tracking can be maintained by eliminating jitter and supplementing data from multiple devices. In dynamic scenarios, the system can adjust the strategy according to speed / distance / safety level, achieving full-scene coverage with "stable static scenarios and flexible dynamic scenarios". The tracking continuity is improved by more than 80% compared to traditional systems.

[0069] 2. Break down data barriers and improve tracking accuracy Traditional single-device tracking relies on data from a single viewpoint, which is prone to positioning errors or tracking loss due to viewpoint deviations and occlusions. The system achieves cross-device data synchronization and correction through a "multi-source data collaboration module" and integrates IMU dynamic data and visual data through a "dynamic-visual fusion tracking module," providing dual assurance of positioning accuracy.

[0070] 3. Strengthen emergency response capabilities and ensure system reliability. Traditional systems often stop tracking directly without emergency response when encountering equipment failure or data interruption. This system monitors equipment and tracking quality in real time. In case of anomalies, it can activate emergency solutions such as local caching, device switching, and range adjustment. For example, when multiple devices fail, it can temporarily track through historical trajectories. When the data link is interrupted, it can switch to offline mode. Anomaly visualization is achieved through alarm management, thus improving the efficiency of operation and maintenance response.

[0071] To optimize the model, this invention also proposes a model optimization subsystem, which includes a data optimization and expansion module, a model feature optimization module, an algorithm fairness management module, and a system monitoring and feedback module. The data optimization and expansion module is fundamental to improving the model's attention to object structure and mitigating sample bias. By enriching data sources and optimizing data distribution, it provides high-quality and diverse datasets for model training, reducing recognition bias caused by data limitations.

[0072] The data optimization and expansion module consists of three units, with the following specific functions: The multi-scene image acquisition unit collects image data covering diverse scenes, angles, and lighting conditions for different recognition tasks, ensuring that the data can comprehensively present the object structure and target group characteristics, providing a wide range of object structure and group characteristics references for model learning.

[0073] The data augmentation processing unit uses data augmentation techniques such as rotation, flipping, scaling, cropping, and adding noise to enhance the acquired original image data, creating more diverse samples and further enriching the diversity of the data. This enables the model to learn the characteristics of object structure under different transformations, avoids overfitting to object structures under specific background, pose, or lighting conditions, and groups of objects with specific appearance features, and improves the model's ability to generalize the learning of object structure and group features.

[0074] To address the imbalance in the number of samples from different classes in the training data, the data balancing unit combines oversampled minority class images with undersampled majority class images to achieve a relative balance in the number of samples from each class. This operation encourages the model to treat the object structures of different classes and the characteristics of different groups equally, preventing the model from being overly biased towards common classes or groups due to data distribution imbalance, while ignoring the object structures of minority classes and the characteristics of minority groups.

[0075] The model feature optimization module focuses on solving the problems of blind extraction of key features and overemphasis on irrelevant features. By improving the model structure, it enhances the model's ability to capture the core structural features of objects, reduces attention to irrelevant features such as background texture, and improves the accuracy of model recognition.

[0076] like Figure 8 As shown, the model feature optimization module consists of three units, with the following specific functions: The structure-aware convolutional unit (SAC) is designed and integrated with a structure-aware convolutional layer on the basis of a convolutional neural network (CNN). When performing convolution operations, this convolutional layer not only considers the local features of pixels, but also focuses on capturing the spatial relationships between different parts of the object's structure through special convolutional kernel design (such as designing convolutional kernels that can capture structural features such as object edges and contours) or unique connection methods (such as cross-layer connections to integrate structural information from different levels). Integrating this convolutional layer into the model replaces or supplements the traditional convolutional layer, enabling the model to prioritize learning the core structural features of the object during training, reducing attention to non-structural features such as background textures and irrelevant decorations, and improving the model's sensitivity to object structure recognition.

[0077] The GNN structure incorporates the idea of ​​Graph Neural Networks (GNNs), abstracting the structure of an object into a graph and integrating it into the model design. Specifically, different parts of the object are treated as nodes in the graph, and the connections between the parts are treated as edges, constructing a graph representation of the object's structure. By learning and reasoning about this graph structure, the model directly understands the structural relationships between the parts of the object, strengthens the learning of the object's core structural features, avoids ignoring the overall structure of the object due to excessive focus on pixel-level local irrelevant features (such as background texture), and reduces recognition errors.

[0078] During model training, the feature attention adjustment unit evaluates the model's attention to different features (such as object structural features and background-independent features) in real time. By introducing a feature attention metric (such as calculating the model's attention weights for object structural features and background-independent features), it monitors the model's feature attention. When it finds that the model's attention to the core structural features of the object has decreased or its attention to irrelevant features has increased, it adjusts the model training parameters in a timely manner (such as adjusting the weights of different features in the loss function and increasing the loss weights for structural features) or optimizes the model structure (such as strengthening the role of structure-aware convolutional layers and GNN structures). This guides the model to focus more attention on the core structural features of the object, ensuring that the model improves the overall recognition accuracy without reducing its attention to key structural features, and avoiding a sharp drop in the recognition rate of specific categories due to feature attention bias.

[0079] The algorithm fairness control module mainly addresses the problem of model bias towards specific groups caused by training data bias. By supervising, evaluating, and optimizing the fairness of the model training process, it ensures that the model maintains a consistent recognition accuracy across different groups, thus avoiding social fairness issues caused by algorithm bias.

[0080] The algorithm fairness management module consists of three units, with the following specific functions: For tasks involving group identification, the data group detection unit detects the feature coverage of different groups in the training data. By statistically analyzing the number of samples and the diversity of sample features for each group, it assesses whether there are problems such as insufficient sample numbers or incomplete features in the data. If an imbalance in group features is found, it promptly feeds back to the "multi-scene image acquisition unit" and the "data balancing processing unit" to urge them to supplement the sample data of the corresponding groups, ensuring that the training data covers each group in a balanced manner.

[0081] After the model is trained and during actual application, the fairness assessment unit regularly evaluates the model's recognition accuracy for different groups to determine whether the model has algorithmic bias. If the recognition accuracy of a certain group is found to be significantly lower than that of other groups, it is determined that the model has a fairness problem. At the same time, for the "groups" that may be involved in the object classification task, the model evaluates the recognition accuracy of each category of objects. If there is a sudden drop in the recognition rate of a certain category of objects, the root cause of the problem is located in time (such as insufficient feature attention or insufficient data).

[0082] When the "Fairness Assessment Unit" detects algorithmic bias in the model, the Bias Correction and Optimization Unit takes targeted measures to correct and optimize it. If the bias stems from data issues, it works with the "Data Optimization and Expansion Module" to supplement the data for that group and retrain the model. If the bias stems from insufficient learning of the characteristics of a specific group, it optimizes the model structure or adjusts the training strategy (such as using a fairness-constrained loss function and adding a term to balance the recognition error of each group in the loss calculation, forcing the model to take into account the recognition performance of each group during training). Through the optimization of this unit, it ensures that the model maintains a consistent recognition standard for all groups, avoids unfair treatment of specific groups due to algorithmic bias, and safeguards social fairness.

[0083] The system monitoring and feedback module is responsible for real-time monitoring of the entire system's operation, collecting information such as model recognition results and data processing effects, forming a feedback mechanism, continuously promoting the optimization of each module of the system, ensuring the long-term stable operation of the system and continuously improving its performance.

[0084] The system monitoring and feedback module consists of two units, with the following specific functions: The operational status monitoring unit monitors the operational status of the "Data Optimization and Expansion Module," "Model Feature Optimization Module," and "Algorithm Fairness Management Module" in real time, including data acquisition progress, data augmentation processing efficiency, model training iteration count, and fairness assessment frequency. By setting operational status thresholds (such as a data acquisition completion rate of over 90% and model training loss values ​​remaining stable within a preset range), when the operational status of a module or unit exceeds the threshold (such as excessively low data augmentation processing efficiency or a continuously rising model training loss value), a corresponding early warning signal is promptly issued to notify relevant personnel to investigate the problem (such as data acquisition equipment failure or incorrect model parameter settings), ensuring the orderly and efficient operation of all aspects of the system.

[0085] The results feedback processing unit collects the recognition results of the model in practical applications (such as object classification results and face recognition pass rate), establishes a user feedback channel, receives user objections to the recognition results, classifies and statistically analyzes recognition error cases (such as recognition errors caused by feature extraction errors and recognition errors caused by group bias), analyzes the causes of errors and feeds them back to the corresponding modules, forming a closed loop of "problem discovery - cause analysis - module optimization" to continuously improve system performance.

[0086] The model optimization subsystem addresses issues in the field of AI visual recognition, such as blind extraction of key features, recognition errors caused by sample bias, excessive focus on irrelevant features, and algorithmic bias towards specific groups. Through the collaborative operation of multiple modules, it comprehensively improves the accuracy and fairness of AI visual recognition from the dimensions of data optimization, model improvement, and fairness control, ensuring that the model operates stably and reliably in various scenarios.

[0087] To verify the system's performance, a 30-day field test was conducted in traffic and security scenarios. The test indicators and results are as follows: Figure 9 As shown.

[0088] When this system is applied to real-world scenarios, the following implementation scenarios exist: Example 1: Multi-target tracking scenario at an urban intersection; At key urban intersections, the system can simultaneously track 15-20 dynamic targets (motor vehicles, non-motor vehicles, and pedestrians), acquire target trajectory and speed information in real time, and support red light violation capture and congestion warning functions. The end-to-end latency of the system has been reduced from 220ms before implementation to 130ms, meeting real-time requirements; the accuracy of red light violation capture has been improved to 98%.

[0089] Example 2: Multi-target tracking scenario for security in a large park For a large industrial park with an area of ​​5km² and 20 cameras, it is necessary to simultaneously track 10-15 targets (personnel, patrol vehicles, and forklifts) and identify abnormal behaviors such as personnel entering restricted areas and vehicles parking illegally in real time. Using this system, the end-to-end latency was reduced from 230ms before implementation to 160ms, which meets the real-time security standards. The response time for abnormal behavior identification was shortened by 40%, and security incidents can be quickly triggered to trigger the handling process. The continuity of target tracking across cameras was improved, avoiding the loss of personnel and vehicles when tracking across areas. The efficiency of handling security incidents in the park was improved by 50%, meeting the needs of 24 / 7 security monitoring.

[0090] In practical applications, even if the target leaves the visual imaging area, the system can still achieve short-term tracking of the target through device sharing or trajectory prediction. Among them, cross-device sharing can not only correct the collected data with the help of multiple devices in some scenarios, but also continue target tracking by using data from other imaging devices in specific situations. At the same time, it avoids the computing power consumption caused by independent tracking by multiple devices, thereby reducing the overall computing power cost. In addition, the frame-by-frame sampling and region-by-region sampling technologies of the system can also reduce computing power consumption and effectively improve real-time performance without affecting the accuracy of the system.

[0091] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.

Claims

1. An AI visual recognition system based on neural networks, characterized in that, It includes an image acquisition and preprocessing subsystem, an image recognition subsystem, a multi-target tracking subsystem, a target tracking collaboration subsystem, a model optimization subsystem, as well as a cloud server and a user terminal; The image recognition subsystem includes a feature extraction module, a classification module, and a pipelined parallel processing module; The multi-target tracking subsystem includes an adaptive sampling module, a sampling region optimization module, a streaming frame tracking processing module, and a multi-task collaborative optimization module. The target tracking collaboration subsystem includes a multi-source data collaboration module, a scene tracking adaptation module, a dynamic vision fusion tracking module, and a system monitoring and emergency response module. The model optimization subsystem includes a data optimization and expansion module, a model feature optimization module, an algorithm fairness management module, and a system monitoring and feedback module.

2. The AI ​​visual recognition system based on neural networks according to claim 1, characterized in that, The image acquisition and preprocessing subsystem includes an image acquisition device and a preprocessing module; The preprocessing module includes a preprocessing acceleration unit, a spectral image fusion unit, an image enhancement unit, a geometric correction unit, and a size normalization unit. The image acquisition device acquires visible light images and NIR images. The preprocessing acceleration unit performs region-of-interest cropping and encoding / decoding on the images. The spectral image fusion unit uses a pixel-level fusion algorithm to fuse the visible light images and NIR images. The image enhancement unit enhances the images. The geometric correction unit corrects the images. The size normalization unit adjusts images of different sizes to a standard size.

3. The AI ​​visual recognition system based on neural networks according to claim 2, characterized in that, The feature extraction module includes a key point extraction unit, a semantic feature extraction unit, and a texture feature extraction unit; The key point extraction unit locates key feature points in the image and captures local features; the semantic feature extraction unit mines features with semantic information in the image based on the model optimized by the model optimization subsystem; and the texture feature extraction unit extracts the texture information of the image. The classification module includes an adaptive computation scheduling unit and a real-time classification unit; The adaptive computation scheduling unit calls the model optimized by the model optimization subsystem based on the scene complexity of the input image and dynamically switches the inference model or the number of network layers. The real-time classification unit inputs the extracted features into the scheduled model to determine the category to which the target belongs and sends the classification result to the cloud server and user terminal.

4. The AI ​​visual recognition system based on neural networks according to claim 3, characterized in that, The pipelined parallel processing module breaks down the visual recognition process into multiple stages, constructs a pipelined parallel mechanism, and connects each stage through a buffer, eliminating waiting time.

5. The AI ​​visual recognition system based on neural networks according to claim 3, characterized in that, The adaptive sampling module includes a region division unit and a dynamic sampling control unit; The region division unit divides each frame of image into several regions to be identified, and the dynamic sampling control unit adjusts the sampling interval according to the number of targets, movement speed and scene complexity. The sampling area optimization module includes an area random sorting unit and an adjacent area avoidance unit. The region random sorting unit labels all the regions to be identified and randomly numbers them before each round of sampling so that undersampling is performed in the order of the numbers. The adjacent region avoidance unit ensures that the regions to be identified in adjacent positions are not in consecutive undersampling sequence numbers. The streaming frame tracking processing module includes an inter-frame tracking execution unit and a streaming data scheduling unit; The inter-frame tracking execution unit performs trajectory association and prediction for multiple targets in the image, and the streaming data scheduling unit splits the video stream into a continuous frame sequence and performs real-time processing in a pipeline mode. The multi-task collaborative optimization module includes a feature sharing processing unit and a task linkage optimization unit; The feature sharing processing unit shares the extracted basic features in the target recognition and tracking task, and the task linkage optimization unit associates the recognition and tracking results to help each other correct deviations.

6. The AI ​​visual recognition system based on neural networks according to claim 5, characterized in that, The multi-source data collaboration module includes a multi-device acquisition unit, a cloud synchronization unit, a cross-device matching unit, a location correction unit, and a shared permission management unit. The multi-device acquisition unit is responsible for accessing the image recognition subsystem and the image acquisition and preprocessing subsystem, and for acquiring image data and target feature data in real time. The cloud synchronization unit uploads the image data and target feature data acquired by all acquisition devices to the cloud server in real time. The cross-device matching unit retrieves the real-time acquisition data of other image acquisition devices through the cloud server, performs cross-device matching based on the core features of the target, and confirms the association relationship of the same target in different image acquisition devices. The position correction unit uses the same target data acquired by multiple devices from different angles, and calculates the actual spatial position of the target by combining the triangulation principle to correct the positioning error. The shared permission management unit manages the permissions and verifies the data of the cross-device shared recognition results. The scene tracking and adaptation module includes a scene recognition unit, a range change detection unit, a static tracking unit, and a dynamic tracking unit; The scene recognition unit analyzes the attributes of the current monitoring scene in real time and distinguishes between static and dynamic scenes. The range change detection unit identifies the type of change in the image acquisition range by comparing the pixel alignment and inter-frame content differences of adjacent multi-frame images. The static tracking unit is adapted to the tracking requirements of static scenes, and the dynamic tracking unit is adapted to the tracking requirements of dynamic scenes.

7. The AI ​​visual recognition system based on a neural network according to claim 6, characterized in that, The static tracking unit includes a jitter elimination subunit, a large-range change processing subunit, and an early warning subunit; When the image acquisition range is shaken, the jitter elimination subunit starts the jitter elimination algorithm to repair the image offset. When the image acquisition range changes, the large-scale change processing subunit calls multi-device data to complete the target position information or predicts the short-term movement direction and position of the target through a deep learning model. When the early warning subunit detects an abnormal change in the image acquisition range, it sends a device abnormality alarm signal. The dynamic tracking unit includes a speed judgment subunit, a slow processing subunit, and a fast processing subunit; The speed judgment subunit distinguishes between "slow change" and "fast change" of the imaging range by analyzing the pixel change amplitude of multiple frames of images. The slow processing subunit simultaneously tracks "targets within the imaging range" and "targets that have just left the imaging range" when the imaging range changes slowly. The fast processing subunit allocates computing power priority based on the target's spatial position and target distance.

8. The AI ​​visual recognition system based on a neural network according to claim 6, characterized in that, The dynamic-visual fusion tracking module includes an inertial measurement unit, a spatial simulation unit, a trajectory prediction unit, a three-point verification unit, and an interference alarm unit; The inertial measurement unit monitors the motion direction, acceleration, and spatial position of the moving subject in real time and generates dynamic data. The spatial simulation unit abstracts the moving subject followed by the imaging device into a line segment model composed of "midpoint + reference point" based on the dynamic data of the moving subject obtained by the inertial measurement unit, and abstracts the target into a single point model, so as to construct a spatial relative position model between the moving subject and the target. The trajectory prediction unit predicts the possible trajectory and potential arrival point of the target in the future based on visual data extracted from historical images. The three-point verification unit verifies whether the reference point, the midpoint and the potential arrival point or the actual arrival point of the target meet the three-point alignment condition. If they meet the condition, the tracking state continues. If they do not meet the condition, the tracking is canceled within a predetermined time. The interference alarm unit calculates the distance between the midpoint and the target's real-time potential arrival point or actual arrival point. If the three points are in a straight line and the distance is less than the threshold, it sends a "target interference alarm" to the user terminal and the cloud server. The actual arrival point is obtained by the multi-source data collaboration module.

9. The AI ​​visual recognition system based on neural networks according to claim 8, characterized in that, The system monitoring and emergency module includes an equipment monitoring unit, a tracking quality assessment unit, an emergency dispatch unit, and an alarm management unit; The equipment monitoring unit monitors the operating status of all equipment in the system in real time and generates equipment health reports periodically. The tracking quality assessment unit evaluates the tracking quality and automatically triggers strategy adjustments based on the tracking quality. The emergency dispatch unit calls corresponding emergency plans for different emergencies. The alarm management unit classifies and manages all alarm signals in the system and stores historical alarm records.

10. The AI ​​visual recognition system based on a neural network according to claim 9, characterized in that, The data optimization and expansion module includes a multi-scene image acquisition unit, a data enhancement processing unit, and a data balancing processing unit. The multi-scene image acquisition unit collects multi-scene image data for different recognition tasks; the data enhancement processing unit enhances the image data; the data balancing processing unit uses a combination of oversampled minority class images and undersampled majority class images to achieve a relative balance in the number of samples in each category; and the model feature optimization module adjusts the model structure. The model feature optimization module includes a structure-aware convolutional unit, a GNN structure integration unit, and a feature attention adjustment unit. The structure-aware convolutional unit is based on a convolutional neural network integrating a structure-aware convolutional layer. The GNN structure integration unit draws on graph neural networks to abstract the object structure into a graph structure and integrate it into the model design. The feature attention adjustment unit evaluates the model's attention to different features in real time during model training and adjusts the model training parameters accordingly. The algorithm's fairness management module includes a data population detection unit, a fairness assessment unit, and a bias correction and optimization unit. The data group detection unit detects the feature coverage of different groups in the training data and feeds it back to the "multi-scene image acquisition unit" and the "data equalization processing unit". The fairness evaluation unit periodically evaluates the model's recognition accuracy for different groups and determines whether the model has algorithmic bias. The bias correction and optimization unit performs adaptive correction and optimization of the model when the fairness evaluation unit detects that the model has algorithmic bias. The system monitoring and feedback module includes an operation status monitoring unit and a result feedback processing unit. The operation status monitoring unit monitors the operation status of the data optimization and expansion module, the model feature optimization module, and the algorithm fairness management module in real time, and issues early warning signals based on the monitoring results. The result feedback processing unit collects user feedback channels established by the model to receive user objections to the recognition results and classifies and statistically analyzes recognition error cases.

Images