Browser AI reasoning system and method based on TensorFlow.js

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a browser-side AI reasoning system and method based on TensorFlow.js. The browser-side AI reasoning system and method based on TensorFlow.js comprises the following steps of model loading and initialization, equipment performance detection and model selection, input data collection and preprocessing, reasoning task scheduling and execution, result analysis output and visual export. Model cache updating maintenance and performance monitoring dynamic adjustment and optimization are carried out; the method has the beneficial effects that a set of complete front-end AI reasoning flow control mechanism is provided; realizing model dynamic adaptation based on equipment performance; various input forms are supported; a main thread is prevented from being blocked by utilizing multi-thread scheduling; a GPU acceleration and mixing precision calculation mechanism is introduced to improve the reasoning efficiency; establishing a model caching mechanism to improve the loading speed and the offline availability; performance monitoring and dynamic tuning functions are provided, and reasoning stability is ensured; providing a structured result output and visual display interface; a model version control and background hot update mechanism is realized; and the security, compatibility and expansibility of the system are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a browser-side AI inference system and method based on TensorFlow.js. BACKGROUND

[0002] Currently, the traditional AI model deployment method widely used in the field of artificial intelligence often excessively relies on the backend server to undertake the key task of inference calculation. However, this conventional deployment mode exposes many problems to be solved in the actual running process, which has a negative impact on the performance, efficiency and user experience of the entire AI application that cannot be ignored.

[0003] Lack of unified inference scheduling mechanism: there are significant differences in hardware performance among different devices, such as processor computing power, memory capacity, graphics processing capability, etc., which are different, and the existing deployment scheme fails to fully consider the actual situation of these hardware conditions, and cannot dynamically and flexibly adjust the loading and execution strategy of the model according to the performance characteristics of the specific device.

[0004] Low model loading efficiency: most systems fail to fully tap and utilize the potential of the browser local storage mechanism. Each time the user uses the related function, the system will download the model file repeatedly, which consumes a large amount of bandwidth resources, making the user face a long waiting time when the network condition is not good.

[0005] Weak multi-thread scheduling capability: blocking the main thread often occurs during the inference process, which means that when the model is performing complex calculation and inference tasks, other interactive operations of the entire page will be severely affected, becoming sluggish or even unresponsive.

[0006] Lack of performance monitoring and adaptive optimization mechanism: the system cannot accurately perceive the resource consumption in the inference process, including CPU usage, memory occupation, network bandwidth consumption and other key indicators. Due to the lack of mastery of these important information, it is difficult to dynamically adjust the model parameters according to the actual resource consumption to achieve the optimal inference performance.

[0007] Poor offline availability: most AI applications based on traditional deployment methods can only run normally in a connected state, and once disconnected, they cannot continue to provide services to users.

[0008] Lack of cross-modal support: current applications mostly focus on image recognition, and other important input forms such as voice and text are not effectively integrated and utilized.

[0009] In view of the problems existing in the above-mentioned traditional AI model deployment mode, it is urgent to propose a new front-end AI inference system and method based on TensorFlow.js, which can comprehensively solve these problems through innovative technical means and optimization strategies, thereby significantly improving the intelligent level of AI inference, enhancing the security of the system, and bringing users a more high-quality, efficient and smooth user experience. SUMMARY

[0010] The purpose of the present application is to provide a browser-side AI inference system and method based on TensorFlow.js to solve the problems raised in the background art.

[0011] To achieve the above-mentioned purpose, the present application provides the following technical solution: a browser-side AI inference system based on TensorFlow.js, comprising:

[0012] A model loading and initialization module is used to trigger the model loading process after the user initiates an AI inference request or the page loading is completed, first check the local cache, if there is a corresponding model, directly load, if not, download from the server and cache to the browser local storage; after the model file is cached successfully, initialize the TensorFlow.js running environment, load the model weight and graph structure to complete the model preparation;

[0013] A device performance detection and model selection module collects multi-dimensional technical parameters of the client device in real time through an intelligent detection mechanism, dynamically classifies the device computing power based on a preset performance evaluation algorithm, and matches a lightweight model architecture for low-end devices and enables a high-performance model for high-end devices.

[0014] Preferably, it further comprises:

[0015] An input data acquisition and preprocessing module acquires real-time original data streams of various input sources through a multi-modal data access module, pre-processes the visual signals using adaptive normalization algorithm, bilinear interpolation method and color space intelligent conversion; for audio data, spectral subtraction, Hamming window framing technology and fast Fourier transform are used to obtain frequency domain feature atlas; after the text information is filtered by stop words, stem extracted and sub-word encoded, all pre-processed data are converted into floating-point three-dimensional tensor structure conforming to TensorFlow.js specification through a feature alignment engine.

[0016] Preferably, it further comprises:

[0017] The inference task scheduling and execution module encapsulates the inference task as an independent function, starts asynchronous execution through Web Worker to avoid blocking the main thread, calls TensorFlow.js in the Worker thread for model inference, uses WebGL to accelerate tensor operations, supports mixed precision calculation and batch processing optimization, and records real-time inference process indicators by the performance monitoring module.

[0018] The inference result analysis and output module analyzes the model output to extract key information, including confidence, class label, and bounding box coordinates, performs post-processing in combination with context semantics, and returns the processed structured results to the front-end application.

[0019] Preferably, it further comprises:

[0020] The result visualization and export module can accurately superimpose bounding boxes with flexible position, size, and style on the original image for image tasks, and add text labels with freely adjustable properties. For speech or text tasks, it supports key content highlighting, result copying, and exporting to various formats. It also provides rich API interfaces for third-party plugin or component integration and secondary development.

[0021] The model caching and updating maintenance module caches model files to local storage for fast loading, periodically sends model version query requests to the server, automatically downloads and replaces the old cache when a new version of the model is detected, and supports user manual switching or restoring the model version.

[0022] Preferably, it further comprises:

[0023] The performance monitoring and dynamic optimization module integrates the performance monitoring module to track and record the resource usage of the inference process in real time. When the memory usage approaches or reaches the overflow threshold or the inference time exceeds, it automatically adjusts the model precision parameters or reduces the resolution. When the device is in a low load or idle state, it actively loads more detailed and high-precision model versions to improve recognition accuracy and effectiveness.

[0024] A method for a browser-based AI inference system based on TensorFlow.js, comprising the following steps:

[0025] Model loading and initialization: When the user initiates an AI inference request or the page is loaded, check if the corresponding model exists in the local cache. If it exists, load it directly from the local cache. If it does not exist, download the model file from the server and cache it to the browser local storage. After the model file is successfully cached, initialize the TensorFlow.js runtime environment, load the model weights and graph structure to complete the model preparation.

[0026] Preferably, it further comprises the following steps:

[0027] Device performance detection and model selection: Real-time collection of multi-dimensional technical parameters of client devices through intelligent detection mechanism, including device type, operating system version, browser kernel and rendering engine version, GPU hardware support status, memory capacity and storage space; Based on the preset performance evaluation algorithm, the device computing power is dynamically classified, and the INT8 quantization compression technology is matched for low-end devices, and the high-performance model with FP16 half-precision operation is enabled for high-end devices.

[0028] Preferably, the method further comprises the following steps:

[0029] Input data collection and preprocessing: Real-time collection of various types of input source original data streams through a multi-modal data access module; For visual signals, an adaptive normalization algorithm is used to map pixel values to the [0, 1] interval, combined with a bilinear interpolation method for size standardization processing, and intelligent color space conversion is performed; For audio data, background noise is suppressed using spectral subtraction, and voiceprints are divided using Hamming window framing technology, and frequency domain feature maps are obtained by performing fast Fourier transform; Text information is filtered by stop words, stem extracted and subword encoded, and a numerical representation system that preserves semantics is constructed; All preprocessed data are dimensionally calibrated by a feature alignment engine and converted into a floating-point three-dimensional tensor structure conforming to TensorFlow.js specifications.

[0030] Preferably, the method further comprises the following steps: inference task scheduling and execution, result analysis and output, and visualization export:

[0031] Inference task scheduling and execution: encapsulate the inference task as an independent function, start asynchronous execution through Web Worker, call TensorFlow.js in the Worker thread for model inference, use WebGL to accelerate tensor operations, support mixed precision calculation and batch processing optimization, and record inference process indicators in real time by a performance monitoring module;

[0032] Inference result analysis and output: analyze the model output, extract confidence, class label, and bounding box coordinate key information, perform post-processing combined with context semantics, and return the processed structured results to the front-end application;

[0033] Result visualization and export: For image tasks, superimpose flexible bounding boxes on the original image and add text labels with freely adjustable properties; For voice or text tasks, support key content highlighting, result copying, and export to multiple formats of documents; Provide rich API interfaces for easy integration and secondary development of third-party plugins or components.

[0034] Preferably, the method further comprises the following steps: model cache update and maintenance, and performance monitoring dynamic optimization:

[0035] Model cache and update maintenance: cache model files to local storage, periodically send model version query requests to the server, detect new version models in the background and automatically download and replace old caches, support user manual switching or restoring model versions;

[0036] Performance monitoring and dynamic optimization: integrate performance monitoring modules to track and record resource usage during inference in real time, automatically adjust model precision parameters or reduce resolution when memory usage approaches or reaches the overflow threshold or inference times out, actively load more detailed and high-precision model versions to improve recognition accuracy and effect when the device is in low load or idle state.

[0037] Compared with the prior art, the beneficial effects of the present application are:

[0038] The browser-side AI inference system and method based on TensorFlow.js provided by the present application provide a complete front-end AI inference process control mechanism, realize dynamic adaptation of models based on device performance, support multiple input forms (images, voice, text, etc.), avoid main thread blocking using multi-thread scheduling, introduce GPU acceleration and mixed precision calculation mechanism to improve inference efficiency, establish model cache mechanism to improve loading speed and offline availability, equip performance monitoring and dynamic optimization functions to ensure inference stability, provide structured result output and visual display interface, realize model version control and background hot update mechanism, and improve the safety, compatibility and expandability of the system. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 The method flowchart of the present application. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solution of the present application clear, complete and more clear and understandable, the embodiments of the present application are further described in detail below in combination with the drawings. It should be understood that the specific embodiments described herein are part of the embodiments of the present application, not all embodiments, and are only used to explain the embodiments of the present application, and do not limit the embodiments of the present application. All other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0041] Embodiment one, the present application provides a technical solution: a browser-side AI inference system based on TensorFlow.js, comprising:

[0042] The model loading and initialization module is used to trigger the model loading process after the user initiates an AI inference request or the page loading is completed. First, the local cache is checked. If the corresponding model exists, it is directly loaded. If it does not exist, it is downloaded from the server and cached to the browser local storage. After the model file caching is successful, the TensorFlow.js runtime environment is initialized, the model weight and graph structure are loaded to complete the model preparation.

[0043] The device performance detection and model selection module collects multi-dimensional technical parameters of the client device in real time through an intelligent detection mechanism, dynamically classifies the device computing power based on a preset performance evaluation algorithm, and matches a lightweight model architecture for low-end devices and enables a high-performance model for high-end devices.

[0044] The input data collection and preprocessing module collects various types of input source raw data streams in real time through a multi-modal data access module. For visual signals, adaptive normalization algorithm, bilinear interpolation method and color space intelligent conversion are used for preprocessing. For audio data, spectral subtraction, Hamming window framing technology and fast Fourier transform are used to obtain frequency domain feature atlas. After stop word filtering, stem extraction and subword encoding processing, all preprocessed data are converted into floating-point three-dimensional tensor structure conforming to TensorFlow.js specification through feature alignment engine.

[0045] The inference task scheduling and execution module encapsulates the inference task as an independent function, starts asynchronous execution through Web Worker to avoid blocking the main thread, calls TensorFlow.js for model inference in the Worker thread, uses WebGL to accelerate tensor operation, supports mixed precision calculation and batch processing optimization, and records the inference process indicators in real time by the performance monitoring module.

[0046] The inference result analysis and output module analyzes the model output to extract key information, including confidence, class label, and bounding box coordinates, and performs post-processing combined with context semantics, and returns the processed structured results to the front-end application.

[0047] The result visualization and export module can accurately superimpose bounding boxes with flexible position, size and style on the original image for image tasks, and add text labels with freely set attributes. For voice or text tasks, it supports key content highlighting, result copying and exporting to various formats of documents. It also provides rich API interfaces for easy integration and secondary development of third-party plugins or components.

[0048] The model caching and updating maintenance module caches the model file to the local storage for fast loading, sends model version query requests to the server regularly, automatically downloads and replaces the old cache when a new version of the model is detected, and also supports user manual switching or restoring the model version.

[0049] The performance monitoring and dynamic tuning module integrates a performance monitoring module to track and record the resource usage of the inference process in real time. When the memory is close to or reaches the overflow threshold or the inference times out, the running model precision parameters are automatically adjusted or the resolution is reduced. When the device is in a low load or idle state, a more detailed and high-precision model version is actively loaded to improve the recognition accuracy and effect.

[0050] In embodiment two, based on embodiment one, a method of browser-side AI inference system based on TensorFlow.js is proposed, including the following steps:

[0051] S1, model loading and initialization

[0052] When the user initiates an AI inference request or the page is loaded, the system will trigger the model loading process. In this process, the system will first check whether the corresponding model already exists in the local cache. If the model is found in the local cache, the system will directly load the model from the local cache, which can improve the loading speed and reduce the waiting time.

[0053] If the corresponding model is not found in the local cache, the system will download the model file from the server. After the download is completed, the system will cache the model file in the local storage of the browser, such as IndexedDB or LocalStorage. The advantage of this is that the next time the model is needed, it can be loaded directly from the local cache without the need to download from the server again, thereby improving efficiency.

[0054] After the model file is successfully cached, the system will initialize the TensorFlow.js running environment. This step is to ensure that subsequent inference tasks can be performed in a stable and efficient environment. Then, the system will load the model's weights and graph structure. The model's weights are the parameters learned by the model during training, and the graph structure describes the model's calculation process. By loading these two parts, the system has completed the preparation of the model and can start executing subsequent inference tasks.

[0055] S2, device performance detection and model selection

[0056] The system collects multi-dimensional technical parameters of the client device in real time through an intelligent detection mechanism, including device type (PC / mobile device), operating system version, browser kernel and rendering engine version, GPU hardware support status, memory capacity and storage space, and other core indicators. Based on the preset performance evaluation algorithm, the system dynamically classifies the device computing power: for low-end devices with old architecture or limited resources, the system preferentially matches a lightweight model architecture, such as a MobileNet model using INT8 quantization compression technology, to balance inference speed and resource occupation; and for high-end devices equipped with high-performance GPUs and sufficient memory, the system enables an EfficientNet model with FP16 half-precision operation to improve feature extraction accuracy while ensuring computing efficiency.

[0057] S3, input data collection and preprocessing

[0058] The system collects raw data streams from various input sources in real time through a multi-modal data access module. For visual signals, an adaptive normalization algorithm is used to map pixel values to the [0, 1] interval, combined with a bilinear interpolation method for size standardization, and simultaneously performs intelligent conversion between RGB and BGR or HSV color spaces; for audio data, spectral subtraction is used to suppress background noise, and the voiceprint is divided into 25ms frame units through Hamming window framing technology, and fast Fourier transform is performed to obtain frequency domain feature maps; text information is processed through three processes of stop word filtering, stem extraction and sub-word encoding to build a numerical representation system that preserves semantics. All preprocessed data are dimensionally calibrated by a feature alignment engine and finally converted into a floating-point three-dimensional tensor structure conforming to TensorFlow.js specifications.

[0059] S4, inference task scheduling and execution

[0060] The system encapsulates the inference task as an independent function and starts asynchronous execution through Web Worker to avoid blocking the main thread. TensorFlow.js is called in the Worker thread to perform model inference, utilizing WebGL to accelerate tensor operations while supporting mixed precision calculation and batch processing optimization. The entire inference process is recorded by the performance monitoring module in real time, including time consumption and memory occupation.

[0061] S5, inference result analysis and output

[0062] After the model output is parsed, key information is accurately extracted, including confidence, which reflects the credibility of the model's identification results; class labels, which clearly identify the object's category; and bounding box coordinates, which accurately define the location range of the identified object in the image or data. The system will combine the context semantics to perform post-processing, such as using NMS (Non-Maximum Suppression) to filter out the best identification results and remove redundant overlapping boxes in image recognition tasks; in speech recognition, scattered speech segments are integrated into complete and coherent text content through text splicing operations. Finally, the system returns the processed structured results to the front-end application, so that UI components can successfully call and clearly display relevant information.

[0063] S6、Result visualization and export

[0064] For image tasks, the system has strong rendering capabilities, capable of accurately superimposing bounding boxes on the original image. The position, size, and style of these bounding boxes can be flexibly adjusted according to specific needs, ensuring perfect integration with image content. At the same time, clear and accurate text labels can be added to the image, with font, color, size, and other properties freely set to provide users with more rich and intuitive information display.

[0065] For speech or text tasks, the system provides a variety of practical functions. In terms of result presentation, it supports highlighting key content, allowing users to quickly focus on important information and improve information acquisition efficiency. In addition, users can easily copy the results, whether it's a single word, paragraph, or entire text content, and easily copy it to other documents or applications for further processing. Moreover, the system supports exporting results to various formats of documents, such as common Word documents and PDF documents, meeting users' needs in different scenarios.

[0066] The system also fully considers the integration needs of third-party plugins or components, providing rich API interfaces. These interfaces have high compatibility and stability, allowing third-party developers to easily call relevant system functions based on their business logic and functional needs, achieving seamless integration with their own plugins or components, thereby facilitating secondary development and expanding the system's functionality and application range.

[0067] S7、Model cache and update maintenance

[0068] The system has the function of caching model files to local storage, and the purpose of this operation is to facilitate users to quickly load models next time. In order to ensure the timely update of the model version, the system will periodically send a model version query request to the server. When the system detects that there is a new version of the model, it will automatically complete the download operation in the background and replace the original old cache. The whole process does not require user intervention. In addition, considering the special needs of users, the system also supports users to manually switch model versions or restore historical versions. In this way, it can ensure that the model is always in the latest and stable state, and meet the user's use needs in different scenarios.

[0069] S8, performance monitoring and dynamic tuning

[0070] The system internally integrates a set of precise performance monitoring modules. This module can continuously and real-time track and record various resource usage conditions in the inference process, including but not limited to CPU occupancy, memory consumption, and processing time, and other key indicators. Once the memory usage approaches or reaches the overflow threshold, or the inference process exceeds the preset time limit, the system will intelligently intervene and automatically adjust the accuracy parameters or reduce the resolution of the current running model, as an immediate response measure to ensure the stable operation of the system. When the device is in a low load or idle state, the system shows its flexibility side and will actively try to load a more detailed and high-precision model version, aiming to fully utilize the idle resources and further improve the recognition accuracy and effect of image or data processing.

[0071] Although embodiments of the present application have been shown and described, it is to be understood that various modifications, substitutions, replacements and changes can be made to these embodiments without departing from the principles and spirit of the present application, and the scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A browser-based AI inference system based on TensorFlow.js, characterized in that: include: The model loading and initialization module is used to trigger the model loading process when a user initiates an AI inference request or after the page loads. It first checks the local cache. If the corresponding model exists, it is loaded directly. If it does not exist, it is downloaded from the server and cached in the browser's local storage. After the model file is successfully cached, the TensorFlow.js runtime environment is initialized, and the model weights and graph structure are loaded to complete the model preparation. The device performance detection and model selection module collects multi-dimensional technical parameters of client devices in real time through an intelligent detection mechanism, dynamically classifies device computing power based on a preset performance evaluation algorithm, matches lightweight model architectures to low-end devices, and enables high-performance models for high-end devices.

2. The browser-based AI inference system based on TensorFlow.js according to claim 1, characterized in that: Also includes: The input data acquisition and preprocessing module collects raw data streams from various input sources in real time through the multimodal data access module. For visual signals, it uses adaptive normalization algorithm, bilinear interpolation, and intelligent color space conversion for preprocessing. For audio data, it uses spectral subtraction, Hamming window framing technology, and fast Fourier transform to obtain frequency domain feature maps. After stop word filtering, stemming, and sub-word encoding, text information is transformed into a floating-point three-dimensional tensor structure conforming to the TensorFlow.js specification through the feature alignment engine.

3. The browser-based AI inference system based on TensorFlow.js according to claim 2, characterized in that: Also includes: The inference task scheduling and execution module encapsulates inference tasks as independent functions, starts asynchronous execution through Web Worker to avoid blocking the main thread, calls TensorFlow.js for model inference in the Worker thread, uses WebGL to accelerate tensor operations, supports mixed precision calculation and batch processing optimization, and the performance monitoring module records inference process metrics in real time. The inference result parsing and output module parses the model output to extract key information, including confidence level, category label, and bounding box coordinates. It then performs post-processing based on contextual semantics and returns the processed structured results to the front-end application.

4. The browser-based AI inference system based on TensorFlow.js according to claim 3, characterized in that: Also includes: The results visualization and export module can accurately overlay bounding boxes with adjustable position, size, and style on the original image for image-based tasks, and add text labels with freely customizable attributes. For audio or text-based tasks, it supports highlighting key content, copying results, and exporting to various document formats. It also provides rich API interfaces to facilitate the integration and secondary development of third-party plugins or components. The model caching and update maintenance module caches model files to local storage for fast loading, periodically sends model version query requests to the server, and automatically downloads and replaces the old cache when a new version model is detected in the background. It also supports users to manually switch or restore the model version.

5. A browser-based AI inference system based on TensorFlow.js according to claim 4, characterized in that: Also includes: The performance monitoring and dynamic optimization module integrates a performance monitoring module to track and record resource usage during the inference process in real time. When memory is close to or reaches the overflow threshold or inference times out, it automatically adjusts the accuracy parameters of the running model or reduces the resolution. When the device is under low load or idle, it actively loads a more refined and high-precision model version to improve recognition accuracy and effect.

6. A method for a browser-based AI inference system based on TensorFlow.js as described in claim 5, characterized in that: Includes the following steps: Model loading and initialization: When a user initiates an AI inference request or after the page has finished loading, check if the corresponding model exists in the local cache; if it exists, load it directly from the local cache; if it does not exist, download the model file from the server and cache it in the browser's local storage. After the model file is successfully cached, the TensorFlow.js runtime environment is initialized, and the model weights and graph structure are loaded to complete the model preparation.

7. A method according to claim 6, characterized in that: It also includes the following steps: Device performance testing and model selection: The system collects multi-dimensional technical parameters of client devices in real time through an intelligent testing mechanism, including device type, operating system version, browser kernel and rendering engine version, GPU hardware support status, memory capacity and storage space; and dynamically classifies device computing power based on a preset performance evaluation algorithm. It matches low-end devices with a lightweight model architecture using INT8 quantization compression technology, and enables high-performance models with FP16 half-precision computing for high-end devices.

8. A method according to claim 7, characterized in that: It also includes the following steps: Input data acquisition and preprocessing: The multimodal data access module acquires raw data streams from various input sources in real time; for visual signals, an adaptive normalization algorithm is used to map pixel values ​​to the [0,1] interval, combined with bilinear interpolation for size standardization, and intelligent color space conversion is performed; for audio data, spectral subtraction is used to suppress background noise, Hamming window framing technology is used to segment the voiceprint, and fast Fourier transform is performed to obtain frequency domain feature maps; text information undergoes stop word filtering, stemming, and sub-word encoding to construct a semantically preserved numerical representation system; all preprocessed data undergoes dimensional calibration through a feature alignment engine and is transformed into a floating-point three-dimensional tensor structure conforming to the TensorFlow.js specification.

9. A method according to claim 8, characterized in that: It also includes the following steps: Inference task scheduling and execution, result parsing and output, and visualization export: Inference task scheduling and execution: Inference tasks are encapsulated as independent functions and executed asynchronously through Web Workers. TensorFlow.js is called in the Worker thread to perform model inference, WebGL is used to accelerate tensor operations, mixed precision calculation and batch processing optimization are supported, and the performance monitoring module records the inference process metrics in real time. Inference result parsing and output: Parse the model output, extract key information such as confidence score, category label, and bounding box coordinates, perform post-processing in combination with contextual semantics, and return the processed structured results to the front-end application; Results visualization and export: For image-based tasks, a flexibly adjustable bounding box is overlaid on the original image, and text labels with freely configurable attributes are added; for audio or text-based tasks, key content is highlighted, results are copied, and exported to various document formats are supported. It provides a rich set of API interfaces to facilitate the integration and secondary development of third-party plugins or components.

10. A method according to claim 9, characterized in that: It also includes the following steps: Model cache update maintenance and dynamic performance monitoring and tuning: Model caching and update maintenance: The model files are cached locally, and model version query requests are sent to the server periodically. When a new version model is detected, it is automatically downloaded in the background and replaces the old cache. Users can manually switch or restore the model version. Performance monitoring and dynamic optimization: The integrated performance monitoring module tracks and records the resource usage during the inference process in real time. When the memory is close to or reaches the overflow threshold or the inference timeout is detected, the accuracy parameters of the running model are automatically adjusted or the resolution is reduced. When the device is under low load or idle, a more refined and high-precision model version is actively loaded to improve the recognition accuracy and effect.

Images