A visual model training and testing integrated method, system, device and medium in an offline environment and false detection filtering

Abstract

The application discloses a visual model training and inference integrated and false detection filtering method, system, device and medium in an offline environment, and relates to the field of artificial intelligence. The method comprises the following steps: obtaining an offline deployment package containing initial weights; obtaining a target small visual model and a target large model with adaptive computing power through local fine-tuning compression; inputting real-time data into the small model to quickly infer and output preliminary detection results; synchronizing the preliminary results to the large model for rejudgment to filter false detection information; receiving feedback labels of the user for the final results and accumulating an incremental sample set; when the sample set meets the standard, triggering local incremental training to iteratively update the large and small models. The application balances inference efficiency and high detection accuracy in a network-free environment, and realizes offline continuous evolution of model performance.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, specifically to a method, system, device, and medium for integrating visual model training and promotion and filtering false positives in an offline environment. Background Technology

[0002] The statements in this section are provided only as background information in connection with this disclosure and may not constitute prior art.

[0003] With the rapid development of computer vision technology, visual models have been widely used in various fields such as security monitoring and industrial quality inspection. The conventional application of these models usually relies on training with large amounts of data in the cloud and then using real-time inference to complete the detection and alarm of target scenes after deployment.

[0004] However, in real-world application scenarios such as classified locations, areas without network coverage, and field engineering projects, model training and inference cannot rely on cloud or networked computing resources and must be carried out in a local offline environment. In a completely offline environment, existing technologies face the following irreconcilable technical contradictions: (1) The contradiction between accuracy in small model deployment: In offline environments, the hardware computing power of local edge computing devices is limited. In order to adapt to this computing power, only lightweight visual small models can usually be deployed. However, this inevitably sacrifices detection accuracy, resulting in a large number of false detections during the inference process (such as false alarms caused by changes in light and shadow), which greatly increases the verification burden of on-site staff.

[0005] (2) The challenge of offline deployment of large models: High-precision large models have stronger semantic understanding and fault tolerance capabilities, but their parameter scale is huge and their computing power requirements are extremely high, making it difficult to deploy and run them directly in offline edge environments. Existing technologies also lack solutions for efficient collaboration between large and small models to filter false detections locally.

[0006] (3) Lack of continuous model evolution capability: In an offline and network-free environment, the research and development unit cannot collect long-tail samples of real scenes in real time and perform secondary training. Once the visual model is deployed, it is in a "static" and solidified state. Its performance is easily affected by changes in the scene (such as seasonal changes and changes in lighting) and will continue to degrade, making it impossible to achieve continuous optimization and updates throughout its life cycle. Summary of the Invention

[0007] The purpose of this invention is to address the technical problems of high false detection rates of small visual models and difficulty in deploying large models due to limited local computing power in offline environments, and the inability of models to continuously optimize based on real-world scene samples after disconnection from the network. This invention provides a method, system, device, and medium for integrated training and inference of visual models and false detection filtering in offline environments. It adopts the technical means of "offline pre-setting and multi-level compression adaptation", "lightweight small visual model initial screening and high-precision large model collaborative re-judgment", and "threshold-based local incremental training triggered by user feedback closed loop". It achieves both high detection accuracy and low inference latency in a completely offline environment without network, and gives the model the ability to spontaneously iterate and continuously evolve using locally added samples in the offline state.

[0008] The technical solution of the present invention is as follows: A method for integrating visual model training and inference and false detection filtering in an offline environment includes: Obtain an offline deployment package pre-installed in the local offline environment. The offline deployment package includes a pre-installed training dataset, initial weights for a lightweight small visual model, and initial weights for a high-precision large model. Based on the pre-set training dataset, the lightweight visual small model and the high-precision large model are fine-tuned and compressed locally to obtain the target visual small model and the target large model adapted to the local hardware computing power. Real-time visual data collected in an offline environment is input into the target visual mini-model for rapid inference, and preliminary detection results are output. The preliminary detection results are then associated and bound with the corresponding real-time visual data and stored. The preliminary detection results and the corresponding real-time visual data are synchronously input into the target large model; the preliminary detection results are re-evaluated by the target large model to filter out false detection information, and the final detection results are output. The system visualizes the final detection results and associated real-time visual data on the user terminal, receives feedback annotations input by the user based on the final detection results, and associates and stores the feedback annotations with the corresponding sample data to continuously accumulate and build an incremental user sample set. When the number of incremental user samples reaches a set threshold, the online training process of the model is automatically triggered; the incremental user sample set is fused with the pre-set training dataset to form a comprehensive incremental training set, and an incremental training strategy is adopted to perform self-iterative updates of the target visual small model and the target large model in an offline environment based on the comprehensive incremental training set.

[0009] Furthermore, based on the pre-set training dataset, the lightweight visual small model and the high-precision large model are fine-tuned and compressed locally to obtain a target visual small model and a target large model adapted to the local hardware computing power, including: The lightweight visual small model and the high-precision large model are fine-tuned and trained locally, and at least one multi-level compression operation among quantization, distillation and pruning is performed during the training process to reduce the model parameter size and computing power consumption. Using the test dataset pre-installed in the offline deployment package, the lightweight visual small model and the high-precision large model, which have been fine-tuned and compressed, are subjected to dual threshold verification of accuracy and speed. If the dual threshold verification passes, the model that passes the verification is used as the target visual small model and the target large model; if it fails, the parameters are readjusted and fine-tuning training is performed again.

[0010] Furthermore, the preliminary detection results include target type, target location, alarm type, alarm location, and confidence level information.

[0011] Furthermore, the preliminary detection results are reviewed using the target large model to filter out false detections, and the final detection results are output, including: By utilizing semantic understanding and scene context analysis capabilities of the target large model, and combining the surrounding environment and lighting influence information in the original visual data, the alarm information in the preliminary detection results is re-evaluated one by one. Identify and filter false detection information in the preliminary detection results, and output the final detection results.

[0012] Furthermore, the system receives feedback annotations input by the user based on the final detection result, associates and stores the feedback annotations with the corresponding sample data, and continuously accumulates and constructs an incremental user sample set, including: The user terminal provides a first type of feedback function button and a second type of feedback function button on its visual interactive interface. In response to the user's triggering operation of the first type of feedback function button, the corresponding final detection result and associated real-time visual data are marked as positive samples; In response to the user's triggering operation of the second type of feedback function button, the corresponding final detection result and associated real-time visual data are marked as negative samples; The positive and negative samples are associated and stored, and continuously accumulated to form the user incremental sample set.

[0013] Furthermore, when the number of incremental user samples reaches a set threshold, the online model training process is automatically triggered, including: When the number of positive samples in the user incremental sample set is greater than or equal to a first preset threshold, and the number of negative samples is greater than or equal to a second preset threshold, the online training process of the model is automatically triggered. The step of using an incremental training strategy based on the comprehensive incremental training set to perform self-iterative updates of the target visual small model and the target large model in an offline environment includes: Based on the comprehensive incremental training set, the target visual small model and the target large model are fine-tuned online respectively. During fine-tuning training, the core parameters of the optimized model are retained, and only the weight parameters related to the newly added samples in the comprehensive incremental training set are updated to complete the self-iterative update.

[0014] Furthermore, after performing offline iterative updates on the target visual small model and the target large model respectively, the method further includes: The detection accuracy of the updated target visual small model and the target large model is verified using the test dataset pre-installed in the offline deployment package; If the improved detection accuracy after verification reaches the set accuracy threshold, the original target visual small model and the target large model will be automatically replaced and put into operation. If the improved detection accuracy after verification does not reach the set accuracy threshold, the original target visual small model and target large model are retained, and the self-iterative update is triggered again after the number of user incremental sample sets accumulates to the set threshold.

[0015] This invention also proposes an offline visual model training and inference integrated system and false detection filtering system, comprising: A pre-configured module is used to obtain an offline deployment package pre-configured in a local offline environment. The offline deployment package includes a pre-configured training dataset, initial weights of a lightweight small visual model, and initial weights of a high-precision large model. The adaptation and optimization module is used to perform fine-tuning training and model compression operations on the lightweight visual small model and the high-precision large model locally based on the pre-set training dataset, so as to obtain the target visual small model and the target large model adapted to the local hardware computing power. The preliminary inference module is used to input real-time visual data collected in the offline environment into the target visual mini-model for rapid inference, output preliminary detection results, and associate and bind the preliminary detection results with the corresponding real-time visual data and store them. The collaborative re-judgment module is used to synchronously input the associated and bound preliminary detection results and the corresponding real-time visual data into the target large model; the target large model re-judges the preliminary detection results to filter false detection information and outputs the final detection results. The feedback interaction module is used to visualize the final detection result and associated real-time visual data on the user terminal, receive feedback annotations input by the user based on the final detection result, and associate and store the feedback annotations with the corresponding sample data to continuously accumulate and build an incremental user sample set. The self-iterative update module is used to automatically trigger the online training process of the model when the number of user incremental samples reaches a set threshold; it merges the user incremental sample set with the pre-set training dataset to form a comprehensive incremental training set, and uses an incremental training strategy to perform self-iterative updates of the target visual small model and the target large model in an offline environment based on the comprehensive incremental training set.

[0016] The present invention also proposes an electronic device, comprising: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor executes the instructions stored in the memory to perform the method described above.

[0017] The present invention also proposes a computer-readable storage medium for storing instructions that, when executed, cause the method described above to be implemented.

[0018] Compared with existing technologies, the advantages of this invention are: 1. Balancing inference efficiency and detection accuracy. An innovative collaborative processing architecture is constructed, combining a lightweight small model for rapid inference with a high-precision large model for accurate re-judgment. The small model handles millisecond-level initial screening of all data, while the large model performs targeted error correction only on the initial screening alarm data in conjunction with the scene context. This solves the problem of high false detection rates when the small model runs alone, and overcomes the offline computing power bottleneck caused by global inference of the large model, achieving a dual guarantee of efficiency and accuracy under demanding hardware conditions.

[0019] 2. Overcoming the barriers to continuous iterative evolution in offline environments. A user interaction feedback mechanism and a threshold-triggered incremental training strategy were designed. Without any external network connection, real business feedback can be transformed into training materials (positive and negative samples). By freezing core parameters and updating only the weights of newly added samples through incremental learning, a fully self-contained loop of "inference-feedback-fine-tuning-deployment" is completed locally, effectively addressing performance degradation caused by scene drift.

[0020] 3. Highly adaptable to edge offline hardware. From the import of the "cloud offline deployment package" at the front end to the localized implementation of multi-level model compression such as quantization, distillation, and pruning, as well as strict dual threshold admission checks for accuracy and speed, the system solution ensures that the target model can be accurately adapted and run efficiently on any limited computing power platform.

[0021] 4. Easy to operate and reduces maintenance costs. The "one-click annotation" design of the system terminal's visual interface significantly lowers the barrier to sample annotation for non-AI professional staff, enabling the rapid and low-cost accumulation of high-quality real business data and continuously nourishing the model's self-iterative lifecycle. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0023] Figure 1 This is a flowchart illustrating a method for integrating visual model training and inference and filtering in an offline environment, as provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0024] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0025] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0026] Example 1 This embodiment provides a method for integrating visual model training and inversion and false detection filtering in an offline environment. It is applicable to local offline environments (such as sensitive locations without network coverage, classified computer rooms, etc.). Please refer to [link to relevant documentation]. Figure 1The method specifically includes the following steps: Step S1: Obtain the offline deployment package pre-installed in the local offline environment. The offline deployment package includes a pre-installed training dataset, initial weights of a lightweight small visual model, and initial weights of a high-precision large model. As a preferred implementation, to enable various business operations in a completely offline local environment, an offline pre-configuration stage is included before step S1: In a networked environment, visual data highly similar to the target application scenario (such as abnormal behavior detection of people in sensitive locations) is collected in advance. For example, 10,000 visual data points of similar scenarios are collected and manually annotated (the annotation content includes information such as target type, target location, and alarm type). 8,000 of these are used as the pre-configured training dataset, and the remaining 2,000 are used as the test dataset. Simultaneously, an appropriate visual model is selected as the base model on the networked end. For example, the MobileNet series is selected as a lightweight visual small model, and BERT-Base (specifically, in the vision domain, its derived visual Transformer large model, such as ViT, etc.) is selected as a high-precision large model. Based on the pre-configured training dataset generated by the above collection and annotation, transfer learning is performed on the selected lightweight visual small model and the high-precision large model to obtain the initial weights of the model adapted to the target scenario. Subsequently, the pre-set training dataset, test dataset, initial weights, and a complete set of offline dependency libraries are packaged together to generate the "offline deployment package," which is then migrated and copied to a local offline server in a sensitive location via physical media (such as a secure hard drive) to complete the pre-setting of basic resources.

[0027] Step S2: Based on the pre-set training dataset, perform fine-tuning training and model compression operations on the lightweight visual small model and the high-precision large model locally to obtain the target visual small model and the target large model adapted to the local hardware computing power. In this embodiment, specifically, step S2 includes a local adaptation optimization stage: The lightweight visual small model and the high-precision large model are fine-tuned and trained locally, respectively. During training, at least one multi-level compression operation among quantization, distillation, and pruning is performed to reduce the model parameter size and computational consumption. In this embodiment, the model is pruned (preferably with an initial pruning rate of 20%) to remove redundant network connections; knowledge distillation is used to guide the learning of the lightweight visual small model with the high-precision large model as the teacher model; and quantization operations (such as FP32 to INT8 conversion) are used to ensure that the compressed model is fully adapted to the limited local hardware computing power.

[0028] After training, the lightweight visual small model and the high-precision large model, which have been fine-tuned and compressed, are subjected to dual threshold verification of accuracy and speed using the test dataset pre-installed in the offline deployment package. In this embodiment, an accuracy threshold for detection accuracy is set (e.g., ≥90% or ≥92%), and a speed threshold for inference latency is set (e.g., ≤500ms or more strictly ≤100ms).

[0029] If the dual threshold verification passes, the verified model is used as the officially deployed target visual small model and target large model (i.e., forming the inference core module); if it fails, the hyperparameters are readjusted (e.g., the pruning rate is reduced) and fine-tuning training is performed again until the model performance and local computing power reach a balance.

[0030] Step S3: Input the real-time visual data collected in the offline environment into the target visual mini-model for rapid inference, output the preliminary detection results, and associate and bind the preliminary detection results with the corresponding real-time visual data and store them; In this step, the local preliminary inference stage begins. Real-time visual data collected by on-site cameras and other sensing devices is streamed and input into the aforementioned optimized target visual mini-model. Due to its lightweight design and extremely low computational consumption, the target visual mini-model can perform millisecond-level rapid inference on the real-time visual data, outputting preliminary detection results. These preliminary detection results specifically include: target type, target location, alarm type (such as "area intrusion," "person falling," or other abnormal behavior), alarm level, alarm location, and confidence level information.

[0031] The system associates and binds the detection information output by the small model with the corresponding raw visual data (such as the current frame image) and stores it on a local server, providing detailed raw backtracking data for subsequent large models. This mechanism solves the problem that small models running alone usually sacrifice detection accuracy, resulting in a large number of false alarms.

[0032] Step S4: Synchronously input the associated preliminary detection results and the corresponding real-time visual data into the target large model; use the target large model to re-evaluate the preliminary detection results to filter out false detection information, and output the final detection results; Specifically, this step is the small-to-large model collaborative review and filtering stage for false detections: The target large model leverages its powerful semantic understanding and scene context analysis capabilities, combined with information on the surrounding environment and lighting effects in the original visual data, to re-evaluate the alarm information in the preliminary detection results one by one. From a working principle perspective, small models typically make coarse judgments based solely on local pixel features (prone to false detections due to sudden changes in lighting or occlusion by similar objects); while the target large model possesses a deeper receptive field and a self-attention mechanism, enabling it to extract global contextual dependencies in the image and determine the logical plausibility of local targets within their surrounding environment (e.g., identifying sudden changes in lighting rather than actual intrusion, or determining whether the fallen form is influenced by surrounding objects).

[0033] Therefore, the target large model can accurately identify and filter false detection information in the preliminary detection results and output the final detection results (accurate detection results), thus perfectly realizing the synergistic technical effect of "small model rapid initial screening (ensuring inference efficiency) + large model accurate error correction and verification (ensuring detection accuracy)".

[0034] Step S5: Visually display the final detection result and associated real-time visual data on the user terminal, receive feedback annotations input by the user based on the final detection result, associate and store the feedback annotations with the corresponding sample data, and continuously accumulate and build an incremental user sample set. Specifically, this step involves constructing a closed-loop mechanism for user sample feedback and active learning: The user terminal (such as the monitoring screen in the duty room) displays the final detection information and original visual image output by the collaborative output of the small and large models on its visual interactive interface. The interface provides a first type of feedback function button (such as the "Confirm Valid" button) and a second type of feedback function button (such as the "Mark False Detection" button).

[0035] In response to a user's (such as a security duty officer) triggering the first type of feedback function button (confirming validity) based on the actual scenario, the system marks the corresponding final detection result and associated real-time visual data as positive samples; in response to a user's triggering the second type of feedback function button (marking false detection), the system marks the corresponding result and data as negative samples.

[0036] The system associates and stores the aforementioned positive and negative samples with the original visual data, continuously accumulating them to form the user incremental sample set. This design significantly reduces the operational threshold for local personnel to perform sample annotation, facilitating the rapid collection of long-tail data from real-world scenes in offline environments.

[0037] Step S6: When the number of user incremental samples reaches a set threshold, the online training process of the model is automatically triggered; the user incremental sample set is fused with the pre-set training dataset to form a comprehensive incremental training set, and an incremental training strategy is adopted to perform self-iterative updates of the target visual small model and the target large model in an offline environment based on the comprehensive incremental training set.

[0038] This step is the core mechanism for enabling the model to "continuously evolve" in an offline environment. Specifically, it includes: The system monitors the amount of data in the incremental user sample set in real time. When the number of positive samples is greater than or equal to a first set threshold (e.g., ≥200 positive samples) and the number of negative samples is greater than or equal to a second set threshold (e.g., ≥50 negative samples), the system automatically triggers the online model training process locally.

[0039] The newly accumulated user incremental sample set is merged with the original offline dataset preset in step S101 to form a comprehensive incremental training set. Based on this comprehensive training set, the target visual small model and the target large model are fine-tuned online respectively.

[0040] It is important to explain the principle of its internal incremental training strategy: In order to quickly complete training on local hardware with limited computing power and prevent the model from "catastrophic forgetting" (i.e. learning new scenes but forgetting old scenes), during the fine-tuning training process, the system retains and freezes the optimized core network backbone parameters of the model, and only updates (or adds lightweight network branches) the weight parameters related to the new samples in the comprehensive incremental training set (this mechanism can be implemented using low-rank fine-tuning techniques such as LoRA or Adapter technology), thereby greatly shortening the local training time and reducing computing power consumption.

[0041] Furthermore, after completing the self-iterative update, the system uses the test dataset in the pre-deployment package to verify the detection accuracy of the updated model: if the improved detection accuracy reaches the set accuracy threshold (e.g., an improvement of ≥2% or ≥3%), the system automatically replaces the original model and goes online; if the accuracy improvement does not reach the threshold, the original model is retained without replacement, and the system silently waits for the user incremental sample set to accumulate again to reach the quantity threshold before triggering the next round of iteration.

[0042] In a practical application case, taking the abnormal behavior detection in a sensitive location as an example, actual tests demonstrate that, using the method of this invention, without any external network support, after five rounds of spontaneous sample accumulation and iterative training, the abnormal behavior detection accuracy of the location increased from 85% at the initial deployment and stabilized at 96%. Simultaneously, due to adaptive optimization, the collaborative inference latency also decreased from 98ms to 93ms. This solution completely solves the persistent problems of local small models being prone to false detections, large models being difficult to deploy, and offline scenarios being unable to continuously optimize, greatly improving the overall efficiency of security duty.

[0043] Example 2 Based on the same inventive concept as Embodiment 1 above, this embodiment provides a visual model training and inference integrated system and false detection filtering system in an offline environment. The system is a virtual device / software functional module architecture that can run on a local offline computing device. Specifically, the system includes: A pre-configured module is used to obtain an offline deployment package pre-configured in a local offline environment. The offline deployment package includes a pre-configured training dataset, initial weights of a lightweight small visual model, and initial weights of a high-precision large model. The adaptation and optimization module is used to perform fine-tuning training and model compression operations on the lightweight visual small model and the high-precision large model locally based on the pre-set training dataset, so as to obtain the target visual small model and the target large model adapted to the local hardware computing power. The preliminary inference module is used to input real-time visual data collected in the offline environment into the target visual mini-model for rapid inference, output preliminary detection results, and associate and bind the preliminary detection results with the corresponding real-time visual data and store them. The collaborative re-judgment module is used to synchronously input the associated and bound preliminary detection results and the corresponding real-time visual data into the target large model; the target large model re-judges the preliminary detection results to filter false detection information and outputs the final detection results. The feedback interaction module is used to visualize the final detection result and associated real-time visual data on the user terminal, receive feedback annotations input by the user based on the final detection result, and associate and store the feedback annotations with the corresponding sample data to continuously accumulate and build an incremental user sample set. The self-iterative update module is used to automatically trigger the online training process of the model when the number of user incremental samples reaches a set threshold; it merges the user incremental sample set with the pre-set training dataset to form a comprehensive incremental training set, and uses an incremental training strategy to perform self-iterative updates of the target visual small model and the target large model in an offline environment based on the comprehensive incremental training set.

[0044] The specific working logic, implementation details, and beneficial technical effects of each module in the system provided in this embodiment correspond one-to-one with steps S1 to S6 in the offline visual model training and induction integration and false detection filtering method described in Embodiment 1. For the sake of brevity, these will not be elaborated here.

[0045] Based on the same technical concept, embodiments of the present invention also provide an electronic device that can implement the offline visual model training and inference integration and false detection filtering method provided in the above embodiments of the present invention. In one embodiment, the electronic device can be a server, a terminal device, or other electronic devices. Figure 2 As shown, the electronic device may include: At least one processor and a memory connected to the at least one processor. In this embodiment of the invention, the specific connection medium between the processor and the memory is not limited. Figure 2 The example used is the connection between the processor and memory via a bus. The bus... Figure 2 The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. Buses can be divided into address buses, data buses, control buses, etc., but for ease of representation, [the specific bus type is not shown here]. Figure 2 The processor is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, a processor can also be called a controller; there are no restrictions on the name.

[0046] In this embodiment of the invention, the memory stores instructions executable by at least one processor. By executing the instructions stored in the memory, the at least one processor can perform the aforementioned method for integrating visual model training and inference in an offline environment, along with false detection filtering. The processor can implement... Figure 2 The functions of each module in the device shown.

[0047] The processor is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory and calling data stored in memory, it can monitor the device's various functions and process data, thereby enabling overall monitoring of the device.

[0048] In an alternative design, the processor may include one or more processing units. The processor may integrate an application processor and a modem processor, wherein the application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may also not be integrated into the processor. In some embodiments, the processor and memory may be implemented on the same chip; in some embodiments, they may also be implemented separately on separate chips.

[0049] The processor can be a general-purpose processor, such as a CPU, digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the offline visual model training and induction integration and false detection filtering method disclosed in the embodiments of this invention can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0050] Memory, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory can include at least one type of storage medium, such as flash memory, hard disk, multimedia cards, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), and electrically erasable programmable read-only memory (EPROM). Only memory (EEPROM), magnetic storage, magnetic disks, optical disks, etc. A memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited thereto. The memory in embodiments of this invention can also be a circuit or any other device capable of performing storage functions for storing program instructions and / or data.

[0051] By designing and programming the processor, the code corresponding to the offline visual model training and induction integration and false detection filtering method described in the foregoing embodiments can be embedded into the chip, enabling the chip to execute the steps of the method described in the foregoing embodiments during runtime. How to design and program the processor is a technique well-known to those skilled in the art and will not be elaborated upon here.

[0052] Based on the same inventive concept, embodiments of the present invention also provide a storage medium storing computer instructions, which, when executed on a computer, cause the computer to perform the aforementioned method for integrating visual model training and inference and filtering false detections in an offline environment.

[0053] In some alternative embodiments, the present invention also provides a method for integrating visual model training and induction and filtering in an offline environment, which can also be implemented as a program product including program code. When the program product is run on a device, the program code is used to cause the control device to perform the steps in the method for integrating visual model training and induction and filtering in an offline environment according to various exemplary embodiments of the present invention as described above.

[0054] It should be noted that although several units or sub-units of the apparatus have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the invention, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units. Furthermore, although the operation of the method of the invention is described in a specific order in the drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0055] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can be implemented in one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs) containing computer-usable program code. The form of a computer program product implemented on ROM, optical memory, etc.

[0056] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a server, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0057] Program code for performing the operations of this invention can be written using any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0058] In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0059] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0060] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0061] Example 3 To further illustrate the practical application effect of the present invention, this embodiment takes the scenario of detecting abnormal human behavior in a sensitive place as an example to explain in detail the specific implementation process of the solution.

[0062] This sensitive location is a local offline environment without network coverage. The business requirement is to detect abnormal human behavior (such as area intrusion, falls, etc.) using a visual model. The scenario requires the final deployed model to have a detection accuracy of ≥95%, inference latency of ≤100ms, and must support local self-iterative optimization based on samples provided by on-duty personnel. The specific implementation steps for this case are as follows: 1) Offline Pre-deployment Stage: In a networked environment, collect 10,000 visual data points from similar scenarios, manually annotate them, and use 8,000 of them as a pre-deployed training dataset and 2,000 as a test dataset; select appropriate lightweight visual small models (such as MobileNet) and high-precision large models (such as BERT-Base) and obtain their initial weights; package the above-mentioned pre-deployed training dataset, test dataset, initial weights and the complete set of offline dependency libraries into an offline deployment package, and migrate it to the local offline server in the sensitive location.

[0063] 2) Local adaptation and optimization stage: Based on the pre-set training dataset, the lightweight visual small model and the high-precision large model are fine-tuned and trained locally, and multi-level compression operations are implemented during the training process (in this case, the initial pruning rate is set to 20%, supplemented by distillation compression and quantization); after training, the accuracy and speed are verified using the test dataset (in this case, the accuracy threshold is set to 92% and the inference latency is 100ms). After the verification is passed, the target visual small model and the target large model adapted to the local hardware computing power are obtained (i.e., forming the inference core module).

[0064] 3) Local Inference Stage: The real-time visual data collected by the on-site camera is input into the target visual mini-model for rapid inference, and the preliminary detection results (including alarm type, alarm location, confidence level, etc.) are quickly output. The preliminary detection results are then associated and bound with the corresponding original real-time visual data and stored.

[0065] 4) Collaborative review stage: The preliminary detection results after association and binding and the corresponding real-time visual data are synchronously input into the target large model; the target large model reviews the preliminary detection results in combination with the scene context of the sensitive place (such as the surrounding environment, lighting effects, etc.), filters out false detection information, and finally outputs accurate final detection results.

[0066] 5) User Sample Feedback Stage: The final detection results are displayed visually on the user's terminal. Users review the detection results through the visual interface, clicking the first type of feedback button (e.g., "Confirm Valid") for accurate detection results and the second type of feedback button (e.g., "Mark False Detection") for false detection results. The system receives the feedback labels, marks the associated data of "Confirm Valid" as positive samples, and marks the associated data of "Mark False Detection" as negative samples, automatically performs association and storage, and continuously accumulates and builds an incremental user sample set.

[0067] 6) Iterative optimization phase: When the number of user incremental samples meets the set trigger threshold (in this case, it is set as: ≥200 positive samples and ≥50 negative samples), the local model online training process is automatically triggered; the user incremental sample set is merged with the original pre-set training dataset to form a comprehensive incremental training set, and the incremental training strategy is used to perform self-iterative updates (fine-tuning training) of the target visual small model and the target large model in the offline environment; after training, the accuracy of the updated model is verified. If the accuracy improvement reaches the set accuracy threshold (i.e., accuracy improvement ≥2%), the original target visual small model and target large model are automatically replaced and put online; otherwise, the original model is retained and waits for the next accumulation of incremental samples.

[0068] Actual test results verification: Real-world testing demonstrates that, using the method provided by this invention, the abnormal behavior detection accuracy of this sensitive location jumped from 85% to 96% after five rounds of local closed-loop self-iterative training without any network support, while the collaborative inference latency further decreased from 98ms to 93ms. These metrics not only perfectly meet the scenario requirements (accuracy ≥95%, latency ≤100ms) but also completely solve the problem of model performance degradation in offline conditions, effectively improving the work efficiency of on-duty personnel.

[0069] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

[0070] This background section is provided to generally present the context of the invention. The work of the currently named inventors, the work to the extent described in this background section, and aspects of this section that did not constitute prior art at the time of application are neither expressly nor impliedly acknowledged as prior art to the invention.

Claims

1. A method for integrating visual model training and inference and false detection filtering in an offline environment, characterized in that, include: Obtain an offline deployment package pre-installed in the local offline environment. The offline deployment package includes a pre-installed training dataset, initial weights for a lightweight small visual model, and initial weights for a high-precision large model. Based on the pre-set training dataset, the lightweight visual small model and the high-precision large model are fine-tuned and compressed locally to obtain the target visual small model and the target large model adapted to the local hardware computing power. Real-time visual data collected in an offline environment is input into the target visual mini-model for rapid inference, and preliminary detection results are output. The preliminary detection results are then associated and bound with the corresponding real-time visual data and stored. The associated preliminary detection results and the corresponding real-time visual data are synchronously input into the target large model; The preliminary detection results are re-evaluated using the target large model to filter out false detections, and the final detection results are output. The system visualizes the final detection results and associated real-time visual data on the user terminal, receives feedback annotations input by the user based on the final detection results, and associates and stores the feedback annotations with the corresponding sample data to continuously accumulate and build an incremental user sample set. When the number of incremental user samples reaches a set threshold, the online model training process is automatically triggered. The user incremental sample set is merged with the pre-set training dataset to form a comprehensive incremental training set. Based on the comprehensive incremental training set, an incremental training strategy is used to perform self-iterative updates of the target visual small model and the target large model in an offline environment.

2. The method for integrating visual model training and inference and false detection filtering in an offline environment according to claim 1, characterized in that, Based on the pre-set training dataset, the lightweight visual small model and the high-precision large model are fine-tuned and compressed locally to obtain target visual small models and target large models adapted to local hardware computing power, including: The lightweight visual small model and the high-precision large model are fine-tuned and trained locally, and at least one multi-level compression operation among quantization, distillation and pruning is performed during the training process to reduce the model parameter size and computing power consumption. Using the test dataset pre-installed in the offline deployment package, the lightweight visual small model and the high-precision large model, which have been fine-tuned and compressed, are subjected to dual threshold verification of accuracy and speed. If the dual threshold verification passes, the model that passes the verification is used as the target visual small model and the target large model; if it fails, the parameters are readjusted and fine-tuning training is performed again.

3. The method for integrating visual model training and inference and false detection filtering in an offline environment according to claim 1, characterized in that, Preliminary detection results include target type, target location, alarm type, alarm location, and confidence level information.

4. The method for integrating visual model training and inference and false detection filtering in an offline environment according to claim 3, characterized in that, The preliminary detection results are reviewed using the target large model to filter out false positives, and the final detection results are output, including: By utilizing semantic understanding and scene context analysis capabilities of the target large model, and combining the surrounding environment and lighting influence information in the original visual data, the alarm information in the preliminary detection results is re-evaluated one by one. Identify and filter false detection information in the preliminary detection results, and output the final detection results.

5. The method for integrating visual model training and inference and false detection filtering in an offline environment according to claim 1, characterized in that, Receive feedback annotations input by the user based on the final detection result, associate and store the feedback annotations with the corresponding sample data, and continuously accumulate and construct an incremental user sample set, including: The user terminal provides a first type of feedback function button and a second type of feedback function button on its visual interactive interface. In response to the user's triggering operation of the first type of feedback function button, the corresponding final detection result and associated real-time visual data are marked as positive samples; In response to the user's triggering operation of the second type of feedback function button, the corresponding final detection result and associated real-time visual data are marked as negative samples; The positive and negative samples are associated and stored, and continuously accumulated to form the user incremental sample set.

6. The method for integrating visual model training and inference and false detection filtering in an offline environment according to claim 5, characterized in that, When the number of incremental user samples reaches a set threshold, the online model training process is automatically triggered, including: When the number of positive samples in the user incremental sample set is greater than or equal to a first preset threshold, and the number of negative samples is greater than or equal to a second preset threshold, the online training process of the model is automatically triggered. The step of using an incremental training strategy based on the comprehensive incremental training set to perform self-iterative updates of the target visual small model and the target large model in an offline environment includes: Based on the comprehensive incremental training set, the target visual small model and the target large model are fine-tuned online respectively. During fine-tuning training, the core parameters of the optimized model are retained, and only the weight parameters related to the newly added samples in the comprehensive incremental training set are updated to complete the self-iterative update.

7. The method for integrating visual model training and inference and false detection filtering in an offline environment according to claim 1, characterized in that, After performing offline iterative updates on the target visual small model and the target large model respectively, the method further includes: The detection accuracy of the updated target visual small model and the target large model is verified using the test dataset pre-installed in the offline deployment package; If the improved detection accuracy after verification reaches the set accuracy threshold, the original target visual small model and the target large model will be automatically replaced and put into operation. If the improved detection accuracy after verification does not reach the set accuracy threshold, the original target visual small model and target large model are retained, and the self-iterative update is triggered again after the number of user incremental sample sets accumulates to the set threshold.

8. A visual model training and inference integrated system with false detection filtering in an offline environment, characterized in that, include: A pre-configured module is used to obtain an offline deployment package pre-configured in a local offline environment. The offline deployment package includes a pre-configured training dataset, initial weights of a lightweight small visual model, and initial weights of a high-precision large model. The adaptation and optimization module is used to perform fine-tuning training and model compression operations on the lightweight visual small model and the high-precision large model locally based on the pre-set training dataset, so as to obtain the target visual small model and the target large model adapted to the local hardware computing power. The preliminary inference module is used to input real-time visual data collected in the offline environment into the target visual mini-model for rapid inference, output preliminary detection results, and associate and bind the preliminary detection results with the corresponding real-time visual data and store them. The collaborative review module is used to synchronously input the associated preliminary detection results and the corresponding real-time visual data into the target large model; The preliminary detection results are re-evaluated using the target large model to filter out false detections, and the final detection results are output. The feedback interaction module is used to visualize the final detection result and associated real-time visual data on the user terminal, receive feedback annotations input by the user based on the final detection result, and associate and store the feedback annotations with the corresponding sample data to continuously accumulate and build an incremental user sample set. The self-iterative update module is used to automatically trigger the online training process of the model when the number of user incremental samples reaches a set threshold; it merges the user incremental sample set with the pre-set training dataset to form a comprehensive incremental training set, and uses an incremental training strategy to perform self-iterative updates of the target visual small model and the target large model in an offline environment based on the comprehensive incremental training set.

9. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which executes the instructions stored in the memory to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store instructions that, when executed, cause the method as described in any one of claims 1-7 to be implemented.

Images