Model construction method and target detection method

Abstract

This application relates to the field of machine learning technology, and provides a model building method and an object detection method. The model building method includes: constructing a lightweight initial detection model by fusing a feature pyramid network, a feature scale fusion network, a high-speed feature super-resolution module, and a triangular detection head network. By sparsely training the initial detection model using a pre-set training sample set, an object detection model is obtained, further simplifying the number of parameters in the object detection model. This makes the object detection model suitable for low-performance monitoring equipment and can easily adapt to various complex online or offline examination environments.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, and particularly relates to a model construction method and a target detection method. BACKGROUND

[0002] With the rapid development of deep learning technology and the improvement of computer hardware performance, target detection models based on deep learning have been widely researched and applied in the invigilation system. However, the existing target detection models have problems such as low accuracy, large amount of calculation, and inability to monitor in real time, and cannot meet the invigilation requirements of online or offline examination room environments with various complex situations (such as the messy examination environment of online examination rooms, the low-performance monitoring devices that cannot provide the computing power required by the existing target detection models, the bright or dark lighting arrangement of the examination room, etc.), thereby leading to a small applicable range of the existing target detection models. SUMMARY

[0003] Therefore, it is necessary to provide a model construction method and a target detection method to solve the problem of a small applicable range in the existing target detection scheme.

[0004] A first aspect of the embodiment of the present application provides a model construction method applied to an electronic device, and the method comprises the following steps: fusing a feature pyramid network, a feature scale fusion network, a high-speed feature super-resolution module, and a triplex detection head network to construct an initial detection model; and performing sparse training on the initial detection model by using a preset training sample set, so that the initial detection model predicts target objects in each preset training sample in the preset training sample set, and obtains a target detection model.

[0005] A second aspect of the embodiment of the present application provides a target detection method based on the target detection model provided in the first aspect, and the method is applied to a monitoring device, and the method comprises the following steps: collecting a monitoring video, and performing preprocessing on the monitoring video to obtain a to-be-detected image; and performing target detection on the to-be-detected image by using the target detection model to determine target objects in the to-be-detected image.

[0006] A third aspect of the embodiment of the present application provides a model construction device, and the model construction device comprises the following components: a design module configured to fuse a feature pyramid network, a feature scale fusion network, a high-speed feature super-resolution module, and a triplex detection head network to construct an initial detection model; and a training module configured to perform sparse training on the initial detection model by using a preset training sample set, so that the initial detection model predicts target objects in each preset training sample in the preset training sample set, and obtains a target detection model.

[0007] The fourth aspect of the embodiment of the present application provides a target detection device, the target detection device comprises: a collection module, configured to collect a monitoring video, and pre-process the monitoring video to obtain a to-be-detected image; a detection module, configured to perform target detection on the to-be-detected image by using the target detection model, and determine a target object in the to-be-detected image.

[0008] The fifth aspect of the embodiment of the present application provides an electronic device, comprising a memory, a processor, and computer readable instructions stored in the memory and executable on the processor, and the processor executes the computer readable instructions to implement the above-mentioned model construction method.

[0009] The sixth aspect of the embodiment of the present application provides a monitoring device, comprising a memory, a processor, and computer readable instructions stored in the memory and executable on the processor, and the processor executes the computer readable instructions to implement the above-mentioned target detection method.

[0010] The seventh aspect of the embodiment of the present application provides one or more computer readable storage media storing computer readable instructions, and the computer readable instructions are executed by one or more processors to make the one or more processors execute the above-mentioned model construction method or target detection method.

[0011] In the model construction method provided by the embodiment of the present application, the electronic device constructs a lightweight initial detection model by fusing a feature pyramid network, a feature scale fusion network, a high-speed feature super-resolution module, and a ternary detection head network, improves the detection accuracy of small target objects, and further simplifies the parameter amount of the target detection model by using a preset training sample set to perform sparse training on the initial detection model, so that the target detection model is suitable for low-performance monitoring devices and can easily adapt to online or offline examination environment of various complex situations. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0013] Figure 1 is the application environment diagram of the model construction method or the target detection method provided by the embodiment of the present application;

[0014] Figure 2 is the implementation flowchart of the model construction method provided by the embodiment of the present application;

[0015] Figure 3 FIG. 1 is a structural example diagram of a target detection model provided by an embodiment of the present application;

[0016] Figure 4 FIG. 2 is a structural example diagram of a high-speed feature super-resolution module provided by an embodiment of the present application;

[0017] Figure 5 FIG. 3 is an example diagram of an anchor box and an anchor point provided by an embodiment of the present application;

[0018] Figure 6 FIG. 4 is an example diagram of an implementation flow of an anchor box configuration method provided by an embodiment of the present application;

[0019] Figure 7 FIG. 5 is an example diagram of an implementation flow of a data enhancement method provided by an embodiment of the present application;

[0020] Figure 8 FIG. 6 is an example diagram of an implementation flow of a sample set manufacturing method provided by an embodiment of the present application;

[0021] Figure 9 FIG. 7 is an example diagram of redundant features in a sparse training provided by an embodiment of the present application;

[0022] Figure 10 FIG. 8 is an example diagram of an implementation flow of a model pruning method provided by an embodiment of the present application;

[0023] Figure 11 FIG. 9 is an example diagram of a model pruning result provided by an embodiment of the present application;

[0024] Figure 12 FIG. 10 is an example diagram of a center point offset of model quantization provided by an embodiment of the present application;

[0025] Figure 13 FIG. 11 is an example diagram of an implementation flow of a model quantization method provided by an embodiment of the present application;

[0026] Figure 14 FIG. 12 is another example diagram of an implementation flow of a model quantization method provided by an embodiment of the present application;

[0027] Figure 15 FIG. 13 is yet another example diagram of an implementation flow of a model quantization method provided by an embodiment of the present application;

[0028] Figure 16 FIG. 14 is still another example diagram of an implementation flow of a model quantization method provided by an embodiment of the present application;

[0029] Figure 17 FIG. 15 is an example diagram of an implementation flow of a model construction method provided by an embodiment of the present application;

[0030] Figure 18 FIG. 16 is a comparison diagram of model precision provided by an embodiment of the present application;

[0031] Figure 19 This is a flowchart illustrating the implementation of the target detection method provided in the embodiments of this application;

[0032] Figure 20 This is an example flowchart of the implementation process of the target detection method provided in the embodiments of this application;

[0033] Figure 21 This is a schematic diagram of the structure of the model detection device provided in the embodiments of this application;

[0034] Figure 22 This is a schematic diagram of the target detection device provided in the embodiments of this application;

[0035] Figure 23 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0036] Figure 24 This is a schematic diagram of the structure of the monitoring device provided in the embodiments of this application. Detailed Implementation

[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] The following is a list of Chinese and English terms used in the embodiments of this application.

[0039] Feature Pyramid Network (FPN) is a neural network architecture based on feature pyramids. It constructs multi-layered feature pyramids using spatial pyramid pooling and fuses the high-resolution feature maps generated during upsampling with those generated during downsampling to produce feature maps with different semantic information at different scales. By fusing information between different feature layers, FPN incorporates more contextual information into the model, effectively improving its receptive field and semantic information representation capabilities, resulting in better robustness and accuracy when handling multi-scale objects.

[0040] The Path Aggregation Network (PAN) is a neural network model used for semantic segmentation tasks. It effectively addresses the problem of varying object sizes by constructing a multi-level feature pyramid and a deformable convolutional module. The PAN network first extracts image features through a backbone network, then concatenates these features using a series of feature fusion modules. During the fusion process, the PAN network employs an adaptive scale fusion strategy, enabling it to handle images with significant differences in object size.

[0041] The Fast Feature Super-Resolution Module (FSR) is a deep learning model component for image super-resolution reconstruction. It enhances and reconstructs images by extracting high-level feature representations from low-resolution images and mapping them to the corresponding high-resolution space.

[0042] The Trident Head network is a deep learning model for object detection tasks. It consists of three parallel convolutional neural networks, which are used to output the classification, localization, and occlusion status of the target object, respectively. This network structure typically employs a multi-level feature pyramid and anchor box mechanism, which can effectively detect target objects of different sizes and shapes, and achieves a balance between detection accuracy and speed.

[0043] Batch Normalization (BN) is a commonly used technique in deep learning that standardizes the output of a single layer of a network to avoid imbalanced input data. This makes training more stable and faster, and helps prevent overfitting.

[0044] Ghost convolutional blocks are lightweight convolutional neural network architectures that offer high accuracy with low computational complexity and a small number of parameters. A key component is the ghost convolutional layer, which reduces computation by splitting the input feature map into two parts and introduces a hyperparameter to control the ratio between the two parts. This application utilizes ghost convolutional blocks as lightweight convolutional base blocks to construct the Featrinet model.

[0045] Post-Training Quantization (PTQ) is a technique that optimizes neural networks by quantizing them after training. It reduces the model's memory footprint and computational requirements by converting high-precision model parameters and activation values ​​to low-precision values, thereby improving the model's speed and efficiency in resource-constrained environments such as mobile devices. PTQ can be applied to various types of neural networks and can significantly reduce model size and improve inference speed without compromising model performance.

[0046] The Featrinet model is a real-time, efficient target detection model designed for online or offline examination scenarios, combining contextual modeling, scale fusion, and scale awareness technologies. It utilizes feature pyramids, feature fusion, and a high-speed feature super-resolution module for small target object features.

[0047] Anchor boxes are typically a set of predefined rectangular boxes defined on an image feature map, used for segmentation and region selection of the input image.

[0048] Anchor point: refers to a point in the image coordinate system, which is usually used to define the position of an object.

[0049] Unlike conventional object detection, object detection algorithms used in exam monitoring need to handle a variety of complex scenarios. These include: densely packed objects in real exam monitoring footage, resulting in a small proportion of objects appearing in the frame if students are using electronic devices; small objects in the frame often suffer from inaccurate localization, occlusion, information loss, noise, and low tolerance for predicted bounding boxes; online exam environments may have clutter, too many irrelevant items, and randomly placed cameras; monitoring equipment may lack sufficient computing power to run existing object detection algorithms, relying on servers and compromising real-time performance; and lighting conditions in online or offline exam environments can be problematic. Dim lighting can blur object outlines and make surface brightness difficult to discern, while excessively bright lighting can cause the background to appear washed out, hindering object detection. Existing object detection algorithms struggle to achieve efficient real-time detection in these complex exam environments.

[0050] To address the aforementioned issues, this application provides a target detection model capable of adapting to various complex examination environments, employing an extremely lightweight model architecture. This application also utilizes techniques such as model pruning and model quantization to further accelerate model inference, enabling the target detection model to be applied to low-end monitoring equipment, achieving real-time detection, and providing rapid feedback on candidates' performance during examinations, effectively improving examination discipline and fairness. Furthermore, the target detection model provided by this application is not limited to remote server detection, significantly reducing real-time degradation caused by network issues and lowering server investment and maintenance costs for enterprises involved in the project.

[0051] Please see Figure 1 The diagram shown illustrates the application environment of the model building method or object detection method provided in this application embodiment. Figure 1 As shown, the electronic device and the monitoring device are connected in communication. The electronic device designs a lightweight model architecture by fusing feature pyramid networks, feature scale fusion networks, high-speed feature super-resolution networks, and triangular detection head networks to obtain an initial detection model. To accelerate the model's inference speed and simplify its structure, the electronic device uses a preset training sample set to perform sparse training on the initial detection model. This training enables the initial detection model to predict target objects in each preset training sample in the preset training sample set, including predictions for target object localization and classification, ultimately resulting in a target detection model. In one embodiment of this application, after determining the target detection model, the electronic device can convert the target detection model into a format supported by the monitoring device, embed the target detection model into an application, and deploy it on the monitoring device so that the monitoring device can use the target detection model to perform target detection tasks. The electronic device can also provide an Application Programming Interface (API), allowing the monitoring device to establish a communication connection with the electronic device through the API and call the target detection model in the electronic device. The monitoring device includes, but is not limited to, various personal computers, laptops, smartphones, tablets, portable wearable devices, and web browsers. Among them, electronic devices can be independent servers or server clusters composed of multiple servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, or terminal devices such as personal computers and computers.

[0052] Please see Figure 2 , Figure 2 The diagram shown is a flowchart illustrating the implementation of the model building method provided in this application embodiment, demonstrating the application of this method in...Figure 2 Taking an electronic device as an example, the explanation includes the following steps:

[0053] S11: Integrate the feature pyramid network, feature scale fusion network, high-speed feature super-resolution module, and trident detection head network to construct the initial detection model.

[0054] In one embodiment of this application, the initial detection model includes a FetriNet model with untrained model parameters.

[0055] Considering the possibility of students carrying small communication devices in both online and offline examination rooms, it is necessary to detect small objects in the examination room monitoring system. These small objects are a key factor in determining whether students are cheating. However, the detection accuracy of small objects in relevant target detection schemes is relatively low. For example, the DyHead detector only has a detection accuracy of 0.283 for small objects on the COCO test set, which is significantly lower than the detection accuracy of 0.503 for medium-sized objects and 0.575 for large objects. The low detection accuracy of small objects may stem from the poor quality of their feature representation, making it very difficult to learn correct feature representations from a limited and distorted set of small objects; the large scale range of small objects, with multiple scales coexisting in the training sample set; and the poor quality of their feature representation, which is often due to their limited size and the usual feature extraction methods. Currently, feature extractors use downsampling methods to reduce spatial redundancy and learn higher-dimensional features, but this inevitably reduces the feature representation of small objects. Furthermore, the features of small objects are easily contaminated by background examples after convolution, making it difficult for the network to capture the discrimination information of small objects that is crucial for subsequent tasks.

[0056] To address the aforementioned issues, this application combines a feature pyramid network, a feature scale fusion network, and a high-speed feature super-resolution module for scale awareness and feature fusion enhancement. Furthermore, considering that the final trained FetriNet model needs to be adapted to low-performance monitoring devices, this application strictly requires that the FetriNet model's blocks be designed to be extremely lightweight, meaning that the computational cost (Floating-Point Operations Per Second, FLOPS) of each block needs to be compressed.

[0057] During training and optimization, conventional convolutional blocks generate a large number of valid but redundant features, which are termed "ghost features." Directly removing these ghost features can severely impact the algorithm's performance. However, using extremely lightweight computation to generate ghost features can significantly reduce the computational cost of convolutional blocks when extracting specific features. Based on this, electronic devices utilize GhostConv blocks as lightweight convolutional blocks to construct the FetriNet model.

[0058] In one embodiment of this application, an initial detection model is constructed by fusing a feature pyramid network, a feature scale fusion network, a high-speed feature super-resolution module, and a triangular detection head network. This includes: using the feature pyramid network, the feature scale fusion network, and the high-speed feature super-resolution module as feature encoding modules, and using the triangular detection head network as feature decoding modules to construct the initial detection model. The feature encoding module is used to extract target features, and the feature decoding module is used to reconstruct the target object corresponding to the target features.

[0059] In one embodiment of this application, the electronic device uses a feature pyramid network, a feature scale fusion network, and a high-speed feature super-resolution module as feature encoding modules to extract features of small, medium, and large target objects from surveillance images or training samples. A tri-branch detection head network serves as the feature decoding module, dividing the features of large, medium, and small target objects into three branches for detection. By using different receptive fields to acquire the feature structures of target objects at different scales, and combining powerful feature context modeling with the backbone feature pyramid network and feature scale fusion network, the FetriNet model can perform efficient target detection even when multiple scale target objects coexist.

[0060] Please see Figure 3 This is a structural example diagram of the target detection model provided in an embodiment of this application. Figure 3As shown, the FeatriNet model architecture includes multiple basic convolutional blocks (Standard Conv), fully connected layers (Linear Mapping), a feature super-resolution module, and a ghost structure. The FeatriNet model combines FPN and PAN networks as its backbone. It extracts high-level semantic features from the RGB (Red, Green, Blue) images input to the object detection model from bottom to top, and then combines these high-level semantic features with shallow features from top to bottom. This invention adds a PAN network to the FPN, mixing shallow and deep semantics. Therefore, the detection of small objects no longer relies solely on shallow features; it can also extract rich high-level semantic information, greatly improving the semantic interaction capability of the FeatriNet model and overcoming the limitation that small objects can only be detected using small receptive fields.

[0061] Meanwhile, considering the difficulty in identifying small objects, the FetriNet model uses a high-speed feature super-resolution module customized for small objects to cope with the detection of small objects in the proctoring environment. Please refer to [link / reference]. Figure 4 This is a structural example diagram of the high-speed feature super-resolution module provided in an embodiment of this application. Figure 4 As shown, the high-speed feature super-resolution module is divided into two stages: the training stage, which uses a complex structure with three branches and different receptive fields for feature extraction; and the testing stage, which uses a simple structure with a single branch and only basic convolutional blocks. In this embodiment, batch normalization (BN) is added after the basic convolutional blocks, which helps to accelerate the convergence of the FeatureNet model.

[0062] In one embodiment of this application, the high-speed feature super-resolution module refers to the super-resolution of the shallow feature map obtained by upsampling the input image. The high-speed feature module shares the same feature extraction structure with the backbone of the model combining FPN and PAN networks. Since high-speed feature super-resolution typically involves upsampling the input image, this would significantly increase the computational load of the model. The FetriNet model, however, uses the high-speed feature super-resolution module to perform super-resolution training on the shallow feature map after multiple downsamplings, greatly reducing the computational burden brought by the high-speed feature super-resolution module. Furthermore, the heavy parameter structure of the FetriNet model further reduces the high-speed feature super-resolution module to a combination of simple convolutional blocks during the inference and deployment phase, enabling the enhancement of features of small target objects with negligible computational increase.

[0063] In addition, such as Figure 3As shown, the FeatriNet model also boasts a powerful tri-branch detection head design, which divides large, medium, and small target objects into three branches for detection, resulting in predictions for small, medium, and large targets. The FeatriNet model can utilize different receptive fields to acquire the feature structures of target objects at different scales. Combined with the robust feature context modeling of the backbone, the FeatriNet model can perform efficient target detection even when faced with multiple scale targets coexisting.

[0064] also, Figure 3 The detection structure in the code is used for operations such as locating and classifying target objects, generating detection results. Non-Maximum Suppression (NMS) is used to remove redundant detection results.

[0065] In the model construction method provided in this application embodiment, an electronic device utilizes a feature pyramid fusion network, a high-speed feature super-resolution module, and other technologies to design a highly efficient detection model, FeatriNet, for online or offline exam proctoring. The FeatriNet model can specifically and accurately detect small cheating communication devices in online exams, such as mobile phones, tablets, and smartwatches. Furthermore, the electronic device employs context modeling, scale fusion, and scale awareness techniques, enabling the FeatriNet model to possess powerful feature detection capabilities, no longer limited to shallow semantic features.

[0066] S12: Use a preset training sample set to perform sparsity training on the initial detection model, so that the initial detection model can predict the target object in each preset training sample in the preset training sample set, and obtain the target detection model.

[0067] In one embodiment of this application, sparse training reduces model complexity and storage requirements by setting certain parameters in the model to zero. The object detection model includes a pre-trained FetriNet model.

[0068] In one embodiment of this application, considering that the performance of monitoring equipment performing object detection in online examinations may not be outstanding, conventional model training methods are not feasible to enable the object detection algorithm to run on low-performance monitoring equipment. Electronic devices further compress the initial detection model through sparse training, allowing the object detection model to cover more types of monitoring equipment in real-world examination scenarios.

[0069] In one embodiment of this application, an initial detection model is sparsely trained using a preset training sample set, enabling the initial detection model to predict target objects in each preset training sample in the preset training sample set, thereby obtaining a target detection model. This includes: determining the anchor box configuration of the target detection model based on an anchor box adaptive algorithm and the preset training sample set, and performing data augmentation on the preset training sample set to obtain an augmented training sample set; calculating the small target loss weight and the large target loss weight during the sparse training of the initial detection model using the anchor box configuration and the augmented training sample set; optimizing the augmented training sample set based on the small target loss weight and the large target loss weight to obtain an optimized training sample set; and performing sparse training of the initial detection model based on the anchor box configuration and the optimized training sample set, enabling the initial detection model to predict target objects in each preset training sample in the preset training sample set, thereby obtaining a target detection model.

[0070] In one embodiment of this application, the anchor box adaptive algorithm includes, but is not limited to, K-Means clustering, differential evolution algorithm for retrieving anchor box shape and scale combinations, Guided Anchors for generating sparse anchor box combination features, AABD using Bayesian optimization and sub-sampling to retrieve the best anchor box, and hyperparameter optimization for searching the best anchor box during training. For complex and variable examination scenarios, it is necessary to consider both fine-grained anchor box retrieval and data processing efficiency. Therefore, this embodiment of the application preferably uses differential evolution algorithm as the adaptive anchor box selection algorithm. Anchor box configuration refers to the anchor box parameter configuration corresponding to the preset training sample set. During the sparsity training process of the initial detection model, the electronic device will calculate the small target loss function and the large target loss function. Through model training, the small target loss function converges so that the target detection model can predict small target objects; through model training, the large target loss function converges so that the target detection model can predict large target objects. The small target loss weight refers to the weight of the small target loss function, and the large target loss weight refers to the weight of the large target loss function.

[0071] Object detection aims to locate all objects of interest in an image and identify their categories. Currently, object detection is generally divided into two types: two-stage detection methods and single-stage detection methods. Specifically, both methods take a single frame of video as input. Two-stage methods first perform a coarse screening of basic features in the frame, then perform a fine-tuning of the selected features for localization and classification. Single-stage methods, on the other hand, directly perform feature localization and classification. Therefore, two-stage methods have high accuracy but are slow; single-stage methods have low accuracy but are fast. In the history of object detection technology development, single-stage methods, due to their speed, have been widely adopted by various industries as the main technology for surveillance and detection. Extensive research based on this has led to the development of various derivative methods, such as anchor-bound and anchor-point-based object detection methods.

[0072] Please see Figure 5 This is an example diagram of the anchor frame and anchor points provided in real time by this application. Figure 5 As shown, in the process of locating and recognizing target objects in image a using the anchor-bound method, the electronic device pre-sets anchor boxes of different sizes on each pixel of the feature map corresponding to image a. This allows for rapid and targeted location and recognition of target objects of different sizes and proportions in image a, thereby achieving target object detection. The anchor-bound method can rapidly reduce target loss during training, allowing the model to quickly approach the labeled values, but the anchor box size needs to be readjusted for different datasets. This has led to the development of another anchor-point-based target detection method, such as... Figure 5 As shown, this method learns the pixels surrounding the center point closest to the object in the image as anchor points, and then uses these anchor points to predict the object's boundary. While the anchor point method solves the cumbersome process of customizing anchor box sizes for different datasets, it also brings unavoidable problems. For example, in image b, when two objects A and B overlap, their center points overlap. In this case, the center point can serve as an anchor point for both object A and object B. In complex situations, when the amount of such overlapping data is large enough, deep learning models based on the anchor point method become very difficult to converge. Furthermore, in complex examination environments, where both online and offline examination rooms have densely packed objects, this embodiment uses an anchor box-based object detection method.

[0073] In one embodiment of this application, the electronic device determines the anchor box configuration of the target detection model based on an anchor box adaptive algorithm and a preset training sample set. See also... Figure 6 This is a flowchart illustrating the implementation process of the anchor frame configuration method provided in this application embodiment. Figure 6 As shown, the optimization objective of the electronic device in optimizing the anchor frame using the differential evolution algorithm is to maximize the sum of the intersection-union ratios of the object frames of all target objects and the matching anchor frames.

[0074] Specifically, such as Figure 6 As shown, initializing the population refers to the electronic device pre-defining anchor frames of different sizes to obtain a preset set of anchor frames. This preset set of anchor frames serves as the population, and each preset anchor frame in the set represents an individual within the population. Mutation involves the electronic device randomly selecting two individuals from the population and calculating their difference to obtain a difference vector. Then, the electronic device randomly selects a third individual from the population and performs a weighted summation of the difference vector between the third individual and the mutated individual to obtain a mutated individual. Crossover involves the electronic device mixing the mutated individual with the aforementioned third individual using parameters to generate experimental individuals. Parameter mixing includes, but is not limited to, weighted summation. Selection involves the electronic device comparing the intersection-union ratio (IU) of the mutated individual with the pre-labeled target bounding box with the IU of the third individual with the target bounding box. The individual with the larger IU is identified as the superior individual, and only the superior individuals are retained for the next round of evolution until a preset evolutionary threshold is reached. In this way, the final determined individuals all possess superior parameters. For example, if the electronic device determines that the intersection-union ratio (IU) of the mutated individual with the target bounding box is greater than that of the third individual with the target bounding box, the electronic device retains the mutated individual and replaces the third individual in the population in the next evolution. If the electronic device determines that the IU of the mutated individual with the target bounding box is less than that of the third individual with the target bounding box, the electronic device determines the third individual as the better individual and still uses the third individual in the next round of evolution. In this way, the electronic device utilizes the characteristics of population evolution to retain the better anchor boxes trained, achieving the optimal anchor box combination through survival of the fittest. Compared with the anchor box clustering method in related technical solutions, the embodiments of this application determine the anchor box parameters corresponding to the preset training sample set through the differential evolution algorithm, without discarding small anchor boxes that may be cheating devices, which can significantly improve the adaptability of the target detection model to the preset training sample set.

[0075] Furthermore, small objects are typically scarce in the pre-set training sample set, contributing little to the overall model loss objective, which can severely impact training imbalance. In some extreme cases, small objects may be considered data noise during training, thus ignoring their features. Therefore, embodiments of this application employ pre-set data augmentation methods, such as mosaic data augmentation, to augment the pre-set training sample set.

[0076] Please see Figure 7 This is an example flowchart illustrating the implementation process of the data augmentation method provided in the application embodiment. Figure 7As shown, to enhance the learning of features of small objects, the electronic device uses a mosaic augmentation method to randomly select data from a preset training sample set, such as data 10, data 98, data 567, and data 884, and performs proportional random scaling and random stitching to obtain an image with a training resolution of 640*640 pixels. This increases the number of small objects and enhances the learning of their features. It is evident that after using data augmentation, large and medium-sized objects in the image will also randomly become small objects, thus significantly expanding the number of small objects at the data level.

[0077] Furthermore, since actual examination environments may contain excessively bright or dark scenes, causing target objects to be contaminated by background changes in brightness, thus affecting target detection, this application employs adaptive histogram equalization to quickly adjust image illumination and contrast to adapt to various scene variations. Specifically, an adaptive histogram equalization method is added to the image processor before modeling to prevent severe overexposure or underexposure of the image input to the target detection model, thereby reducing the contamination of target objects, especially small target objects, by background changes in brightness. During the sparse training of the initial detection model using a preset training sample set, the electronic device's data augmentation of the preset training sample set also includes using the adaptive histogram equalization method to augment the preset training sample set.

[0078] In one embodiment of this application, after determining the anchor box configuration corresponding to the target detection model and enhancing a preset training sample set, the electronic device performs sparse training on the initial detection model using the anchor box configuration and the enhanced training sample set. Based on the small target loss weights and large target loss weights during training, the enhanced training sample set is then optimized to obtain an optimized training sample set. The electronic device performs sparse training on the initial detection model based on the anchor box configuration and the optimized training sample set, enabling the initial detection model to predict the target object in each preset training sample in the preset training sample set, thus obtaining the target detection model.

[0079] In one embodiment of this application, data optimization of the augmented training sample set based on small target loss weights and large target loss weights is performed to obtain an optimized training sample set. This includes: determining whether there is a loss imbalance in the initial detection model during sparse training based on the small target loss weights and large target loss weights; if it is determined that there is a small target loss imbalance in the initial detection model during sparse training, optimizing the augmented training sample set using a preset image augmentation method to obtain an optimized training sample set; if it is determined that there is a large target loss imbalance in the initial detection model during sparse training, adjusting the size of each augmented training sample in the augmented training sample set to obtain an optimized training sample set; if it is determined that there is no loss imbalance in the initial detection model during sparse training, using the augmented training sample set as the optimized training sample set.

[0080] In one embodiment of this application, the preset image enhancement method includes, but is not limited to, mosaic enhancement. This embodiment uses mosaic enhancement as an example for illustration. If an electronic device uses mosaic enhancement to perform data enhancement on a preset training sample set, random samples in the preset training sample set will be scaled and stitched together during the data enhancement process, which may lead to an imbalance in the loss of large targets. Therefore, the electronic device also needs to balance the target loss during model training to consider whether to use mosaic enhancement.

[0081] Based on the above problems, the electronic device in this application calculates the small target loss weights and large target loss weights during the sparse training of the initial detection model using anchor frame configuration and augmented training sample sets, and determines whether there is a loss imbalance in the initial detection model during sparse training based on the small target loss weights and large target loss weights. If the electronic device determines that there is a small target loss imbalance in the initial detection model during sparse training, it uses a preset image augmentation method to augment the training sample set. If the electronic device determines that there is a large target loss imbalance in the initial detection model during sparse training, it adjusts the size of each augmented training sample in the augmented training sample set to obtain an optimized training sample set. If the electronic device determines that there is no loss imbalance in the initial detection model during sparse training, it uses the augmented training sample set as the optimized training sample set.

[0082] Please see Figure 8 This is an example flowchart illustrating the implementation process of the sample set creation method in an embodiment of this application. Figure 8As shown, this embodiment of the application considers the anchor box size of the target detection algorithm, using a differential evolution algorithm to adaptively determine the anchor box configuration corresponding to an arbitrary preset training sample set; from the perspective of data imbalance of small target objects, a mosaic data augmentation method is used to smooth the proportion of target objects of different sizes; and an adaptive histogram transformation is used to adapt to the input of training samples with different lighting conditions, reducing the impact of bright and dark training samples on target detection. Specifically, in the process of determining the anchor box configuration of the target detection model, the electronic device first initializes the anchor box configuration, and then uses a differential evolution algorithm to determine the optimal anchor box configuration. In the process of data augmentation of the preset training sample set, the electronic device first uses adaptive histogram equalization to adjust the lighting and contrast of the training samples to obtain an enhanced training sample set. The electronic device uses the anchor box configuration and the enhanced training sample set to perform sparse training on the initial detection model, and calculates the loss weights for small targets and large targets during the training process, and judges whether there is a loss imbalance in the initial detection model during the sparse training process based on the loss weights for small targets and large targets. If the electronic device determines that the initial detection model suffers from a loss imbalance due to small targets during sparse training, it uses a mosaic augmentation method to randomly scale and stitch the augmented training samples in the augmented training sample set to obtain training samples of a preset training size, such as training samples with a preset training size of 640*640 pixels, thus obtaining an optimized training sample set. If the electronic device determines that the initial detection model suffers from a loss imbalance due to large targets during sparse training, it adjusts the size of each augmented training sample in the augmented training sample set to the preset training size, thus obtaining an optimized training sample set.

[0083] This application's embodiments utilize data processing techniques combining mosaic stitching, adaptive differential evolutionary search of anchor frames, and brightness transformation to address the challenges of complex online examination scenarios that current monitoring technologies cannot cover, such as cluttered objects and excessively bright or dim lighting. Furthermore, common target detection problems in complex scenarios, such as the variable aspect ratio of detection boxes for small objects like examinees' mobile devices, the difficulty in standardizing object sizes, and data scarcity, are also further optimized.

[0084] In one embodiment of this application, an initial detection model is sparsely trained using a preset training sample set, enabling the initial detection model to predict target objects in each preset training sample in the preset training sample set, thereby obtaining a target detection model. This includes: sparsely training the initial detection model using the preset training sample set to compress model parameters and obtain a trained detection model; detecting the detection accuracy of the trained detection model; if the detection accuracy is greater than or equal to a preset accuracy threshold, pruning the trained detection model to obtain a pruned detection model; detecting the model size of the pruned detection model; if the model size meets a preset model size requirement, quantizing the pruned detection model to obtain a target detection model; if the model size does not meet the preset model size requirement, returning to the step of sparsely training the initial detection model using the preset training sample set to compress model parameters and obtain a trained target detection model; and if the detection accuracy is less than a preset accuracy threshold, returning to the step of fusing a feature pyramid network, a feature scale fusion network, a high-speed feature super-resolution module, and a triangular detection head network to construct the initial detection model.

[0085] In one embodiment of this application, both the preset accuracy threshold and the preset model size requirement are custom values. This application does not limit the preset accuracy threshold and the preset model size requirement.

[0086] In one embodiment of this application, considering that the performance of monitoring equipment performing object detection in online examinations may not be outstanding, conventional model training methods are not feasible in order to enable the object detection algorithm to run on low-performance monitoring equipment. This application further compresses the initial detection model by sparse training, thereby enabling the object detection model to cover more monitoring equipment in real examination scenarios.

[0087] In one embodiment of this application, the initial detection model typically generates a large number of redundant features during training. These redundant features are content that is ineffective in representing the detection results. For example... Figure 9The example diagram of redundant features in the sparse training provided in this application embodiment shows that even when using ghost convolution blocks, a large number of redundant features are generated. Unlike ghost features, redundant features are invalid and superfluous features. Therefore, deleting these feature representations from the initial detection model will not have a significant impact on the target detection results. A practical and feasible method is to sparsify the model parameters, then prune and remove unimportant weights to obtain a pruned detection model, and then fine-tune it. This process is repeated until the number of parameters of the initial detection model can be successfully deployed on a low-performance monitoring device. If it is found that the accuracy of the pruned detection model has decreased significantly during this period, but the model size has not yet met the preset model size requirements, it indicates that the designed initial detection model has structural defects, and the initial detection model needs to be redesigned.

[0088] As an example, please refer to Figure 10 This is an example flowchart illustrating the implementation process of the model pruning method provided in this application embodiment. During training, the FeatriNet model specifies the step size for each model parameter, uniformly compressing it to near 0. Some model parameters with strong representational ability will move away from 0 as much as possible, while those with poor representational ability will move closer to 0. After sparse training, the model parameter matrix exhibits a form with many zeros, thus becoming a sparse matrix. However, it should be noted that directly deleting small weight values ​​from the sparse matrix will make the convolution matrix incomplete, thereby violating the existing convolution calculation rules. A common approach is to calculate the mean of the sparse convolution matrix; if the mean is less than a set threshold, the convolution matrix is ​​directly deleted. The formula for the pruning constraint is expressed as:

[0089]

[0090] In the formula, y represents the direction of the model parameter X, R represents a real number, the regularization term β imposes a regularization constraint on the model parameter X, and ε is the regularization coefficient, representing the magnitude of the constraint on the model parameter X. Clearly, the larger the value of ε, the greater the degree to which the model parameters are compressed towards zero. When ε = 0, sparse training becomes regular training; while when ε >> 1, the regularization term replaces the optimization direction of the objective function, at which point almost all parameters are zero; N represents the number of model parameters.

[0091] Therefore, it is necessary to impose regularization constraints on parameter X while ensuring that parameter X has strong expressive power, such as... Figure 10As shown, this application employs a dynamic adjustment pruning method for training / sparse training of the initial detection model, which significantly reduces the complexity of manually adjusting model parameters and improves training efficiency. It should be noted that when the initial detection model does not meet the accuracy requirements, there are two scenarios: I. Insufficient expressive power due to model construction; II. The model is too constrained by regularization, resulting in insufficient expressive power due to severe sparsity.

[0092] For insufficient expressive power caused by model construction issues, electronic devices can redesign the initial detection model, i.e., solve the problem through model reconstruction. However, for insufficient expressive power caused by excessive regularization constraints and severe sparsity, a method of dynamically adjusting the regularization term β can be used for re-sparse training. For example... Figure 10 As shown, the electronic device uses a preset training sample set to perform sparsification training on the initial detection model, obtaining a trained detection model. The detection accuracy of the trained model is then checked. If the detection accuracy meets the requirements, the trained detection model undergoes sparsification verification and model pruning to obtain a pruned detection model. The electronic device then checks the size of the pruned detection model. If the model size does not meet the preset model size requirements, the model is too constrained by regularization, resulting in insufficient expressive power due to severe sparsity. In this case, the electronic device increases the regularization term β and returns to the step of sparsifying the initial detection model using the preset training sample set. If the detection accuracy is less than the preset accuracy threshold and does not meet the accuracy requirements, it indicates that the regularization term β is too large, resulting in insufficient model expressive power. The electronic device needs to reconstruct the model to reduce the regularization term β.

[0093] The divergence between the two scenarios described above dynamically depends on the number of times the model calls method II. If repeatedly scheduling method II fails to yield a significant improvement in accuracy, it indicates that the model itself lacks high-precision expressive capabilities, necessitating a model redesign. Therefore, for method I, only dynamic refactoring hints are required.

[0094] Furthermore, after the model has completed pruning, it is necessary to reassess whether the pruned detection model meets the inference requirements of low-performance monitoring equipment. If it does not meet the inference requirements, there may be two situations: low pruning rate or excessively large model size. Therefore, model reconstruction or increasing the regularization coefficient for more intensive pruning is required. It should be noted that the model accuracy usually decreases significantly after pruning. This application employs a three-stage training approach of sparsification, pruning, and fine-tuning; therefore, a fine-tuning training of the model is also required after pruning.

[0095] Please see Figure 11The image shows an example of the model pruning results provided in this application embodiment. It can be observed that after adjusting appropriate regularization constraints and undergoing multiple training rounds, the weights of most model parameters are squeezed to near 0. At this point, amplitude pruning of the parameters can achieve a better model compression effect.

[0096] The FetriNet model provided in this application is truly lightweight. In addition to adopting a low-redundancy structural design, features that do not affect the main body of the model are further pruned after training, making it extremely simple and lightweight, and very friendly to low-performance monitoring devices.

[0097] In one embodiment of this application, if the model size meets the preset model size requirement, in order to further accelerate the training of the target detection model to enable real-time detection on low-performance monitoring devices, the electronic device also needs to perform quantization processing on the model. Quantization is the process of approximating a continuous signal through a set of discrete symbols or integer values. However, techniques that reduce accuracy do not always achieve the expected speedup. For example, due to the additional operations of quantization and dequantization, INT8 inference cannot achieve a precise 4x speedup on 32-bit floating-point. For example, TensorFlow Lite and Tensor RT using INT8 inference speeds up the model by approximately 2-3 times. It significantly improves data turning speed, and in mobile devices with limited storage, quantization can reduce model storage size by 4 times. Historically, quantization techniques can be categorized into three types: Dynamic Quantization (DQ), Post-Training Quantization (PTQ), and Quantization-Aware Training (QAT). This application prioritizes post-training quantization for model quantization, primarily for two reasons: 1. The target detection model applied in an examination proctoring environment does not have high requirements for regression localization, and PTQ does not significantly reduce classification accuracy; 2. PTQ technology is relatively easy to implement and can be easily integrated into the entire target detection model implementation process. The PTQ logic used in this application embodiment will be briefly introduced below.

[0098] No quantization technique can avoid the impact of decreased quantization precision. To put it simply, representing a number using Float32 bits with Int8 reduces its dynamic range by a factor of four. This introduces additional problems. For example, Int8 typically represents 256 values: {-128, -127, ..., 0, ..., 127}. The inclusion of 0 prevents +128 from being represented by Int8, leading to a rule similar to the rule in division where the dividend cannot be 0. One direct approach is to remove -128 to achieve a symmetrical representation range, but this further reduces the already degraded dynamic range. Another approach is to use odd-number mapping, spreading the 256 numbers across a larger dynamic range, such as {-255, -253, ..., 253, 255}. Since 0 is an even number, it is not within the representable range. The advantage of this approach is that it simplifies the dynamic range expansion, but it also introduces the problem of not allowing zero values, indicating that this method cannot achieve an identity mapping. In fact, for neural networks, the center point of the values ​​is often not zero, so a center point shift is needed, that is, moving the representable center range from 0 to other positions, such as... Figure 12 The example image shown is of the center point offset in the model quantization, where min(x) f ) represents the minimum value within the representable range, max(x) f ) indicates the maximum value that can be represented.

[0099] When an electronic device uses a center point offset, odd-number mapping can be used to represent low-bit numbers. In addition, this application also uses the scaling factor S from dynamic quantization to expand the representation range of low-bit numbers, as simply expressed in equation (1): WX≈S w (W int -Z w )S x (X int -Z x (1).

[0100] Among them, Z w Z x The number system used is the center point offset, where W represents the model parameters and X represents the input features. int X represents the integer model parameter. int This represents the integer input feature. Further expansion of equation (1) yields equation (2), which is expressed as follows: S w S x (W int X int )+S w S x Zx W int +S w Z w S x Z x +S w S x Z w X int (2).

[0101] It can be observed that after the model training is completed, Z w Z x S w S x W int Since they are all scalars, they can be stored in the model as offline quantizations and await computation. What cannot be quantized offline is the input feature X. int It changes with the input, therefore the main computational cost in the quantization process comes from having X. int Two items. To simplify quantization calculations, electronic devices make Z... w =0, it can be seen that after performing this operation, equation (2) is equivalent to equation (3): WX≈ωX int +β(3).

[0102] Among them, ω and β are constants for offline quantization, which can be calculated first and then stored in the model, thereby significantly reducing the amount of computation caused by quantization during inference.

[0103] Based on the above example, please refer to Figure 13 This is a flowchart illustrating the implementation process of the model quantization method provided in this application. Figure 13As can be seen, each convolution (Conv) requires quantization and de-quantization. During quantization, the maximum and minimum values ​​of the current input X must be calculated to correctly obtain the low-bit center offset and dynamic scaling factor. When the input X is large enough, the overhead of traversing it becomes extremely high. In this case, the Min and Max values ​​of the input X can also be quantized offline and included in the model, but this requires decoupling Min and Max from their dependence on the input X. PTQ's approach is to use more than 100 images as identically distributed replacements for the test set before performing model quantization, thereby calculating the Min and Max values ​​of the data and successfully storing them in the model. However, this also presents a problem: can 100 or 1000 images truly represent the distribution of tens of thousands of test sets? If there is a significant deviation, it will lead to errors in the calculation of Min and Max values, resulting in a larger quantization error in the input X, and consequently a significant decrease in the accuracy of the final model. However, as analyzed earlier in this application, regression is not so important for exam room monitoring; the core is the items appearing in the exam room and their confidence levels. Therefore, the PTQ scheme is feasible. (See the offline quantization scheme for PTQ.) Figure 14 The diagram shows another implementation example of the model quantization method.

[0104] exist Figure 14 In this context, "calibration" refers to model-level correction of the input X using additional data from the PTQ (Physical Level Quantization). "Quantization," on the other hand, refers to... Figure 13 As shown, the PTQ presented here performs offline quantization with the data-dependent Min and Max values ​​to further improve the speed of quantization computation.

[0105] Please see Figure 15 This is another example flowchart illustrating the model quantization method provided in this application. Figure 15 As shown, the inverse quantization of the first convolutional layer and the quantization of the second convolutional layer are combined, allowing the two quantized convolutions, Conv1 and Conv2, to directly perform numerical calculations. If M and N represent the quantization of Conv1 and Conv2 respectively, it's easy to see that this is actually performing calculations on two low-bit values ​​with different dynamic ranges and offsets, such as {-9, -7, -5, 3} and {75, 77, 79, 81}. The calculation result must be represented using the same low-bit value P. In other words, this calculation requires finding a P that perfectly represents M and N, which is obviously impossible. This application addresses this problem using a PTQ solution that extends the dynamic range; see the details for reference. Figure 16The diagram shows another implementation example of the model quantization method.

[0106] like Figure 16 As shown, after calculating the two low bits M and N with inconsistent dynamic ranges, P with a larger dynamic range (such as Int32) is used for perfect representation. Then, the dynamic scaling factor of P is recalculated, and P is requantized to Int8. This solves the problem of numerical calculation of the two low bits in PTQ.

[0107] Taking the ImageNet classification model as an example, the quantization results after training are shown in Table 1 below. It can be seen that quantization achieves a very high compression ratio for the model size. The MNN (Mobile Neural Network) quantization tool is used to compress and optimize deep learning models. The TF (TensorFlow) quantization tool is a set of tools for compressing and optimizing TensorFlow models, including tools for quantizing weights, activation functions, etc. (reducing precision) to reduce model size and accelerate the inference process.

[0108]

[0109] Table 1. Examples of quantization results after training.

[0110] This application's embodiments further accelerate the inference speed of the FetriNet model by employing techniques such as model pruning and model quantization. This enables the application to cope with the low-performance monitoring devices of online test takers, provide rapid feedback on the test takers' situation during the exam, and effectively improve exam discipline and fairness.

[0111] In a model construction method provided in this application embodiment, an electronic device constructs a lightweight initial detection model by fusing a feature pyramid network, a feature scale fusion network, a high-speed feature super-resolution module, and a triangular detection head network, thereby improving the detection accuracy of small target objects. Furthermore, by using a preset training sample set to perform sparse training on the initial detection model, a target detection model is obtained, further simplifying the number of parameters in the target detection model. This makes the target detection model suitable for low-performance monitoring equipment and can easily adapt to various complex online or offline examination environments.

[0112] In one embodiment of this application, after sparsifying the initial detection model using a preset training sample set so that the initial detection model can predict the target object in each preset training sample in the preset training sample set and obtain the target detection model, the model construction method further includes: deploying the target detection model to a monitoring device; monitoring the inference time of the monitoring device using the target detection model to perform the target detection task; if the inference time does not meet the preset real-time requirements, returning to the step of executing the fusion feature pyramid network, feature scale fusion network, high-speed feature super-resolution module, and trident detection head network to construct the initial detection model.

[0113] In one embodiment of this application, after determining the target detection model, the electronic device can convert the target detection model into a format supported by the monitoring device, embed the target detection model into an application, and deploy it on the monitoring device so that the monitoring device can use the target detection model to perform target detection tasks. The electronic device can also provide an API, enabling the monitoring device to establish a communication connection with the electronic device and call the target detection model in the electronic device. The preset real-time requirement can be set to a preset duration. When the inference time exceeds the preset duration, it indicates that the inference time does not meet the preset real-time requirement; when the inference time is less than or equal to the preset duration, it also indicates that the inference time does not meet the preset real-time requirement.

[0114] In one embodiment of this application, when verifying whether the inference time of the monitoring device using the target monitoring model to perform the target detection task meets the preset real-time requirements, the monitoring device can use a low-performance device to ensure that the target detection model can detect in real time while enabling the target detection model to be applied to the low-performance monitoring device.

[0115] In one embodiment of this application, the electronic device can set a timer to periodically monitor the inference time of the monitoring device in performing the target detection task using the target monitoring model, determine whether the inference time meets the preset real-time requirements, and perform model reconstruction when the inference time does not meet the preset real-time requirements.

[0116] Please see Figure 17 This is an example flowchart illustrating the implementation process of the model building method provided in this application embodiment. Figure 17As shown, the problem addresses the complexities of examination monitoring environments, such as chaotic online or offline testing environments, low-performance monitoring equipment incapable of running existing object detection algorithms, and excessively bright or dim lighting. The aim is to construct a real-time, efficient object detection model for monitoring examinations by performing object detection on the monitoring video. Furthermore, the constructed object detection model can be applied to low-performance monitoring equipment. Based on the proposed problem, this application constructs an object detection model following the steps of selecting a technical solution, data production, model design, training and sparse training and model pruning, model quantization, and client-side inference deployment.

[0117] Specifically, this application embodiment employs an anchor-box-based object detection algorithm and utilizes a differential evolution algorithm to generate the anchor-box configuration corresponding to the object detection model. Furthermore, the electronic device uses mosaic enhancement and adaptive histogram equalization to augment the preset training sample set to improve data applicability, covering as many real-world scenarios as possible in online and offline examination rooms, thereby improving the object detection model's accuracy in detecting objects in complex scenes. In the model design process, this application embodiment selects the optimal combination from state-of-the-art technologies such as Feature Pyramid Network (FPN), context modeling, scale fusion, scale awareness, and attention, combined with a fast feature super-resolution specifically optimized for small targets in examination rooms, thus creating an extremely lightweight model architecture. After repeated performance verification, the theoretically derived object detection model most suitable for online or offline examination scenarios is named FeatureNet.

[0118] Considering that some low-performance monitoring devices cannot run existing target detection algorithms, the electronic device further performs sparsification training on the designed FetriNet model and tests the detection accuracy of the sparsified model. If the detection accuracy meets the requirements, the sparsified model is validated and pruned to obtain a pruned detection model. The electronic device then checks the size of the pruned model to determine if it meets the preset size requirements. If the size meets the requirements, the pruned model is trained and then quantized; if the size does not meet the requirements, the process returns to the training / sparse training steps; if the detection accuracy does not meet the requirements, the electronic device returns to the steps of selecting the technical solution and redesigns the model. By performing sparsification training, pruning, and quantization on the designed FetriNet model, the model is compressed, allowing the model's parameters to be deployed on low-performance monitoring devices. After training and quantizing the model, the electronic device deploys the final FetriNet model to the monitoring device and verifies the detection results of the monitoring device using the FetriNet model under various examination scenarios. The inference time of the monitoring device using the FetriNet model for object detection is also monitored. When the electronic device detects that the inference time does not meet the real-time requirements, it returns to the steps selected in the technical solution and redesigns the model until the inference time meets the real-time requirements, thereby improving the real-time performance of the object detection model.

[0119] Please see Figure 18 The figures provided are a comparison of the model accuracy provided in the embodiments of this application, and Table 2 shows a comparison of the accuracy of the FetriNet model with mainstream object detection algorithms. Figure 18 The object detection algorithms or models shown in Table 2 all use Tencent Neural Network (TNN) as the inference framework and an ARM Snapdragon 845-configured device for object detection testing. Figure 18 As shown, it can be seen that, in terms of both accuracy (mAP) on exam data and execution speed, the FetriNet model constructed by the electronic device in this embodiment of the application is more balanced and has a greater advantage over existing advanced object detection algorithms.

[0120]

[0121] Table 2. Accuracy Comparison of Mainstream Target Detection Algorithms

[0122] For ease of comparison, Figure 18The mAP in Table 2 is displayed using the standard method of shifting the decimal point to two places. According to the data in Table 2, it can be seen that after performing INT8 quantization on the FetriNet model, the speed is improved by 2.4 times with only a 6% loss in accuracy. This demonstrates that model quantization can effectively accelerate model inference.

[0123] Please see Figure 19 , Figure 19 The diagram shown is a flowchart illustrating the implementation of the target detection method provided in this application embodiment, applied to... Figure 2 Taking the monitoring equipment in the example, the following steps are included:

[0124] S21: Acquire surveillance video and preprocess the surveillance video to obtain the image to be detected.

[0125] In one embodiment of this application, the surveillance video refers to the examination room video captured in real time by the monitoring device. In other embodiments, the monitoring device may also be the examination room video received by the monitoring device from other devices. These other devices include, but are not limited to, camera equipment, monitoring equipment, servers, etc. The image to be detected refers to the examination room image corresponding to the surveillance video.

[0126] In one embodiment of this application, after the electronic device deploys the target detection model to the monitoring device, the monitoring device uses the target detection model to perform invigilation. Specifically, since the monitoring device uses the target detection model to perform target detection on the image to detect whether there are any prohibited items in the examination room during the examination, thereby achieving invigilation, the monitoring device needs to preprocess the monitoring video after collecting the monitoring video of the examination room to obtain the image to be detected.

[0127] In one embodiment of this application, preprocessing the surveillance video to obtain the image to be detected includes: converting the surveillance video into at least one video frame image, and performing format conversion on the at least one video frame image to obtain at least one target frame image; using a filter to filter out valid frame images from the at least one target frame image, and performing image enhancement on the valid frame images to obtain the image to be detected.

[0128] In one embodiment of this application, the monitoring device can use tools such as OpenCV library and FFmpeg to convert monitoring video into video frame images. To facilitate subsequent target detection tasks, the monitoring device can further convert the video frame images to obtain at least one target frame image, and store each target frame image as a separate image file. Since invalid frame images, such as blank frames, may exist in the monitoring video, these invalid frame images do not contribute to the target detection results and instead affect the target detection efficiency. Therefore, the monitoring device uses a filter to filter out valid frame images from the at least one target frame image, and performs target detection only on the valid frame images. After determining the valid frame images, the monitoring device performs image enhancement on the valid frame images to further improve detection accuracy, obtaining the frame image to be detected. For example, adaptive histogram equalization is used to adaptively adjust the brightness of the valid frame images to reduce the impact of image brightness on target detection.

[0129] S22: Use the object detection model to detect objects in the image to be detected and determine the target objects in the image to be detected.

[0130] In one embodiment of this application, the target object refers to an object of a specific category in the image to be detected, such as a prohibited device: a test taker, a mobile phone, a wearable device, etc. The target detection model typically outlines the target object using a predicted bounding box and then classifies it to achieve target detection. This target detection model is the aforementioned Featrinet model.

[0131] In one embodiment of this application, the monitoring device uses a target detection model to perform target detection on the image to be detected frame by frame, thereby determining the target object in each image to be detected.

[0132] In one embodiment of this application, a target detection model is used to detect targets in an image to be detected, and the target objects in the image to be detected are determined. This includes: using the target detection model to detect targets in the image to be detected, generating prediction boxes corresponding to the target objects in the image to be detected, the prediction boxes being used to characterize the position and category of the target objects; and determining the target objects in the image to be detected based on the prediction boxes.

[0133] In one embodiment of this application, the predicted bounding box can also represent the shape, size, etc. of the target object. During the target detection process of the monitoring device using a target detection model, features are extracted from the image using FPN networks, PAN networks, etc., to obtain a feature map with high-order feature representation. The monitoring device generates candidate regions on the feature map using methods such as anchor boxes. Then, the monitoring device classifies the candidate regions, i.e., determines the category to which each candidate region belongs. For candidate regions determined to be of the target object category, the position of the target object is accurately located using methods such as bounding box regression. In one embodiment of this application, after determining the predicted bounding box corresponding to the target object in the image to be detected, the monitoring device can further filter and merge multiple overlapping and similar predicted bounding boxes to further optimize the detection results, and finally output the optimized detection results, determining the predicted bounding box corresponding to the target object, thereby identifying the target object in the image to be detected.

[0134] In the target detection method provided in this embodiment, the monitoring equipment achieves real-time and efficient target detection of the monitoring video by utilizing a lightweight Featrinet model, and provides rapid feedback on the situation of candidates during the examination, effectively improving the discipline and fairness of the examination.

[0135] In one embodiment of this application, after determining the target object in the image to be detected based on the predicted bounding box, the target detection method further includes: scaling the predicted bounding box corresponding to the image to be detected based on the object size of the target object in the image to be detected, and determining the predicted target bounding box corresponding to the image to be detected; drawing the target bounding box in the corresponding image to be detected based on the predicted target bounding box to obtain the target image; and integrating all target images to obtain the target video.

[0136] In one embodiment of this application, since the image to be detected may be cropped or scaled during target detection, the actual size of the predicted bounding box, such as its width and height, may not match the actual size of the corresponding target object in the image. Therefore, the monitoring device also needs to scale the predicted bounding box corresponding to the image to be detected based on the object size of the target object in the image. The predicted bounding box has the same size as the target object. The target bounding box is then drawn in the corresponding image to be detected based on the predicted bounding box, resulting in a target image labeled with the target bounding box. The target bounding box is used to characterize the position, category, shape, and size of the target object in the target image. Finally, the monitoring device integrates all target images to obtain the target video. As an example, the monitoring device can use tools such as OpenCV to integrate all target images; this application does not limit the integration method.

[0137] Please see Figure 20This is an example flowchart illustrating the implementation process of the target detection method provided in this application embodiment. Figure 20 As shown, after receiving the surveillance video, the monitoring equipment uses a preprocessor to preprocess the video, converting it into target frame images. Then, the monitoring equipment uses a filter to filter the target frame images and performs image enhancement on the filtered valid frames to determine the image to be detected. During the target detection process using the target detection model, the monitoring equipment downsamples the image multiple times, using FPN networks, PAN networks, etc., to extract and fuse features to obtain high-level feature maps (such as small-size, medium-size, and large-size features). Since small target objects in the image have inherently low resolution, multiple downsampling may lead to the loss of some small target object features, reducing the number of small target samples and affecting the detection of small target objects. Therefore, the Featrinet model adds a high-speed feature super-resolution module. By performing high-speed super-resolution on the feature maps corresponding to small-size features, high-quality features of small target objects are added with a small increase in computation, improving the efficiency of small target object detection. Figure 20 In this context, small-size features refer to feature maps corresponding to small-sized objects, medium-size features refer to feature maps corresponding to medium-sized objects, and large-size features refer to feature maps corresponding to large-sized objects. After determining the feature maps corresponding to objects of different sizes, the monitoring equipment performs target localization and classification on each feature map to determine the predicted bounding box corresponding to the target object in the image to be detected. Further, based on the object size of the target object in the image to be detected, the monitoring equipment performs scale transformation on the predicted bounding box corresponding to the image to be detected to determine the predicted target bounding box. Then, based on the predicted target bounding box, the monitoring equipment performs target drawing on the corresponding image to be detected, drawing the target bounding box corresponding to the target object, thus obtaining the target image. The monitoring equipment integrates all target images through a post-processor to obtain the target video and outputs the target video.

[0138] In one embodiment of this application, after acquiring the target video, the monitoring device can also send the target video to the proctoring device, so that the proctor can further review the target video and identify any violations during the examination.

[0139] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0140] In one embodiment of this application, a model building apparatus 100 is provided, which corresponds one-to-one with the model building methods in the above embodiments. For example... Figure 21As shown, the model building device includes a design module 101 and a training module 102. Detailed descriptions of each functional module are as follows:

[0141] Design module 101 is used to fuse the feature pyramid network, feature scale fusion network, high-speed feature super-resolution module, and trident detection head network to build an initial detection model.

[0142] The training module 102 is used to perform sparsity training on the initial detection model using a preset training sample set, so that the initial detection model can predict the target object in each preset training sample in the preset training sample set, thereby obtaining the target detection model.

[0143] Specific limitations regarding the model building device can be found in the limitations of the model building method described above, and will not be repeated here. Each module in the aforementioned model building device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in the electronic device, or stored in the memory of the electronic device in software form, so that the processor can call and execute the operations corresponding to each module.

[0144] In one embodiment of this application, a target detection device 200 is provided, which corresponds one-to-one with the target detection methods described in the above embodiments. For example... Figure 22 As shown, the target detection device includes a data acquisition module 201 and a detection module 202. Detailed descriptions of each functional module are as follows:

[0145] The acquisition module 201 is used to acquire surveillance video and preprocess the surveillance video to obtain the image to be detected.

[0146] The detection module 202 is used to perform target detection on the image to be detected using a target detection model, and to determine the target object in the image to be detected.

[0147] Specific limitations regarding the target detection device can be found in the limitations of the target detection method described above, and will not be repeated here. Each module in the aforementioned target detection device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the monitoring equipment, or stored in software in the memory of the monitoring equipment, so that the processor can call and execute the corresponding operations of each module.

[0148] In one embodiment, an electronic device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 23As shown, the electronic device includes a processor, memory, network interface, etc., connected via a system bus. The processor provides computing and control capabilities. The memory includes a computer-readable storage medium and internal memory. The computer-readable storage medium stores computer-readable instructions. The internal memory provides an environment for the execution of the computer-readable instructions in the computer-readable storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer-readable instructions are executed by the processor, they implement a model building method. The computer-readable storage medium provided in this embodiment includes both non-volatile and volatile readable storage media.

[0149] In one embodiment, an electronic device is provided, which may be a terminal device. The electronic device may include a processor, memory, and a network interface connected via a system bus. The processor of the electronic device provides computing and control capabilities. The memory of the electronic device includes a computer-readable storage medium and internal memory. The computer-readable storage medium stores an operating system and computer-readable instructions. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the computer-readable storage medium. The network interface of the electronic device is used to communicate with an external server via a network connection. When the computer-readable instructions are executed by the processor, they implement a model building method. The computer-readable storage medium provided in this embodiment includes non-volatile readable storage media and volatile readable storage media.

[0150] In one embodiment, an electronic device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, it performs the following steps: fusing a feature pyramid network, a feature scale fusion network, a high-speed feature super-resolution module, and a triangular detection head network to construct an initial detection model; and performing sparse training on the initial detection model using a preset training sample set so that the initial detection model can predict the target object in each preset training sample in the preset training sample set, thereby obtaining a target detection model.

[0151] In one embodiment, one or more computer-readable storage media storing computer-readable instructions are provided. The computer-readable storage media provided in this embodiment include non-volatile readable storage media and volatile readable storage media. The computer-readable storage media stores computer-readable instructions, which, when executed by one or more processors, implement the following steps: fusing a feature pyramid network, a feature scale fusion network, a high-speed feature super-resolution module, and a triangular detection head network to construct an initial detection model; and performing sparse training on the initial detection model using a preset training sample set, so that the initial detection model predicts the target object in each preset training sample in the preset training sample set, thereby obtaining a target detection model.

[0152] In one embodiment, a monitoring device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 24 As shown, the monitoring device may include a processor, memory, and network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a computer-readable storage medium and internal memory. The computer-readable storage medium stores an operating system and computer-readable instructions. The internal memory provides an environment for the operation of the operating system and computer-readable instructions in the storage medium. The network interface is used to communicate with an external server via a network connection. When the computer-readable instructions are executed by the processor, a target detection method is implemented. The computer-readable storage medium provided in this embodiment includes both non-volatile and volatile readable storage media.

[0153] In one embodiment, a monitoring device is provided, including a memory, a processor, and computer-readable instructions stored in the memory and executable on the processor. When the processor executes the computer-readable instructions, it performs the following steps: acquiring monitoring video and preprocessing the monitoring video to obtain an image to be detected; and using an object detection model to perform object detection on the image to be detected to determine the target object in the image to be detected.

[0154] In one embodiment, one or more computer-readable storage media storing computer-readable instructions are provided. The computer-readable storage media provided in this embodiment include non-volatile readable storage media and volatile readable storage media. The computer-readable storage media stores computer-readable instructions, which, when executed by one or more processors, perform the following steps: acquiring surveillance video and preprocessing the surveillance video to obtain an image to be detected; using an object detection model to perform object detection on the image to be detected, and determining the target object in the image to be detected.

[0155] Those skilled in the art will understand that implementing all or part of the processes in the methods of the above embodiments can be accomplished by instructing related hardware with computer-readable instructions. These computer-readable instructions can be stored in a non-volatile readable storage medium or a volatile readable storage medium. When executed, these computer-readable instructions can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0156] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0157] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A model building method applied to electronic devices, characterized in that, The model construction method includes: An initial detection model is constructed by fusing a feature pyramid network, a feature scale fusion network, a high-speed feature super-resolution module, and a trident detection head network. The initial detection model is sparsely trained using a preset training sample set to predict target objects in each preset training sample in the preset training sample set, thereby obtaining a target detection model. This includes: determining the anchor box configuration of the target detection model based on an anchor box adaptive algorithm and the preset training sample set, and performing data augmentation on the preset training sample set to obtain an augmented training sample set; calculating the small target loss weight and the large target loss weight during the sparse training of the initial detection model using the anchor box configuration and the augmented training sample set; determining whether there is a loss imbalance in the initial detection model during the sparse training process based on the small target loss weight and the large target loss weight; if it is determined that there is a small target loss imbalance in the initial detection model during the sparse training process, using a preset graph... The augmentation method optimizes the augmented training sample set to obtain the optimized training sample set. If it is determined that the initial detection model has a large target loss imbalance during sparse training, the size of each augmented training sample in the augmented training sample set is adjusted to obtain the optimized training sample set. If it is determined that the initial detection model does not have a loss imbalance during sparse training, the augmented training sample set is used as the optimized training sample set to obtain the optimized training sample set. Based on the anchor box configuration and the optimized training sample set, the initial detection model is sparsely trained so that the initial detection model predicts the target object in each preset training sample in the preset training sample set to obtain the target detection model. The preset training samples include images, and the target objects include objects of a specific category in the images.

2. The model construction method as described in claim 1, characterized in that, After training the initial detection model sparsely using a preset training sample set, so that the initial detection model can predict the target object in each preset training sample in the preset training sample set, and obtaining the target detection model, the model construction method further includes: Deploy the target detection model to the monitoring equipment; Monitor the inference time of the monitoring equipment when performing the target detection task using the target detection model; If the inference time does not meet the preset real-time requirements, return to the step of executing the fusion feature pyramid network, feature scale fusion network, high-speed feature super-resolution module, and trident detection head network to construct the initial detection model.

3. The model construction method as described in claim 1, characterized in that, The fusion feature pyramid network, feature scale fusion network, high-speed feature super-resolution module, and triangular detection head network are used to construct the initial detection model, including: The initial detection model is constructed using a feature pyramid network, a feature scale fusion network, and a high-speed feature super-resolution module as feature encoding modules, and a tri-path detection head network as feature decoding modules. The feature encoding module is used to extract target features, and the feature decoding module is used to reconstruct the target object corresponding to the target features.

4. The model construction method as described in claim 1, characterized in that, The step of sparsifying the initial detection model using a preset training sample set, so that the initial detection model can predict the target object in each preset training sample in the preset training sample set, to obtain the target detection model, includes: The initial detection model is trained using a preset training sample set to reduce sparsity and compress the model parameters, resulting in a trained detection model. The detection accuracy of the trained detection model is then tested. If the detection accuracy is greater than or equal to a preset accuracy threshold, the trained detection model is pruned to obtain a pruned detection model. The size of the pruned detection model is detected; If the model size meets the preset model size requirements, the pruned detection model is quantized to obtain the target detection model; If the model size does not meet the preset model size requirement, return to the step of performing sparsification training on the initial detection model using the preset training sample set, compressing the model parameters, and obtaining the trained target detection model. If the detection accuracy is less than the preset accuracy threshold, return to the step of executing the fusion feature pyramid network, feature scale fusion network, high-speed feature super-resolution module, and trident detection head network to construct the initial detection model.

5. A target detection method based on the target detection model according to any one of claims 1 to 4, applied to monitoring equipment, characterized in that, The target detection method includes: Acquire surveillance video and preprocess the surveillance video to obtain the image to be detected; The target detection model is used to detect the target object in the image to be detected, thereby identifying the target object in the image to be detected.

6. The target detection method as described in claim 5, characterized in that, The preprocessing of the surveillance video to obtain the image to be detected includes: The surveillance video is converted into at least one video frame image, and the at least one video frame image is format-converted to obtain at least one target frame image; The effective frame images are filtered out from the at least one target frame image using a filter, and the effective frame images are enhanced to obtain the image to be detected.

7. The target detection method as described in claim 5, characterized in that, The step of using the target detection model to perform target detection on the image to be detected, and determining the target object in the image to be detected, includes: The target detection model is used to detect targets in the image to be detected, and a prediction box corresponding to the target object in the image to be detected is generated. The prediction box is used to represent the position and category of the target object. The target object in the image to be detected is determined based on the prediction box.

8. The target detection method as described in claim 7, characterized in that, After determining the target object in the image to be detected based on the prediction bounding box, the target detection method further includes: Based on the object size of the target object in the image to be detected, the prediction bounding box corresponding to the image to be detected is scaled to determine the prediction target bounding box corresponding to the image to be detected. Based on the predicted target bounding box, draw the target bounding box in the corresponding image to be detected to obtain the target image; By integrating all the target images, the target video is obtained.

Images