Detection model construction method, monitoring method, and monitoring system

Abstract

The application provides a detection model construction method, a monitoring method and a monitoring system. The detection model construction method can include: constructing scene data; training an initial detection model based on the scene data to obtain a target detection model; the target detection model is used for target detection on monitored images when the monitoring system is in an awake state; the target detection model is subjected to lightweight processing to determine a lightweight detection model; the quantization bit width of parameters of the lightweight detection model is lower than that of the target detection model; and the lightweight detection model is used for target detection on monitored images when the monitoring system is in a sleep stage. The method can reduce part of the calculation amount while maintaining the required accuracy.

Description

Technical Field

[0001] This application relates to the field of monitoring technology, and more specifically, to a detection model construction method, a monitoring method, and a monitoring system. Background Technology

[0002] In the field of surveillance, target monitoring (such as people and animals) is typically achieved by identifying targets in the video captured by surveillance cameras. However, surveillance cameras usually capture data continuously, meaning that a relatively large amount of video data needs to be identified. If a computationally intensive model is continuously used to identify targets in the images, the computational load may be relatively large. Summary of the Invention

[0003] The purpose of this application is to provide a detection model construction method, a monitoring method, and a monitoring system that can use different detection models for the dormant and awake states of the monitoring system, thereby reducing the amount of computation and maintaining accuracy even in the awake state.

[0004] In a first aspect, embodiments of this application provide a method for constructing a detection model, the method comprising: constructing scene data; training an initial detection model based on the scene data to obtain a target detection model; wherein the target detection model is used to perform target detection on monitored images when the monitoring system is in a wake-up state; performing lightweight processing on the target detection model to determine a lightweight detection model; wherein the quantization bit width of the parameters of the lightweight detection model is lower than the quantization bit width of the parameters of the target detection model, and the lightweight detection model is used to perform target detection on monitored images when the monitoring system is in a sleep state.

[0005] In the above implementation, different precision detection models are used for the sleep and wake states of the monitoring system to detect targets during the detection model construction. Since a relatively low-precision, lightweight detection model is used in the sleep state, it allows for the use of models with lower computational and hardware resource requirements while still meeting monitoring needs, thus improving the efficiency and reducing the computational burden of target detection. When the monitoring system is in the wake state, a more precise target detection model can be used to improve the accuracy of target detection.

[0006] In an optional implementation, training the initial detection model based on the scene data to obtain a target detection model includes: pre-training the initial detection model using the scene data to obtain a pre-trained detection model; wherein the training period for the pre-training is a first iteration; fine-tuning the pre-trained detection model using the scene data to obtain a fine-tuned detection model; wherein the training period for the fine-tuning is a second iteration, which is less than the first iteration; and distilling the fine-tuned detection model based on a teacher model to obtain a target detection model, wherein the distillation loss used in the distillation training is determined based on the loss of the fine-tuned detection model and the divergence loss between the outputs of the fine-tuned detection model and the teacher model; the training period for the distillation training is a third iteration, which is less than the second iteration.

[0007] In the above implementation, three stages of training are used in training the detection model, which can achieve good training results with relatively few training sessions.

[0008] In an optional implementation, the step of pre-training the initial detection model using the scene data to obtain a pre-trained detection model includes: pre-training the initial detection model using the scene data based on initial training parameters to obtain a current detection model; during training, verifying the model accuracy of the current detection model using validation data determined from the scene data every preset number of training iterations; updating the initial training parameters to obtain updated training parameters if the model accuracy does not meet the requirements; wherein the updated training parameters have a lower learning rate; and pre-training the initial detection model using the scene data based on the updated training parameters to obtain a pre-trained detection model.

[0009] The above implementation method also allows for monitoring of training effectiveness. If the training effectiveness does not meet the requirements, the learning rate can be adaptively adjusted, thereby improving the training accuracy of the detection model.

[0010] In an optional implementation, the initial detection model includes: a sampling module that outputs four feature maps; a feature fusion module for fusing the four feature maps; an attention mechanism module; and a result prediction module for outputting prediction results.

[0011] In an optional implementation, the sampling module includes: a first downsampling layer; a first group of RepVGG-X modules; a second group of RepVGG-X modules; a second downsampling layer disposed between the first group of RepVGG-X modules and the second group of RepVGG-X modules; a third downsampling layer disposed after the second group of RepVGG-X modules; a pooling layer; a fourth downsampling layer disposed after the pooling layer; a third group of RepVGG-X modules; a fifth downsampling layer disposed after the third group of RepVGG-X modules; a fourth group of RepVGG-X modules; and a first upsampling layer disposed after the fourth group of RepVGG-X modules; wherein the four feature maps output by the sampling module are the outputs of the third downsampling layer, the fourth downsampling layer, the fifth downsampling layer, and the first upsampling layer, respectively.

[0012] In the above implementation, the sampling module can output more feature maps, thereby obtaining more features of the image, which can make the recognition ability of the trained model relatively higher.

[0013] In an optional implementation, the result prediction module includes: a detection unit and a classification unit; the detection unit is used to determine detection boxes containing targets and to determine a confidence level for each detection box; the classification unit is used to classify the targets in the detection boxes based on the confidence levels of the detection boxes; the formula of the detection unit includes: ; in, This represents the confidence score of the i-th detection box; (i,j) represents the intersection-union ratio of the i-th detection box and the highest confidence detection box j; Indicates the IoU threshold; Indicates the attenuation coefficient; when At that time, the confidence level of the detection box is decayed through an exponential function.

[0014] In the above implementation method, by performing secondary calculation on the confidence level, the situation of erroneous deletion of some detection boxes can be reduced, the probability of the target being detected can be increased, and the accuracy of target detection can also be improved.

[0015] In an alternative implementation, the formula for the confidence threshold used by the classification unit includes: ; ; in, Indicates the baseline confidence threshold; L represents the ambient light intensity. Maximum illumination intensity; S represents the area of ​​the target pixel in the image; Indicates the maximum target size; Indicates the illumination adjustment factor; Indicates the size adjustment factor; This represents the minimum confidence threshold. This represents the maximum confidence threshold.

[0016] In the above implementation method, the confidence threshold can also be adaptively adjusted to adapt to the actual detection environment, thereby improving the accuracy of classification.

[0017] In an optional implementation, the step of lightweighting the target detection model to determine a lightweight detection model includes: converting a first precision of the parameters of the target detection model into a second precision; wherein the second precision is a lower precision than the first precision; pruning the channels of the convolutional layers in the target detection model according to a preset ratio to obtain a pruned detection model; and training the pruned detection model for a preset number of iterations to obtain a lightweight detection model.

[0018] In the above implementation, lightweight processing can include adjusting the quantization bit width of the model parameters, reducing the channels of the convolutional layers, and supplementing the training after the reduction. This can achieve a smaller model size while maintaining relatively high object detection accuracy.

[0019] In an optional implementation, the step of pruning the channels of the convolutional layers in the target detection model according to a preset ratio to obtain a pruned detection model includes: determining the weight norms of the channels in the convolutional layers of the target detection model; sorting the weight norms from smallest to largest; and deleting the channels of the convolutional layers corresponding to the weight norms at the top preset ratio to obtain the pruned detection model.

[0020] In the above implementation, the pruning process first sorts the channels and then deletes those with relatively smaller weight norms. This approach better preserves channels that contribute significantly to identification and maintains the accuracy of the detection model.

[0021] Secondly, embodiments of this application provide a monitoring method applied to a monitoring system. The monitoring method includes: in the sleep state of the monitoring system, using a lightweight detection model to detect a first image and determine a first detection result, wherein the lightweight detection model is constructed using the aforementioned detection model construction method; when the first detection result indicates that a target exists in the first image, waking up the monitoring system; in the awakened state of the monitoring system, using a target detection model to detect a second image and determine a monitoring result; wherein the target detection model is constructed using the aforementioned detection model construction method, and the parameter precision of the lightweight detection model is less than the parameter precision of the target detection model.

[0022] Thirdly, embodiments of this application provide a monitoring system, including: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the monitoring system is running, the machine-readable instructions are executed by the processor to perform the steps of the above-described method.

[0023] Fourthly, embodiments of this application provide a server, including: a processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the server is running, the machine-readable instructions are executed by the processor to perform the steps of the detection model construction method described above.

[0024] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the method described above.

[0025] Sixthly, embodiments of this application provide a computer program product comprising a computer program that, when executed by a processor, implements the above-described method. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a block diagram of a monitoring system provided in an embodiment of this application; Figure 2 A flowchart of the detection model construction method provided in the embodiments of this application; Figure 3An optional flowchart of step 220 of the detection model construction method provided in the embodiments of this application; Figure 4 This is a partial flowchart of the detection model construction method provided in the embodiments of this application.

[0028] Figure 5 An optional flowchart of step 230 of the detection model construction method provided in the embodiments of this application; Figure 6 A flowchart of the monitoring method provided in the embodiments of this application. Detailed Implementation

[0029] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0030] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0031] With the rapid development of artificial intelligence and Internet of Things technologies, intelligent monitoring systems have been widely used in security, wildlife observation, traffic management and other fields. However, traditional monitoring systems require continuous operation of high-definition cameras, video encoders and main processors, resulting in high energy consumption. This is particularly unfavorable for field monitoring scenarios powered by batteries or solar power. If the battery runs out, timely monitoring cannot be achieved. In solar-powered monitoring scenarios, the monitoring system may fail to monitor at night or when sunlight is weak. Current low-power monitoring solutions for monitoring systems mainly include: (1) periodically capturing images of the surrounding environment of the monitoring system, for example, taking images at preset time intervals, but this method may miss key events; (2) triggering recording by detecting changes in heat sources, but this is easily affected by the environment and has a high false alarm rate; (3) detecting moving objects by frame difference method, but it cannot distinguish between specific targets and non-targets (such as swaying leaves, changes in light and shadow), resulting in a large number of false triggers. Although the above solutions can meet some of the low-power requirements, it is difficult to guarantee high accuracy and there may be a large number of false alarms or missed alarms.

[0032] Based on the above research, the embodiments of this application can provide a detection model construction method, a monitoring method, and a monitoring system that can use different detection models for the sleep and wake-up states of the monitoring system, thereby reducing the amount of computation and maintaining accuracy even in the wake-up state.

[0033] To facilitate understanding of this embodiment, the monitoring system that implements the detection model construction method and monitoring method disclosed in this application embodiment will first be described in detail.

[0034] like Figure 1 The diagram shown is a block diagram of a monitoring system. The monitoring system 100 may include a memory and a processor. The memory stores machine-readable instructions executable by the processor. When the monitoring system is running, the machine-readable instructions are executed by the processor to perform the steps in the detection model construction method and monitoring method provided in this application embodiment.

[0035] The monitoring system 100 is divided into a hibernation monitoring module 110 and a full-function working module 120 according to their functions.

[0036] The sleep monitoring module 110 can provide low-power monitoring when the monitoring system is in sleep mode. The full-function module 120 can provide efficient and continuous monitoring.

[0037] The sleep monitoring module 110 is the core of the monitoring system's low-power operation. It can continuously perceive the surrounding environment and make preliminary judgments about target objects. While the monitoring system is in standby mode, it continuously runs a lightweight recognition algorithm. The wake-up process is triggered only when a potential target object is detected, waking up the full-function working module 120. The hardware configuration of the sleep monitoring module 110 needs to balance ultra-low power consumption with basic target detection performance. The sleep monitoring module 110 can run a lightweight detection model constructed using the detection model construction method provided in this application embodiment.

[0038] Optionally, the sleep monitoring module 110 may include a first processor, which may include a neural network processing unit (Micro NPU). The first processor may be a chip whose power consumption can be controlled within 0.1-0.5W (e.g., 0.1W, 0.2W, 0.3W, 0.5W), and whose computing power can reach 0.5-1 TOPS (INT8 quantization precision), supporting fixed-point arithmetic and lightweight model inference acceleration.

[0039] Optionally, the chip of the first processor can integrate a hardware-level power-saving unit, support dynamic clock gating technology, and automatically shut down redundant computing units when idle, further reducing static power consumption.

[0040] In this embodiment, the sleep monitoring module 110 may further include an image sensor. Exemplarily, this image sensor may be paired with a monochrome image sensor with a first range of pixel global shutter speeds. The image sensor may provide a low-resolution operating mode, where the first range can be a range with relatively low pixel values, such as 2-5 megapixels. Exemplarily, the image sensor's resolution may support two adjustable levels: 640×480@5fps and 1280×720@3fps. The global shutter of the image sensor may be designed to avoid motion blur.

[0041] The monochrome image sensor used can achieve a signal-to-noise ratio of no less than 55dB in low-light environments (0.01 lux), making it suitable for outdoor nighttime and indoor low-light scenarios. The image sensor supports a low-power wake-up mode, which only initiates data acquisition during the listening cycle and immediately enters sleep mode after acquisition is completed, with a single acquisition power consumption of no more than 0.05W.

[0042] Optionally, the sleep monitoring module 110 may include a first memory, which may operate at a voltage of 1.1V. This first memory may be used to store a computer program for a lightweight recognition algorithm. For example, the computer program for the lightweight recognition algorithm may occupy less than 1MB of memory. The first memory may also be used for temporary data acquisition (single frame image data size not exceeding 0.5MB) and algorithm runtime caching.

[0043] For example, the first memory can be 128MB-256MB LPDDR4 memory, operating at 1.1V, with a bandwidth of not less than 1600Mbps. Optionally, the first memory can support a partial self-refresh mode, maintaining low-power refresh only in the model storage area during hibernation, while powering down the remaining areas, thereby reducing the power consumption of the first memory.

[0044] In this embodiment, the sleep monitoring module 110 may include an auxiliary sensing unit.

[0045] Optionally, the auxiliary sensing unit may include sensors such as infrared sensors and sound sensors to form a multimodal sensing matrix. The power consumption of the infrared sensor is no more than 0.03W, and the power consumption of the sound sensor is no more than 0.02W. For example, the detection range of the infrared sensor can be 5-10 meters, and the temperature response range is -20℃ to 80℃, used to assist in identifying heat source targets. For example, the sound sensor can support sound acquisition in the 100-1000Hz frequency band, with a sensitivity of no less than -40dB, and can detect sound signals generated by target movement, which can be fused with image data to improve the initial target identification accuracy.

[0046] In this embodiment, the sleep monitoring module 110 may include an interface module. For example, this interface module may be equipped with I2C or SPI low-power interfaces for communicating with the adaptive power management module 130 and transmitting target detection results and wake-up requests. For example, this interface module may integrate a GPIO interface to support hardware-level wake-up signal output with a response latency of no more than 10ms.

[0047] In this embodiment, the full-function working module 120 can be started after the monitoring system is awakened. The full-function working module 120 can be used to acquire high-definition images or videos, and to achieve accurate identification of target objects, dissemination of identification data, and tracking of target objects based on high-definition images or videos. The full-function working module 120 can run a target detection model constructed by the detection model construction method provided in this application embodiment, and is used to detect targets.

[0048] The full-function module 120 may include a second processor. Exemplarily, the second processor may be a multi-core ARM Cortex-A series processor.

[0049] Optionally, the second processor can have a clock speed of 1.5-2.0 GHz and 4-8 cores. This second processor can be equipped with 1-4 GB of LPDDR4 memory (with a bandwidth of no less than 2133 Mbps), supports the NEON accelerated instruction set and hardware floating-point operations, achieving a computing power of 5-10 TOPS, meeting the needs of complex tasks such as high-definition video encoding, multi-target tracking, and face recognition.

[0050] Optionally, the second processor can support Dynamic Voltage and Frequency Scaling (DVFS) technology, which dynamically adjusts the main frequency (300MHz-2.0GHz) and core voltage (0.8V-1.2V) according to the task load, and the idle cores automatically go into sleep mode.

[0051] In this embodiment, the full-function working module 120 may include a camera. This camera may be a high-definition camera. For example, the camera may be a high-definition dual-light image sensor with 8-12 megapixels. Exemplarily, the camera may support both color and monochrome dual-mode switching. In one example, the camera's resolution in color mode may be 4K (3840×2160)@30fps, and the resolution in monochrome mode may be 1080P (1920×1080)@60fps, with a dynamic range of not less than 120dB. Exemplarily, in low light conditions (e.g., 0.1 lux), the signal-to-noise ratio in color mode is not less than 45dB, and the signal-to-noise ratio in monochrome mode is not less than 60dB.

[0052] Optionally, the camera can also be equipped with an infrared filter switching function, automatically switching to infrared mode at night, and working with an 850nm infrared fill light (for example, the power can be adjusted in the range of 1-3W) to achieve clear imaging at night within 20 meters.

[0053] In this embodiment, the full-function working module 120 may include a video encoder.

[0054] For example, the video encoder can be an integrated H.265 / H.264 hardware encoder, supporting multi-bitrate adaptive encoding with an adjustable bitrate range of 1-10Mbps, and supporting mode switching between Constant Bitrate (CBR) and Variable Bitrate (VBR). H.265 encoding is 30% more energy-efficient than H.264. Specifically, the power consumption for 1080P@30fps video encoding is no more than 1W, and for 4K@30fps video encoding, the power consumption is no more than 2W. It supports ROI (Region of Interest) encoding, encoding only the target area in high definition while degrading the background area, further reducing encoding power consumption and data volume.

[0055] In this embodiment, the full-function working module 120 may include a gimbal tracking system.

[0056] For example, the gimbal tracking system can use a miniaturized PTZ gimbal (e.g., weighing no more than 500g). The gimbal tracking system supports horizontal rotation of 0-355° (speed can be 0.5-10° / s) and vertical rotation of -10° to 90° (speed can be 0.5-8° / s), with a positioning accuracy of no more than 0.1°, and supports automatic target tracking and preset point navigation. The gimbal driver of the gimbal tracking system can use a stepper motor, with static power consumption of no more than 0.1W and dynamic operating power consumption of no more than 2W, and supports a power-off self-locking function to prevent target loss.

[0057] In this embodiment, the full-function working module 120 may include a communication module. This communication module can integrate multi-mode communication interfaces, such as 4G LTE Cat.4 / Cat.6 (power consumption no greater than 2W), 5G NR (power consumption no greater than 3W), Wi-Fi 6 (802.11ax, power consumption no greater than 1.5W), Ethernet (10 / 100 / 1000M, power consumption no greater than 0.8W), and a Bluetooth module, such as Bluetooth 5.0 (power consumption no greater than 0.3W).

[0058] Optionally, the communication module can support wired / wireless dual-mode switching. The 4G / 5G module can support remote data upload and command reception, with an upload rate of not less than 10Mbps; the Wi-Fi 6 module supports local high-speed data transmission (rate ≥1.2Gbps), adapting to short-range scenarios such as smart homes; the Bluetooth module is used for local debugging and linkage with low-power devices.

[0059] In this embodiment, the full-function working module 120 may include a second memory. The second memory may be local storage (e.g., 128GB-1TB eMMC 5.1 with a power consumption of no more than 0.5W) and a cloud storage interface.

[0060] Optionally, the local storage supported by the second storage device can be in a circular storage mode. For example, it supports circular overwrite (with an adjustable storage period). The cloud storage of the second storage device can support encrypted transmission (e.g., AES-256 encryption) and resume interrupted transmission, meeting the requirements for data security and long-term retention.

[0061] In this embodiment, the full-function working module 120 may include an alarm and linkage module. This alarm and linkage module may include an audible and visual alarm unit (buzzer power not greater than 0.5W, LED indicator power not greater than 0.1W), a relay output interface (supporting control of external devices such as infrared lights and sirens), and linkage interfaces (RS485, CAN), which can be linked with access control systems, fire-fighting equipment, drones, etc., to achieve multi-device collaborative security.

[0062] In this embodiment, the monitoring system may further include an adaptive power management module 130. The adaptive power management module 130 can be used to control the power supply, regulate the voltage, monitor the power consumption, and process the wake-up signal for each module. Through the collaboration of hardware circuits and software algorithms, it can achieve fine-grained power consumption management.

[0063] The power supply for each module of the monitoring system can support multiple power inputs, including lithium batteries (e.g., 3.7V / 12V, capacity 10-100Ah), solar panels (e.g., 10-50W), Power over Ethernet (PoE) (e.g., 802.3af / at, 48V input), and 12V DC power, adapting to the power supply requirements of different application scenarios. For power supply functionality, a built-in power management chip can be incorporated, with a conversion efficiency of no less than 95%, supporting multiple independent outputs (3.3V, 5V, 12V) to power each module separately.

[0064] In this embodiment, the adaptive power management module 130 can provide dynamic voltage regulation. For example, a dual mechanism of DVFS+DVS (Dynamic Voltage Regulation) can be used to adjust the supply voltage according to the operating state of each module. For instance, the sleep monitoring module 110 operates at a supply voltage of 3.3V, which drops to 1.8V during sleep. The second processor operates at 0.8V under light load, rising to 1.2V under heavy load. Through coordinated regulation of voltage and frequency, optimal matching of power consumption and performance is achieved.

[0065] In this embodiment, the adaptive power management module 130 may include a power monitoring unit. For example, the power monitoring unit may include a high-precision current sensor (e.g., accuracy ±1%) and a voltage sensor (e.g., accuracy ±0.5%), which monitors the current and voltage data of each module in real time, with a sampling frequency of 100Hz, calculates the real-time power consumption of each module (P=U×I), and uploads the data to the second processor for input to the adaptive power adjustment algorithm.

[0066] In this embodiment, the adaptive power management module 130 may include a wake-up control unit. The wake-up control unit may include a dedicated wake-up controller (power consumption not greater than 0.01W), which receives the wake-up request from the sleep monitoring module 110, triggers the power supply start-up of the full-function working module 120 according to preset logic, and supports hardware-level fast wake-up (start-up time not greater than 50ms) and software-level delayed wake-up (for example, the delay can be adjusted in the range of 0-1000ms), avoiding power waste caused by false triggering.

[0067] In this embodiment, the adaptive power management module 130 may include an energy storage and protection unit. For example, the energy storage and protection unit may be a supercapacitor (e.g., with a capacity of 1-5F) to cope with sudden power outages or voltage fluctuations, ensuring the safe storage of critical data (such as target detection results and video clips); it integrates overvoltage, overcurrent, and overtemperature protection circuits, automatically cutting off power supply to protect hardware devices when the supply voltage exceeds the rated value by 15%, the current exceeds the rated value by 20%, or the module temperature exceeds 65°C.

[0068] In this embodiment, in order to make the monitoring system more adaptable to the environment, more devices that adapt to the environment can be set up.

[0069] Optionally, each component of the monitoring system (the sleep monitoring module 110 and the components included in the full-function working module 120) can be equipped with a housing. Optionally, the housing is made of aluminum alloy (thermal conductivity not less than 200W / (m·K)), and the housing has built-in heat sinks and thermal pads. In high-temperature environments (60℃), heat dissipation is assisted by a fan (e.g., fan power not greater than 0.5W, temperature-controlled start), and in low-temperature environments (-40℃), a heating film (power not greater than 1W, temperature-controlled start) ensures the normal operation of the components.

[0070] In this embodiment, the PCB board of the monitoring system can adopt a three-proof coating (moisture-proof, salt spray-proof, mildew-proof), the interface adopts a waterproof aviation plug (IP67 protection level), and the shell is sealed. It supports long-term operation in environments with relative humidity of 10%~90% (non-condensation), and is suitable for humid scenarios such as the seaside and rainforest.

[0071] In this embodiment, the power supply circuit of the monitoring system adopts an EMC filter module to suppress electromagnetic interference; the communication interface adopts differential signal transmission, which improves the anti-interference capability by 30%; the image sensor lens is equipped with an anti-fog and anti-reflection coating to avoid the impact of severe weather (rain, snow, fog) on ​​the imaging quality.

[0072] The monitoring system 100 in this embodiment can be used to execute various steps in the various methods provided in the embodiments of this application. The implementation process of the detection model construction method and the monitoring method is described in detail below through several embodiments.

[0073] Please see Figure 2 This is a flowchart of the detection model construction method provided in this application embodiment. The detection model construction method provided in this application embodiment can be applied to a monitoring system or run on a server, through which the steps in the detection model construction method are executed. After the server completes the model construction, the model can be transmitted to the monitoring system for execution.

[0074] The following will be about Figure 2 The specific process shown will be explained in detail.

[0075] Step 210: Construct scene data.

[0076] Scene data can include multiple sample images. Some sample images may contain various targets, while others may not. The backgrounds of the sample images containing targets can be diverse.

[0077] To improve the applicability of models trained based on scene data, data from various scenes can be used for training. Optionally, scene data can be constructed from scenes such as the wild (e.g., forests, grasslands, mountains), urban roads, smart homes (e.g., courtyards, doorways, living rooms), and extreme environments (rainy days, snowy days, nighttime).

[0078] Depending on the specific monitoring needs, the monitoring target may vary. For example, the monitoring target might be a human figure, such as an adult, a child, or a person in different postures. For example, the monitoring target might be a vehicle, such as a car, truck, motorcycle, or bicycle. Of course, depending on the specific requirements, the monitoring target can also include both human and vehicle figures.

[0079] To better train the model's recognition capabilities, scene data can include not only different targets but also different backgrounds. For example, the background of images in scene data can include swaying leaves, changes in light and shadow, small animals, rustling grass, and different types of buildings.

[0080] For example, the number of images in the scene data can be pre-set according to training requirements. For instance, the number of images in the scene data can be no less than 50,000. Among them, there can be at least 25,000 sample images of human targets (e.g., no less than 20,000 positive sample images and no less than 5,000 negative sample images), no less than 20,000 sample images of vehicle targets (e.g., 16,000 positive sample images and 4,000 negative sample images), and 5,000 background interference samples. Of course, based on actual needs, the number of each type of sample image can be more or less.

[0081] Optionally, the sample images in the scene data can be in the following formats: image resolution 640×640, format JPG / PNG, annotation format VOC (XML) or COCO (JSON), and the annotation box standard can ensure that the selected target is complete and does not exceed the target range.

[0082] To improve the generalization ability of models trained on scene data, we can first obtain original sample images by taking pictures, and then enhance the original sample images to make the obtained sample images richer.

[0083] Optionally, the enhancement process may include geometric transformations. Geometric transformations may include random cropping, for example, a cropping ratio of 0.6-1.0. Geometric transformations may include random flipping, for example, flipping a portion of the sample image horizontally or vertically. Geometric transformations may include random rotation, for example, a rotation angle of -15° to 15°. Geometric transformations may include scaling the sample image, for example, scaling it by 0.8 to 1.2 times.

[0084] Optionally, the enhancement process may include color transformation. Color transformation may include randomly adjusting brightness, for example, by ±20%. Color transformation may include adjusting contrast, by ±20%. Color transformation may include adjusting saturation, by ±20%. Color transformation may include adjusting hue, by ±10°. Color transformation may also include simulating different lighting conditions. It is understood that the aforementioned proportions for various color transformation adjustments are merely exemplary, and actual adjustments may be more or less than the aforementioned proportions.

[0085] Optionally, the enhancement process may include adding noise. Adding noise may involve adding Gaussian noise, for example, with a variance of 0.001-0.01. Adding noise may involve adding salt-and-pepper noise, with a noise density of 0.001-0.005. Adding noise may simulate sensor imaging noise. It is understood that the aforementioned amounts of added noise are merely illustrative, and actual adjustments may be more or less than the amounts described above.

[0086] Optionally, the enhancement process may include occlusion simulation. For example, randomly adding rectangular occlusions, with an occlusion area of ​​5%-15%, to simulate a scene where part of the target is occluded.

[0087] Enhancement processing can increase the number of sample images from 50,000 to 200,000, ensuring sufficient training for the model.

[0088] In the embodiments provided in this application, the obtained scene data can be divided into training data, validation data, and test data based on training requirements. For example, the ratio of training data, validation data, and test data can be 7:2:1, respectively. Taking a total of 200,000 sample images in the scene data as an example, the training data can contain 140,000 sample images for model parameter learning; the validation data can contain 40,000 sample images for hyperparameter adjustment and model selection during training; and the test data can contain 20,000 sample images for final model performance evaluation. The test data does not participate in the training and validation processes.

[0089] Step 220: Train the initial detection model based on the scene data to obtain the target detection model.

[0090] The target detection model is used to detect targets in the monitored images when the monitoring system is in a wake-up state.

[0091] The initial detection model can be trained using the training data from the aforementioned scene data.

[0092] The initial detection model provided in this application can be a model determined based on YOLOv5. The difference between the initial detection model in this application and the traditional YOLOv5 model is that the backbone network C3 module of YOLOv5 is replaced with the RepVGG-X module.

[0093] The initial detection model outputs four feature maps: 80×80×256 (for small target detection), 40×40×512 (for medium target detection), 30×30×512 (for medium to large target detection), and 20×20×1024 (for large target detection). These four feature map sizes can cover target sizes ranging from 10×10px to 640×640px.

[0094] Step 230: Lightweight the target detection model to determine the lightweight detection model.

[0095] Optionally, lightweight processing may include adjusting the quantization bit width of the parameters of the object detection model. Lightweight processing may include pruning the channels of the object detection model. Lightweight processing may include compressing the network structure of the object detection model. Lightweight processing may also include distillation training on the detection model obtained after parameter adjustment and pruning to achieve higher detection accuracy of the lightweight model.

[0096] The lightweight detection model uses a lower quantization bit width than the target detection model. It performs target detection on images captured by the monitoring system while the system is in sleep mode. For example, the quantization bit width of the lightweight detection model can be INT8, while the target detection model can use FP32. Of course, other precisions can be selected based on the hardware capabilities of the actual monitoring system. For instance, if the hardware performance of the monitoring system is relatively lower in sleep mode, a precision lower than INT8 can be used.

[0097] Alternatively, lightweight processing can include making the object detection model relatively less computationally intensive, or requiring less storage resources to store the relevant data of the object detection model.

[0098] By implementing the above methods, the monitoring system can enable models with different levels of precision in different states. In sleep mode, a lightweight detection model that requires less computing and storage resources can be used; in wake-up mode, a more precise target detection model can be used, which can better meet the actual needs of the monitoring system in different states.

[0099] In the embodiments provided in this application, to improve training effectiveness and efficiency, the training of the detection model can be implemented using three stages: pre-training, fine-tuning, and distillation. Based on this, as... Figure 3 As shown, step 220 above may include steps 221 to 223.

[0100] Step 221: Use scene data to pre-train the initial detection model to obtain a pre-trained detection model.

[0101] The training cycle for pre-training is the first iteration round.

[0102] Optionally, step 221 above may include pre-training the initial detection model using scene data based on the initial training parameters to obtain the current detection model. During training, every preset number of training iterations, the accuracy of the current detection model is verified using validation data determined from the scene data. If the model accuracy does not meet the requirements, the initial training parameters are updated to obtain updated training parameters. Based on the updated training parameters, the initial detection model is pre-trained using scene data to obtain a pre-trained detection model.

[0103] The updated training parameters have a lower learning rate.

[0104] For example, before training the initial detection model, the weights of the initial detection model can be initialized. Optionally, pre-trained weights from YOLOv5s can be used, and the newly added parameters of the RepVGG-X module in the initial detection model can be uniformly initialized using Xavier. For example, the Xavier formula can be expressed as: ; a represents the number of input channels, and b represents the number of output channels.

[0105] The batch size used to train the initial detection model can be adjusted according to the server's GPU memory. For example, if the server has a single GPU (e.g., 16GB), the batch size can be set to 32; if it has multiple GPUs (e.g., 4×16GB), the batch size can be set to 128.

[0106] The initial learning rate used to train the initial detection model can be 0.01 for the backbone network and 0.02 for the head network. To adapt to changing training requirements, a cosine annealing learning rate adjustment strategy can be used.

[0107] Stochastic gradient descent (SGD) can be used to adjust the parameters of the detection model during training. For example, the momentum of SGD is 0.9, and the weight decay can be 0.0005. Of course, the values ​​of momentum and weight decay can be chosen according to different actual needs.

[0108] The loss function used for pre-training can be bounding box regression loss (CIoU loss) + Focal loss (classification loss) + non-maximum suppression (DIOU-NMS).

[0109] The first iteration of the training cycle can be 100 epochs.

[0110] The preset number of training iterations can be a pre-set value to verify the effectiveness of model training during the pre-training process. This preset number of training iterations can be 10 epochs, 15 epochs, or 18 epochs.

[0111] During pre-training, the mean average precision (mAP) and loss value can be calculated using validation data from the scene data. When the mean average precision does not improve after a preset number of training iterations, the current learning rate can be reduced. For example, the learning rate can be reduced to 1 / 10 of the original.

[0112] Step 222: Use scene data to fine-tune the pre-trained detection model to obtain the fine-tuned detection model.

[0113] The training cycle for fine-tuning is the second iteration, which is shorter than the first iteration.

[0114] In fine-tuning training, training and validation data from the scenario data can be used.

[0115] By adjusting the hyperparameters used in pre-training, the learning rate can be adjusted to 0.001. The batch size can be kept at 32, and the second iteration of the training cycle can be 50 epochs, 60 epochs, 65 epochs, etc.

[0116] During fine-tuning training, a Dropout layer (e.g., dropout rate=0.1) and Label Smoothing (smoothing coefficient=0.1) can be added to prevent overfitting.

[0117] Optionally, during the fine-tuning training phase, the model with the highest average accuracy on the validation data can be selected as the fine-tuned model for testing. Alternatively, during the fine-tuning training phase, a model with an average accuracy of not less than 90% on the validation data can be selected as the fine-tuned model for testing.

[0118] Step 223: Based on the teacher model, perform distillation training on the fine-tuned detection model to obtain the target detection model.

[0119] The distillation loss used in the distillation training is determined based on the loss of the fine-tuned detection model and the divergence loss between the output of the fine-tuned detection model and the teacher model; the training cycle for the distillation training is the third iteration, which is shorter than the second iteration.

[0120] The teacher model can be a large model. For example, the teacher model can be a YOLOv5x model, which has 8.9 MB of parameters, 13.2 GFLOPs of computation, and mAP@0.5=95%.

[0121] The fine-tuned detection model obtained during the fine-tuning training phase can be used as a student model.

[0122] The distillation loss function can be expressed by the following formula: ; in, This represents the loss between the student model and the real label, which is the loss of the fine-tuned detection model after further distillation training. This represents the KL divergence loss between the student model and the teacher model outputs. These are the weighting coefficients.

[0123] The value can range from 0.3 to 0.5.

[0124] The third iteration of the distillation training cycle can be 30 epochs, 35 epochs, etc. The initial learning rate can be set to 0.0005, and the batch size can be set to 32.

[0125] The detection model obtained after distillation training can achieve a mAP@0.5 of not less than 85%, and the computational cost can be reduced by more than 96% compared to the teacher model.

[0126] By employing the three training stages described above, the detection model can be trained with relatively fewer iterations.

[0127] In the embodiments provided in this application, the initial detection model includes: a sampling module that outputs four feature maps; a feature fusion module for fusing the four feature maps; an attention mechanism module; and a result prediction module for outputting prediction results.

[0128] Optionally, the sampling module may include six samplings and feature processing from multiple RepVGG-X modules.

[0129] The sampling module may include a first downsampling layer, which may be a Conv2d layer, and the Conv2d layer may be a 3×3 convolution with a stride of 2. The first downsampling layer is used to implement the first downsampling, and the size of the output feature map of the first downsampling layer may be 320×320×64.

[0130] The sampling module may include a first group of RepVGG-X modules positioned after the first downsampling layer; the first group of RepVGG-X modules may contain two RepVGG-X modules. A second downsampling layer, which can also be a Conv2d layer, is positioned after the first group of RepVGG-X modules. This second downsampling layer performs a second downsampling, and the output feature map size can be 160×160×128.

[0131] Among them, the RepVGG-X module in the RepVGG-X module group can be used to extract features, and the RepVGG-X module does not change the size.

[0132] After the second downsampling layer, a second group of RepVGG-X modules can be applied; after the second group of RepVGG-X modules, a third downsampling layer is applied; the third downsampling layer can be a Conv2d layer. After the third downsampling layer, the output feature map size can be 80×80×256.

[0133] A pooling layer is set after the third downsampling layer; a fourth downsampling layer is set after the pooling layer.

[0134] This pooling layer can be a Spatial Pyramid Pooling-Fast (SPPF) module.

[0135] The fourth downsampling layer performs the fourth downsampling, and the size of the output feature map can be 40×40×512.

[0136] A third RepVGG-X module group is set after the fourth downsampling layer; a fifth downsampling layer is set after the third RepVGG-X module group.

[0137] The third group of RepVGG-X modules can include two RepVGG-X modules. The fifth downsampling layer can perform a fifth downsampling, and the size of the output feature map can be 20×20×1024.

[0138] A fourth RepVGG-X module group is set after the fifth downsampling layer; a first upsampling layer is set after the fourth RepVGG-X module group.

[0139] The fourth group of RepVGG-X modules can contain one RepVGG-X module. The first upsampling layer can perform one upsampling operation, and the size of the output feature map can be 30×30×512.

[0140] The four feature maps output by the sampling module are the outputs of the third downsampling layer, the fourth downsampling layer, the fifth downsampling layer, and the first upsampling layer.

[0141] In the embodiments provided in this application, the RepVGG-X module may include three parallel branches: a main branch, a first auxiliary branch, and a second auxiliary branch.

[0142] Optionally, the main branch can be a branch containing 3×3 convolutions, batch normalization layers (BatchNorm), and activation functions. The activation function can be a rectified linear unit (ReLU).

[0143] The main branch serves as the core feature extraction channel, used to extract the core features of the feature map. The 3×3 convolution is the fundamental structure for image feature extraction, and combined with batch normalization and activation functions, it ensures the effective capture of core semantic features (such as human and vehicle contours).

[0144] Optionally, the first auxiliary branch can be a branch containing 1×1 convolutions, a batch normalization layer (BatchNorm), and an activation function. The activation function can be a rectified linear unit (ReLU). As a lightweight dimensionality reduction branch, the first auxiliary branch can reduce the number of convolutional kernels by a reduction ratio r (e.g., the reduction ratio can be a value between 4 and 16), and the computational cost is only a low proportion of the original 3×3 convolution (e.g., when r=4, the computational cost is 1 / 16 of that of a 3×3 convolution). While reducing the training burden, it supplements fine-grained features and avoids the omission of core features.

[0145] Optionally, the second auxiliary branch can be a direct connection branch. This second auxiliary branch serves as a low-level feature preservation channel, and is only activated when the number of input and output channels and the stride are consistent. It is equivalent to an "identity mapping" and can directly transmit the original low-dimensional features, enhancing the integrity of feature representation and alleviating the gradient vanishing problem in deep networks.

[0146] In one optional implementation, the three branches of the RepVGG-X module run in parallel during the training phase. During the inference phase, the RepVGG-X module, which contains three parallel branches, can be transformed into a single-branch 3×3 convolutional structure using structural reparameterization techniques.

[0147] like Figure 4 As shown, the implementation of converting the RepVGG-X module containing three parallel branches into a single-branch 3×3 convolutional structure may include steps 310 to 320.

[0148] Step 310: Convert the three branches of the RepVGG-X module into a structure of the target size.

[0149] The target size can be a 3×3 convolution kernel. For the first auxiliary branch, the 1×1 convolution kernel can be expanded into a 3×3 convolution kernel (with the original parameters in the center and zeros padded elsewhere) to achieve the same size as the 3×3 convolution kernel in the main branch.

[0150] For the second auxiliary branch, when the input and output channels are consistent, it is equivalent to a 3×3 convolution kernel (with 1 in the center and 0s padded elsewhere), thus completing the structural alignment.

[0151] Step 320: Perform a fusion calculation on each branch.

[0152] The 3×3 convolution kernels of the three branches are merged into a unified 3×3 convolution kernel by weighted summation of the convolution kernels of each branch and the scaling factor (γ / σ) of the batch normalization layer.

[0153] The parameters of the batch normalization layer are integrated by weighted calculation of the mean and standard deviation of each branch, and the normalization effect is finally incorporated into the convolution kernel. During inference, the batch normalization layer can be directly removed to further reduce power consumption.

[0154] The fusion formula for convolution kernels can be expressed as: ; Wherein, K represents the unified 3×3 convolution kernel after fusion (used in the inference phase). , , These are the main branch, the first auxiliary branch, and the second auxiliary branch (which are the branches used during the training phase). , , These represent the scaling factors of the batch normalization layer corresponding to the main branch, the first auxiliary branch, and the second auxiliary branch, respectively. , , These represent the standard deviations of the batch normalization layers corresponding to the main branch, the first auxiliary branch, and the second auxiliary branch, respectively.

[0155] By using the above fusion formula for convolution kernels, the normalization parameters (γ / σ) of each branch are weighted and fused into the convolution kernels of the three branches, eliminating multi-branch redundancy. In the inference stage, feature extraction can be completed with only one 3×3 convolution.

[0156] The fusion formula for the batch normalization layer can include: The formula for fusing the mean values ​​of the batch normalized layers for each branch can be given to me: ; Standard deviation fusion of batch normalization layers for each branch: ; Fixed parameters: γ=1, β=μ Where μ, σ, γ, and β represent the parameters of the fused batch normalized layer; ~ These represent the mean values ​​of the batch normalization layers corresponding to the main branch, the first auxiliary branch, and the second auxiliary branch, respectively. ~ This represents the offset coefficient of the batch normalization layer corresponding to the main branch, the first auxiliary branch, and the second auxiliary branch.

[0157] By using the fusion formula of the batch normalization layer described above, the normalized feature distribution information of the three branches can be integrated, and the effect of the batch normalization layer can be incorporated into the fused convolutional kernel. During inference, the batch normalization layer can be directly removed, further reducing the amount of computation.

[0158] During the training phase, three parallel branches can be used to ensure the feature representation capability. When using the trained detection model for inference after training, a single branch simplifies the calculation process, ultimately reducing the amount of computation and shortening the inference time.

[0159] The feature fusion module in this embodiment can be used to fuse feature maps of various sizes output by the sampling module. The four sizes of feature maps output by the sampling module can be: (80×80×C) (40×40×C) (30×30×C) (20×20×C).

[0160] In this embodiment, the feature map can first be adjusted to a uniform size H×W×C through interpolation or downsampling, and then fused into a final feature map through adaptive weight calculation. .

[0161] The feature fusion module can be implemented using Adaptive Spatial Feature Fusion (ASFF-X). For example, the fusion formula used by the feature fusion module may include: ; ; ; in, Let (i, j) represent the feature value after adjustment of the feature map at the k-th scale, (i, j) represent the pixel coordinates of the feature map, and c represent the channel index. This represents the adaptive weight of the feature map at position (i,j,c) of the k-th scale, with values ​​ranging from [0,1], satisfying... ; The weighted score is represented by a 1×1 convolutional layer (Conv) and a fully connected layer ( This is the weight matrix. Calculated with bias; , representing a learnable parameter, which adaptively adjusts the contribution of features at different scales through training.

[0162] The formula of the feature fusion module mentioned above achieves dynamic fusion of multi-scale features through adaptive weights at the pixel and channel levels. Compared with traditional fixed weight fusion, it can adjust the weights of features at each scale according to the target position and size. Normalization using the Softmax function ensures that the sum of the weights is 1, thus preventing a feature at a certain scale from becoming overly dominant. Local feature information is extracted through 1×1 convolution, and the fully connected layer learns weight scores, making the weight allocation strongly correlated with the features of the monitored target (such as small target regions). Higher weight, larger target area (Higher weighting). The number of channels C is set to 256 / 512 / 1024 (consistent with the feature map channels) to ensure that feature information is not lost during the fusion process. Improvement effect: ASFF-X technology improves the detection accuracy of small targets, and the detection accuracy of medium and large targets can also be improved, thereby achieving an overall improvement in detection accuracy.

[0163] In the embodiments provided in this application, the attention mechanism module can be a simplified channel attention module (SCA-Module), which enhances effective features and suppresses ineffective features by adaptively adjusting the weights of feature channels.

[0164] The input to the attention mechanism module is a feature map, which can be represented as: The output is a weighted feature map. .

[0165] The calculation formula for the attention mechanism module may include: ; ; in, This represents the attention weight of the c-th channel. The value range is [0,1]. This represents the Sigmoid activation function, used to normalize weights to the [0,1] interval. The formula is... ; ReLU activation function, used to introduce nonlinearity, is expressed by the formula: ; This represents the weight matrix of the first fully connected layer. represents the weight matrix of the second fully connected layer; r represents the channel reduction ratio, which can range from 4 to 16 and is used to reduce the computational cost of the fully connected layer. Let H represent the feature value of the c-th channel at position (i,j) of the feature map, and H and W represent the height and width of the feature map, respectively. This represents the global average pooling result of the c-th channel, reflecting the global feature strength of that channel; This represents the feature value at position (i,j) of the c-th channel after weighting.

[0166] Through the global average pooling operation in the formula Under the influence of this mechanism, global information for each channel can be extracted, avoiding the bias of local features and making the attention weights more globally adaptable. In the weight matrix of the first fully connected layer... Under the influence of this, the number of channels can be reduced from C to C / r, decreasing the number of parameters in the fully connected layer (from C×C to C×(C / r)+(C / r)×C=2C² / r). When r=8, the number of parameters is only 1 / 4 of the original. In the weight matrix of the second fully connected layer... Under the influence of this mechanism, the number of channels can be restored from C / r to C, ensuring that the attention weights are consistent with the number of input channels, thus achieving channel-by-channel weighting. (This is achieved through the ReLU activation function.) Introducing nonlinearity enables the model to learn complex channel dependencies. This is achieved through the Sigmoid function. The weights are normalized so that the weights of effective channels approach 1 (enhancement) and the weights of ineffective channels approach 0 (suppression). The value of the reduction ratio r needs to balance computational load and accuracy. For example, when r=4, the computational load increases by 4.8% and the accuracy improves by 3%; when r=16, the computational load increases by 1.2% and the accuracy improves by 2%. The reduction ratio r can be adjusted based on actual accuracy requirements or flexibly selected based on hardware computing power.

[0167] With the help of the attention mechanism module, the false alarm rate of the detection model can be reduced in complex backgrounds (such as swaying leaves and changes in light and shadow), and the detection accuracy of low-contrast targets (such as light-colored vehicles at night) can also be improved. At the same time, the amount of computation increases only slightly, which can be adapted to the low computing power deployment of the monitoring system in the dormant state.

[0168] The result prediction module in the embodiments provided in this application includes a detection unit and a classification unit.

[0169] The detection unit is used to identify detection boxes containing targets and to determine the confidence level for each detection box; the classification unit is used to classify the targets in the detection boxes based on the confidence level of the detection boxes.

[0170] In the embodiments provided in this application, the formula for the confidence threshold used by the classification unit includes: ; ; in, Indicates the baseline confidence threshold; L represents the ambient light intensity. Maximum illumination intensity; S represents the area of ​​the target pixel in the image; Indicates the maximum target size; Indicates the illumination adjustment factor; Indicates the size adjustment factor; This represents the minimum confidence threshold. This represents the maximum confidence threshold.

[0171] Optionally, the confidence threshold can be a pre-set value, for example, the confidence threshold can be 0.5, 0.6, etc.

[0172] The maximum value of ambient light intensity can be the light intensity in a strong light environment. =10000 lux. The maximum target size can be any value. =640×640=409600px².

[0173] Optionally, size adjustment factor It can be between 0.1 and 0.15.

[0174] Optionally, the minimum confidence threshold can be set in the range of 0.28 to 0.35, for example, =0.3, by setting a minimum threshold, excessive false alarms can be prevented. The maximum confidence threshold can be set in the range of 0.65 to 0.75, for example, =0.7, by setting the maximum threshold, false negatives can be prevented.

[0175] By using the above formula for the confidence threshold, the confidence threshold can be adjusted by combining the dynamic feedback of ambient light and target size. In strong light and large target scenarios, the threshold can be increased to reduce false alarms, while in weak light and small target scenarios, the threshold can be decreased to avoid missed alarms, thus adapting the threshold to the detection needs of different scenarios.

[0176] To avoid the accidental deletion of multiple overlapping detection boxes, the determined confidence score can be further adjusted using the following formula: ; in, This represents the confidence score of the i-th detection box; (i,j) represents the intersection-union ratio of the i-th detection box and the highest confidence detection box j; Indicates the IoU threshold; Indicates the attenuation coefficient; when At that time, the confidence level of the detection box is decayed through an exponential function.

[0177] The value can range from 0.3 to 0.5. Attenuation coefficient. The value can range from 0.5 to 1.0; when In this case, instead of setting the confidence level directly to zero, the detection box confidence is decayed by an exponential function, thus preserving the detection boxes of overlapping targets and improving the accuracy of multi-target detection.

[0178] By combining the confidence adjustment formula mentioned above, the detection accuracy of overlapping targets (such as dense crowds or multiple vehicles) is improved, and the occurrence of false deletion of detection boxes is reduced.

[0179] Through the above implementation logic, a target detection model with relatively higher accuracy can be obtained. However, for monitoring systems in a dormant state, the processing power of the dormant listening module 110 activated in this state is relatively low. Therefore, the target detection model is further lightweighted for the dormant state. Based on this, as... Figure 5 As shown, step 230 above may include steps 231 to 233.

[0180] Step 231: Convert the first precision of the target detection model parameters to the second precision.

[0181] The second precision is a precision that is lower than the first precision.

[0182] For example, the first precision can be FP32, and the second precision can be FP32.

[0183] In the parameter precision quantization method, the quantization bit width of the target detection model parameters is converted from FP32 precision to INT8 precision. During the precision conversion process, validation data from the scene data can be used for calibration, and the minimum mean square error (MSE) calibration method is adopted to ensure that the quantization precision loss is relatively low, while the computational load in the inference stage can be significantly reduced.

[0184] The quantization tools used for precision conversion in step 231 above can be tools such as TensorRT or ONNX Runtime. Using step 231, the number of model parameters can be significantly reduced, inference speed can be improved, and inference power consumption can be significantly reduced.

[0185] Step 232: Prune the channels of the convolutional layer in the target detection model according to a preset ratio to obtain a pruned detection model.

[0186] Step 233: Train the pruning detection model for a preset number of iterations to obtain a lightweight detection model.

[0187] Step 232 above may include: assigning weight norms to the channels of the convolutional layers in the object detection model; sorting the weight norms from smallest to largest; deleting the channels of the convolutional layers corresponding to the weight norms at the top of the sorted list by a predetermined proportion, thereby obtaining a pruned detection model.

[0188] The preset ratio can be a value set according to actual needs, for example, the preset ratio can be 30-40%.

[0189] After pruning, the computational cost of the model can be significantly reduced, and the number of model parameters can also be significantly reduced.

[0190] In this embodiment, after model training is complete, the model format can be converted from PyTorch to ONNX format, and then converted to a proprietary format supported by the NPU (e.g., Horizon.bin format, Black Sesame.om format, etc.) using vendor tools. To improve inference efficiency, the inference task can be split into multiple computing units of the NPU, while overlapping data reading and inference computation to reduce latency.

[0191] Please see Figure 6 This is a flowchart of a monitoring method provided in an embodiment of this application. The monitoring method provided in this application can be applied to a monitoring system, through which the monitoring system executes the steps in the monitoring method. The following will describe... Figure 6 The specific process shown will be explained in detail.

[0192] Step 410: In the sleep state of the monitoring system, the lightweight detection model is used to detect the obtained first image and determine the first detection result.

[0193] The lightweight detection model is a model constructed using the detection model construction method.

[0194] The first image can be an image captured by the monitoring system in sleep mode.

[0195] Step 420: If the first detection result indicates that a target exists in the first image, the monitoring system is activated.

[0196] Step 430: In the wake-up state of the monitoring system, the target detection model is used to detect the obtained second image to determine the monitoring result.

[0197] The target detection model is constructed using the detection model construction method.

[0198] The parameter accuracy of lightweight detection models is lower than that of target detection models.

[0199] The second image can be an image captured by the monitoring system in a wake-up state.

[0200] In this embodiment, the monitoring system acquires images more frequently when it is awake than when it is asleep.

[0201] To adapt to the detection needs of new scenarios and new targets, online updates of the detection model can be detected.

[0202] Optionally, online updates to the detection model may include the server training a new model: training a new model according to the training process provided in the aforementioned embodiments based on newly added scene data (such as newly added pet targets or special vehicle targets).

[0203] Optionally, after an online update of the detection model, the server transmits the encrypted new model (e.g., AES-256) to the monitoring system via 4G / 5G / Wi-Fi. Resume transmission at breakpoints to avoid interruptions. Upon receiving the updated model, the monitoring system verifies its integrity (e.g., using MD5 checksum). If successful, the old model is backed up and replaced with the new model. After obtaining the new model, the monitoring system can use a small amount of locally collected data (100-200 images) for incremental fine-tuning to adapt the updated model to the local scenario. This fine-tuning process can consume relatively little power and does not affect normal system monitoring.

[0204] Furthermore, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when run by a processor, executes the steps of the detection model construction method or monitoring method described in the above method embodiments.

[0205] The computer program product of the detection model construction method and monitoring method provided in this application includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the steps of the detection model construction method or monitoring method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0206] In the several embodiments provided in this application, it should be understood that the disclosed methods can also be implemented in other ways. The method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0207] In addition, the method steps in the various embodiments of this application can be integrated together to form an independent part for execution, or each method step can be executed by a separate module, or two or more steps can be formed into an independent part for execution.

[0208] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. It should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0209] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0210] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for constructing a detection model, characterized in that, The method includes: Build scenario data; The initial detection model is trained based on the scene data to obtain a target detection model; wherein, the target detection model is used to perform target detection on the monitored image when the monitoring system is in a wake-up state; The target detection model is lightweighted to determine a lightweight detection model; wherein the quantization bit width of the parameters of the lightweight detection model is lower than that of the target detection model, and the lightweight detection model is used to perform target detection on the monitored images when the monitoring system is in a dormant phase.

2. The method according to claim 1, characterized in that, The step of training the initial detection model based on the scene data to obtain the target detection model includes: The initial detection model is pre-trained using the scene data to obtain a pre-trained detection model; wherein the training cycle of the pre-training is the first iteration round. The pre-trained detection model is fine-tuned using the scene data to obtain a fine-tuned detection model; wherein the training cycle of the fine-tuning training is the second iteration round, which is shorter than the first iteration round; Based on the teacher model, the fine-tuned detection model is trained by distillation to obtain a target detection model. The distillation loss used in the distillation training is determined based on the loss of the fine-tuned detection model and the divergence loss of the outputs of the fine-tuned detection model and the teacher model. The training cycle of the distillation training is the third iteration round, which is shorter than the second iteration round.

3. The method according to claim 2, characterized in that, The step of using the scene data to pre-train the initial detection model to obtain a pre-trained detection model includes: Based on the initial training parameters, the initial detection model is pre-trained using the scene data to obtain the current detection model; During the training process, at preset training iterations, the accuracy of the current detection model is verified using the verification data determined from the scene data. If the model accuracy does not meet the requirements, the initial training parameters are updated to obtain updated training parameters; wherein, the learning rate of the updated training parameters is lower. Based on the updated training parameters, the initial detection model is pre-trained using the scene data to obtain a pre-trained detection model.

4. The method according to claim 1, characterized in that, The initial detection model includes: A sampling module that outputs four feature maps; A feature fusion module for fusing four feature maps; Attention mechanism module; and, The result prediction module is used to output the prediction results.

5. The method according to claim 4, characterized in that, The sampling module includes: First downsampling layer; The first group of RepVGG-X modules; The second group of RepVGG-X modules; A second downsampling layer is set between the first group of RepVGG-X modules and the second group of RepVGG-X modules; The third downsampling layer is set after the second group of RepVGG-X modules; Pooling layer; A fourth downsampling layer is set after the pooling layer; The third group of RepVGG-X modules; The fifth downsampling layer is set after the third group of RepVGG-X modules; The fourth group is the RepVGG-X module group; The first upsampling layer is set after the fourth group of RepVGG-X modules; The four feature maps output by the sampling module are the outputs of the third downsampling layer, the fourth downsampling layer, the fifth downsampling layer, and the first upsampling layer.

6. The method according to claim 4, characterized in that, The result prediction module includes: Detection unit and classification unit; The detection unit is used to determine the detection boxes containing the target and to determine the confidence level for each detection box; the classification unit is used to classify the target in the detection box based on the confidence level of the detection box. The formula of the detection unit includes: ； in, This represents the confidence score of the i-th detection box; (i,j) represents the intersection-union ratio of the i-th detection box and the highest confidence detection box j; Indicates the IoU threshold; Indicates the attenuation coefficient; when At that time, the confidence level of the detection box is decayed through an exponential function.

7. The method according to claim 6, characterized in that, The formula for the confidence threshold used by the classification unit includes: ； ； in, Indicates the baseline confidence threshold; L represents the ambient light intensity. Maximum illumination intensity; S represents the area of ​​the target pixel in the image; Indicates the maximum target size; Indicates the illumination adjustment factor; Indicates the size adjustment factor; This represents the minimum confidence threshold. This represents the maximum confidence threshold.

8. The method according to claim 1, characterized in that, The step of lightweighting the target detection model to determine a lightweight detection model includes: The first precision of the parameters of the target detection model is converted into a second precision; wherein the second precision is a lower precision than the first precision. According to a preset ratio, the channels of the convolutional layer in the target detection model are pruned to obtain a pruned detection model; The pruning detection model is trained through a preset number of iterations to obtain a lightweight detection model.

9. The method according to claim 8, characterized in that, The step of pruning the channels of the convolutional layer in the target detection model according to a preset ratio to obtain a pruned detection model includes: The weight norm of the channels in the convolutional layers of the target detection model; Sort the weight norms from smallest to largest; The channels of the convolutional layers corresponding to the weight norms at the top of the ranking are deleted to obtain the pruned detection model.

10. A monitoring method, characterized in that, The monitoring method is applied to a monitoring system and includes: In the dormant state of the monitoring system, a lightweight detection model is used to detect the obtained first image and determine a first detection result, wherein the lightweight detection model is constructed using the method described in any one of claims 1 to 9; If the first detection result indicates the presence of a target in the first image, the monitoring system is activated; In the wake-up state of the monitoring system, the obtained second image is detected using a target detection model to determine the monitoring result; wherein, the target detection model is constructed using the method described in any one of claims 1 to 8, and the parameter accuracy of the lightweight detection model is less than that of the target detection model.

11. A monitoring system, characterized in that, include: The processor and memory, wherein the memory stores machine-readable instructions executable by the processor, and when the monitoring system is running, the machine-readable instructions are executed by the processor to perform the steps of the method as described in any one of claims 1 to 10.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the method as described in any one of claims 1 to 10.

13. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 10.

Images