Image recognition secondary rechecking method based on cloud edge collaboration
By employing a cloud-edge collaborative image recognition secondary verification method, which combines edge detection and verification with cloud-based detection, the problems of low efficiency and poor accuracy in railway video surveillance have been solved, achieving efficient and accurate image recognition results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-04-14
AI Technical Summary
In ultra-large-scale video surveillance scenarios such as railways, image recognition methods are inefficient and inaccurate, making them difficult to adapt to complex application scenarios. In particular, the false detection rate is high due to low image quality and environmental interference, and existing cloud-edge collaborative solutions have failed to effectively solve this problem.
A cloud-edge collaborative image recognition secondary verification method is adopted. The image sequence quality is evaluated and preliminarily detected at the edge, the target image sequence is selected and sent to the cloud for verification, and finally high-resolution detection results and change feature information are obtained. Combined with background modeling and confidence threshold adjustment, the detection accuracy is improved.
It reduces systematic false positives, improves the accuracy and efficiency of image recognition, reduces cloud computing load, enhances adaptability to complex scenes, and effectively avoids false positives caused by lighting interference and environmental factors.
Smart Images
Figure CN121861418A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, and in particular to a secondary verification method for image recognition based on cloud-edge collaboration. Background Technology
[0002] With the widespread application of deep learning technology in video surveillance, intelligent security systems have achieved high accuracy in target detection. However, in large-scale video surveillance scenarios such as railways, the sheer number of cameras generates a massive amount of images to be inspected daily. Low image quality and environmental interference can easily lead to false detections. Therefore, improving the efficiency and accuracy of image recognition in complex application scenarios is a critical issue that urgently needs to be addressed.
[0003] Currently, mainstream solutions mainly fall into two categories: The first is a cloud-edge-device collaborative solution, which improves the final detection accuracy but fails to address the high false alarm rate at the edge due to low image quality and environmental interference, resulting in numerous invalid alarms consuming uplink bandwidth and cloud computing power. The second involves deploying multiple models locally using a voting mechanism to improve image detection stability through ensemble learning, but this incurs huge computational overhead, making it difficult to deploy on resource-constrained edge devices, and offering limited improvement over systemic false alarms caused by image quality. Furthermore, deep learning methods, with different feature extraction schemes focusing on different feature angles, struggle to adapt to features with distinct domain attributes. Summary of the Invention
[0004] This invention provides a cloud-edge collaborative image recognition secondary verification method to address the shortcomings of existing image recognition methods, such as low efficiency, poor accuracy, and difficulty in adapting to complex application scenarios.
[0005] This invention provides a cloud-edge collaborative image recognition secondary verification method, applied to the edge, comprising: The quality of multiple image sequences to be detected is evaluated to obtain an evaluation result for each image sequence. Each image sequence and its evaluation result are then input into a first image detection model to obtain a preliminary detection result for each image sequence output by the first image detection model. Based on the preliminary detection results of each of the image sequences, multiple target image sequences are selected from the multiple image sequences; Each target image sequence and its preliminary detection result are sent to the cloud so that the cloud can review the preliminary detection results of each target image sequence to obtain the final recognition result.
[0006] In some embodiments, the step of verifying the preliminary detection results of each target image sequence to obtain the final recognition result includes: Each of the target image sequences is input into the second image detection model to obtain a high-resolution detection result for each of the target image sequences output by the second image detection model; Background modeling is performed on each of the target image sequences to obtain the change feature information of each target image sequence; the change feature information includes the proportion of the changed region. Based on the high-resolution detection results of each target image sequence and the variation feature information of each target image sequence, the preliminary detection results of each target image sequence are reviewed to obtain the final recognition result.
[0007] In some embodiments, the step of reviewing the preliminary detection results of each target image sequence based on the high-resolution detection results of each target image sequence and the variation feature information of each target image sequence to obtain the final recognition result includes: Based on the high-resolution detection results of each target image sequence and the change feature information of each target image sequence, the preliminary detection results of each target image sequence are filtered to obtain the recognition results of each target image sequence; the recognition results of each target image sequence include multiple tracking targets; the target image sequence includes multiple frames of images arranged in time sequence; Based on the recognition results of each target image sequence, the number of frames in which each tracked target appears in the target image sequence is counted; If the number of frames corresponding to each tracked target is greater than or equal to a preset threshold, the tracked target is determined to be a valid target, and all valid targets in each target image sequence are taken as the final recognition result of each target image sequence.
[0008] In some embodiments, the second image detection model is obtained by improving the YOLOv8l network. The improvement includes adding a P2 layer to the YOLOv8l network to improve the detection accuracy of small-scale targets. The P2 layer corresponds to a 1 / 4 downsampling scale of the input image. The P2 layer is connected to the original P3 layer of the YOLOv8l network. The second image detection model is trained based on target image sequence samples and the high-resolution detection result labels of the target image sequence samples.
[0009] In some embodiments, the first image detection model is trained based on image sequence samples, the evaluation results of the image sequence samples, and the preliminary detection result labels of the image sequence samples.
[0010] This invention also provides a cloud-edge collaborative image recognition secondary verification method, applied in the cloud, including: The system receives each target image sequence and a preliminary detection result for each target image sequence sent from the edge terminal; the target image sequence is selected from multiple image sequences based on the preliminary detection result of each image sequence; the preliminary detection result of each image sequence is obtained by a first image detection model performing target detection on each image sequence based on the evaluation result of each image sequence. Based on each target image sequence, the preliminary detection results of each target image sequence are reviewed to obtain the final recognition result.
[0011] In some embodiments, the step of verifying the preliminary detection results of each target image sequence to obtain the final recognition result includes: Each of the target image sequences is input into the second image detection model to obtain a high-resolution detection result for each of the target image sequences output by the second image detection model; Background modeling is performed on each of the target image sequences to obtain the change feature information of each target image sequence; the change feature information includes the proportion of the changed region. Based on the high-resolution detection results of each target image sequence and the variation feature information of each target image sequence, the preliminary detection results of each target image sequence are reviewed to obtain the final recognition result.
[0012] In some embodiments, the step of reviewing the preliminary detection results of each target image sequence based on the high-resolution detection results of each target image sequence and the variation feature information of each target image sequence to obtain the final recognition result includes: Based on the high-resolution detection results of each target image sequence and the change feature information of each target image sequence, the preliminary detection results of each target image sequence are filtered to obtain the recognition results of each target image sequence; the recognition results of each target image sequence include multiple tracking targets; the target image sequence includes multiple frames of images arranged in time sequence; Based on the recognition results of each target image sequence, the number of frames in which each tracked target appears in the target image sequence is counted; If the number of frames corresponding to each tracked target is greater than or equal to a preset threshold, the tracked target is determined to be a valid target, and all valid targets in each target image sequence are taken as the final recognition result of each target image sequence.
[0013] In some embodiments, the second image detection model is obtained by improving the YOLOv8l network. The improvement includes adding a P2 layer to the YOLOv8l network to improve the detection accuracy of small-scale targets. The P2 layer corresponds to a 1 / 4 downsampling scale of the input image. The P2 layer is connected to the original P3 layer of the YOLOv8l network. The second image detection model is trained based on target image sequence samples and the high-resolution detection result labels of the target image sequence samples.
[0014] In some embodiments, the first image detection model is trained based on image sequence samples, the evaluation results of the image sequence samples, and the preliminary detection result labels of the image sequence samples.
[0015] The cloud-edge collaborative image recognition secondary verification method provided by this invention evaluates the quality of multiple image sequences to be detected, obtaining an evaluation result for each image sequence. Each image sequence and its evaluation result are then input into a first image detection model to obtain preliminary detection results for each image sequence output by the first image detection model. This reduces systematic false detections and improves the accuracy of preliminary detection. Based on the preliminary detection results of each image sequence, multiple target image sequences are selected from the multiple image sequences. Each target image sequence and its preliminary detection result are then sent to the cloud for verification, resulting in a final recognition result. This reduces the computational load on the cloud and improves the efficiency, accuracy, and adaptability of image recognition. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0017] Figure 1 This is one of the flowcharts of the image recognition secondary verification method based on cloud-edge collaboration provided in the embodiments of the present invention.
[0018] Figure 2 This is the second flowchart of the image recognition secondary verification method based on cloud-edge collaboration provided in this embodiment of the invention.
[0019] Figure 3 This is a schematic diagram of the process for verifying the preliminary detection results of each target image sequence provided in an embodiment of the present invention.
[0020] Figure 4 This is a schematic diagram of the structure of the second image detection model provided in an embodiment of the present invention. Detailed Implementation
[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0022] The terms "first," "second," etc., used in this invention are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first" and "second" are generally of the same class, not limited in number; for example, the first object can be one or more. Furthermore, in this invention, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.
[0023] Currently, deep learning-based image recognition methods focus on different feature dimensions across their feature extraction networks, often making it difficult to effectively adapt to visual features with significant domain-specific characteristics. Taking railway industry monitoring as an example, the captured railway tracks typically span the entire frame, representing global structural features rather than local details. This makes it difficult for conventional shallow networks to fully learn and represent local features. Furthermore, in nighttime environments, train headlights sweeping across the monitored area cause large-scale brightness and content abrupt changes in the image, easily triggering false alarms even when no actual target intrudes. Simultaneously, traditional detection methods based on single-frame images cannot utilize the temporal continuity information in video sequences. Especially in outdoor nighttime scenes, birds and insects create obvious movement trajectories under supplementary lighting; when they approach the camera, they are easily misidentified as targets due to their prominent imaging, further increasing the false detection rate.
[0024] To address this, the present invention provides a cloud-edge collaborative image recognition secondary verification method. This method evaluates the quality of multiple image sequences to be detected, obtaining an evaluation result for each sequence. Each image sequence and its evaluation result are then input into a first image detection model to obtain preliminary detection results for each sequence. This reduces systematic false detections and improves the accuracy of preliminary detection. Based on the preliminary detection results of each image sequence, multiple target image sequences are selected from the multiple image sequences. Each target image sequence and its preliminary detection result are then sent to the cloud for verification, resulting in a final recognition result. This reduces the computational load on the cloud and improves the efficiency, accuracy, and adaptability of image recognition.
[0025] Figure 1 This is one of the flowcharts illustrating the cloud-edge collaborative image recognition secondary verification method provided in this embodiment of the invention. Figure 1 As shown, a cloud-edge collaborative image recognition secondary verification method is provided, applied to the edge, including the following steps: step 110, step 120, and step 130. This method's steps are merely one possible implementation of the invention.
[0026] Step 110: Evaluate the quality of the multiple image sequences to be detected, obtain the evaluation result of each image sequence, input each image sequence and the evaluation result of each image sequence into the first image detection model, and obtain the preliminary detection result of each image sequence output by the first image detection model.
[0027] Optionally, monitoring video is acquired in real time through monitoring equipment, and the monitoring video is processed to obtain multiple image sequences; each image sequence includes multiple consecutive frames of images, which helps to distinguish between transient interference and real targets by utilizing temporal information.
[0028] The evaluation results can be one or more quantitative evaluation scores or grades, or they can be image categories, such as normal images, low-quality images caused by heavy rain or fog, or low-quality images caused by changes in camera field of view.
[0029] Optionally, the quality of multiple image sequences to be detected is evaluated according to multiple preset quality evaluation indicators; the multiple quality evaluation indicators include, but are not limited to: sharpness, brightness, contrast, noise level, whether there is large area occlusion or strong light interference, etc.
[0030] Specifically, the quality of each image in each image sequence is evaluated based on data such as the size of each image and the Tenengrad gradient value. Image file size is the basic scale of its information carrier; the smaller the image file, the less detailed information it contains. The Tenengrad gradient value reflects the edge sharpness of the image; the smaller the gradient value, the more blurred the image.
[0031] Image file size is an important indicator reflecting the integrity of image information and the shooting environment. Under normal circumstances, if the file size of an image taken in the wild is less than 300KB, the image may have been compressed or lost in detail; if the file size is less than 200KB, the image is very likely to have been taken in black and white mode under low light conditions, or severely affected by severe weather such as heavy rain or fog, resulting in a significant decrease in image clarity.
[0032] The Tenengrad gradient value is a key parameter for measuring the focus accuracy and edge sharpness of an image. When this value is below 0.8, the image is very likely to have problems with inaccurate focus and overall blurriness; if the value is further below 0.6, the image is severely blurred and it is difficult to identify its specific content. This situation is usually caused by unexpected changes in the camera's field of view.
[0033] Specifically, the judgment logic for image quality assessment is as follows: If Tenengrad(P) < 0.6, then the image sequence P is determined to be blurry. If Getsize(P) < 200KB and Tenengrad(P) < 0.6 at the same time, it is determined that the image sequence P has low clarity due to the influence of weather such as heavy rain and fog. In all other cases, the image sequence P is judged to be of normal quality.
[0034] Here, Getsize(P) represents the file size of image sequence P, and Tenengrad(P) represents the Tenengrad gradient value of image sequence P, which is its clarity quantization parameter.
[0035] Optionally, the first image detection model can be a YOLOV8s model. In order to reduce the false negative rate of the target, the first image detection model adopts a high recall mode and adjusts the confidence threshold to conf=0.2.
[0036] Optionally, the first image detection model can select different detection strategies based on the evaluation results of different image sequences to perform target detection on different image sequences.
[0037] In some embodiments, the first image detection model is trained based on image sequence samples, the evaluation results of image sequence samples, and the preliminary detection result labels of image sequence samples.
[0038] Optionally, the training process of the first image detection model includes: Acquire multiple image sequence samples and determine the preliminary detection result label for each image sequence sample; The quality of each image sequence sample is evaluated to obtain the evaluation result for each image sequence sample; Using each image sequence sample and its evaluation result as training samples, and the preliminary detection result label of each image sequence sample as sample label, a first initial image detection model is trained. After training, the first image detection model is obtained.
[0039] Step 120: Based on the preliminary detection results of each image sequence, select multiple target image sequences from multiple image sequences.
[0040] The target image sequence refers to an image sequence containing the target to be detected. The target image sequence includes at least the target image and multiple images associated with the target image, and the target image contains the target to be detected.
[0041] For example, the target image sequence is p i-15 p i-1 p i-0.5 p i p i+0.5 p i+1 Where p i-15 For the image 15 seconds before the target image occurred, p i-1 For the 23rd frame image preceding the target image, p i-0.5 For the 11th frame image before the target image, p i For the target image, p i+0.5 For the 11th frame image after the target image, p i+1 This is the image in the 23rd frame following the target image.
[0042] Optionally, if the image sequence does not contain the target to be detected, it can be removed without further processing.
[0043] Understandably, by filtering out multiple target image sequences from multiple image sequences based on the preliminary detection results of each image sequence, irrelevant images can be eliminated, reducing the amount of computation in the cloud and improving the efficiency of image recognition.
[0044] Step 130: Send each target image sequence and its preliminary detection result to the cloud so that the cloud can review the preliminary detection result of each target image sequence to obtain the final recognition result.
[0045] Optionally, the cloud can use model review and / or background modeling to review the preliminary detection results of each target image sequence.
[0046] Optionally, a second image detection model is deployed in the cloud, and the confidence threshold used by the second image detection model is higher than that used by the first image detection model; the second image detection model is used to perform refined target detection on each target image sequence.
[0047] In this embodiment of the invention, the quality of multiple image sequences to be detected is evaluated to obtain an evaluation result for each image sequence. Each image sequence and its evaluation result are then input into a first image detection model to obtain a preliminary detection result for each image sequence output by the first image detection model. This reduces systematic false detections and improves the accuracy of preliminary detection. Based on the preliminary detection results of each image sequence, multiple target image sequences are selected from the multiple image sequences. Each target image sequence and its preliminary detection result are then sent to the cloud for the cloud to review the preliminary detection results of each target image sequence to obtain the final recognition result. This reduces the computational load on the cloud and improves the efficiency, accuracy, and adaptability of image recognition.
[0048] In some embodiments, based on each target image sequence, the preliminary detection results of each target image sequence are reviewed to obtain the final recognition result, including: Each target image sequence is input into the second image detection model to obtain the high-resolution detection result of each target image sequence output by the second image detection model; Background modeling is performed on each target image sequence to obtain the change feature information of each target image sequence; the change feature information includes the proportion of the changed region. Based on the high-resolution detection results of each target image sequence and the variation feature information of each target image sequence, the preliminary detection results of each target image sequence are reviewed to obtain the final recognition result.
[0049] Optionally, the second image detection model can be an improved YOLOV8l model; the confidence threshold of the second image detection model can be increased to conf=0.6 to ensure the accuracy of the identified content; a P2 layer can be added to the detection branch of the existing YOLOV8l model to improve the response capability of the second image detection model to small targets.
[0050] In some embodiments, the second image detection model is derived from an improvement on the YOLOv8l network. The improvement includes adding a P2 layer to the YOLOv8l network to improve the detection accuracy of small-scale targets. The P2 layer corresponds to a 1 / 4 downsampling scale of the input image, and the P2 layer is connected to the original P3 layer of the YOLOv8l network. The second image detection model is trained based on target image sequence samples and high-resolution detection result labels of the target image sequence samples.
[0051] The percentage of the changing region refers to the ratio of the area of continuously changing pixels within the detection box to the total area of the detection box.
[0052] Optionally, the proportion of the changed area can be compared with a preset proportion threshold to determine whether there is a target within the detection box.
[0053] For example, the target image p i The content of the detection box is compared with the image p 15 seconds before the target image occurred. i-15 The content of the detection frame is compared, and the ratio of the area of the changed region of the detection frame to the total area of the detection frame is calculated.
[0054] In some embodiments, based on the high-resolution detection results of each target image sequence and the variation feature information of each target image sequence, the preliminary detection results of each target image sequence are reviewed to obtain the final recognition result, including: Based on the high-resolution detection results of each target image sequence and the variation feature information of each target image sequence, the preliminary detection results of each target image sequence are filtered to obtain the recognition results of each target image sequence; the recognition results of each target image sequence include multiple tracked targets; the target image sequence includes multiple frames of images arranged in time sequence; Based on the recognition results of each target image sequence, the number of frames in which each tracked target appears in the target image sequence is counted; If the number of frames corresponding to each tracked target is greater than or equal to a preset threshold, the tracked target is determined as a valid target, and all valid targets in each target image sequence are taken as the final recognition result of each target image sequence.
[0055] Optionally, if the high-resolution detection results of the target image sequence show that a target has appeared, and the proportion of the changed area of the detection box in the target image sequence is less than a preset proportion threshold, such as 50%, then the target is determined to exist; otherwise, the preliminary detection results at the edge are false alarms.
[0056] For example, for each target image sequence, if at least two out of five images with the same time interval contain the target and the targets in the two images have an overlap, then the target is determined to exist and is considered a valid target, effectively avoiding false detections caused by raindrops, snowflakes, flying insects, or flying birds entering the field of view.
[0057] In this embodiment of the invention, a cloud-edge collaborative image recognition secondary verification method and a background modeling method are used to jointly verify the preliminary detection results. This effectively avoids false detections of targets caused by lighting interference such as vehicle headlights and shadows, and improves the accuracy of image verification. Based on the recognition results of each target image sequence, the number of frames in which each tracked target appears in the target image sequence is counted. If the number of frames corresponding to each tracked target is greater than or equal to a preset threshold, the tracked target is determined to be a valid target. This effectively avoids false detections caused by environmental factors such as raindrops, snowflakes, flying insects, and birds, and improves the accuracy of image recognition.
[0058] Figure 2 This is the second flowchart illustrating the image recognition secondary verification method based on cloud-edge collaboration provided in this embodiment of the invention. Figure 2 As shown, a cloud-edge collaborative image recognition secondary verification method is provided, applied in the cloud, including the following steps: step 210, step 220. This method's steps are merely one possible implementation of the invention.
[0059] Step 210: Receive each target image sequence and the preliminary detection result of each target image sequence sent by the edge terminal; the target image sequence is selected from multiple image sequences based on the preliminary detection result of each image sequence; the preliminary detection result of each image sequence is obtained by the first image detection model performing target detection on each image sequence based on the evaluation result of each image sequence.
[0060] The target image sequence refers to an image sequence containing the target to be detected. The target image sequence includes at least the target image and multiple images associated with the target image, and the target image contains the target to be detected.
[0061] Optionally, each image sequence includes multiple consecutive frames, which helps to utilize temporal information to distinguish between transient interference and the real target. The evaluation result can be one or more quantified evaluation scores or grades, or it can be an image category, such as a normal image, a low-quality image caused by heavy rain or fog, or a low-quality image caused by changes in camera field of view.
[0062] Optionally, the evaluation result is obtained by evaluating the quality of multiple image sequences to be detected based on multiple preset quality evaluation indicators; the multiple quality evaluation indicators include, but are not limited to: sharpness, brightness, contrast, noise level, whether there is large area occlusion or strong light interference, etc.
[0063] Optionally, multiple quality assessment metrics also include image file size, Tenengrad gradient value, and other metrics. Image file size is the basic scale of its information carrier; the smaller the image file, the less detailed information it contains. Tenengrad gradient value reflects the edge sharpness of the image; the smaller the gradient value, the more blurred the image.
[0064] Optionally, the first image detection model can be a YOLOv8s model, and the confidence threshold of the first image detection model can be set to conf=0.2 to reduce the false negative rate of the target. The first image detection model can select different detection strategies based on the evaluation results of different image sequences to perform target detection on different image sequences.
[0065] In some embodiments, the first image detection model is trained based on image sequence samples, the evaluation results of image sequence samples, and the preliminary detection result labels of image sequence samples.
[0066] Step 220: Based on each target image sequence, review the preliminary detection results of each target image sequence to obtain the final recognition result.
[0067] Optionally, model review and / or background modeling can be used to review the preliminary detection results of each target image sequence.
[0068] Optionally, a second image detection model can be used to perform refined target re-inspection on each target image sequence; the confidence threshold used by the second image detection model is higher than the confidence threshold used by the first image detection model.
[0069] In this embodiment of the invention, by reviewing the preliminary detection results of each target image sequence based on each target image sequence, the final recognition result is obtained, thereby improving the efficiency, accuracy and adaptability of image recognition.
[0070] Figure 3 This is a schematic diagram illustrating the process of verifying the preliminary detection results of each target image sequence according to an embodiment of the present invention. Figure 3 As shown, in some embodiments, step 220 verifies the preliminary detection results of each target image sequence based on each target image sequence to obtain the final recognition result, including: Step 221: Input each target image sequence into the second image detection model to obtain the high-resolution detection result of each target image sequence output by the second image detection model.
[0071] Optionally, the second image detection model is an improvement based on the YOLOv8l network. The improvement includes adding a P2 layer to the YOLOv8l network to improve the detection accuracy of small-scale targets. The P2 layer corresponds to a 1 / 4 downsampling scale of the input image, and the P2 layer is connected to the original P3 layer of the YOLOv8l network. The second image detection model is trained based on target image sequence samples and the high-resolution detection result labels of the target image sequence samples.
[0072] Optionally, the second image detection model can be an improved YOLOV8l model; the confidence threshold of the second image detection model can be increased to conf=0.6 to ensure the accuracy of the identified content; a P2 layer can be added to the detection branch of the existing YOLOV8l model to improve the response capability of the second image detection model to small targets.
[0073] Step 222: Perform background modeling for each target image sequence to obtain the change feature information of each target image sequence; the change feature information includes the proportion of the change area.
[0074] The percentage of the changing region refers to the ratio of the area of continuously changing pixels within the detection box to the total area of the detection box.
[0075] Step 223: Based on the high-resolution detection results of each target image sequence and the change feature information of each target image sequence, the preliminary detection results of each target image sequence are reviewed to obtain the final recognition results.
[0076] Optionally, the proportion of the changed area can be compared with a preset proportion threshold to determine whether there is a target within the detection box.
[0077] In some embodiments, step 223 verifies the preliminary detection results of each target image sequence based on the high-resolution detection results of each target image sequence and the variation feature information of each target image sequence, to obtain the final recognition result, including: Step 2231: Based on the high-resolution detection results of each target image sequence and the change feature information of each target image sequence, filter the preliminary detection results of each target image sequence to obtain the recognition results of each target image sequence; the recognition results of each target image sequence include multiple tracking targets; the target image sequence includes multiple frames of images arranged in time sequence.
[0078] Optionally, if the high-resolution detection results of the target image sequence show that a target has appeared, and the proportion of the changed area of the detection box in the target image sequence is less than a preset proportion threshold, then the target is determined to exist; otherwise, the preliminary detection results at the edge are false alarms.
[0079] Step 2232: Based on the recognition results of each target image sequence, count the number of frames in which each tracked target appears in the target image sequence; Step 2233: If the number of frames corresponding to each tracked target is greater than or equal to a preset threshold, the tracked target is determined as a valid target, and all valid targets in each target image sequence are taken as the final recognition result of each target image sequence.
[0080] For example, for each target image sequence, if at least two out of five images with the same time interval contain the target and the targets in the two images have an overlap, then the target is determined to exist and is considered a valid target, effectively avoiding false detections caused by raindrops, snowflakes, flying insects, or flying birds entering the field of view.
[0081] In this embodiment of the invention, a cloud-edge collaborative image recognition secondary verification method and a background modeling method are used to jointly verify the preliminary detection results, which can effectively avoid false detections of targets caused by illumination interference and improve the accuracy of image verification. Based on the recognition results of each target image sequence, the number of frames in which each tracked target appears in the target image sequence is counted. If the number of frames corresponding to each tracked target is greater than or equal to a preset threshold, the tracked target is determined to be a valid target, which can effectively avoid false detections caused by environmental factors and improve the accuracy of image recognition.
[0082] Figure 4 This is a schematic diagram of the structure of the second image detection model provided in an embodiment of the present invention. Figure 4 As shown, in some embodiments, the second image detection model is obtained by improving the YOLOv8l network. The improvement includes: adding a P2 layer to the YOLOv8l network to improve the detection accuracy of small-scale targets. The P2 layer corresponds to a 1 / 4 downsampling scale of the input image. The P2 layer is connected to the original P3 layer of the YOLOv8l network. The second image detection model is trained based on target image sequence samples and the high-resolution detection result labels of the target image sequence samples.
[0083] The second image detection model includes multiple feature extraction layers: a C1 layer at a 1 / 2 scale, a C2 layer at a 1 / 4 scale, a C3 layer at a 1 / 8 scale, a C4 layer at a 1 / 16 scale, and a C5 layer at a 1 / 32 scale. The C2 layer is connected to the P2 layer, and the C3 layer is connected to the P3 layer.
[0084] Optionally, the training process of the second image detection model includes: Acquire multiple image sequence samples; The quality of each image sequence sample is evaluated to obtain the evaluation result for each image sequence sample; Each image sequence sample and its evaluation result are input into the first image detection model to obtain the preliminary detection result of each image sequence sample output by the first image detection model. Based on the preliminary detection results of each image sequence sample, multiple target image sequence samples are selected from multiple image sequence samples; Determine the high-resolution detection result label for each target image sequence sample; Using each target image sequence sample as a training sample and the high-resolution detection result label of each target image sequence sample as a sample label, a second initial image detection model is trained. After training, the second image detection model is obtained.
[0085] Specifically, a training dataset is obtained, which includes 32,000 precisely labeled images of people and 62,000 background images. The models YOLOV8s, YOLOV8l, and YOLOV8l-E are trained based on the training dataset, where YOLOV8l-E represents the YOLOV8l model with p2 layers added.
[0086] The training parameters include: epochs=100, imgsz=640, batch=16, optimizer="auto", lr0=0.01, weight_decay=0.0005. After 100 rounds of training, the model is exported, and the model of YOLOV8s (conf=0.2) is denoted as M1, the model of YOLOV8s (conf=0.45) is denoted as M11, the model of YOLOV8l-E (conf=0.6) is denoted as M2, and the model of YOLOV8l (conf=0.45) is denoted as M22.
[0087] Specifically, 10 cameras were connected to the test line for real-time video analysis testing. A cloud-edge collaborative video analysis system was built using two servers. Model M1 was deployed on the edge server, and model M2 was deployed on the cloud server. In order to verify the performance comparison between this invention and YoloV8s and YoloV8l, models M11 and M22 were deployed on the edge server. Each model was packaged in an independent image and used different graphics cards. The test showed that there was no significant impact between them.
[0088] Table 1 is a comparison table of the test effects of different detection schemes provided in the embodiments of the present invention.
[0089] As shown in Table 1, after 7 days of continuous operation testing with 10 network cameras on site, the standard YOLOv8s model, i.e., the M11 model, reported 655 alarms, of which 37 were false positives, resulting in a positive detection rate of 94.4%; the standard YOLOv8l model, i.e., the M22 model, reported 646 alarms, with 29 false positives, resulting in a positive detection rate of 95.5%; the cloud-edge collaborative image recognition secondary verification method proposed in this embodiment of the invention, which uses the detection scheme of M1 and M2 models, reported 625 alarms, with 8 false positives, resulting in a positive detection rate of 98.72%, significantly improving the alarm accuracy. There were no false alarms caused by light and shadow, nighttime flying insects, or birds.
[0090] Table 1. Comparison of test results of different detection schemes provided in the embodiments of the present invention.
[0091] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A secondary verification method for image recognition based on cloud-edge collaboration, characterized in that, Applied to the edge, including: The quality of multiple image sequences to be detected is evaluated to obtain an evaluation result for each image sequence. Each image sequence and its evaluation result are then input into a first image detection model to obtain a preliminary detection result for each image sequence output by the first image detection model. Based on the preliminary detection results of each of the image sequences, multiple target image sequences are selected from the multiple image sequences; Each target image sequence and its preliminary detection result are sent to the cloud so that the cloud can review the preliminary detection results of each target image sequence to obtain the final recognition result.
2. The image recognition secondary verification method based on cloud-edge collaboration according to claim 1, characterized in that, The step of reviewing the preliminary detection results of each target image sequence to obtain the final recognition result includes: Each of the target image sequences is input into the second image detection model to obtain a high-resolution detection result for each of the target image sequences output by the second image detection model; Background modeling is performed on each of the target image sequences to obtain the change feature information of each target image sequence; the change feature information includes the proportion of the changed region. Based on the high-resolution detection results of each target image sequence and the variation feature information of each target image sequence, the preliminary detection results of each target image sequence are reviewed to obtain the final recognition result.
3. The image recognition secondary verification method based on cloud-edge collaboration according to claim 2, characterized in that, The preliminary detection results of each target image sequence are reviewed based on the high-resolution detection results of each target image sequence and the variation feature information of each target image sequence to obtain the final recognition result, including: Based on the high-resolution detection results of each target image sequence and the change feature information of each target image sequence, the preliminary detection results of each target image sequence are filtered to obtain the recognition results of each target image sequence; the recognition results of each target image sequence include multiple tracking targets; the target image sequence includes multiple frames of images arranged in time sequence; Based on the recognition results of each target image sequence, the number of frames in which each tracked target appears in the target image sequence is counted; If the number of frames corresponding to each tracked target is greater than or equal to a preset threshold, the tracked target is determined to be a valid target, and all valid targets in each target image sequence are taken as the final recognition result of each target image sequence.
4. The image recognition secondary verification method based on cloud-edge collaboration according to claim 1, characterized in that, The first image detection model is trained based on image sequence samples, the evaluation results of the image sequence samples, and the preliminary detection result labels of the image sequence samples.
5. A secondary verification method for image recognition based on cloud-edge collaboration, characterized in that, Applied to the cloud, including: The system receives each target image sequence and a preliminary detection result for each target image sequence sent from the edge terminal; the target image sequence is selected from multiple image sequences based on the preliminary detection result of each image sequence; the preliminary detection result of each image sequence is obtained by a first image detection model performing target detection on each image sequence based on the evaluation result of each image sequence. Based on each target image sequence, the preliminary detection results of each target image sequence are reviewed to obtain the final recognition result.
6. The image recognition secondary verification method based on cloud-edge collaboration according to claim 5, characterized in that, The step of reviewing the preliminary detection results of each target image sequence to obtain the final recognition result includes: Each of the target image sequences is input into the second image detection model to obtain a high-resolution detection result for each of the target image sequences output by the second image detection model; Background modeling is performed on each of the target image sequences to obtain the change feature information of each target image sequence; the change feature information includes the proportion of the changed region. Based on the high-resolution detection results of each target image sequence and the variation feature information of each target image sequence, the preliminary detection results of each target image sequence are reviewed to obtain the final recognition result.
7. The image recognition secondary verification method based on cloud-edge collaboration according to claim 6, characterized in that, The preliminary detection results of each target image sequence are reviewed based on the high-resolution detection results of each target image sequence and the variation feature information of each target image sequence to obtain the final recognition result, including: Based on the high-resolution detection results of each target image sequence and the change feature information of each target image sequence, the preliminary detection results of each target image sequence are filtered to obtain the recognition results of each target image sequence; the recognition results of each target image sequence include multiple tracking targets; the target image sequence includes multiple frames of images arranged in time sequence; Based on the recognition results of each target image sequence, the number of frames in which each tracked target appears in the target image sequence is counted; If the number of frames corresponding to each tracked target is greater than or equal to a preset threshold, the tracked target is determined to be a valid target, and all valid targets in each target image sequence are taken as the final recognition result of each target image sequence.
8. The image recognition secondary verification method based on cloud-edge collaboration according to claim 5, characterized in that, The first image detection model is trained based on image sequence samples, the evaluation results of the image sequence samples, and the preliminary detection result labels of the image sequence samples.