Method, device, and storage medium for detecting large resolution traffic images

Abstract

The application provides a large-resolution traffic image detection method, device and storage medium, the method comprising: performing cut image processing on a large-resolution traffic image; performing target detection on the cut image through a pre-trained target detection model to obtain a detection result; the target detection model is trained based on a sample image and a supplementary sample image; and the supplementary sample image is obtained according to the training result of the sample image. The target detection model used in the method provided by the application for detecting a large-resolution traffic image is trained based on a sample image and a supplementary sample image, and the supplementary sample image is obtained according to the training result of the sample image. Through the optimization and training of the target detection model by taking the supplementary sample image as a training sample, the recognition effect of the target detection model on the large-resolution traffic image can be improved, the false detection rate and the recall rate of the target detection model can be effectively reduced, and the detection quality of the large-resolution traffic image is ensured.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, device, and storage medium for detecting high-resolution traffic images. Background Technology

[0002] With the development of artificial intelligence, terms such as smart cities, smart transportation, and smart city brains have gradually emerged and become hot topics of discussion. Among them, artificial intelligence is the cornerstone of these intelligent scenarios, deep learning is an important component of artificial intelligence, and object detection is a crucial part of deep learning.

[0003] The application of target detection algorithms can effectively identify various targets in traffic scenarios, form events from the targets through time series or spatial relationships, and return the results of event analysis to autonomous vehicles, traffic management departments, control centers and other control parties, enabling intervention and scheduling of autonomous vehicles or traffic, and efficiently serving the public.

[0004] Commonly used object detection algorithms can only handle small image sizes (e.g., 640*640, 480*480), which is extremely unfavorable for high-resolution roadside images (e.g., 3840*2160). During training, the model scales down the image, causing the labeled bounding boxes to become extremely small or even disappear, resulting in poor model recognition performance and severe false negatives and missed detections. Therefore, there is an urgent need for a detection method specifically designed for high-resolution traffic images. Summary of the Invention

[0005] To address one of the aforementioned technical deficiencies, this application provides a method, apparatus, and storage medium for detecting high-resolution traffic images.

[0006] The first aspect of this application provides a method for detecting high-resolution traffic images, the method comprising:

[0007] Image slicing processing is performed on high-resolution traffic images;

[0008] The image after being sliced ​​is used to detect objects by a pre-trained object detection model, and the detection results are obtained.

[0009] The object detection model is trained based on sample images and supplementary sample images;

[0010] The supplementary sample images are obtained based on the training results of the sample images.

[0011] Optionally, before obtaining the detection results, the process includes performing object detection on the cropped image using a pre-trained object detection model, and further includes:

[0012] The high-resolution traffic images of the samples are cropped.

[0013] Based on the image after image cropping, obtain sample images;

[0014] The target detection model is trained on the sample images to obtain the detection results;

[0015] Based on the detection results, a false detection analysis is conducted to determine the false detection rate;

[0016] When the false detection rate is higher than the preset false detection rate threshold, supplementary sample images are determined based on the false detection images obtained from the false detection analysis; sample images are obtained based on the image after image cropping and the supplementary sample images; the steps of training the target detection model on the sample images and subsequent steps are repeated until the target detection model meets the preset effect, and the current target detection model is used as the trained target detection model.

[0017] When the false detection rate is not higher than the preset false detection rate threshold, the current target detection model is used as the trained target detection model.

[0018] Optionally, based on the detection results, a false detection analysis is performed to determine the false detection rate, including:

[0019] Determine the camera identifier to which each false detection image obtained from the false detection analysis belongs;

[0020] Determine the number of false positive images corresponding to each camera identifier within a preset time period;

[0021] The false detection rate of each camera identifier is determined by the quotient of the number of falsely detected images corresponding to each camera identifier within a preset time period and the total number of falsely detected images within the preset time period; or, the false detection rate of each camera identifier is determined by the number of falsely detected images that are continuously detected by each camera identifier within a preset time period.

[0022] Optionally, based on the false detection images obtained from the false detection analysis, supplementary sample images are determined, including:

[0023] Based on the false detection images obtained from the false detection analysis, the target false detection image is obtained; the false detection rate of the camera identifier to which the target false detection image belongs is higher than the preset false detection rate threshold.

[0024] Determine the false detection scenario type of the target false detection image; the false detection type is either fixed scenario false detection, general scenario false detection, or non-fixed scenario false detection.

[0025] Based on the type of false detection scenario of the target false detection image, supplementary sample images are determined.

[0026] Optionally, supplementary sample images are determined based on the false detection scenario type of the target false detection image, including:

[0027] If the false detection scene type of the target false detection image is fixed scene false detection, then when the camera identifier of the target false detection image is not in the reported traffic incident report, a target thumbnail is pasted into the target false detection image; the image after pasting the target thumbnail is determined as a supplementary sample image;

[0028] The target thumbnail represents the detected object in the false positive image, and the pasting position of the target thumbnail is determined based on the scaling factor, rotation angle, and the position of the geometric center of the target thumbnail in the false positive image. The scaling factor is determined based on the width of the false positive image, the height of the false positive image, and the position of the geometric center of the target thumbnail in the false positive image.

[0029] Optionally, supplementary sample images are determined based on the false detection scenario type of the target false detection image, including:

[0030] If the false detection scenario type of the target false detection image is a general scenario false detection, then the target false detection image will be used as a supplementary sample image.

[0031] Optionally, supplementary sample images are determined based on the false detection scenario type of the target false detection image, including:

[0032] If the false detection scene type of the target false detection image is a non-fixed scene false detection, then if the camera identifier to which the target false detection image belongs is not in the reported traffic incident report table, and the target detection score of the target false detection image is less than the score threshold, the target false detection image will be used as a supplementary sample image.

[0033] Optionally, the image slicing process includes: target image slicing, nearest neighbor image slicing, sliding window overlapping image slicing, and whole image preservation.

[0034] A second aspect of this application provides an electronic device, comprising:

[0035] Memory;

[0036] Processor; and

[0037] Computer programs;

[0038] The computer program is stored in the memory and configured to be executed by the processor to implement the method described in the first aspect above.

[0039] A third aspect of this application provides a computer-readable storage medium having a computer program stored thereon; the computer program is executed by a processor to implement the method described in the first aspect above.

[0040] This application provides a method, device, and storage medium for detecting high-resolution traffic images. The method includes: slicing the high-resolution traffic image; performing object detection on the sliced ​​image using a pre-trained object detection model to obtain detection results; the object detection model is trained based on sample images and supplementary sample images; the supplementary sample images are obtained based on the training results of the sample images. The object detection model used in the method provided by this application for detecting high-resolution traffic images is trained based on sample images and supplementary sample images, and the supplementary sample images are obtained based on the training results of the sample images. By using supplementary sample images as training samples, the object detection model is optimized and trained, which can improve the recognition effect of the object detection model for high-resolution traffic images, effectively reduce the false detection rate and recall rate of the object detection model, and ensure the detection quality of high-resolution traffic images. Attached Figure Description

[0041] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0042] Figure 1 A flowchart illustrating a method for detecting high-resolution traffic images provided in an embodiment of this application;

[0043] Figure 2 This is a schematic diagram of the image slicing process provided in the embodiments of this application;

[0044] Figure 3 This is a schematic diagram illustrating the mosaic enhancement effect provided in an embodiment of this application.

[0045] Figure 4 This is a schematic diagram illustrating the enhancement effect of mixup provided in an embodiment of this application;

[0046] Figure 5 This is a schematic diagram illustrating the HSV color space enhancement effect provided in an embodiment of this application.

[0047] Figure 6 This is a schematic diagram illustrating the image flipping enhancement effect provided in an embodiment of this application;

[0048] Figure 7 A schematic diagram illustrating the tiling prediction method provided in the embodiments of this application;

[0049] Figure 8 A schematic diagram of supplementary sample images for false detections in fixed scenes provided in the embodiments of this application;

[0050] Figure 9 A schematic diagram illustrating false detections in non-fixed scenarios provided in the embodiments of this application;

[0051] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0052] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0053] In developing this application, the inventors discovered that commonly used object detection algorithms can only handle small image sizes, which is extremely unfavorable for high-resolution roadside images. Because the model scales down the image during training, the labeled bounding boxes become extremely small or even disappear, resulting in poor model recognition performance and severe false negatives and missed detections. Therefore, there is an urgent need for a detection method specifically for high-resolution traffic images.

[0054] To address the aforementioned issues, this application provides a method, device, and storage medium for detecting high-resolution traffic images. The method includes: slicing the high-resolution traffic image; performing object detection on the sliced ​​image using a pre-trained object detection model to obtain detection results; the object detection model is trained based on sample images and supplementary sample images; the supplementary sample images are obtained based on the training results of the sample images. The method provided in this application uses an object detection model trained on sample images and supplementary sample images when detecting high-resolution traffic images. The supplementary sample images are obtained based on the training results of the sample images. By using supplementary sample images as training samples to optimize the object detection model, the recognition performance of the object detection model for high-resolution traffic images can be improved, effectively reducing the false detection rate and recall rate of the object detection model, and ensuring the detection quality of high-resolution traffic images.

[0055] See Figure 1 The implementation process of the high-resolution traffic image detection method provided in this embodiment is as follows:

[0056] 101. Perform image slicing on high-resolution traffic images.

[0057] The image slicing process includes: target image slicing, nearest neighbor image slicing, sliding window overlapping image slicing, and whole image preservation.

[0058] In the specific implementation, the high-resolution traffic image is labeled before step 101. For example, according to the labeling requirements, the labeling tool is used to label the target bounding box information on the high-resolution image, and the overall labeling result is returned. Then, step 101 is executed to perform image tiling processing on the labeled result, and the labeled coordinates can also be mapped accordingly based on the tiling method.

[0059] In addition, step 101 is also a stage of image processing, so four image slicing methods will be used: target slicing, nearest neighbor slicing, sliding window overlapping slicing, and whole image preservation. Regardless of the slicing method, the minimum size of the sliced ​​image must be greater than 56 pixels in both width and height.

[0060] Among them, such as Figure 2 As shown, target cropping is a method that involves appropriately expanding the boundaries of a single labeled target in an image before cropping. The cropped image obtained through this method ensures the target detection model's ability to recognize large targets.

[0061] Nearest neighbor cropping is a method that merges the n (n>=2) closest labeled targets in a single image, appropriately expands the boundary after merging, and then crops the image. The cropped image obtained by this method can ensure the target detection model's ability to recognize medium and large-sized targets.

[0062] Sliding window overlapping image slicing is a method that sets the sliding window size and overlap range, and slices the image from the top left corner to the bottom right corner. The sliced ​​image obtained using this method can ensure the target detection model's ability to recognize small to medium-sized targets.

[0063] The whole-image preservation process directly uses the annotation results of the original labeled data. The cropped image obtained by this method can ensure the target detection model's ability to recognize small-sized targets.

[0064] In practical implementation, asynchronous processing methods can be used for target image tiling, nearest neighbor image tiling, sliding window overlapping image tiling, and whole image retention.

[0065] In addition, data preprocessing methods such as mosaic enhancement, mixup enhancement, HSV color space enhancement, and image flipping enhancement will be added. In the actual implementation, all of these methods will be used.

[0066] The mosaic enhancement process involves selecting multiple images, performing operations such as flipping, scaling, and color gamut changes on each image, and then arranging the processed images in different positions to form a single image. Figure 3 As shown.

[0067] The mixup enhancement process involves merging two or more images according to a certain ratio to create a single image, such as... Figure 4 As shown.

[0068] The HSV color space enhancement process involves converting the image from RGB to HSV space and adjusting the corresponding h (hue), s (saturation), and v (brightness) values ​​to achieve data enhancement. Figure 5 As shown.

[0069] The image flipping enhancement process involves flipping the image horizontally or vertically, resulting in a new image, such as... Figure 6 As shown.

[0070] 102. The pre-trained object detection model is used to perform object detection on the sliced ​​image to obtain the detection results.

[0071] The object detection model is trained based on sample images and supplementary sample images. The supplementary sample images are obtained based on the training results of the sample images.

[0072] Since step 102 uses a pre-trained object detection model for object detection, the model is trained before executing step 102. This training process can be performed on each execution. Figure 1 In the high-resolution traffic image detection method shown, the detection is performed before step 102. Alternatively, it can be performed when the execution conditions are met (such as every preset time interval, or when the detection accuracy of the target detection model is lower than a certain threshold, or when a new scene detection requirement appears). This embodiment does not limit the training time or number of training iterations of the target detection model.

[0073] The training process of the object detection model is as follows:

[0074] 1. Perform image slicing on the high-resolution traffic images of the samples.

[0075] The image slicing process here is the same as that in step 101, and will not be described in detail here. Please refer to step 101 for details.

[0076] In addition, the high-resolution traffic images of the samples will be labeled before step 1. Data preprocessing methods such as mosaic enhancement, mixup enhancement, HSV color space enhancement, and image flipping enhancement will also be added.

[0077] 2. Based on the image after image slicing, obtain sample images.

[0078] All images after image cropping can be used as sample images, or only a portion of the images after image cropping (such as images of better quality) can be used as sample images. This embodiment does not limit this.

[0079] 3. Train the target detection model on the sample images to obtain the detection results.

[0080] The ultimate goal of an object detection model is to detect objects. The detection results need to be based on the actual scene requirements. The detection results depend on the image resolution. If the image resolution is too high, the object will shrink or even be lost when it is fed into a smaller object detection model after normalization, resulting in missed detections. Therefore, in specific implementations, a slice prediction method can be used for object detection. That is, the sample image is sliced ​​first (the purpose is to increase the size of the object detected by the object detection algorithm and improve the recall rate), and then the object detection algorithm is used to detect the object.

[0081] in,

[0082] 1) You can choose the appropriate image slicing method according to the size of the sample image. You can use methods such as whole image prediction, image n equal parts overlap prediction, sliding window overlap prediction, etc.

[0083] as follows Figure 7 As shown, whole-image prediction: The model is used to predict the whole image, and the model output is the prediction result. This method can achieve good recognition results for large and medium-sized targets.

[0084] Image n-part overlap prediction: The image is divided into n (n>=2, n=1 for whole-image prediction) equal parts with overlap. Each part is then fed into the model for prediction. The prediction results are mapped back to the original image based on the coordinates of the parts. Non-maximum suppression is then applied, and the processed result is the final result. This method shows good recognition performance for targets of different sizes (large, medium, small, micro) depending on the setting of n.

[0085] Sliding window overlap prediction: For images with overlapping areas, prediction is performed by sliding the window from left to right and top to bottom according to a set window size. The predicted coordinates are mapped back to the original image coordinates. Non-maximum suppression is then applied, and the processed result is the final result. This method shows good recognition performance for targets of different sizes (large, medium, small, micro) depending on the sliding window size.

[0086] In practice, the appropriate slicing method can be selected based on the pixel characteristics of the input image (i.e., the sliced ​​image).

[0087] For example,

[0088] a) When the longest side of the input image (i.e., the sliced ​​image) is less than or equal to 1280 pixels, select whole image prediction.

[0089] b) When the longest side pixel of the input image (i.e. the image after slicing) is greater than 1280 and less than or equal to 2560, select n equal parts of the image for overlapping prediction, where n = 4.

[0090] c) When the longest side pixel of the input image (i.e. the image after slicing) is greater than 2560 and less than or equal to 3840, select n equal parts of the image for overlapping prediction, where n = 9.

[0091] d) When the longest side of the input image (i.e. the image after cropping) is greater than 3840, a sliding window overlapping prediction is adopted. The longest side of the sliding window is 2160, the aspect ratio is consistent with the input size of the network structure, and the overlap ratio is 5% of the corresponding aspect ratio.

[0092] It should be noted that there are two image slicing processes during the training of the object detection model. The first is image slicing of the high-resolution traffic images in step 1, and the second is image slicing of the sample images in step 3 before object detection is performed using the object detection algorithm. The data sources for the two image slicing processes are different. The image slicing in step 1 is for model training, and the image data is offline. The image slicing in step 3 is for object detection, and the image data is online; the sliced ​​data is directly used for online object detection.

[0093] In other words, the image slicing process in step 101 preprocesses the high-resolution traffic image to ensure that the aspect ratio of the sliced ​​image is consistent, thereby ensuring that the aspect ratio of the image input to the target detection model in step 102 is consistent. The image slicing method in this step involves slicing the input image (i.e., the image after slicing in step 101) again before prediction. The purpose of this slicing is to perform prediction and obtain the prediction result. Therefore, the slicing method in step 102 does not need to ensure that the aspect ratio of the sliced ​​image is consistent; it can be any of several aspect ratios. Once a certain size is determined, an image is cropped according to this size.

[0094] In this step, after slicing the image using the slicing method, the resulting slice prediction will be merged into the original image.

[0095] Directly predicting with high-resolution images can lead to missed detections. This is because high-resolution images are scaled down to the network's specified size (e.g., 512x512, 1280x1280) when input into the network structure, causing the target to be lost during the scaling process, resulting in missed detections. The image slicing method described in this step avoids this problem and ensures the accuracy of subsequent predictions.

[0096] 2) The target detection algorithm used can be an existing algorithm, such as one-stage, two-stage, etc. This embodiment does not limit the target detection algorithm used in the target detection model.

[0097] 4. Based on the detection results, conduct false detection analysis to determine the false detection rate.

[0098] There are several methods for determining the false positive rate.

[0099] For example, one method for determining the false detection rate is to use the quotient of the number of falsely detected images corresponding to each camera identifier within a preset time period and the total number of falsely detected images within the preset time period as the false detection rate of each camera identifier.

[0100] In practical implementation,

[0101] 1) Determine the camera identifier to which each false detection image obtained from the false detection analysis belongs.

[0102] Each false positive image is captured by a camera, and the process identifies the camera that captured each false positive image (such as an ID or IP address).

[0103] In addition, false detection images can be identified using existing methods or through manual verification. For example, if a construction cone is detected on an image that does not contain one, then the construction cone is a false detection.

[0104] During the training of object detection models, false positive image recognition is frequently performed on specified recall and false positive sets.

[0105] 2) Determine the number of false positive images corresponding to each camera identifier within a preset time period.

[0106] 3) The quotient of the number of falsely detected images corresponding to each camera identifier and the total number of falsely detected images within a preset time period is determined as the false detection rate of each camera identifier.

[0107] For example, for camera identifier i, count the number of images captured by camera identifier i that are identified as false positives NPi within a certain period. Obtain the total number NP of all images identified as false positives within that period, and determine the value of NPi / NP as the false positive rate of camera identifier i.

[0108] Another method for determining the false detection rate is to determine the number of falsely detected images of each camera identifier within a preset time period as the false detection rate of each camera identifier.

[0109] In practical implementation,

[0110] 1) Determine the camera identifier of a falsely detected image in each preset cycle (e.g., 1 minute).

[0111] For example, every minute, the camera identifier of each false detection image obtained from the false detection analysis is determined.

[0112] 2) The number of falsely detected images of each camera identifier within a preset time period is determined as the false detection rate of each camera identifier.

[0113] For example, for camera identifier i, the number of falsely detected images that are continuously identified as falsely detected within a preset time (e.g., images captured by camera identifier i are identified as falsely detected images within 3 consecutive minutes) is determined as the false detection rate of each camera identifier.

[0114] 5. When the false detection rate exceeds the preset false detection rate threshold (where the preset false detection rate threshold corresponds to the method for determining the false detection rate; for example, if the false detection rate is a ratio, then the false detection rate threshold is also a ratio. If the false detection rate is the number of consecutive recognitions, then the false detection rate threshold is also a number of recognitions), it indicates that the image captured by the corresponding camera identifier is being continuously recognized. Therefore, the following steps are executed:

[0115] 1) Based on the false detection images obtained from the false detection analysis, determine the supplementary sample images.

[0116] Specifically,

[0117] (1) Obtain the target false detection image based on the false detection image obtained from the false detection analysis.

[0118] Among them, the false detection rate of the camera identifier to which the target false detection image belongs is higher than the preset false detection rate threshold.

[0119] In step (1), the false detection images collected by the camera with a false detection rate higher than the preset false detection rate threshold will be used as the target false detection images. The false detection images collected by the camera with a false detection rate not higher than the preset false detection rate threshold will not be processed.

[0120] (2) Determine the false detection scenario type of the target false detection image.

[0121] The false detection types are: fixed scenario false detection, general scenario false detection, or non-fixed scenario false detection.

[0122] In practice, the false detection type is not determined based on a single false detection image of the target, but rather based on multiple false detection images of the target.

[0123] The scheme for determining the false detection scene type can adopt existing methods (such as clustering methods, calculating the distance between the target false detection image and each false detection scene type, and clustering it into the false detection scene type with the smallest distance, which is then the false detection scene type. Alternatively, the false detection scene type can be determined manually). This embodiment does not limit the scheme for determining the false detection scene type.

[0124] For example, when detecting construction cones (i.e., the object being detected is a construction cone), if the same location at the same intersection is frequently falsely detected at different times, this is called a fixed-scene false detection. Fixed-scene false detection manifests as the target detection model continuously falsely detecting the same location in images captured by the same camera.

[0125] For example, fire hydrants in different locations in different pictures are mistakenly detected as construction cones; this is a common scenario of location misdetection.

[0126] For example, in one image, flowers and plants are mistakenly identified as construction cones; in another, pedestrians are mistakenly identified as construction cones; and in yet another, vehicles and road markings are mistakenly identified as construction cones. This broad category is called non-fixed scene misdetection. Non-fixed scene misdetection means that the model is misdetected in images taken by different cameras.

[0127] (3) Based on the false detection scenario type of the target false detection image, determine the supplementary sample image.

[0128] Specifically,

[0129] • If the false detection scene type of the target false detection image is a fixed scene false detection, then

[0130] When the camera identifier of a falsely detected image is not listed in the reported traffic incident report, a small target image is pasted onto the falsely detected image. The image with the pasted target image is then designated as a supplementary sample image. Figure 8 As shown.

[0131] In this context, the target thumbnail represents the detected object (e.g., a construction cone) in the false positive image, and the pasting position of the target thumbnail is determined based on the scaling factor, rotation angle, and the position of the geometric center of the target thumbnail within the false positive image. The scaling factor is determined based on the width, height, and geometric center of the target thumbnail within the false positive image.

[0132] For falsely detected images of targets in fixed scenes, if the camera identifier of the target is not listed in the reported traffic incident report, it indicates that the target has been misidentified. Therefore, by placing a small image of the target next to the false detection location (e.g., pasting a small image of the target category outside the false detection location area), labeling the data, and then using the image with the pasted target image as a supplementary sample image, and finally adding the supplementary sample image as a positive sample to the training set to train the model, false detections at fixed locations can be effectively reduced or even eliminated. If the camera identifier of the target is listed in the reported traffic incident report, it indicates that the target has been correctly identified and no further processing is required.

[0133] When pasting the target thumbnail, it can be appropriately scaled and rotated according to the image resolution and its position on the false positive image. That is, the pasting position of the target thumbnail is based on the scaling factor k, the rotation angle θ, and the position of the geometric center of the target thumbnail in the false positive image (t). x ,t y The scaling factor k is determined based on the width (WIDTH) of the target false positive image, the height (HIGHT) of the target false positive image, and the position (t) of the geometric center of the target thumbnail within the target false positive image. x,t y )Determined.

[0134] For example, the pasting position of the target small image is calculated by the following formula.

[0135]

[0136]

[0137]

[0138] Among them, x' and y' are the pasting positions of the target small image (i.e., the positions after scaling and rotation processing), θ is the rotation angle, greater than zero indicates counterclockwise rotation, less than zero indicates clockwise rotation, and θ ∈ [-180°, 180°]. t x ,t y is the position of the geometric center of the target small image in the target mis-detected image (i.e., the position where the geometric center of the target small image is pasted on the original image). ± indicates whether to flip, + indicates not to flip, - indicates to flip. The plus or minus sign in the first row indicates whether to flip horizontally, and the plus or minus sign in the second row indicates whether to flip vertically. k is the scaling factor, k ∈ (0, +∞). When 0 < k < 1, it is for shrinking processing, and when k ≥ 1, it is for expansion processing. is the floor operation.

[0139] · If the mis-detection scene type of the target mis-detected image is a general scene mis-detection, then

[0140] Use the target mis-detected image as a supplementary sample image.

[0141] Specifically, all target mis-detected images can be used as supplementary sample images, or the target mis-detected images can be clustered, and the target mis-detected images corresponding to several categories with more target mis-detected images can be used as supplementary sample images.

[0142] · If the mis-detection scene type of the target mis-detected image is an unfixed scene mis-detection, then

[0143] When the camera identifier of the target mis-detected image is not in the reported traffic event report form and the target detection score of the target mis-detected image is less than the score threshold, use the target mis-detected image as a supplementary sample image.

[0144] For the target mis-detected image with unfixed scene mis-detection, when its camera identifier is not in the reported traffic event report form and the target detection score of the target mis-detected image is less than the score threshold, then use it as a supplementary sample image. When its camera identifier is in the reported traffic event report form, or the target detection score of the target mis-detected image is not less than the score threshold, although it is a mis-detection, no sample supplementation is required and no processing is performed.

[0145] In other words, for a target false detection image whose false detection scenario type is not fixed, whether to use it as a supplementary sample image depends on whether the target detection score of the target false detection image is less than the score threshold. If it is less than the threshold, it is used as a supplementary sample image; if it is not less than the threshold, it is not used as a sample image.

[0146] Furthermore, if a scoring method is also used to determine whether an image is a false positive, for example, if the object detection score of an image is less than a threshold (let's call it threshold 1 for convenience), then the image is determined to be a false positive. In this case, the scoring threshold (let's call it threshold 2 for convenience) will be less than the threshold used to determine a false positive, i.e., threshold 2 < threshold 1.

[0147] 2) Obtain sample images based on the image after image tiling and supplementary sample images.

[0148] • For false detections of target images in a fixed scene,

[0149] The supplementary sample image is used as a positive sample, and the image after cropping is used together as the sample image.

[0150] • For false positives in general scenarios,

[0151] The supplementary sample image is used as a negative sample, and the image after cropping is used together as the sample image.

[0152] • For false detections of targets in non-fixed scenes,

[0153] The supplementary sample image is used as a new category, and the image after image cropping is used together as the sample image.

[0154] 3) Repeat the steps of training the object detection model on the sample images (i.e., step 3) and subsequent steps until the object detection model meets the preset effect. Then, take the current object detection model (i.e., the object detection model that meets the preset effect) as the trained object detection model.

[0155] When the false detection rate exceeds a preset threshold, for falsely detected images of targets in fixed scenes, if the camera identifier to which the image belongs is not in the reported traffic incident table, a detection target is placed next to the false detection location, the data is labeled, and the target detection model is retrained as a positive sample. This can effectively reduce or even eliminate false detections at fixed locations. For falsely detected images of targets in general scenes, the falsely detected images are clustered, and the top few categories with the most false detections are used as negative samples to retrain the target detection model. This can effectively and significantly reduce false detections in similar scenes. For falsely detected images of targets in non-fixed scenes, if the camera identifier to which the image belongs is not in the reported traffic incident table, and the target detection score is less than the score threshold, all falsely detected images are treated as a new category, and the falsely detected images are added to the training set for retraining the target detection model (e.g., all falsely detected images are treated as a new category, and the original image and bounding box information are reused and added to the training set to train the model). This method can adapt to false detections in any scene.

[0156] For false detections of targets in non-fixed scenes, such as Figure 9 As shown, the score is 0.24. Assume there are currently 3 detection categories, 0, 1, and 2. If the camera identifier of the falsely detected image is not in the traffic event table, determine if the score of 0.24 is less than the preset false detection rate threshold (assuming 0.5). If it is less, record the false detection box location information, and set the target category to the new category 3 (all false detection categories are set to false detection 3). Then, input the original detection image, target box information, and category information into the training set for training.

[0157] In addition, the expected results are pre-set. For example, the recall effect of the recall set exceeds 95%, and the false detection effect of the false detection set is less than 0.5%. Reaching this indicator is considered to have achieved the expected results (this indicator is mostly based on actual business and is generally different for different businesses).

[0158] 6. When the false detection rate is not higher than the preset false detection rate threshold, the current target detection model is used as the trained target detection model.

[0159] The high-resolution traffic image detection method provided in this embodiment can effectively improve the target detection performance on high-resolution traffic images and effectively reduce false detections. Furthermore, the high-resolution traffic image detection method provided in this embodiment has general applicability, offering solutions and optimization methods for target detection algorithms on general high-resolution images.

[0160] This embodiment provides a method for detecting high-resolution traffic images. The method includes: slicing the high-resolution traffic image; performing object detection on the sliced ​​image using a pre-trained object detection model to obtain detection results; the object detection model is trained based on sample images and supplementary sample images; the supplementary sample images are obtained based on the training results of the sample images. The method provided in this embodiment uses an object detection model trained on sample images and supplementary sample images when detecting high-resolution traffic images. The supplementary sample images are obtained based on the training results of the sample images. By using the supplementary sample images as training samples to optimize the object detection model, the recognition performance of the object detection model for high-resolution traffic images can be improved, effectively reducing the false detection rate and recall rate of the object detection model, and ensuring the detection quality of high-resolution traffic images.

[0161] Based on the same inventive concept as detection methods for high-resolution traffic images, this embodiment provides an electronic device, which, as follows: Figure 10 As shown, it includes: a memory 1001, a processor 1002, and a computer program.

[0162] The computer program is stored in memory 1001 and configured to be executed by processor 1002 to implement the above-described method for detecting high-resolution traffic images.

[0163] Specifically,

[0164] High-resolution traffic images are sliced.

[0165] The image after being sliced ​​is used to detect objects by a pre-trained object detection model, and the detection results are obtained.

[0166] The object detection model is trained based on sample images and supplementary sample images.

[0167] The supplementary sample images are obtained based on the training results of the sample images.

[0168] Optionally, before obtaining the detection results, the process includes performing object detection on the cropped image using a pre-trained object detection model, and further includes:

[0169] The high-resolution traffic images of the samples are sliced.

[0170] Based on the image after image slicing, obtain sample images.

[0171] The target detection model is trained on the sample images to obtain the detection results.

[0172] Based on the detection results, a false detection analysis is performed to determine the false detection rate.

[0173] When the false detection rate exceeds a preset false detection rate threshold, supplementary sample images are determined based on the false detection images obtained from the false detection analysis. Sample images are then acquired based on the sliced ​​images and the supplementary sample images. The steps of training the object detection model on the sample images and subsequent steps are repeated until the object detection model meets the preset performance. The current object detection model is then used as the trained object detection model.

[0174] When the false detection rate is not higher than the preset false detection rate threshold, the current target detection model is used as the trained target detection model.

[0175] Optionally, based on the detection results, a false detection analysis is performed to determine the false detection rate, including:

[0176] Determine the camera identifier to which each false detection image obtained from the false detection analysis belongs.

[0177] The false detection rate for each camera identifier is determined by dividing the number of falsely detected images corresponding to each camera identifier within a preset time period by the total number of falsely detected images within that preset time period. Alternatively, the false detection rate for each camera identifier is determined by the number of consecutively falsely detected images generated within a preset time period.

[0178] Optionally, based on the false detection images obtained from the false detection analysis, supplementary sample images are determined, including:

[0179] Based on the false detection images obtained from the false detection analysis, the target false detection image is acquired. The false detection rate of the camera identifier to which the target false detection image belongs is higher than a preset false detection rate threshold.

[0180] Determine the false detection scenario type of the target false detection image. The false detection type is either a fixed scenario false detection, a general scenario false detection, or a non-fixed scenario false detection.

[0181] Based on the type of false detection scenario of the target false detection image, supplementary sample images are determined.

[0182] Optionally, supplementary sample images are determined based on the false detection scenario type of the target false detection image, including:

[0183] If the false detection scene type of the target false detection image is a fixed scene false detection, then if the camera identifier of the target false detection image is not in the reported traffic incident report table, a target thumbnail is pasted into the target false detection image. The image after pasting the target thumbnail is determined as a supplementary sample image.

[0184] In this context, the target thumbnail represents the detected object in the false positive image, and its pasting position is determined based on the scaling factor, rotation angle, and the position of the target thumbnail's geometric center within the false positive image. The scaling factor is determined based on the width, height, and geometric center of the target thumbnail within the false positive image.

[0185] Optionally, supplementary sample images are determined based on the false detection scenario type of the target false detection image, including:

[0186] If the false detection scenario type of the target false detection image is a general scenario false detection, then the target false detection image will be used as a supplementary sample image.

[0187] Optionally, supplementary sample images are determined based on the false detection scenario type of the target false detection image, including:

[0188] If the false detection scene type of the target false detection image is a non-fixed scene false detection, then if the camera identifier to which the target false detection image belongs is not in the reported traffic incident report table, and the target detection score of the target false detection image is less than the score threshold, the target false detection image will be used as a supplementary sample image.

[0189] Optionally, the image slicing process includes: target image slicing, nearest neighbor image slicing, sliding window overlapping image slicing, and whole image preservation.

[0190] The electronic device provided in this embodiment has a computer program executed by a processor. When detecting high-resolution traffic images, the target detection model is trained based on sample images and supplementary sample images. The supplementary sample images are obtained based on the training results of the sample images. By using the supplementary sample images as training samples, the target detection model is optimized and trained, which can improve the recognition effect of the target detection model for high-resolution traffic images, effectively reduce the false detection rate and recall rate of the target detection model, and ensure the detection quality of high-resolution traffic images.

[0191] Based on the same inventive concept as the detection method for high-resolution traffic images, this embodiment provides a computer on which a computer program is stored. The computer program is executed by a processor to implement the aforementioned detection method for high-resolution traffic images.

[0192] Specifically,

[0193] High-resolution traffic images are sliced.

[0194] The image after being sliced ​​is used to detect objects by a pre-trained object detection model, and the detection results are obtained.

[0195] The object detection model is trained based on sample images and supplementary sample images.

[0196] The supplementary sample images are obtained based on the training results of the sample images.

[0197] Optionally, before obtaining the detection results, the process includes performing object detection on the cropped image using a pre-trained object detection model, and further includes:

[0198] The high-resolution traffic images of the samples are sliced.

[0199] Based on the image after image slicing, obtain sample images.

[0200] The target detection model is trained on the sample images to obtain the detection results.

[0201] Based on the detection results, a false detection analysis is performed to determine the false detection rate.

[0202] When the false detection rate exceeds a preset false detection rate threshold, supplementary sample images are determined based on the false detection images obtained from the false detection analysis. Sample images are then acquired based on the sliced ​​images and the supplementary sample images. The steps of training the object detection model on the sample images and subsequent steps are repeated until the object detection model meets the preset performance. The current object detection model is then used as the trained object detection model.

[0203] When the false detection rate is not higher than the preset false detection rate threshold, the current target detection model is used as the trained target detection model.

[0204] Optionally, based on the detection results, a false detection analysis is performed to determine the false detection rate, including:

[0205] Determine the camera identifier to which each false detection image obtained from the false detection analysis belongs.

[0206] The false detection rate for each camera identifier is determined by dividing the number of falsely detected images corresponding to each camera identifier within a preset time period by the total number of falsely detected images within that preset time period. Alternatively, the false detection rate for each camera identifier is determined by the number of consecutively falsely detected images generated within a preset time period.

[0207] Optionally, based on the false detection images obtained from the false detection analysis, supplementary sample images are determined, including:

[0208] Based on the false detection images obtained from the false detection analysis, the target false detection image is acquired. The false detection rate of the camera identifier to which the target false detection image belongs is higher than a preset false detection rate threshold.

[0209] Determine the false detection scenario type of the target false detection image. The false detection type is either a fixed scenario false detection, a general scenario false detection, or a non-fixed scenario false detection.

[0210] Based on the type of false detection scenario of the target false detection image, supplementary sample images are determined.

[0211] Optionally, supplementary sample images are determined based on the false detection scenario type of the target false detection image, including:

[0212] If the false detection scene type of the target false detection image is a fixed scene false detection, then if the camera identifier of the target false detection image is not in the reported traffic incident report table, a target thumbnail is pasted into the target false detection image. The image after pasting the target thumbnail is determined as a supplementary sample image.

[0213] In this context, the target thumbnail represents the detected object in the false positive image, and its pasting position is determined based on the scaling factor, rotation angle, and the position of the target thumbnail's geometric center within the false positive image. The scaling factor is determined based on the width, height, and geometric center of the target thumbnail within the false positive image.

[0214] Optionally, supplementary sample images are determined based on the false detection scenario type of the target false detection image, including:

[0215] If the false detection scenario type of the target false detection image is a general scenario false detection, then the target false detection image will be used as a supplementary sample image.

[0216] Optionally, supplementary sample images are determined based on the false detection scenario type of the target false detection image, including:

[0217] If the false detection scene type of the target false detection image is a non-fixed scene false detection, then if the camera identifier to which the target false detection image belongs is not in the reported traffic incident report table, and the target detection score of the target false detection image is less than the score threshold, the target false detection image will be used as a supplementary sample image.

[0218] Optionally, the image slicing process includes: target image slicing, nearest neighbor image slicing, sliding window overlapping image slicing, and whole image preservation.

[0219] The computer-readable storage medium provided in this embodiment contains a computer program executed by a processor. When detecting high-resolution traffic images, the target detection model is trained based on sample images and supplementary sample images. The supplementary sample images are obtained based on the training results of the sample images. By using the supplementary sample images as training samples, the target detection model is optimized and trained, which can improve the recognition effect of the target detection model for high-resolution traffic images, effectively reduce the false detection rate and recall rate of the target detection model, and ensure the detection quality of high-resolution traffic images.

[0220] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0221] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0222] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0223] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0224] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0225] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method of detecting a large resolution traffic image, characterized by, The method comprises: cutting processing is performed on a large-resolution traffic image; based on the image after cutting processing, a sample image is obtained; target detection model training is performed on the sample image to obtain a detection result; based on the detection result, false detection analysis is performed to determine a false detection rate and determine the camera identifier to which each false detection image obtained by false detection analysis belongs; the quotient of the number of false detection images corresponding to each camera identifier within a preset time and the total number of false detection images within the preset time period is determined as the false detection rate of each camera identifier; or the number of false detection images continuously detected by false detection analysis within a preset time for each camera identifier is determined as the false detection rate of each camera identifier; when the false detection rate is higher than a preset false detection rate threshold, a target false detection image is obtained according to the false detection image obtained by false detection analysis; the false detection rate of the camera identifier to which the target false detection image belongs is higher than the preset false detection rate threshold; the false detection scene type of the target false detection image is determined; the false detection type is fixed scene false detection, general scene false detection, or non-fixed scene false detection; a supplementary sample image is determined according to the false detection scene type of the target false detection image; based on the image after cutting processing and the supplementary sample image, a sample image is obtained; the steps of target detection model training on the sample image and subsequent steps are repeatedly performed until the target detection model meets a preset effect, and the current target detection model is taken as a trained target detection model; when the false detection rate is not higher than the preset false detection rate threshold, the current target detection model is taken as a trained target detection model; wherein determining the supplementary sample image according to the false detection scene type of the target false detection image comprises: if the false detection scene type of the target false detection image is fixed scene false detection, then when the camera identifier to which the target false detection image belongs is not located in a reported traffic event report table, a target small image is pasted in the target false detection image; the image after pasting the target small image is determined as the supplementary sample image; wherein the target small image represents the detection object of the target false detection image, and the pasting position of the target small image is determined based on a scaling factor, a rotation angle, and the position of the geometric center of the target small image in the target false detection image; the scaling factor is determined based on the width of the target false detection image, the height of the target false detection image, and the position of the geometric center of the target small image in the target false detection image; target detection is performed on the cut image by a pre-trained target detection model to obtain a detection result.

2. The method of claim 1, wherein, Determining the supplementary sample image according to the false detection scene type of the target false detection image comprises: if the false detection scene type of the target false detection image is general scene false detection, then the target false detection image is taken as the supplementary sample image.

3. The method of claim 1, wherein, Determining the supplementary sample image according to the false detection scene type of the target false detection image comprises: if the false detection scene type of the target false detection image is non-fixed scene false detection, then when the camera identifier to which the target false detection image belongs is not located in a reported traffic event report table and the target detection score of the target false detection image is less than a score threshold, the target false detection image is taken as the supplementary sample image.

4. The method of claim 1, wherein, The cutout processing includes: target cutout processing, nearest neighbor cutout processing, sliding window overlapping cutout processing, and whole image reservation processing.

5. An electronic device, comprising: Comprise: a memory; a processor; and a computer program; wherein the computer program is stored in the memory and configured to be executed by the processor to implement the method of any one of claims 1-4.

6. A computer readable storage medium characterized by, having a computer program stored thereon; the computer program being executed by a processor to implement the method of any one of claims 1-4.

Images