Data set updating method, device, electronic device and storage medium

Abstract

The present application provides a data set updating method, device, electronic device and storage medium, which relates to the field of video image processing technology. The method includes: reading the acquired video data to obtain a specified frame image and a corresponding data set; detecting the specified frame image to determine the target detection information contained in the specified frame image; and updating the data set according to the target detection information. The present application detects the specified frame image in the video data, and can update the video data set based on the target detection information corresponding to one or more targets in the detected specified frame image, so as to improve the efficiency of data set updating, improve the quality of data set, accelerate the generation and iteration of deep learning models based on the optimized data set, improve the training and evaluation quality of deep learning models, and improve the precision and accuracy of target detection and recognition.

Description

Technical Field

[0001] The present application relates to the technical field of video image processing, and in particular to a data set updating method, device, electronic device and storage medium. Background Art

[0002] With the continuous development of science and technology, deep learning has gradually shown its advantages in the field of target recognition. After the neural network has developed to a certain level, the accuracy of target recognition needs to rely on the data and quality of the labeled data set. Improving the quality of the data set can improve the quality of model training and the accuracy of prediction.

[0003] In the existing technology, traditional open source datasets are basically based on laboratory research, with low accuracy in detecting and identifying specific targets, and are difficult to adapt to complex and changing engineering application scenarios. In addition, target datasets built based on actual engineering application scenarios are mostly manually labeled, which requires a lot of manpower and time costs in the manual labeling process. Manual work also makes the quality of dataset labels uneven, resulting in low efficiency and poor quality of dataset updates, and unable to accelerate the generation and iteration of deep learning models. Summary of the invention

[0004] In view of this, an object of the embodiments of the present application is to provide a data set updating method, device, electronic device and storage medium to improve the problem of low data set updating efficiency existing in the prior art.

[0005] In order to solve the above problems, in a first aspect, an embodiment of the present application provides a method for updating a data set, including:

[0006] Read the acquired video data to obtain the specified frame image and the corresponding data set;

[0007] Detecting the designated frame image to determine target detection information contained in the designated frame image;

[0008] The data set is updated according to the target detection information.

[0009] In the above implementation, by detecting the specified frame image in the video data, the video data image dataset can be updated based on the target detection information corresponding to one or more targets in the specified frame image obtained by detection, so as to improve the efficiency of dataset update, improve the quality of dataset, accelerate the generation and iteration of deep learning models based on the optimized dataset, improve the training and evaluation quality of deep learning models, and improve the precision and accuracy of target detection and recognition.

[0010] Optionally, the step of reading the acquired video data to acquire a specified frame image and a corresponding data set includes:

[0011] Get the data set corresponding to the video data;

[0012] Determine whether the frame image to be detected is a designated frame image according to the frame number of the frame image to be detected in the video data, and if so, obtain the designated frame image; wherein, the designated frame image is the nth frame image in the video data, the modulo n of m is 0, m is a preset integer, and the image frames in the video data other than the designated frame image are non-designated frame images.

[0013] In the above implementation, when acquiring the designated frame, the frame number of the frame image to be detected in the video data is used to determine whether the frame image to be detected is the designated frame, so that the designated frame and the non-designated frame in the video data can be quickly classified. The method of selecting the designated frame image by the frame number can be selected and adjusted by the staff according to the actual situation and engineering experience.

[0014] Optionally, the target detection information includes confidence and comprehensive similarity; the detecting the designated frame image to determine the target detection information contained in the designated frame image includes:

[0015] Identify the designated frame image based on the target detection model to obtain a target set corresponding to the designated frame image, wherein the target set corresponding to any designated frame image includes one or more targets in the designated frame image and a confidence level corresponding to each target;

[0016] Get the template target set of the n-1th frame image;

[0017] The comprehensive similarity between a first target in the target set and a second target in the template target set is calculated, wherein the first target is any target in the target set, and the second target is any template target in the template target set.

[0018] In the above implementation, when the current frame image is determined to be a designated frame image, the designated frame image is identified to obtain a target set of one or more targets in the designated frame image, and the target detection information may also include target information corresponding to each target, such as confidence and other information. By obtaining the previous frame image of the designated frame image, that is, the template target set of the n-1 frame image, and comparing any target in the target set with any template target in the template target set, the comprehensive similarity between the first target and the second target can be calculated to detect and identify the changes in the targets in the previous and next frame images, effectively improving the accuracy of target detection and recognition. Among them, after the comprehensive similarity of the first target and all the second targets in the template target set is calculated, the second target with the largest comprehensive similarity can be taken as the reference template, and the comprehensive similarity of the maximum value is taken as the comprehensive similarity in the target detection information.

[0019] Optionally, calculating the comprehensive similarity between the first target in the target set and the second target in the template target set includes:

[0020] Calculating an image intersection-over-union ratio between the first target and the second target according to the contour data of the first target and the second target;

[0021] Calculating the structural similarity between the first target and the second target;

[0022] The comprehensive similarity between the first target and the second target is calculated according to the image intersection-over-union ratio and the structural similarity.

[0023] In the above implementation, when calculating the comprehensive similarity, by calculating the image intersection and union ratio of the first target and the second target, and calculating the structural similarity of the first target and the second target, the comprehensive similarity can be obtained by combining the image intersection and union ratio and the structural similarity. Combining the two data to obtain the comprehensive similarity can simultaneously reflect the similarity between the first target and the second target in terms of both image and structure, effectively improving the effectiveness and accuracy of the comprehensive similarity.

[0024] Optionally, calculating the structural similarity between the first target and the second target includes:

[0025] Based on the image intersection-over-union ratio, normalizing the images of the first target and the second target to obtain a target image corresponding to the first target and a template target image corresponding to the second target, respectively;

[0026] Calculating a first pixel average value of the target image and a second pixel average value of the template target image;

[0027] Calculating a first pixel standard deviation of the first target based on the first pixel average value;

[0028] Calculating a second pixel standard deviation of the second target based on the second pixel average value;

[0029] Calculating a brightness correlation value between the first target and the second target based on the first pixel average value and the second pixel average value;

[0030] Calculating a contrast correlation value between the first target and the second target based on the first pixel standard deviation and the second pixel standard deviation;

[0031] Calculating a structural correlation value between the first target and the second target based on the first pixel standard deviation and the second pixel standard deviation;

[0032] Based on the brightness correlation value, the contrast correlation value and the structure correlation value, a structural similarity between the first object and the second object is calculated.

[0033] In the above implementation, when calculating the structural similarity, by calculating the first pixel average of the first target and the second pixel average of the second target, the brightness correlation value of the first target and the second target can be calculated based on the first pixel average and the second pixel average. By calculating the first pixel standard deviation of the first target and the second pixel standard deviation of the second target, the contrast correlation value and the structure correlation value of the first target and the second target can be calculated based on the first pixel standard deviation and the second pixel standard deviation. The similarity between the first target and the second target can be compared from multiple aspects of brightness, contrast and structure, thereby improving the diversity and effectiveness of the structural similarity.

[0034] Optionally, updating the data set according to the target detection information includes:

[0035] Determining whether the first target belongs to the template target set according to the target detection information;

[0036] When the similarity between the first target and the second target is within a first numerical interval, and the confidence of the first target is within a second numerical interval, determining that the first target does not belong to the template target set, and adding the first target to the current template target set of the specified frame image as a new target;

[0037] When the similarity between the first target and the second target is within a third numerical interval, and the confidence of the first target is within a fourth numerical interval, determining that the first target belongs to the template target set, and updating the first target to the current template target set of the designated frame image;

[0038] When the similarity between the first target and the second target is within the fifth numerical interval, or the confidence of the first target is within the sixth numerical interval, the first target is not processed;

[0039] When the first target does not belong to the template target set, the target detection information is added to the data set to update the data set.

[0040] In the above implementation, it is determined whether to update the data set based on the target detection information. When the similarity and confidence in the target detection information are in different numerical ranges, it is determined whether the first target belongs to the template target set. If it does, the target detection information is added to the current template target set of the specified frame image to update the information of the second target. If it does not, the target detection information is added to the current template target set of the specified frame image as a new target. When the first target does not belong to the template target set, the target detection information can also be added to the data set to update the video image data set of the video. The relationship between the first target and the template target set can be judged based on the similarity and confidence, so as to decide whether to update the data set, improve the efficiency of data set update, and improve the quality of the data set.

[0041] Optionally, the method further comprises:

[0042] When acquiring the non-designated frame image, detecting the non-designated frame image based on a target detection algorithm to obtain template target detection information;

[0043] The template target set is updated based on the template target detection information.

[0044] In the above implementation, when the acquired frame image to be detected is a non-specified frame image, in order to improve the quality of the data set, it is not necessary to update the data set according to the non-specified frame image. The non-specified frame image is detected by the target detection algorithm to obtain one or more template target detection information contained in the non-specified frame image, and the current template target set is updated based on the template target detection information, so that multiple template targets in the template target set can be updated in a timely manner, thereby improving the real-time and effectiveness of the template target set and improving the accuracy of subsequent comprehensive similarity calculations.

[0045] In a second aspect, the present application further provides a data set updating device, the device comprising:

[0046] An acquisition module is used to read the acquired video data and obtain a specified frame image and a corresponding data set;

[0047] A determination module, configured to detect the specified frame image and determine target detection information contained in the specified frame image;

[0048] An updating module is used to update the data set according to the target detection information.

[0049] In the above implementation, the specified frame image and data set of the video data are acquired through the acquisition module, the specified frame image is detected through the determination module to determine the target detection information contained in the specified frame image, and the data set is updated based on the acquired target detection information through the update module. The data set can be updated in a timely and accurate manner, which improves the efficiency of data set update and the quality of data set. Based on the optimized data set, the generation and iteration of the deep learning model is accelerated, the training and evaluation quality of the deep learning model is improved, and the precision and accuracy of target detection and recognition are improved.

[0050] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores program instructions, and when the processor reads and runs the program instructions, it executes the steps in any implementation of the above-mentioned data set updating method.

[0051] In a fourth aspect, an embodiment of the present application further provides a readable storage medium, wherein the readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps in any implementation of the above-mentioned data set updating method are executed.

[0052] In summary, the present application provides a data set update method, device, electronic device and storage medium, which can timely and accurately update the image data set of the video by identifying and detecting multiple frames of images in the video data, effectively improving the efficiency of data set update and improving the quality of data set. Based on the optimized data set, the generation and iteration of the deep learning model is accelerated, the training and evaluation quality of the deep learning model is improved, and the precision and accuracy of target detection and recognition are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0054] Figure 1 A flowchart of a method for updating a data set provided in an embodiment of the present application;

[0055] Figure 2 A detailed flow chart of step S1 provided in an embodiment of the present application;

[0056] Figure 3 A detailed flow chart of step S2 provided in an embodiment of the present application;

[0057] Figure 4 A detailed flow chart of step S23 provided in an embodiment of the present application;

[0058] Figure 5 A detailed flow chart of step S232 provided in an embodiment of the present application;

[0059] Figure 6 A detailed flow chart of step S3 provided in an embodiment of the present application;

[0060] Figure 7 A schematic diagram of the structure of a data set updating device provided in an embodiment of the present application.

[0061] Icons: 400 - data set updating device; 410 - acquisition module; 420 - determination module; 430 - update module. DETAILED DESCRIPTION

[0062] The technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments of the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the embodiments of the present application.

[0063] An embodiment of the present application provides a data set updating method, which is applied to a server. The server may be a personal computer (PC), a tablet computer, a smart phone, a personal digital assistant (PDA), or other electronic device with logical computing functions. The server can process video data and update the video image data set at the same time to improve the updating efficiency and quality of the data set, thereby improving the training efficiency and evaluation quality of a deep learning model trained based on the data set.

[0064] See also Figure 1 , Figure 1 A flowchart of a method for updating a data set provided in an embodiment of the present application, the method may include the following steps:

[0065] Step S1, reading the acquired video data to obtain a specified frame image and a corresponding data set.

[0066] The sequence of video data to be updated may be read, the total number of frames of the video data is recorded as M, and a data set (Data Set) corresponding to the video data is obtained, that is, a set composed of data.

[0067] Optionally, refer to Figure 2 , Figure 2 A detailed flow chart of step S1 provided in an embodiment of the present application, step S1 may also include steps S11-S12:

[0068] Step S11, obtaining a data set corresponding to the video data.

[0069] The data set is a video image data set, which is used as the data basis for training deep learning models. When deep learning models identify targets, the target recognition accuracy depends on the data and quality of the labeled data set. The quality of the data set will affect the quality of deep learning model training and the accuracy of prediction.

[0070] Step S12, judging whether the frame image to be detected is a designated frame image according to the frame number of the frame image to be detected in the video data, and if so, acquiring the designated frame image.

[0071] Among them, the designated frame image is the nth frame image in the video data, the frame number is n, the modulo result of n times m is 0, m is a preset integer, and the image frames other than the designated frame image in the video data are non-designated frame images.

[0072] Optionally, when classifying designated frames and non-designated frames, designated frame images can be selected by selecting the number of frames, and a calculation method in which the result of n modulo m is 0 is adopted. The value of m can be selected and adjusted by the staff according to actual conditions and engineering experience, so as to realize rapid classification of designated frames and non-designated frames in the video data, and quickly determine whether the current frame to be detected is a designated frame.

[0073] exist Figure 2 In the illustrated embodiment, multiple frames of images can be quickly classified to obtain a designated frame of image in real time for subsequent detection operations.

[0074] After executing step S1, continue to execute step S2.

[0075] Step S2: Detect the designated frame image based on the target detection model to determine the target detection information contained in the designated frame image.

[0076] The target detection information includes detection confidence data and similarity data, as well as target-related detection information such as contour data and category ID obtained by target detection. The target can be a moving vehicle or a moving pedestrian or other multiple targets. The target detection information can also include vehicle license plate information, vehicle number information, vehicle movement trajectory, vehicle time information, and vehicle position information on the image, as well as pedestrian identification information, pedestrian number information, pedestrian movement trajectory, pedestrian time information, and pedestrian position information in the image.

[0077] Optionally, see Figure 3 , Figure 3 A detailed flow chart of step S2 provided in an embodiment of the present application, step S2 may also include steps S21-S23:

[0078] Step S21, identifying the designated frame image based on the target detection model to obtain a target set corresponding to the designated frame image.

[0079] The target set corresponding to any specified frame image includes one or more targets in the specified frame image, and related information such as the confidence level corresponding to each target.

[0080] Optionally, the target detection model may be a YOLO (You Only Look Once, a target detection algorithm) model, and a specified frame image is used as an input parameter of the YOLO model to obtain target detection information.

[0081] Step S22, obtaining the template target set of the n-1th frame image.

[0082] Among them, the video data also has a template target set, denoted as T n-1 The template target set includes one or more template targets, and the template target set corresponding to the previous frame image of the currently detected specified frame image is selected to be compared with the one or more targets in the specified frame image.

[0083] Optionally, when acquiring a non-specified frame image, in order to improve the quality of the data set, it is not necessary to update the data set according to the target in the non-specified frame image. The non-specified frame image can be detected based on the target detection algorithm to obtain one or more template target detection information contained in the non-specified frame image, and the template target set can be updated based on the template target detection information. The template target detection information can also include contour data, category ID and other detection information related to the target obtained by target detection. Multiple template targets in the template target set can be updated in a timely manner, thereby improving the real-time and effectiveness of the template target set and improving the accuracy of subsequent comprehensive similarity calculations.

[0084] Among them, the target detection algorithm can be a target detection algorithm such as the YOLO algorithm. The YOLO algorithm has the advantages of being simple and fast, and can perform convolution on the entire image. It has a larger field of view when detecting targets, is not prone to misjudgment of the background, and improves the accuracy of detection.

[0085] Step S23, calculating the comprehensive similarity between the first target in the target set and the second target in the template target set.

[0086] The first target is any target in the target set, and the second target is any template target in the template target set. By selecting any target in the target set as the first target and any template target in the template target set as the second target, the comprehensive similarity between the first target and the second target can be calculated, thereby improving the accuracy of the comprehensive similarity.

[0087] Optionally, the first target can be compared with all targets in the template target set in turn. After the comprehensive similarity between the first target and all second targets in the template target set is calculated, the second target with the largest comprehensive similarity can be taken as the reference template, and the maximum comprehensive similarity can be used as the comprehensive similarity in the target detection information.

[0088] exist Figure 3 In the illustrated embodiment, by detecting and identifying the changes of the target in the previous and next frame images, the accuracy of target detection and identification can be effectively improved.

[0089] Optionally, see Figure 4 , Figure 4 A detailed flow chart of step S23 provided in an embodiment of the present application, step S23 may also include steps S231-S233:

[0090] Step S231, calculating the image intersection-over-union ratio of the first target and the second target according to the contour data of the first target and the second target.

[0091] The first target is denoted as d i , the second target is t j , d i The rectangle of the contour data is Rd i , t j The rectangle of the contour data is Rt j , based on Rd i and Rt j Calculate d i With t j The image intersection-over-Union ratio (IOU) is used to measure d i With t jThe degree of overlap between two rectangular frames in the image is calculated as follows:

[0092]

[0093] Among them, Ad i is d i The outline rectangle Rd i The area occupied, At j t j The outline rectangle Rt j The area occupied, A dt Rd i and Rt j The area of ​​the intersection of two rectangles.

[0094] Step S232: Calculate the structural similarity between the first target and the second target.

[0095] The structural similarity between the first target image and the second target image can be calculated from multiple aspects such as brightness, contrast and structure, which can effectively improve the diversity and effectiveness of the structural similarity.

[0096] Optionally, see Figure 5 , Figure 5 A detailed flow diagram of step S232 provided in an embodiment of the present application, step S232 may also include steps S2321-S2327:

[0097] Step S2321: based on the image intersection-over-union ratio, normalize the images of the first target and the second target to obtain a target image corresponding to the first target and a template target image corresponding to the second target, respectively.

[0098] Among them, due to the differences in image size, angle, viewing angle, etc., the image sizes of the first target and the second target are also different. The images of the first target and the second target need to be normalized to normalize the sizes of the images occupied by the first target and the second target to the same scale space to obtain target images and template target images of the same size.

[0099] Step S2322, calculating a first pixel average value of the target image and a second pixel average value of the template target image.

[0100] Among them, the pixel average is calculated based on the pixels of the target image and the template target image, and the calculation method is as follows:

[0101]

[0102]

[0103] Among them, xp The first target i The pth pixel value of the target image, μ d is the first pixel average, y p The second target t j The pth pixel value of the template target image, μ t is the second pixel average value, and N is the total value of the pixel values ​​in the target image and the template target image.

[0104] Step S2323, calculating a first pixel standard deviation of the first target based on the first pixel average value; calculating a second pixel standard deviation of the second target based on the second pixel average value.

[0105] Among them, according to the calculated first pixel average μ d and the second pixel average μ t , calculate the first target d i The first pixel standard deviation and the second target t j The second pixel standard deviation is calculated as follows:

[0106]

[0107]

[0108] Among them, σ d is the first pixel standard deviation, σ t is the second pixel standard deviation.

[0109] Step S2324: Calculate the brightness correlation value between the first target and the second target based on the first pixel average value and the second pixel average value.

[0110] The calculation method is as follows:

[0111]

[0112] C1=(K1×H) 2

[0113] Among them, L(d i , t j ) is the brightness related value, C1 is the first constant, which can ensure the stability of the equation when the denominator is zero. C1 can be selected and adjusted by the staff according to the pixel value dynamic range H of the target image and the template target image.

[0114] Optionally, when the target image and the template target image are 8-bit grayscale images, the pixel value H=255, K1 is the proportionality coefficient, and can be 0.01.

[0115] Step S2325: Calculate a contrast correlation value between the first target and the second target based on the first pixel standard deviation and the second pixel standard deviation.

[0116] The calculation method is as follows:

[0117]

[0118] C2=(K2×H) 2

[0119] Among them, P(d i , t j ) is the contrast-related value, C2 is the second constant, which can ensure the stability of the equation when the denominator is zero. C2 can be selected and adjusted by the staff according to the pixel value dynamic range H of the target image and the template target image.

[0120] Optionally, when the target image and the template target image are 8-bit grayscale images, the pixel value H=255, K2 is the proportionality coefficient, and can be 0.03.

[0121] Step S2326: Calculate a structural correlation value between the first target and the second target based on the first pixel standard deviation and the second pixel standard deviation.

[0122] The calculation method is as follows:

[0123]

[0124]

[0125] Among them, σ dt is the pixel standard value of the intersection area between the target image and the template target image, S(d i , t j ) is a structure-related value, and C3 is the third constant, which can ensure the stability of the equation when the denominator is zero.

[0126] Step S2327: Calculate the structural similarity between the first target and the second target based on the brightness correlation value, the contrast correlation value and the structure correlation value.

[0127] Among them, the structural similarity of the target image and the template target image is calculated by combining the relevant values ​​of brightness, contrast and structure. The similarity between the first target and the second target can be compared from the aspects of brightness, contrast and structure, thus improving the diversity and effectiveness of structural similarity. The calculation method is as follows:

[0128] SSIM(d i , t j )=[L(di , t j ) a ][P(d i , t j ) b ][S(d i , t j ) c ]

[0129] Among them, SSIM(d i , t j ) is the structural similarity, a, b, c are weight coefficients, which are used to adjust the weights of the three modules of brightness, contrast and structure respectively. For engineering application optimization calculation, a=b=c=1, C3=0.5C2 can be set, and we can get:

[0130]

[0131] Step S233: Calculate the comprehensive similarity between the first target and the second target according to the image intersection-over-union ratio and the structural similarity.

[0132] Among them, the comprehensive similarity is calculated by combining the image intersection ratio and structural similarity, which can simultaneously reflect the similarity between the first target and the second target in terms of both image and structure, effectively improving the effectiveness and accuracy of the comprehensive similarity. The calculation method is as follows:

[0133] Z=α×I(d i , t j )+β×SSIM(d i , t j )

[0134] Among them, Z is the comprehensive similarity, α and β are relevant weighting coefficients, and the values ​​of α and β can be selected and adjusted by the staff according to the actual situation and engineering experience. The reference values ​​are α=0.4 and β=0.6.

[0135] Optionally, since the first target needs to be consistent with the template target set T n-1 All targets in the template target set are compared in turn. After the comprehensive similarity between the first target and all second targets in the template target set is calculated, the comprehensive similarity value of the second target with the largest comprehensive similarity can be taken as the comprehensive similarity in the target detection information. The calculation method is as follows:

[0136]

[0137] Among them, u is the template target set T n-1 The number of all template targets in Z max As the final comprehensive similarity.

[0138] After executing step S2, continue to execute step S3.

[0139] Step S3: updating the data set according to the target detection information.

[0140] Among them, the relationship between the target in the specified frame image and the target in the template target set is judged according to the target detection information obtained by detection, so as to update the data set, improve the efficiency of data set updating, improve the quality of data set, and accelerate the generation and iteration of deep learning models based on the optimized data set.

[0141] Optionally, see Figure 6 , Figure 6 A detailed flow chart of step S3 provided in an embodiment of the present application, step S3 may also include steps S31-S35:

[0142] Step S31: determining whether the first target belongs to the template target set according to the target detection information.

[0143] Among them, according to the confidence of the first target in the target detection information and the maximum comprehensive similarity between the first target and multiple second targets in the template target set, it can be judged whether the first target belongs to the template target set.

[0144] Optionally, the determined numerical range may be selected and adjusted according to actual conditions and engineering experience.

[0145] Step S32, when the similarity between the first target and the second target is within a first numerical interval, and the confidence of the first target is within a second numerical interval, it is determined that the first target does not belong to the template target set, and the first target is added to the current template target set of the specified frame image as a new target.

[0146] Among them, the similarity between the first target and the second target is within the first numerical range, such as Z max <0.25, and the confidence of the first target is within the second numerical interval, such as when the confidence w≥0.8, the similarity between the first target and the second target is low, and the first target is judged to be a new target and does not belong to the template target set T n-1 , update the first target to the current template target set T of the specified frame image n It is used as a new target for subsequent comparison without updating the dataset.

[0147] Step S33, when the similarity between the first target and the second target is within the third numerical interval, and the confidence of the first target is within the fourth numerical interval, it is determined that the first target belongs to the template target set, and the first target is updated to the current template target set of the specified frame image.

[0148] Among them, the similarity between the first target and the second target is within the third numerical range, such as Z max ≥0.75, and the confidence of the first target is within the fourth numerical interval, such as when the confidence w≥0.8, the similarity between the first target and the second target is high, and the confidence is high, and it is judged that the first target and the second target belong to the same target, and the first target belongs to the template target set T n-1 , there is no need to collect relevant information of the first target, and only needs to update the first target to the current template target set T of the specified frame image n middle.

[0149] Optionally, when the confidence level of the first target is high, the second numerical interval and the fourth numerical interval may be the same numerical interval.

[0150] Step S34: when the similarity between the first target and the second target is within the fifth numerical interval, or the confidence of the first target is within the sixth numerical interval, the first target is not processed.

[0151] Among them, the similarity between the first target and the second target is within the fifth numerical interval, that is, 0.25≤Z max <0.75, or the confidence of the first target is within the sixth numerical interval, that is, w<0.8, it is impossible to compare the first target with the template target set T n-1 The relationship between them is judged, so the first target is not processed.

[0152] Step S35: When the first target does not belong to the template target set, the target detection information is added to the data set to update the data set.

[0153] Among them, when determining that the first target does not belong to the template target set T n-1 When the target detection information of the first target, such as the outline data, the category ID and other related information, is updated to the data set, so as to update the data set in real time.

[0154] It is worth noting that when a detection or update failure occurs, the server can also receive manual intervention instructions sent by the staff and verify the instructions. After successful verification, the target detection is directly performed on the nth frame image currently detected, and the detected target is directly updated to the current template target set T n In the process of detecting and locating objects, the corresponding target detection information is directly updated to the data set. In the event of a failure, the data set can still be updated in real time, improving the real-time and accuracy of the data set.

[0155] Optionally, after the nth frame image of the current designated frame image is detected and updated, the n+1th frame image is read continuously, and the detection and update process is repeated until all frame images in the video data are processed.

[0156] exist Figure 1 In the illustrated embodiment, the video data image dataset can be updated based on the target detection information corresponding to one or more targets in the specified frame image detected, so as to improve the efficiency of dataset updating and improve the quality of the dataset, thereby accelerating the generation and iteration of the deep learning model based on the optimized dataset, improving the training and evaluation quality of the deep learning model, and improving the precision and accuracy of target detection and recognition.

[0157] See also Figure 7 , Figure 7 This is a schematic diagram of the structure of a data set updating device provided in an embodiment of the present application. The data set updating device 400 may include: an acquisition module 410 , a determination module 420 and an update module 430 .

[0158] The acquisition module 410 is used to read the acquired video data to obtain a specified frame image and a corresponding data set;

[0159] A determination module 420 is used to detect the specified frame image and determine the target detection information contained in the specified frame image;

[0160] The updating module 430 is used to update the data set according to the target detection information.

[0161] In an optional implementation, the acquisition module 410 may further include an acquisition submodule and a judgment submodule:

[0162] The acquisition submodule is used to obtain the data set corresponding to the video data;

[0163] A judgment submodule is used to judge whether the frame image to be detected is a designated frame image according to the frame number of the frame image to be detected in the video data, and if so, obtain the designated frame image; wherein, the designated frame image is the nth frame image in the video data, the modulo result of n times m is 0, m is a preset integer, and the image frames in the video data other than the designated frame image are non-designated frame images.

[0164] In an optional implementation, the determination module 420 may further include an identification submodule, a template submodule, and a calculation submodule:

[0165] A recognition submodule, configured to recognize the designated frame image based on a target detection model to obtain a target set corresponding to the designated frame image, wherein the target set corresponding to any designated frame image includes one or more targets in the designated frame image and a confidence level corresponding to each target;

[0166] The template submodule is used to obtain the template target set of the n-1th frame image;

[0167] The calculation submodule is used to calculate the comprehensive similarity between the first target in the target set and the second target in the template target set, wherein the first target is any target in the target set and the second target is any template target in the template target set.

[0168] In an optional implementation, the calculation submodule may further include an image unit and a structure unit:

[0169] An image unit, configured to calculate an image intersection-over-union ratio between the first target and the second target based on the contour data of the first target and the second target;

[0170] A structural unit, used to calculate the structural similarity between the first target and the second target;

[0171] The calculation submodule may also be used to calculate the comprehensive similarity between the first target and the second target according to the image intersection-over-union ratio and the structural similarity.

[0172] In an optional embodiment, the structural unit can also be used to normalize the images of the first target and the second target based on the image intersection-over-union ratio, and obtain a target image corresponding to the first target and a template target image corresponding to the second target, respectively; calculate a first pixel average value of the target image and a second pixel average value of the template target image; calculate a first pixel standard deviation of the first target based on the first pixel average value; calculate a second pixel standard deviation of the second target based on the second pixel average value; calculate a brightness correlation value between the first target and the second target based on the first pixel average value and the second pixel average value; calculate a contrast correlation value between the first target and the second target based on the first pixel standard deviation and the second pixel standard deviation; calculate a structure correlation value between the first target and the second target based on the first pixel standard deviation and the second pixel standard deviation; calculate the structural similarity between the first target and the second target based on the brightness correlation value, the contrast correlation value and the structure correlation value.

[0173] In an optional implementation, the update module 430 may further include a classification submodule and an update submodule:

[0174] a classification submodule, used for judging whether the first target belongs to the template target set according to the target detection information; when the similarity between the first target and the second target is within a first numerical interval, and the confidence of the first target is within a second numerical interval, judging that the first target does not belong to the template target set, and adding the first target to the current template target set of the specified frame image as a new target; when the similarity between the first target and the second target is within a third numerical interval, and the confidence of the first target is within a fourth numerical interval, judging that the first target belongs to the template target set, and updating the first target to the current template target set of the specified frame image; when the similarity between the first target and the second target is within a fifth numerical interval, or the confidence of the first target is within a sixth numerical interval, not processing the first target;

[0175] An updating submodule is used to add the target detection information to the data set to update the data set when the first target does not belong to the template target set.

[0176] In an optional embodiment, the data set updating device 400 may also include a template updating module, which is used to detect the non-specified frame image based on the target detection algorithm when acquiring the non-specified frame image to obtain template target detection information; and update the template target set based on the template target detection information.

[0177] Since the principle of solving the problem by the device in the embodiment of the present application is similar to that of the embodiment of the aforementioned data set updating method, the implementation of the device in this embodiment can refer to the description in the embodiment of the aforementioned method, and the repeated parts will not be repeated.

[0178] exist Figure 7 In the illustrated embodiment, timely and accurate updating of the video image data set is achieved through various modules.

[0179] An embodiment of the present application further provides an electronic device, which includes a memory and a processor, wherein the memory stores program instructions, and when the processor reads and runs the program instructions, the steps of any one of the data set updating methods provided in this embodiment are executed.

[0180] It should be understood that the electronic device may be a personal computer, a tablet computer, a smart phone, a personal digital assistant, or other electronic device with logic computing capabilities.

[0181] The embodiment of the present application further provides a readable storage medium, wherein the readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the steps in the data set updating method are executed.

[0182] In summary, the embodiments of the present application provide a method, device, electronic device, and storage medium for updating a data set, which can timely and accurately update the image data set of a video by identifying and detecting multiple frames of images in the video data, effectively improving the efficiency of updating the data set and improving the quality of the data set. Based on the optimized data set, the generation and iteration of the deep learning model is accelerated, the training and evaluation quality of the deep learning model is improved, and the precision and accuracy of target detection and recognition are improved.

[0183] In several embodiments provided in the present application, it should be understood that the disclosed device can also be implemented in other ways. The device embodiments described above are merely schematic, for example, the block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices according to the multiple embodiments of the present application. In this regard, each box in the block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram, and the combination of the block diagrams, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0184] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0185] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Therefore, this embodiment also provides a computer program instruction stored in a readable storage medium, and when the computer program instruction is read and executed by a processor, the steps in any method described in the block data storage method are executed. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RanDom Access Memory), disk or optical disk and other media that can store program codes.

[0186] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0187] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

[0188] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for updating a data set, characterized in that: include: Read the acquired video data to obtain the specified frame image and the corresponding data set; Detecting the designated frame image to determine target detection information contained in the designated frame image; updating the data set according to the target detection information; The step of reading the acquired video data to acquire the designated frame image and the corresponding data set includes: acquiring the data set corresponding to the video data; judging whether the frame image to be detected is the designated frame image according to the frame number of the frame image to be detected in the video data, and if so, acquiring the designated frame image; wherein the designated frame image is the nth frame image in the video data, the modulo result of n times m is 0, m is a preset integer, and the image frames in the video data other than the designated frame image are non-designated frame images; The target detection information includes confidence and comprehensive similarity; the detecting the designated frame image to determine the target detection information contained in the designated frame image includes: identifying the designated frame image based on a target detection model to obtain a target set corresponding to the designated frame image, wherein the target set corresponding to any designated frame image includes one or more targets in the designated frame image and the confidence corresponding to each target; obtaining a template target set of the n-1th frame image; calculating the comprehensive similarity between a first target in the target set and a second target in the template target set, wherein the first target is any one target in the target set, and the second target is any one template target in the template target set; The calculating of the comprehensive similarity between the first target in the target set and the second target in the template target set includes: calculating the image intersection-over-union ratio of the first target and the second target based on the contour data of the first target and the second target; calculating the structural similarity between the first target and the second target; and calculating the comprehensive similarity between the first target and the second target based on the image intersection-over-union ratio and the structural similarity.

2. The method according to claim 1, characterized in that The calculating the structural similarity between the first target and the second target includes: Based on the image intersection-over-union ratio, normalizing the images of the first target and the second target to obtain a target image corresponding to the first target and a template target image corresponding to the second target, respectively; Calculating a first pixel average value of the target image and a second pixel average value of the template target image; Calculating a first pixel standard deviation of the first target based on the first pixel average value; Calculating a second pixel standard deviation of the second target based on the second pixel average value; Calculating a brightness correlation value between the first target and the second target based on the first pixel average value and the second pixel average value; Calculating a contrast correlation value between the first target and the second target based on the first pixel standard deviation and the second pixel standard deviation; Calculating a structural correlation value between the first target and the second target based on the first pixel standard deviation and the second pixel standard deviation; Based on the brightness correlation value, the contrast correlation value and the structure correlation value, a structural similarity between the first object and the second object is calculated.

3. The method according to claim 1, characterized in that The updating of the data set according to the target detection information includes: Determining whether the first target belongs to the template target set according to the target detection information; When the similarity between the first target and the second target is within a first numerical interval, and the confidence of the first target is within a second numerical interval, determining that the first target does not belong to the template target set, and adding the first target to the current template target set of the specified frame image as a new target; When the similarity between the first target and the second target is within a third numerical interval, and the confidence of the first target is within a fourth numerical interval, determining that the first target belongs to the template target set, and updating the first target to the current template target set of the designated frame image; When the similarity between the first target and the second target is within the fifth numerical interval, or the confidence of the first target is within the sixth numerical interval, the first target is not processed; When the first target does not belong to the template target set, the target detection information is added to the data set to update the data set.

4. The method according to claim 1, characterized in that: The method further comprises: When acquiring the non-designated frame image, detecting the non-designated frame image based on a target detection algorithm to obtain template target detection information; The template target set is updated based on the template target detection information.

5. A data set updating device, characterized in that: The device comprises: An acquisition module is used to read the acquired video data and obtain a specified frame image and a corresponding data set; A determination module, configured to detect the specified frame image and determine target detection information contained in the specified frame image; An updating module, used for updating the data set according to the target detection information; The acquisition module includes an acquisition submodule and a judgment submodule; the acquisition submodule is used to acquire a data set corresponding to the video data; the judgment submodule is used to determine whether the frame image to be detected is a designated frame image according to the frame number of the frame image to be detected in the video data, and if so, acquire the designated frame image; wherein the designated frame image is the nth frame image in the video data, the modulo result of n times m is 0, m is a preset integer, and the image frames in the video data other than the designated frame image are non-designated frame images; The determination module includes a recognition submodule, a template submodule and a calculation submodule; the recognition submodule is used to recognize the specified frame image based on the target detection model to obtain a target set corresponding to the specified frame image, wherein the target set corresponding to any specified frame image includes one or more targets in the specified frame image and the confidence corresponding to each target; the template submodule is used to obtain the template target set of the n-1th frame image; the calculation submodule is used to calculate the comprehensive similarity between the first target in the target set and the second target in the template target set, wherein the first target is any one target in the target set, and the second target is any one template target in the template target set; The calculation submodule includes an image unit and a structure unit; the image unit is used to calculate the image intersection-over-union ratio of the first target and the second target based on the contour data of the first target and the second target; the structure unit is used to calculate the structural similarity between the first target and the second target; based on the image intersection-over-union ratio and the structural similarity; the calculation submodule is also used to calculate the comprehensive similarity between the first target and the second target.

6. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory stores program instructions, and when the processor runs the program instructions, the steps in the method according to any one of claims 1 to 4 are executed.

7. A readable storage medium, characterized in that: The readable storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the steps in the method according to any one of claims 1 to 4 are executed.

Images