Method for determining whether the objects detected in the two frames of video are the same
By calculating the area ratio, coincidence and color difference of objects in the two frames before and after the video, the problem of repeated recording of objects recognition in the video is solved, which improves statistical accuracy and reduces the computational burden.
Patent Information
- Application Number
- CN202111002431.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-30
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2041-08-30
AI Technical Summary
The prior art is difficult to accurately determine whether the same object is detected in the two frames of the video before and after the video, resulting in the problem of repeated recording in the target number statistics.
By recording the coordinates and RGB values of the two frame images, the ratio of area, coincidence and color difference of the object are calculated, and whether it is the same object is determined based on the set threshold.
It improves the accuracy of target quantity statistics, reduces the amount of calculation, has low hardware requirements, and can effectively avoid duplicate recording.
Smart Images

Figure CN113724316B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of video detection, and in particular to a method for judging whether objects detected in two frames of video are the same. Background Art
[0002] As deep learning-based video detection technology continues to mature, the technology of integrating cameras with deep learning models and recording and storing relevant data of detected target objects has gradually matured. Therefore, by embedding target detection models in cameras, long-term unmanned monitoring of regional target changes can be achieved.
[0003] In order to obtain the number of targets in the detected target area over a period of time, the data related to the detected target objects needs to be recorded. When the system needs to count the number of targets over a period of time, the same objects in different frames need to be eliminated, so it is necessary to determine whether the objects detected in the previous and next frames are the same object. Summary of the invention
[0004] The technical problem to be solved by the present invention is to provide a method for determining whether the objects detected in the two frames of video are the same, which can more accurately record the relevant data of the detected objects and avoid repeated recording.
[0005] The technical solution adopted by the present invention to solve the technical problem is: to provide a method for determining whether the objects detected in the two frames of video are the same, including the following steps:
[0006] (1) recording a first coordinate group and a first RGB value of a target object detected by the target detection model for the current frame image, where the first RGB value is an average value of the RGB values of the target object detected by the current frame image;
[0007] (2) recording a second coordinate group and a second RGB value of the target object detected by the target detection model for the next frame of image, where the second RGB value is an average value of the RGB values of the target object detected for the next frame of image;
[0008] (3) calculating the area ratio, overlap ratio and color difference of the target object detected by the two frames of images according to the first coordinate group, the first RGB value, the second coordinate group and the second RGB value;
[0009] (4) Compare the area ratio, overlap ratio and color difference with a set threshold value, and determine whether they are the same object based on the comparison result.
[0010] In step (3), Calculate the ratio of the area of the target object detected by the two frames of images, where q面 is the ratio of the area of the target object detected by the previous and next two frames of images, S0 represents the area of the target object detected by the current frame of image, S0 = |(x 02 -x 01 )*(y 02 -y 01 )|,(x 01 ,y 01 ) and (x 02 ,y 02 ) are the upper left coordinate and the lower right coordinate in the first coordinate group, S1 represents the area of the target object detected in the next frame image, S1=|(x 12 -x 11 )*(y 12 -y 11 )|,(x 11 ,y 11 ) and (x 12 ,y 12 ) are the upper left coordinate and the lower right coordinate in the second coordinate group.
[0011] In step (3), Calculate the ratio of the overlap of the target objects detected by the two frames of images, where q 重 is the ratio of the overlap of the target object detected by the previous and next two frames of images, S0 represents the area of the target object detected by the current frame of image, S0 = |(x 02 -x 01 )*(y 02 -y 01 )|,(x 01 ,y 01 ) and (x 02 ,y 02 ) are the upper left coordinate and the lower right coordinate in the first coordinate group, S 重 S represents the area of the overlapping part of the target object detected by the two frames of images. 重 =(min(x 02 ,x 12 )-max(x 01 -x 11 ))*(min(y 02 ,y 12 )-max(y 01 -y 11 )),(x 11 ,y 11 ) and (x 12 ,y 12 ) are the upper left coordinate and the lower right coordinate in the second coordinate group.
[0012] Before calculating the ratio of the overlap of the target objects detected by the two frames of images, It is determined whether the target objects detected by the two frames of images overlap, where: If true, it is 1; otherwise, it is 0. When it is true, it is 1, otherwise it is 0. && represents logical AND. When result = 1, it means there is an overlap, and result = 0 means there is no overlap. sx 前 =|x 01 -x 02 |,sx 后 =|x 11 -x 12 |,sy 前 =|y 01 -y 02 |,sy 后 =|y 11 -y 12 |.
[0013] In step (3), Calculate the color difference of the target object detected by the two frames of images, where (R0, G0, B0) represents the first RGB value and (R1, G1, B1) represents the second RGB value.
[0014] The first RGB value and the second RGB value are obtained as follows: according to the coordinates of the target object, the position of the center point of the target object is determined, and with the center point of the target object as the midpoint, a rectangular area with a length and width of 20% of the length and width of the target object is taken, and the average value of the RGB values within the rectangular area is calculated.
[0015] The step (4) is specifically as follows: when the area ratio exceeds a first threshold, the overlap ratio exceeds a second threshold, and the color difference is lower than a third threshold, it is determined that they are the same object.
[0016] The first threshold is 90%, the second threshold is 50%, and the third threshold is 15%.
[0017] Beneficial Effects
[0018] Due to the adoption of the above technical solution, the present invention has the following advantages and positive effects compared with the prior art: the present invention determines whether the objects detected in the previous and next frames are the same object by judging the ratio of the areas, the degree of overlap and the degree of color difference, and the accuracy of the target quantity statistics can be improved by this method. The present invention also has the advantages of small calculation amount and low hardware requirements. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 is a flow chart of an embodiment of the present invention;
[0020] Figure 2 is a schematic diagram of detecting a target object in a current frame in an embodiment of the present invention;
[0021] Figure 3 It is a schematic diagram showing objects detected in two frames before and after in the same picture according to an embodiment of the present invention. DETAILED DESCRIPTION
[0022] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.
[0023] The embodiment of the present invention relates to a method for determining whether the objects detected in the two frames of video are the same, comprising the following steps: first, all objects in the frame images are detected by a target detection model, and the upper left and lower right coordinates of the detected objects and the RGB color values of the target objects are recorded. The upper left and lower right coordinates and RGB values of the objects in the two frames are processed to determine whether they are the same object, wherein the judgment is based on: if the area ratio, overlap ratio and color difference of the objects in the two frames meet the conditions, it indicates the same object. Figure 1 As shown, the specific steps are as follows:
[0024] (1) Recording the first coordinate group and the first RGB value of the target object detected by the target detection model for the current frame image, wherein the first RGB value is the average value of the RGB values of the target object detected by the current frame image, wherein the first coordinate group includes the upper left coordinate (x 01 ,y 01 ) and the lower right coordinate (x 02 ,y 02 ). Figure 2 Schematic diagram of the target object detected in the current frame, where box A is the detection box of the target object.
[0025] (2) Recording a second coordinate group and a second RGB value of the target object detected by the target detection model for the next frame of image, wherein the second RGB value is the average value of the RGB values of the target object detected by the next frame of image, wherein the second coordinate group includes the upper left coordinate (x 11 ,y 11 ) and the lower right coordinate (x 12 ,y 12 ). Figure 3This is a schematic diagram showing the objects detected in the previous and next frames in the same image, where box A is the detection box of the target object in the current frame image, and box B is the detection box of the target object in the next frame image.
[0026] In this embodiment, the method for obtaining the first RGB value and the second RGB value is as follows: first, the position of the center point of the target object is determined according to the upper left and lower right coordinates of the target object, and then the center point of the target object is taken as the midpoint, and a rectangular area with a length and width of 20% of the length and width of the target object is taken, and the average value of the RGB values in the rectangular area is calculated, which are recorded as (R0, G0, B0) and (R1, G1, B1) respectively.
[0027] (3) Calculating the area ratio, overlap ratio and color difference of the target object detected by the two frames of images according to the first coordinate group, the first RGB value, the second coordinate group and the second RGB value.
[0028] When calculating the area ratio, let S0 represent the area of the target object detected by the current frame image, and S1 represent the area of the target object detected by the next frame image, then:
[0029] S0=|(x 02 -x 01 )*(y 02 -y 01 )|
[0030] S1=|(x 12 -x 11 )*(y 12 -y 11 )|
[0031] In order to make the standard uniform, the ratio of the control area is always less than or equal to 1, so the ratio of the area is obtained as follows:
[0032]
[0033] Before calculating the overlap ratio, it is usually necessary to determine whether there is overlap. This implementation adopts the following method to make the determination:
[0034] Let lx represent the absolute value of the difference between the midpoint values of the X axis of the two frames before and after, and ly represent the absolute value of the difference between the midpoint values of the Y axis of the two frames before and after, then:
[0035]
[0036]
[0037] At the same time, let sx 前 、sx 后Respectively represent the absolute value of the X-axis length of the object detected in the previous and next frames, sy 前 ,sy 后 Respectively represent the absolute value of the Y-axis length of the object detected in the previous and next frames, that is:
[0038] sx 前 =|x 01 -x 02 |
[0039] sx 后 =|x 11 -x 12 |
[0040] sy 前 =|y 01 -y 02 |
[0041] sy 后 =|y 11 -y 12 |
[0042] Therefore, the formula for determining whether two objects overlap is:
[0043]
[0044] in, If true, it is 1; otherwise, it is 0. It is 1 when it is true, otherwise it is 0. && represents logical AND. When result = 1, it means there is an overlap, and result = 0 means there is no overlap.
[0045] In actual target detection applications, when the speed is not high, the time interval between the two frames before and after the video is not large, and the target density is not large, if they are the same object, there must be overlapping parts. The area calculation formula of the overlapping parts of the two objects is:
[0046] S 重 =(min(x 02 ,x 12 )-max(x 01 -x 11 ))*(min(y 02 ,y 12 )-max(y 01 -y 11 ))
[0047] Therefore, it can be concluded that the calculation formula for the overlap ratio is:
[0048]
[0049] When calculating the color difference, the color difference between two objects in the previous and next frames is defined as:
[0050]
[0051] (4) Compare the area ratio, overlap ratio and color difference with the set thresholds, and determine whether they are the same object based on the comparison results. Specifically, when the area ratio exceeds the first threshold, the overlap ratio exceeds the second threshold and the color difference is lower than the third threshold, it is determined to be the same object. In order to determine the specific values of the above three thresholds, this embodiment uses a total of 20 videos of surface garbage in flowing waters shot in Shanghai Fuxing Island Canal, Shanghai Nanhui New Town Beach and Lianyungang Yanwei Port as test data. After testing 20 videos, this embodiment finally concluded that when the set thresholds of the area ratio, overlap ratio and color difference are greater than 90%, greater than 50% and less than 15% respectively, the average accuracy of judgment in the 20 videos is as high as 95%.
[0052] (5) The floating object quantity data in the statistical data shall be revised based on whether it is finally determined to be the same object.
[0053] It is not difficult to find that the present invention determines whether the object is the same object by judging the area ratio, overlap and color difference of the object detected in the previous and next two frames, and the accuracy of target quantity statistics can be improved by this method. The present invention also has the advantages of small calculation amount and low hardware requirements.
Claims
1. A method for determining whether the objects detected in two frames of video are the same, characterized in that: The following steps are involved: (1) recording a first coordinate group and a first RGB value of a target object detected by the target detection model for the current frame image, where the first RGB value is an average value of the RGB values of the target object detected by the current frame image; (2) recording a second coordinate group and a second RGB value of the target object detected by the target detection model for the next frame of image, where the second RGB value is an average value of the RGB values of the target object detected for the next frame of image; (3) calculating the area ratio, overlap ratio and color difference of the target object detected by the two frames of images according to the first coordinate group, the first RGB value, the second coordinate group and the second RGB value; wherein, Calculate the ratio of the area of the target object detected by the two frames of images, where q 面 is the ratio of the area of the target object detected by the previous and next two frames of images, S0 represents the area of the target object detected by the current frame of image, S0 = |(x 02 -x 01 )*(y 02 -y 01 )|,(x 01 ,y 01 ) and (x 02 ,y 02 ) are the upper left coordinate and the lower right coordinate in the first coordinate group, S1 represents the area of the target object detected in the next frame image, S1=|(x 12 -x 11 )*(y 12 -y 11 )|,(x 11 ,y 11 ) and (x 12 ,y 12 ) are the upper left coordinate and the lower right coordinate in the second coordinate group; pass Calculate the ratio of the overlap of the target objects detected by the two frames of images, where q 重 is the ratio of the overlap of the target object detected by the two frames of images, S 重 S represents the area of the overlapping part of the target object detected by the two frames of images. 重 =(min(x 02 ,x 12 )-max(x 01 -x 11 ))*(min(y 02 ,y 12 )-max(y 01 -y 11 )); before calculating the ratio of the overlap of the target object detected by the two frames of images, It is determined whether the target objects detected by the two frames of images overlap, where: If true, it is 1, otherwise, it is 0. When it is true, it is 1, otherwise it is 0. && represents logical AND. When result = 1, it means there is an overlap, and result = 0 means there is no overlap. sx 前 =|x 01 -x 02 |, sx 后 =|x 11 -x 12 |, and 前 =|y 01 -y 02 |, and 后 =|y 11 -y 12 |; pass Calculate the color difference of the target object detected by the two frames of images, where (R0, G0, B0) represents the first RGB value and (R1, G1, B1) represents the second RGB value; the first RGB value and the second RGB value are obtained as follows: according to the coordinates of the target object, determine the position of the center point of the target object, and take the center point of the target object as the midpoint, take a rectangular area with a length and width of 20% of the length and width of the target object respectively, and calculate the average value of the RGB values in the rectangular area; (4) Compare the area ratio, overlap ratio and color difference with a set threshold value, and determine whether they are the same object based on the comparison result.
2. The method for determining whether the objects detected in the two frames of video are the same as each other according to claim 1, characterized in that: The step (4) is specifically as follows: when the area ratio exceeds a first threshold, the overlap ratio exceeds a second threshold, and the color difference is lower than a third threshold, it is determined that they are the same object.
3. The method for determining whether the objects detected in the two frames of video are the same as each other according to claim 2, characterized in that: The first threshold is 90%, the second threshold is 50%, and the third threshold is 15%.
