Formation detection method and device for sports formations in large-scale activities

Abstract

The present invention provides a formation detection method and device for a moving array of a large-scale activity, the formation detection method for a moving array of a large-scale activity comprising: obtaining a target video of a target moving array in a target environment; generating a jitter factor corresponding to the target video frame based on a target video frame in the target video and a diagonal grayscale projection standard array; when the jitter factor is lower than a target jitter threshold, generating a target image of the target moving array based on the target video frame. The formation detection method for a moving array of a large-scale activity of the present invention generates a jitter factor based on a target video frame in the target video and a diagonal grayscale projection standard array, and screens the target video frames based on the jitter factor to retain target video frames with lower jitter amplitudes, and processes the target video frames with lower jitter amplitudes to generate a target image of the target moving array, which effectively improves the clarity of imaging and helps to improve the detection results.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a formation detection method and device for a moving array of a large-scale activity. Background Art

[0002] Real-time detection of motion formations based on video images in large-scale activities mainly involves capturing video images with a panoramic camera installed at a higher position in an outdoor venue, and performing real-time detection of motion formations with a larger area. However, in the related art, since the shooting equipment (especially outdoor) is easily affected by external factors such as strong winds or shaking of the support frame, it will cause jitters in the video image and ghosting, causing the position of objects in the image to shift and the image to be distorted, affecting the clarity of the image and thus affecting the detection results. Summary of the invention

[0003] The present invention provides a formation detection method and device for a moving array in a large-scale activity, which are used to solve the defect of poor detection effect of the moving array in the prior art and improve the detection effect.

[0004] The present invention provides a formation detection method for a sports formation in a large-scale activity, comprising:

[0005] Acquire target video of target motion array in target environment;

[0006] Based on a target video frame in the target video and a diagonal grayscale projection standard array, generating a jitter factor corresponding to the target video frame;

[0007] When the jitter factor is lower than a target jitter threshold, a target image of the target motion matrix is ​​generated based on the target video frame.

[0008] According to a formation detection method for a moving array of a large-scale activity provided by the present invention, the jitter factor corresponding to the target video frame is generated based on the target video frame in the target video and the diagonal grayscale projection standard array, including:

[0009] Based on a target area of ​​the target video frame, generating a first diagonal grayscale projection array of the target area, wherein the target area does not include features of the motion matrix;

[0010] The dither factor is generated based on the first diagonal grayscale projection array and the diagonal grayscale projection standard array.

[0011] According to a formation detection method for a moving array of a large-scale activity provided by the present invention, generating the jitter factor based on the first diagonal grayscale projection array and the diagonal grayscale projection standard array includes:

[0012] Application formula:

[0013]

[0014] Generate the jitter factor, where θ is the jitter factor, the Gr_bi(m) is the diagonal grayscale projection standard value of the m-th position in the diagonal grayscale projection standard array, the Gr_ci(m) is the first diagonal grayscale projection value of the m-th position in the first diagonal grayscale projection array, and d is the side length of the target area.

[0015] According to a formation detection method for a moving square array of a large-scale activity provided by the present invention, generating a target image of the target moving square array based on the target video frame includes:

[0016] Performing a frame difference operation on the target video frame and the target background model to generate a first image;

[0017] Performing binarization and noise reduction processing on the first image to generate a third image;

[0018] A foreground image in the target video frame is extracted based on the third image, and a target image of the target motion matrix is ​​generated based on the foreground image.

[0019] According to a formation detection method for a moving array of a large-scale activity provided by the present invention, the step of generating a target image of the target moving array based on the foreground image comprises:

[0020] Converting the foreground image into a target format to generate a fourth image;

[0021] The fourth image is subjected to shadow region segmentation processing and erosion and dilation processing based on a target shadow threshold to generate a target image of the target motion matrix.

[0022] According to a formation detection method for a moving array of a large-scale activity provided by the present invention, before acquiring a target video of a target moving array in a target environment, the method further includes:

[0023] Acquire multiple frames of environment background images within a first target time period in the target environment;

[0024] Based on the multiple frames of environmental background images, generating a target background model;

[0025] Based on the target background model, a diagonal grayscale projection standard array corresponding to the target background model is generated.

[0026] The present invention also provides a formation detection device for a sports formation in a large-scale activity, comprising:

[0027] A first acquisition module is used to acquire a target video of a target motion matrix in a target environment;

[0028] A first generating module, configured to generate a jitter factor corresponding to the target video frame based on the target video frame in the target video and a diagonal grayscale projection standard array;

[0029] The second generating module is used to generate a target image of the target motion matrix based on the target video frame when the jitter factor is lower than a target jitter threshold.

[0030] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for detecting the formation of a moving array of a large-scale activity as described in any one of the above are implemented.

[0031] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a formation detection method for a moving array of a large-scale activity as described in any one of the above.

[0032] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements the formation detection method of a moving array in a large-scale activity as described in any of the above.

[0033] The formation detection method and device of a moving array of a large-scale activity provided by the present invention generates a jitter factor based on a target video frame in a target video and a diagonal grayscale projection standard array, and screens the target video frame based on the jitter factor to retain the target video frame with a lower jitter amplitude, and processes the target video frame with a lower jitter amplitude to generate a target image of the target moving array, thereby overcoming the defect that other grayscale projection methods are easily affected by local motion, realizing effective jitter detection of target video frames in motion, significantly improving the rapidity and real-time performance of video frame jitter detection, and improving the imaging effect of the target image, thereby helping to improve the accuracy and precision of the detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 This is one of the flow charts of the method for detecting the formation of a sports formation in a large-scale activity provided by the present invention;

[0036] Figure 2 This is the second flow chart of the method for detecting the formation of a sports formation in a large-scale activity provided by the present invention;

[0037] Figure 3 This is one of the principle schematic diagrams of the method for detecting the formation of a sports array in a large-scale activity provided by the present invention;

[0038] Figure 4 This is the second principle schematic diagram of the method for detecting the formation of a sports array in a large-scale activity provided by the present invention;

[0039] Figure 5 This is the third principle schematic diagram of the method for detecting the formation of a sports array in a large-scale activity provided by the present invention;

[0040] Figure 6 This is the fourth principle schematic diagram of the method for detecting the formation of a sports array in a large-scale activity provided by the present invention;

[0041] Figure 7 This is the fifth principle schematic diagram of the method for detecting the formation of a sports array in a large-scale activity provided by the present invention;

[0042] Figure 8 This is the sixth principle schematic diagram of the method for detecting the formation of a sports array in a large-scale activity provided by the present invention;

[0043] Fig. 9 It is a structural schematic diagram of a formation detection device for a sports formation of a large-scale activity provided by the present invention;

[0044] Fig.10 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0046] Combine the following Figures 1 to 6 The present invention describes a formation detection method for a moving array of a large-scale activity.

[0047] The execution subject of the method for detecting the formation of the sports array of the large-scale activity may be a server, or a user's terminal, such as a mobile phone or a computer.

[0048] like Figure 1As shown, the formation detection method of the moving array of the large-scale activity includes: step 110, step 120 and step 130.

[0049] Step 110, obtaining a target video of a target motion array in a target environment;

[0050] In this step, the target environment is the scene where the target motion matrix is ​​located.

[0051] The target motion matrix is ​​a matrix that needs to be detected among multiple motion matrixes.

[0052] The target video is a real-time video stream including a target motion matrix. The target video may include multiple frames of images including the target motion matrix.

[0053] For example, in a sports meeting, the target environment is the environment of the sports field, and the sports formation is the sports formation composed of athletes.

[0054] It is understandable that the motion square can be in motion.

[0055] The target video may be collected by an image sensor, wherein the image sensor may be a panoramic camera or other image sensors, which is not limited in this application.

[0056] In the actual implementation process, the image sensor needs to be installed at a higher position to ensure that the image of the entire target moving array can be captured.

[0057] In some embodiments, after step 110, the method may further include: performing distortion correction on each video frame of the target video.

[0058] In this embodiment, the distortion correction includes radial distortion and tangential distortion.

[0059] The formula can be:

[0060] Where (x 2 +y 2 =r 2 );

[0061] To correct radial distortion,

[0062] Among them, (x', y') represents the distorted pixel coordinates of each frame in the target video, (x, y) represents the pixel coordinates after the distortion is corrected, and k1, k2 and k3 are the distortion parameters obtained by calibrating the image sensor.

[0063] The formula can be:

[0064] Where (x 2 +y2 =r 2 );

[0065] To correct tangential distortion,

[0066] Among them, (x', y') represents the distorted pixel coordinates of each frame in the target video, (x, y) represents the pixel coordinates after the distortion is corrected, and p1 and p3 are the distortion parameters obtained by calibrating the image sensor.

[0067] It should be noted that the distortion parameters can be obtained by calibrating the image sensor.

[0068] In the actual implementation process, before step 110 , the image sensor may be calibrated first to obtain the distortion parameters and the intrinsic parameter matrix of the image sensor.

[0069] In this embodiment, by performing distortion correction on the target video, picture distortion caused during image acquisition using a panoramic camera can be avoided, which helps to improve the imaging effect and the authenticity of the imaging.

[0070] After acquiring the target video, the target video can be stored in a local server or a cloud server and can be retrieved when needed.

[0071] Step 120: Generate a jitter factor corresponding to the target video frame based on the target video frame in the target video and the diagonal grayscale projection standard array;

[0072] In this step, the target video frame is the video frame that needs to be detected.

[0073] The target video frame may be any image frame in the target video.

[0074] The diagonal grayscale projection standard array is the diagonal grayscale projection value of the image when no dithering occurs, that is, the initial value of the diagonal grayscale projection value.

[0075] The diagonal grayscale projection standard array is a value determined in advance before detection. The present invention will describe in detail the steps of determining the diagonal grayscale projection standard array in subsequent embodiments, which will not be described in detail here.

[0076] The jitter factor is used to characterize the jitter degree of the target video.

[0077] It is understandable that, affected by external environmental factors, different video frames in the same target video may have different jitter degrees.

[0078] In this embodiment, based on the target video frame in the target video and the diagonal grayscale projection standard array, the jitter factor corresponding to the target video frame can be determined.

[0079] The specific implementation method of this step is described below.

[0080] In some embodiments, step 120 may include:

[0081] Based on the target area of ​​the target video frame, a first diagonal grayscale projection array of the target area is generated, wherein the target area does not include features of the motion matrix;

[0082] A dither factor is generated based on the first diagonal grayscale projection array and the diagonal grayscale projection standard array.

[0083] In this embodiment, the target area is any area in the target video frame except the motion matrix area, and the number of the target area may be one or more.

[0084] The target area can be customized based on the user.

[0085] For example, the target area can be set to square areas at the four corners of the target video frame, with the side length of each square area being d; or the target area can be set to square areas at any two corners of the target video frame; or the target area can be set to a square area at any one corner of the target video frame.

[0086] By selecting the square areas at the four corners of the target video frame, the influence on the jitter detection caused by the movement of the large moving square matrix in the middle of the video frame can be avoided.

[0087] The first diagonal grayscale projection array is the actual diagonal grayscale projection value corresponding to the target video frame in the target video.

[0088] The first diagonal grayscale projection array can be represented by Gr, wherein the number of the first diagonal grayscale projection arrays is consistent with the number of target areas.

[0089] For example, when the square areas at the four corners of the target video frame are determined as the target areas, the first diagonal grayscale projection arrays of the four target areas of the target video frame are calculated, and are represented by Gr_c1, Gr_c2, Gr_c3 and Gr_c4 respectively. Among them, Gr_c1 is used to represent the diagonal grayscale projection array of the upper left corner area of ​​the target video, Gr_c2 is used to represent the diagonal grayscale projection array of the upper right corner area of ​​the target video, Gr_c3 is used to represent the diagonal grayscale projection array of the lower right corner area of ​​the target video, and Gr_c4 is used to represent the diagonal grayscale projection array of the lower left corner area of ​​the target video.

[0090] Then, the weighted difference square factors between the first diagonal grayscale projection array and the diagonal grayscale projection standard array of the four target areas of the current target video frame are calculated respectively.

[0091] In some embodiments, the maximum value among the weighted difference square factors of the four target areas may be selected as the jitter factor θ to reflect the most realistic shaking situation.

[0092] That is,

[0093] θ max =max(θ1,θ2,θ3,θ4);

[0094] Among them, θ max is the jitter factor, θ1, θ2, θ3, θ4 are the weighted difference square factors corresponding to the four target areas respectively.

[0095] It can be understood that, when there are multiple groups of diagonal grayscale projection standard arrays, the first diagonal grayscale projection array and each diagonal grayscale projection standard array are calculated respectively, and the maximum value of the multiple dither factors obtained is used as the final dither factor.

[0096] For example, when there are 1000 sets of diagonal grayscale projection standard arrays, the first diagonal grayscale projection array and the 1000 sets of diagonal grayscale projection standard arrays are calculated respectively, and the maximum value is selected as the dithering factor.

[0097] In some embodiments, generating a dither factor based on the first diagonal grayscale projection array and the diagonal grayscale projection standard array includes:

[0098] Application formula:

[0099]

[0100] Generate a jitter factor, where θ is the jitter factor, Gr_bi(m) is the diagonal grayscale projection standard value of the m-th position in the diagonal grayscale projection standard array corresponding to the target area i, Gr_ci(m) is the first diagonal grayscale projection value of the m-th position in the first diagonal grayscale projection array corresponding to the target area i, d is the side length of the target area, and "?m≤d-1:2d-m-1" is used to determine the value of m at the denominator position in the formula, specifically: when m≤d-1, m=d-1 in the denominator "m+1"; otherwise, m=2d-m-1 in the denominator "m+1".

[0101] After the jitter factor is obtained, step 130 may be executed.

[0102] During the research and development process, the inventors discovered that in the related technologies, there are methods based on feature point matching and methods based on optical flow to detect jitter, but both rely on the detected feature points, which usually involve a very large amount of calculation. The quality of feature point detection will also affect the accuracy of the optical flow algorithm, and the movement of objects in the picture will also cause the optical flow algorithm to make incorrect estimates, resulting in poor image effects in the final display.

[0103] In the present application, by extracting the target area of ​​the target video frame (such as the four corners of the target video frame) for jitter detection, the influence of local motion (such as the influence of the central motion matrix of the target video frame) can be effectively avoided, which is suitable for dynamic shooting environments.

[0104] In addition, selecting the value with the largest jitter degree as the jitter degree of the entire target video frame can effectively avoid the interference of the moving target in the target video frame on the jitter detection.

[0105] Step 130: When the jitter factor is lower than the target jitter threshold, a target image of the target motion matrix is ​​generated based on the target video frame.

[0106] In this step, the target jitter threshold is the maximum jitter value of the image when the image is approximately considered to be in a static state.

[0107] When the jitter factor is lower than the target jitter threshold, the target video frame corresponding to the jitter factor can be approximately considered to be still.

[0108] The target image is an image obtained by processing the original image, including the moving pedestrians in the target moving square and the shadows of the pedestrians.

[0109] The target image is used to characterize the real-time status of the target motion matrix.

[0110] In this embodiment, when the jitter factor of the target video frame is lower than the target jitter threshold, the target video frame is approximately considered to be non-jittered, and the target video frame is processed in the next step.

[0111] In other embodiments, when the jitter factor of the target video frame is not lower than the target jitter threshold, it is considered that the jitter degree of the target video frame is large, and the target video frame corresponding to the jitter factor is skipped to detect the next video frame.

[0112] Next, the method of generating the target image in this step is described in detail.

[0113] In some embodiments, step 130 may further include:

[0114] Performing frame difference operation on the target video frame and the target background model to generate a first image;

[0115] Performing binarization and noise reduction processing on the first image to generate a third image;

[0116] A foreground image in the target video frame is extracted based on the third image, and a target image of the target motion matrix is ​​generated based on the foreground image.

[0117] In this embodiment, the foreground image includes the moving pedestrians in the target moving square and the projections of the moving pedestrians in the target moving square on the ground.

[0118] In the actual implementation process, for the target video frame without jitter, the coarse-grained detection of the foreground area can be performed first.

[0119] For example, Figure 4 As shown, a frame difference operation is performed between the target video frame and the target background model to generate a first image.

[0120] The target background model is a model established in advance, and the steps for establishing the target background model will be described in subsequent embodiments, which will not be described in detail here.

[0121] Then, the result after frame difference is first binarized and then subjected to noise reduction processing to eliminate the influence of noise in the first image and the interference of small moving objects to generate a third image, such as Figure 5 As shown, the third image is used to characterize the contour features of the motion matrix in the target video frame;

[0122] The noise reduction process may include basic morphological operations of corrosion and dilation.

[0123] Then, a foreground image in the target video frame that is within the contour corresponding to the third image is extracted based on the third image, and a target image is generated based on the foreground area, such as Figure 8 shown.

[0124] At this time, the foreground area will include the pedestrians in the square and their shadows, such as Figure 6 shown.

[0125] The formation detection method of a moving array in a large-scale activity provided by an embodiment of the present invention realizes improvement in details by combining a background modeling method with a frame difference method for moving target detection.

[0126] In some embodiments, generating a target image of the target motion matrix based on the foreground image may further include:

[0127] converting the foreground image into a target format to generate a fourth image;

[0128] The fourth image is subjected to shadow region segmentation processing and erosion and dilation processing based on the target shadow threshold value to generate a target image of the target motion matrix.

[0129] In this embodiment, the foreground image after the foreground area coarse-grained detection can also be subjected to color space conversion and square area fine detection to eliminate the shadow of the foreground area and make the foreground extraction more accurate.

[0130] The target shadow threshold can be customized by the user.

[0131] For example, the third image is converted from RGB color space to HSV color space to generate a fourth image, wherein the HSV color space includes color (H), saturation (S) and brightness (V).

[0132] By converting to the HSV color space, the square area and the shadow area in the foreground area of ​​the third image can be effectively distinguished based on the brightness value. Figure 7 As shown, the difference between the two in brightness value (V space) is quite obvious.

[0133] By utilizing the distribution characteristics of the third image in the HSV color space, after converting the third image from the RBG color space to the HSV color space to generate the fourth image, a threshold is set in the brightness V space for the image in the foreground area of ​​the fourth image to segment the shadow area, and then an erosion and dilation operation is performed after segmentation, so as to obtain the target image after eliminating the shadow interference, as shown in FIG. Figure 8 shown.

[0134] In this embodiment, the distortion correction and corrosion dilation methods are combined to overcome the influence of jitter, lighting, shadow and other interferences under outdoor natural conditions, and finally achieve real-time detection of moving array formations based on video frames.

[0135] In some embodiments, the method may further include: updating the target shadow threshold every third target time period.

[0136] For example, we can also use the numerical distribution of the foreground area (square and shadow area) in the brightness space V. Based on prior knowledge, we can determine that the value range of the pixel values ​​in the shadow area in the V space is between 50-100. Then, we take the interval of 10-200 of the V space numerical distribution, use the least squares method to fit the curve, and take the minimum value of the trough as the target shadow threshold Ψ for segmenting the shadow area. At the same time, we also update the target shadow threshold Ψ every third target time period.

[0137] The image pixel values ​​corresponding to the brightness space V that are lower than the target shadow threshold Ψ are all set to 0, and then the basic morphological operations of erosion and dilation are performed in sequence to eliminate the influence of image noise, and then a more refined foreground area and the target image are extracted.

[0138] According to the formation detection method for moving arrays in large-scale activities provided by an embodiment of the present invention, the target shadow threshold is updated in real time through an "adaptive" target shadow threshold update method, and the target shadow threshold can be adjusted to the optimal threshold in time based on the light intensity at different times, thereby avoiding the situation where the fixed threshold cannot maintain the best effect when the natural light intensity changes, and also eliminating the problem that the image shadow in the outdoor natural environment is easily affected by the change of light intensity, so that it can adapt to the formation detection of moving arrays under long-term outdoor natural conditions, which helps to improve the segmentation effect of the shadow area, thereby improving the imaging effect of the target image, and further improving the detection effect.

[0139] In this step, by processing the target video frame multiple times, it is possible to effectively reduce noise and eliminate shadow interference, thereby obtaining a clear target image, which helps to improve the accuracy of the detection results.

[0140] like Figure 2 As shown, in some embodiments, in step 130, after determining that the jitter factor is lower than the target jitter threshold, the method may further include:

[0141] Get the definition corresponding to the target video frame;

[0142] When the definition is not lower than a definition threshold, a target image of the target motion matrix is ​​generated based on the target video frame.

[0143] When the definition is lower than the definition threshold, the jitter factor of the next target video frame is detected.

[0144] In this embodiment, the clarity threshold is the minimum value of the clarity of the image when the image is approximately considered to be clear.

[0145] The clarity threshold can be customized based on the user.

[0146] In the actual implementation process, the Laplacian gradient function can be used to evaluate the clarity of the target video frame.

[0147] The specific formula is:

[0148]

[0149] Determine, where M and N represent the width and height of the target video frame respectively, T is the given edge detection threshold, and z(x,y) is the convolution of the Laplacian operator at the pixel point (x,y).

[0150] The Laplacian operator matrix is ​​defined as follows:

[0151]

[0152] It can be understood that, for a target video frame that has been converted into a grayscale image, the clearer the image of the target video frame input into the convolution, the larger the Laplacian gradient function value.

[0153] After obtaining the definition Ω corresponding to the target video frame t Then, the clarity is compared with the clarity threshold Ω. t ≥Ω, it is determined that the target video frame is clear, and based on the target video frame, a target image of the target motion matrix is ​​generated.

[0154] In other embodiments, in Ω t <Ω, it is determined that the target video frame is unclear, and the target video frame is skipped, and the jitter factor of the next video frame is detected.

[0155] In this embodiment, by detecting the clarity of the target video frame, on the basis of satisfying that the jitter factor is lower than the target jitter threshold, the target video frame that does not meet the clarity requirement is skipped, thereby avoiding the influence of camera shake, overexposure and the like on the array detection, thereby improving the clarity of the imaging and improving the accuracy of the detection result.

[0156] According to the formation detection method for a moving array of a large-scale event provided by an embodiment of the present invention, a jitter factor is generated based on a target video frame in a target video and a diagonal grayscale projection standard array, and the target video frame is screened based on the jitter factor to retain the target video frame with a lower jitter amplitude, and the target video frame with a lower jitter amplitude is processed to generate a target image of the target moving array, thereby overcoming the defect that other grayscale projection methods are easily affected by local motion, realizing effective jitter detection of target video frames in motion, significantly improving the rapidity and real-time performance of video frame jitter detection, and improving the imaging effect of the target image, thereby helping to improve the accuracy and precision of the detection results.

[0157] The steps of generating the target background model are described below through a specific embodiment.

[0158] In some embodiments, before step 110, the method further includes:

[0159] Acquire multiple frames of environment background images within a first target time period in a target environment;

[0160] Generate a target background model based on multiple frames of environmental background images;

[0161] Based on the target background model, a diagonal grayscale projection standard array corresponding to the target background model is generated.

[0162] In this embodiment, the first target time period may be based on user customization.

[0163] The environment background image is the same as the target video background image.

[0164] The environmental background image can be collected by an image sensor, such as a panoramic camera.

[0165] The number of environment background images can be user-defined, such as set to 1000 frames.

[0166] After the environmental background image is collected, the environmental background image may be subjected to distortion correction first. The specific implementation method is the same as that in the above embodiment and will not be described in detail here.

[0167] For example, in the actual implementation process, the Gaussian background model may be first used to model the first 1000 consecutive frames of environmental background images to generate a target background model.

[0168] After the target background model is generated, it can be saved in a local server or a cloud server and can be called when needed.

[0169] The following describes a specific implementation method for generating a diagonal grayscale projection standard array corresponding to a target background model based on the target background model.

[0170] First, each frame of the environmental background image is converted into a grayscale image, and four square areas of the same size with a side length of d are taken from the upper left, lower left, upper right and lower right corners of the background model image as target areas to avoid the influence of large moving targets in the middle of the future picture on jitter detection.

[0171] It should be noted that the target area in this embodiment needs to be consistent with the position and number of the target area in the target video frame extracted in the above embodiment.

[0172] Histogram equalization is performed on the four target areas respectively.

[0173] The specific formula is:

[0174]

[0175] Perform histogram equalization processing, where f is a monotone nonlinear mapping, G I represents the gray value of the pixel in the environmental background image I, L represents the gray level, and I n is the number of pixels in the environment background image, H I represents the grayscale histogram distribution of the environmental background image I, and u represents the grayscale value.

[0176] L is usually 256.

[0177] After the histogram equalization process is performed, grayscale projection is then performed on the four regions after the histogram equalization to generate grayscale projection array standard values.

[0178] like Figure 3 As shown, the grayscale projection array standard value is used to characterize the comprehensive grayscale information of the four corner areas of the current environment background image in the horizontal and vertical directions.

[0179] In some embodiments, the formula:

[0180]

[0181] Calculate the grayscale mean of the environment background image, where I_mean represents the grayscale value of the pixel in the environment background image. represents the gray value of the pixel (x, y) in the environmental background image, d represents the side length of each target area, and I n is the number of pixels in the environment background image.

[0182] After obtaining the grayscale mean of the environmental background image, based on the grayscale mean, the formula is:

[0183]

[0184] The standard value of the diagonal grayscale projection array corresponding to each target area can be calculated;

[0185] Among them, Gr_b1 represents the standard value of the kth position of the diagonal grayscale projection array of the upper left corner area of ​​the environmental background image, d represents the side length of the target area, Represents the grayscale value of pixel (i, j) in the environmental background image.

[0186] It can be understood that Gr_b2 represents the standard value of the kth position of the diagonal grayscale projection array of the upper right corner area of ​​the environmental background image, Gr_b3 represents the standard value of the kth position of the diagonal grayscale projection array of the lower right corner area of ​​the environmental background image, and Gr_b4 represents the standard value of the kth position of the diagonal grayscale projection array of the lower left corner area of ​​the environmental background image.

[0187] After the diagonal grayscale projection standard array is obtained, the diagonal grayscale projection standard array can be saved for use in subsequent calculations in an actual detection process.

[0188] In some embodiments, the target background model may be updated every second target time period t1 to avoid changes in information such as background brightness caused by the outdoor environment being susceptible to sunlight.

[0189] In this embodiment, by combining methods such as distortion correction and histogram equalization, it is helpful to overcome the influence of interference such as jitter, lighting and shadow under outdoor natural conditions.

[0190] According to the object detection method of the motion policy provided by the embodiment of the present invention, a diagonal grayscale projection standard array is generated based on multiple frames of environmental background images in a first target time period, which is used for jitter detection of subsequent target video frames, and has high flexibility and high detection accuracy.

[0191] The following is a description of the formation detection device for a moving array of a large-scale event provided by the present invention. The formation detection device for a moving array of a large-scale event described below and the formation detection method for a moving array of a large-scale event described above can be referenced to each other.

[0192] like Fig. 9 As shown, the formation detection device for the moving array of the large-scale activity includes: a first acquisition module 910, a first generation module 920 and a second generation module 930.

[0193] The first acquisition module 910 is used to acquire a target video of a target motion matrix in a target environment;

[0194] A first generating module 920, configured to generate a jitter factor corresponding to a target video frame based on a target video frame in a target video and a diagonal grayscale projection standard array;

[0195] The second generating module 930 is used to generate a target image of a target motion matrix based on a target video frame when the jitter factor is lower than a target jitter threshold.

[0196] According to the object detection device of the motion policy provided by the embodiment of the present invention, a jitter factor is generated based on the target video frame in the target video and the diagonal grayscale projection standard array, and the target video frame is screened based on the jitter factor to retain the target video frame with a lower jitter amplitude, and the target video frame with a lower jitter amplitude is processed to generate a target image of the target motion matrix, which effectively improves the clarity of the imaging and helps to improve the detection result.

[0197] In some embodiments, the first generating module 920 is used to:

[0198] Based on the target area of ​​the target video frame, generating a first diagonal grayscale projection array of the target area;

[0199] A dither factor is generated based on the first diagonal grayscale projection array and the diagonal grayscale projection standard array.

[0200] In some embodiments, the first generation module 920 is used to: apply the formula:

[0201]

[0202] Generate a dither factor, where θ is the dither factor, Gr_bi(m) is the diagonal grayscale projection standard value of the m-th position in the diagonal grayscale projection standard array, Gr_ci(m) is the first diagonal grayscale projection value of the m-th position in the first diagonal grayscale projection array, and d is the side length of the target area.

[0203] In some embodiments, the second generating module 930 is configured to:

[0204] Performing frame difference operation on the target video frame and the target background model to generate a first image;

[0205] Performing binarization and noise reduction processing on the first image to generate a third image;

[0206] A foreground image in the target video frame is extracted based on the third image, and a target image of the target motion matrix is ​​generated based on the foreground image.

[0207] In some embodiments, the second generating module 930 is configured to:

[0208] converting the foreground image into a target format to generate a fourth image;

[0209] The fourth image is subjected to shadow region segmentation processing and erosion and dilation processing based on the target shadow threshold value to generate a target image of the target motion matrix.

[0210] In some embodiments, the apparatus further comprises:

[0211] The second acquisition module is used to acquire multiple frames of environment background images within a first target time period in the target environment before acquiring the target video of the target motion array in the target environment;

[0212] A third generation module is used to generate a target background model based on multiple frames of environmental background images;

[0213] The fourth generating module is used to generate a diagonal grayscale projection standard array corresponding to the target background model based on the target background model.

[0214] Fig.10 An example of a physical structure diagram of an electronic device is shown in FIG. Fig.10As shown, the electronic device may include: a processor 1010, a communication interface 1020, a memory 1030 and a communication bus 1040, wherein the processor 1010, the communication interface 1020 and the memory 1030 communicate with each other through the communication bus 1040. The processor 1010 may call the logic instructions in the memory 1030 to execute a formation detection method for a moving array of a large-scale activity, the method comprising: obtaining a target video of a target moving array in a target environment; generating a jitter factor corresponding to the target video frame based on a target video frame in the target video and a diagonal grayscale projection standard array; generating a target image of the target moving array based on the target video frame when the jitter factor is lower than a target jitter threshold.

[0215] In addition, the logic instructions in the above-mentioned memory 1030 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0216] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the formation detection method of a motion array of a large-scale activity provided by the above-mentioned methods, and the method includes: obtaining a target video of a target motion array in a target environment; generating a jitter factor corresponding to the target video frame based on a target video frame in the target video and a diagonal grayscale projection standard array; and generating a target image of the target motion array based on the target video frame when the jitter factor is lower than a target jitter threshold.

[0217] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned methods for detecting formations of motion arrays of large-scale activities, the method comprising: acquiring a target video of a target motion array in a target environment; generating a jitter factor corresponding to the target video frame based on a target video frame in the target video and a diagonal grayscale projection standard array; and generating a target image of the target motion array based on the target video frame when the jitter factor is lower than a target jitter threshold.

[0218] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0219] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0220] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting the formation of a sports formation in a large-scale activity, characterized in that: include: Acquire target video of target motion array in target environment; Based on a target video frame in the target video and a diagonal grayscale projection standard array, generating a jitter factor corresponding to the target video frame; When the jitter factor is lower than a target jitter threshold, generating a target image of the target motion matrix based on the target video frame; The step of generating a jitter factor corresponding to the target video frame based on the target video frame in the target video and a diagonal grayscale projection standard array includes: Based on the target area of ​​the target video frame, generating a first diagonal grayscale projection array of the target area, the target area does not include the features of the motion matrix, the first diagonal grayscale projection array is an actual diagonal grayscale projection value corresponding to the target area of ​​the target video frame, and the number of the first diagonal grayscale projection arrays is consistent with the number of the target areas; The step of generating the jitter factor based on the first diagonal grayscale projection array and the diagonal grayscale projection standard array comprises: Application formula: Generate the jitter factor, wherein θ is the jitter factor, the Gr_bi(m) is the diagonal grayscale projection standard value of the m-th position in the diagonal grayscale projection standard array, the Gr_ci(m) is the first diagonal grayscale projection value of the m-th position in the first diagonal grayscale projection array, and d is the side length of the target area; In the formula, (m+1)?m≤d-1:2d-m-1 means that the denominator is m+1, when m≤d-1, m=d-1, and when m>d-1, m=2d-m-1.

2. The method for detecting the formation of a sports formation in a large-scale activity according to claim 1, characterized in that: Before acquiring the target video of the target motion matrix in the target environment, the method further includes: Acquire multiple frames of environment background images within a first target time period in the target environment; Based on the multiple frames of environmental background images, generating a target background model; Based on the target background model, a diagonal grayscale projection standard array corresponding to the target background model is generated.

3. The formation detection method of a sports formation in a large-scale activity according to claim 2, characterized in that: The step of generating a target image of the target motion matrix based on the target video frame comprises: Performing a frame difference operation on the target video frame and the target background model to generate a first image; Performing binarization and noise reduction processing on the first image to generate a third image; A foreground image in the target video frame is extracted based on the third image, and a target image of the target motion matrix is ​​generated based on the foreground image.

4. The method for detecting the formation of a sports formation in a large-scale activity according to claim 3, characterized in that: The step of generating the target image of the target motion matrix based on the foreground image comprises: Converting the foreground image into a target format to generate a fourth image; The fourth image is subjected to shadow region segmentation processing and erosion and dilation processing based on a target shadow threshold to generate a target image of the target motion matrix.

5. A formation detection device for a sports formation in a large-scale activity, characterized in that: include: A first acquisition module is used to acquire a target video of a target motion matrix in a target environment; A first generating module, configured to generate a jitter factor corresponding to the target video frame based on the target video frame in the target video and a diagonal grayscale projection standard array; A second generating module is used to generate a target image of the target motion matrix based on the target video frame when the jitter factor is lower than a target jitter threshold; The first acquisition module, the first generation module and the second generation module are used to execute the formation detection method of a moving formation of a large-scale activity as described in any one of claims 1 to 4.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method for detecting the formation of a moving array of a large-scale activity as claimed in any one of claims 1 to 4 is implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting the formation of a moving array of a large-scale activity as claimed in any one of claims 1 to 4 is implemented.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for detecting the formation of a moving array of a large-scale activity as claimed in any one of claims 1 to 4 is implemented.

Images