Image processing method, video processing method, device, equipment and storage medium
By performing edge extraction and feature value analysis on the image, the problem of difficulty in determining the frosted glass border and misjudgment in the prior art is solved, and the accuracy and timeliness of image processing are taken into account, and the efficiency of image recognition is improved.
Patent Information
- Application Number
- CN201910473179.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-05-31
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2039-05-31
AI Technical Summary
The prior art is difficult to effectively determine the frosted glass border in the image, and traditional methods are prone to misjudgment that the meaningful area is meaningless, which is poorly robust, and the accuracy of the deep learning method depends on the number of training samples and is not good in time.
By edge extraction of the original image, an edge image is obtained, and meaningless areas are determined based on longitudinal and transverse eigenvalues. The edge image is used to highlight important features, reduce the subsequent data processing volume, and achieve accurate and fast meaningless areas recognition.
It improves the accuracy and timeliness of meaningless areas, reduces the data processing volume of image analysis, and enhances the accuracy and efficiency of image recognition.
Smart Images

Figure CN112017203B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present invention relate to image processing technology, and in particular to an image processing method, a video processing method, an apparatus, a device and a storage medium. Background Art
[0002] In short videos or live video broadcasts, if black and white borders or frosted glass borders appear on the edge of the image, it will affect viewing. These black and white borders or frosted glass borders on the edge of the image are called meaningless areas. In image processing, meaningless areas increase the amount of calculation and may affect the image processing effect. Therefore, determining the meaningless areas in the image can improve the efficiency and accuracy of image processing. Among them, the black and white borders include at least one of the black border and the white border. The image can be a single image or a video frame in a video.
[0003] The following two methods can be used to determine whether there are meaningless areas in the video. Method 1: A pixel threshold determination method based on traditional computer vision determines whether there are meaningless areas in the image by determining whether the pixels reach a preset threshold. Method 2: A classification model regression method based on deep learning obtains a classification model by training a large number of training samples, and then determines whether there are meaningless areas in the image based on the classification model.
[0004] However, there are at least the following problems in the prior art: Method 1 can effectively determine whether there is a black and white border in the image, but it is difficult to effectively determine whether there is a frosted glass border in the image; and this method is prone to misidentify some meaningful areas with too dark or too light colors as meaningless areas, so the robustness is poor. Method 2 can effectively determine whether there is a frosted glass border in the image, but the accuracy of the determination result depends on the number of training samples, and a high-accuracy determination result requires a large number of training samples; and because there is a training process for the classification model, the timeliness of the determination is poor. Summary of the invention
[0005] The embodiments of the present invention provide an image processing method, a video processing method, an apparatus, a device and a storage medium to achieve a balance between accuracy and timeliness in determining a meaningless region.
[0006] In a first aspect, an embodiment of the present invention provides an image processing method, the method comprising:
[0007] Extract the edge of the original image to obtain an edge image;
[0008] The determination result of the meaningless area of the original image is determined according to the longitudinal characteristic value and the transverse characteristic value of the edge image.
[0009] In a second aspect, an embodiment of the present invention further provides a video processing method, the method comprising:
[0010] Extract each video frame from the video as an original image;
[0011] Execute the image processing method as described in the first aspect of the embodiment of the present invention to determine the determination result of the meaningless area of each video frame;
[0012] The determination result of the meaningless area of the video is determined according to the determination result of the meaningless area of each video frame.
[0013] In a third aspect, an embodiment of the present invention further provides an image processing device, the device comprising:
[0014] The edge image acquisition module is used to extract the edge of the original image to obtain the edge image;
[0015] The first determination result module is used to determine the determination result of the meaningless area of the original image according to the longitudinal characteristic value and the transverse characteristic value of the edge image.
[0016] In a fourth aspect, an embodiment of the present invention further provides a video processing device, the device comprising:
[0017] A video frame extraction module is used to extract each video frame from the video as an original image;
[0018] A second determination result module, configured to execute the image processing method as described in the first aspect of the embodiment of the present invention, and determine a determination result of a meaningless area of each video frame;
[0019] The third result determination module is used to determine the determination result of the meaningless area of the video according to the determination result of the meaningless area of each video frame.
[0020] In a fifth aspect, an embodiment of the present invention further provides a device, the device comprising:
[0021] one or more processors;
[0022] A memory for storing one or more programs;
[0023] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in the first aspect or the second aspect of the embodiment of the present invention.
[0024] In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect or the second aspect of the embodiment of the present invention.
[0025] The embodiment of the present invention extracts the edge of the original image to obtain an edge image, and determines the determination result of the meaningless area of the original image according to the longitudinal eigenvalue and the lateral eigenvalue of the edge image. As the edge image can highlight the important features in the original image, reduce the amount of data processing for subsequent image analysis, and make the understanding and recognition of the image easier, therefore, according to the eigenvalue of the edge image, the meaningless area of the original image can be accurately and quickly determined, that is, the accuracy and timeliness of the determination of the meaningless area are taken into consideration. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 is a flow chart of an image processing method in an embodiment of the present invention;
[0027] Figure 2 is a schematic diagram of calculating a characteristic value of an edge image in an embodiment of the present invention;
[0028] Figure 3 is a schematic diagram of an edge image in an embodiment of the present invention;
[0029] Figure 4 is a schematic diagram showing a characteristic value in an embodiment of the present invention;
[0030] Figure 5 is a schematic diagram of displaying a meaningless area in an embodiment of the present invention;
[0031] Figure 6 is a schematic diagram of another edge image in an embodiment of the present invention;
[0032] Figure 7 is a schematic diagram showing another characteristic value in an embodiment of the present invention;
[0033] Figure 8 is another schematic diagram of displaying a meaningless area in an embodiment of the present invention;
[0034] Fig. 9 is a flow chart of another image processing method in an embodiment of the present invention;
[0035] Fig.10 is a flow chart of a video processing method in an embodiment of the present invention;
[0036] Fig.11 is a flowchart of another video processing method in an embodiment of the present invention;
[0037] Fig.12 is an application schematic diagram of a video processing method in an embodiment of the present invention;
[0038] Fig.13 is a structural schematic diagram of an image processing device in an embodiment of the present invention;
[0039] Fig.14 is a structural schematic diagram of a video processing device in an embodiment of the present invention;
[0040] Fig.15 It is a structural schematic diagram of a device in an embodiment of the present invention. DETAILED DESCRIPTION
[0041] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention, and the various features recorded in the embodiments can be combined to form multiple optional solutions. It is also necessary to explain that, for the convenience of description, only the parts related to the present invention are shown in the accompanying drawings, rather than all structures.
[0042] Example
[0043] Figure 1 This is a flowchart of an image processing method provided by an embodiment of the present invention. This embodiment is applicable to the situation of accurately and quickly determining meaningless areas in an image. The method can be executed by an image processing device, which can be implemented in software and / or hardware. The device can be configured in a device, such as a computer. Figure 1 As shown, the method specifically comprises the following steps:
[0044] Step 110: extract edges from the original image to obtain an edge image.
[0045] In an embodiment of the present invention, in order to achieve a balance between accuracy and timeliness in determining meaningless areas, a method of analyzing an edge image obtained by performing edge extraction on the original image instead of analyzing the original image may be adopted. The original image may be understood as an image for which it is necessary to determine whether there is a meaningless area, and the meaningless area may be understood as a black and white border or a frosted glass border that appears on the edge of the image. It should be noted that the original image may be a single image or a video frame in a video. If the original image is a video frame of a video frame, edge extraction is performed on the original image to obtain an edge image, which may be understood as extracting each video frame from the video, performing edge extraction on each video frame respectively, and obtaining an edge image corresponding to each video frame. If the original image is a single image, edge extraction is performed on the original image to obtain an edge image, which may be understood as performing edge extraction on the single image to obtain an edge image corresponding to the single image.
[0046] The edge extraction algorithm can be used to extract the edge of the original image to obtain the edge image of the original image. Edge extraction involves the feature extraction of the target object in the image, that is, how to identify the outline of the object in the image, which is the premise of digital image analysis and processing. If the edge of the image can be successfully extracted, the subsequent image processing will be greatly simplified, and image recognition and understanding will also become simpler. Since the edge of the image is invariant, that is, it will not be affected by the change of light or other external factors, and the human visual system is most sensitive to the edge of the image, therefore, edge extraction can affect the difficulty and accuracy of subsequent image processing.
[0047] The edge of an image refers to the part where the pixel value of an image pixel changes significantly, which is the result of the discontinuity of the pixel value of the pixel. The edge of an image exists between the target object and the background, between the target object and the target object, and between regions and regions, and is often caused by the change of the physical properties of the target object in the image. Due to the different changes in physical properties, different edges are generated. The edge of an image has two characteristics: direction and amplitude. Parallel to the edge direction, the pixel value of the pixel changes slowly; perpendicular to the edge direction, the pixel value of the pixel changes dramatically and presents a step or slope shape. The essence of edge extraction is to use a certain algorithm to extract the boundary line between the target object and the background, the boundary line between the target object and the target object, and the boundary line between regions in the image. The above boundary line is the area with the largest change in pixel value. Usually, an image can be regarded as an image grayscale function. The line connecting the singular points and mutation points of the image grayscale function can reflect the above-mentioned boundary line. The singular points and mutation points of the image grayscale function can be obtained by the gradient of the image grayscale function.
[0048] Edge extraction algorithms can be divided into local image-based maximum gradient judgment algorithms, filter-based edge extraction algorithms, and image-based edge analysis algorithms. Among them, the principle of the local image-based maximum gradient judgment algorithm is: select a small window containing a certain number of pixels, calculate the gradient value of the grayscale value of the pixels in this window, and the pixel corresponding to the maximum gradient value is the edge. The principle of the filter-based edge extraction algorithm is: by performing different filtering processes on the image, a filtered image is obtained, and then the gradient value of the filtered image is calculated. The pixel corresponding to the maximum gradient value is the edge. According to the characteristic that the first-order derivative of the image has a maximum value at the edge, a variety of edge extraction operators are obtained, and the edge extraction operator can include Canny operator, Sobel operator, LoG (Laplacian of Gaussian, Laplace Gaussian) operator, Laplace operator, Robert operator and Robinson operator. The theoretical basis of the edge analysis algorithm based on the global image is mainly mathematical analysis. It uses the two types of algorithms described above to extract the local edges of the image, and then synthesizes each local area, analyzes these local data information from a global perspective, and extracts the edge with the global optimal evaluation standard. It should be noted that the specific edge extraction algorithm used to extract the edge of the original image can be selected according to actual conditions and is not specifically set here.
[0049] Since edge images can highlight important features of images, reduce the amount of subsequent data processing, and also make it easier to understand and recognize images, the edge images can be subsequently analyzed to determine whether there are meaningless areas in the original image, and, if so, the specific locations of the meaningless areas.
[0050] Step 120: Determine the result of determining the meaningless area of the original image according to the longitudinal eigenvalue and the transverse eigenvalue of the edge image.
[0051] In an embodiment of the present invention, the characteristic value of the edge image can be determined by the pixel value of the pixel point. The characteristic value of the edge image can be determined in the following manner, specifically: the edge image can be mapped into a plane rectangular coordinate system, that is, an XY coordinate system. Correspondingly, each pixel point in the edge image has a corresponding coordinate value. The coordinate value of each pixel point can include a horizontal coordinate value (i.e., X) and a vertical coordinate value (i.e., Y). The longitudinal characteristic value of the edge image can be determined in the following manner: along the horizontal direction (or X-axis direction) of the edge image, according to the pixel values of each pixel point with the same horizontal coordinate value, the longitudinal characteristic value of the edge image is determined. The transverse characteristic value of the edge image can be determined in the following manner: along the longitudinal direction (or Y-axis direction) of the edge image, according to the pixel values of each pixel point with the same vertical coordinate value, the transverse characteristic value of the edge image is determined. Among them, along the horizontal direction of the edge image, according to the pixel values of each pixel point with the same horizontal coordinate value, the longitudinal characteristic value of the edge image is determined as follows: for each longitudinal characteristic value of the edge image, the pixel values of each pixel point with the same horizontal coordinate value are added along the horizontal direction of the edge image to obtain the longitudinal characteristic value of the edge image. Alternatively, for each longitudinal eigenvalue of the edge image, the average value of the pixel values of each pixel point having the same horizontal coordinate value is calculated along the horizontal direction of the edge image to obtain the longitudinal eigenvalue of the edge image. Each longitudinal eigenvalue can be determined in the above manner. Similarly, along the longitudinal direction of the edge image, the transverse eigenvalue of the edge image is determined based on the pixel values of each pixel point having the same vertical coordinate value, which can be understood as follows: for each transverse eigenvalue of the edge image, along the longitudinal direction of the edge image, the pixel values of each pixel point having the same vertical coordinate value are added to obtain the transverse eigenvalue of the edge image. Alternatively, for each transverse eigenvalue of the edge image, the average value of the pixel values of each pixel point having the same vertical coordinate value is calculated along the longitudinal direction of the edge image to obtain the transverse eigenvalue of the edge image. Each transverse eigenvalue can be determined in the above manner.
[0052] For example, Figure 2 As shown, a schematic diagram of calculating the eigenvalues of an edge image is given. Figure 2 The size of the edge image is 4×4, which includes 16 pixels. The edge image is mapped to a plane rectangular coordinate system. The coordinates of the 16 pixels are pixel 1 (l x,1 , l y,1 ), pixel 2(l x,1 , l y,2 ), pixel 3(l x,1 , l y,3 ), pixel 4(l x,1 , l y,4 ), pixel 5(l x,2 , l y,1 ), pixel 6(l x,2, l y,2 ), pixel 7(l x,2 , l y,3 ), pixel 8(l x,2 , l y,4 ), pixel 9(l x,3 , l y,1 ), pixel 10(l x,3 , l y,2 ), pixel 11(l x,3 , l y,3 ), pixel 12(l x,3 , l y,4 ), pixel 13(l x,4 , l y,1 ), pixel 14(l x,4 , l y,2 ), pixel 15(l x,4 , l y,3 ) and pixel 16(l x,4 , l y,4 ).
[0053] Along the horizontal direction of the edge image (i.e. along the X-axis direction), for the horizontal coordinate value l x,1 , according to the pixel value of pixel point 1, the pixel value of pixel point 2, the pixel value of pixel point 3 and the pixel value of pixel point 4, the first longitudinal characteristic value is obtained; for the horizontal coordinate value l x,2 , according to the pixel value of pixel 5, the pixel value of pixel 6, the pixel value of pixel 7 and the pixel value of pixel 8, the second longitudinal eigenvalue is obtained; for the horizontal coordinate value l x,3 , according to the pixel value of pixel point 9, the pixel value of pixel point 10, the pixel value of pixel point 11 and the pixel value of pixel point 12, the third longitudinal eigenvalue is obtained; for the horizontal coordinate value l x,4 , according to the pixel value of pixel 13, the pixel value of pixel 14, the pixel value of pixel 15 and the pixel value of pixel 16, a fourth longitudinal eigenvalue is obtained.
[0054] Along the longitudinal direction of the edge image (i.e. along the Y-axis direction), for the ordinate value l y,1 , according to the pixel value of pixel point 1, the pixel value of pixel point 5, the pixel value of pixel point 9 and the pixel value of pixel point 13, the first horizontal characteristic value is obtained; for the vertical coordinate value l y,2 , according to the pixel value of pixel point 2, the pixel value of pixel point 6, the pixel value of pixel point 10 and the pixel value of pixel point 14, the second horizontal characteristic value is obtained; for the horizontal coordinate value l y,3 , according to the pixel value of pixel point 3, the pixel value of pixel point 7, the pixel value of pixel point 11 and the pixel value of pixel point 15, the third horizontal characteristic value is obtained; for the horizontal coordinate value l y,4, according to the pixel value of pixel point 4, the pixel value of pixel point 8, the pixel value of pixel point 12 and the pixel value of pixel point 16, the fourth horizontal characteristic value is obtained.
[0055] After obtaining each longitudinal eigenvalue and each transverse eigenvalue of the edge image, the meaningless area of the original image can be determined according to each longitudinal eigenvalue and each transverse eigenvalue. The above can be understood as follows: according to the longitudinal eigenvalue greater than or equal to the longitudinal threshold and the transverse eigenvalue greater than or equal to the transverse threshold, the border of the edge image is obtained, and the meaningless area of the original image is determined according to the border of the edge image. This is because the border can be understood as including at least one edge, that is, the border is an area composed of at least one edge. For each edge in the border, the edge includes at least one pixel point. If the edge is perpendicular to the transverse direction of the image (i.e., perpendicular to the horizontal axis), the horizontal coordinate values of the pixels constituting the edge in the border are the same. If the edge is perpendicular to the longitudinal direction of the image (i.e., perpendicular to the vertical axis), the vertical coordinate values of the pixels constituting the edge in the border are the same. For each longitudinal eigenvalue, it is formed by pixels having the same horizontal coordinate value. For each transverse eigenvalue, it is formed by pixels having the same vertical coordinate value. If a certain edge of a certain border appears in a certain area, and the edge is perpendicular to the longitudinal direction of the image (i.e., perpendicular to the ordinate axis), then the horizontal eigenvalue corresponding to the area will be greater than or equal to the horizontal threshold. If a certain edge of a certain border appears in a certain area, and the edge is perpendicular to the horizontal direction of the image (i.e., perpendicular to the abscissa axis), then the vertical eigenvalue corresponding to the area will be greater than or equal to the vertical threshold. Therefore, the border of the edge image can be determined based on the horizontal and vertical eigenvalues of the edge image. In addition, since the size of the edge image is the same as that of the original image, the position of the border in the edge image is the same as the position of the border in the original image. The border can be understood as a meaningless area. Therefore, after determining the border in the edge image, the meaningless area of the original image can be determined based on the border of the edge image.
[0056] It should be noted that the original image containing the unintentional area can be marked as a low-quality image to facilitate subsequent other services.
[0057] The technical solution of this embodiment is to obtain an edge image by performing edge extraction on the original image, and determine the determination result of the meaningless area of the original image according to the longitudinal eigenvalue and the transverse eigenvalue of the edge image. As the edge image can highlight the important features in the original image, reduce the amount of data processing for subsequent image analysis, and make the understanding and recognition of the image easier, therefore, according to the eigenvalue of the edge image, the meaningless area of the original image can be accurately and quickly determined, that is, the accuracy and timeliness of the determination of the meaningless area are taken into consideration.
[0058] Optionally, based on the above technical solution, the determination result of the meaningless area of the original image is determined according to the longitudinal eigenvalue and the transverse eigenvalue of the edge image, which may specifically include: determining the border in the edge image according to the longitudinal eigenvalue greater than or equal to the longitudinal threshold and the transverse eigenvalue greater than or equal to the transverse threshold. If the border satisfies at least one of the following conditions: the distance between the edge of the border and the edge in the corresponding edge image is greater than or equal to the distance threshold, and the distance difference between the relative edge of the border and the central axis of the edge image is less than or equal to the error threshold, then the meaningless area of the original image is determined according to the border. Otherwise, it is determined that there is no meaningless area in the original image.
[0059] In the embodiment of the present invention, both the vertical threshold and the horizontal threshold can be used as the standard for determining whether a region has a border edge. The specific values of the vertical threshold and the horizontal threshold can be set according to actual conditions and are not specifically limited here.
[0060] According to the foregoing, the number of horizontal eigenvalues is at least one, and the number of vertical eigenvalues is also at least one. Correspondingly, the number of horizontal eigenvalues greater than or equal to the horizontal threshold may be zero or at least one, and the number of vertical eigenvalues greater than or equal to the vertical threshold may also be zero or at least one. Since there may be a border edge at the opposite sides close to the edge image, when the edge image is mapped to a plane rectangular coordinate system, the above embodiment is that there may be a border edge along the direction from small to large coordinate values and along the direction from large to small coordinate values. And since the border is usually located near the edge position of the edge image, the so-called edge position can be understood as a position close to the edge. Therefore, if there is a border edge along the direction from small to large horizontal coordinate values, and the edge is perpendicular to the horizontal coordinate axis, the vertical eigenvalue corresponding to the area where the above edge appears is the first vertical eigenvalue greater than or equal to the vertical threshold value along the direction from small to large horizontal coordinate values. If there is a border edge along the direction from large to small horizontal coordinate values, and the edge is perpendicular to the horizontal axis, then the longitudinal feature value corresponding to the region where the edge appears is the first longitudinal feature value greater than or equal to the longitudinal threshold value along the direction from large to small horizontal coordinate values. Similarly, if there is a border edge along the direction from small to large vertical coordinate values, and the edge is perpendicular to the vertical axis, then the transverse feature value corresponding to the region where the edge appears is the first transverse feature value greater than or equal to the transverse threshold value along the direction from small to large vertical coordinate values. If there is a border edge along the direction from large to small vertical coordinate values, and the edge is perpendicular to the vertical axis, then the transverse feature value corresponding to the region where the edge appears is the first transverse feature value greater than or equal to the transverse threshold value along the direction from large to small vertical coordinate values. In other words, if there is a border edge in a region, and the edge is perpendicular to the horizontal axis, then the longitudinal feature value corresponding to the region of the edge is the first longitudinal feature value greater than or equal to the longitudinal threshold value along the direction from small to large horizontal coordinate values, or, the longitudinal feature value corresponding to the region of the edge is the first transverse feature value greater than or equal to the transverse threshold value along the direction from large to small horizontal coordinate values. If a region has a border edge that is perpendicular to the ordinate axis, then the lateral feature value corresponding to the region of the edge is the first lateral feature value that is greater than or equal to the lateral threshold along the direction from small to large ordinate values, or the lateral feature value corresponding to the region of the edge is the first lateral feature value that is greater than or equal to the lateral threshold along the direction from large to small ordinate values.
[0061] Based on the above, it can be understood that the border in the edge image can be determined based on the longitudinal eigenvalue greater than or equal to the longitudinal threshold and the transverse eigenvalue greater than or equal to the transverse threshold. Specifically, it can be understood as follows: along the direction from small to large transverse coordinate values, the first longitudinal eigenvalue greater than or equal to the longitudinal threshold can be determined as the first target longitudinal eigenvalue, and the first target transverse coordinate value of each pixel point corresponding to the first target longitudinal eigenvalue can be determined based on the first target longitudinal eigenvalue. Along the direction from large to small transverse coordinate values, the first longitudinal eigenvalue greater than or equal to the longitudinal threshold can be determined as the second target longitudinal eigenvalue, and the second target transverse coordinate value of each pixel point corresponding to the second target longitudinal eigenvalue can be determined based on the second target longitudinal eigenvalue. Along the direction from small to large longitudinal coordinate values, the first longitudinal eigenvalue greater than or equal to the transverse threshold can be determined as the first target transverse eigenvalue, and the first target longitudinal coordinate value of each pixel point corresponding to the first target transverse eigenvalue can be determined based on the first target transverse eigenvalue. Along the direction from large to small vertical coordinate values, the first vertical feature value greater than or equal to the horizontal threshold value can be determined as the second target horizontal feature value, and the second target vertical coordinate value of each pixel point corresponding to the second target horizontal feature value can be determined according to the second target horizontal feature value. If there is a first target horizontal coordinate value, a second target horizontal coordinate value, a first target vertical coordinate value, and a second target vertical coordinate value, the coordinates of the four vertices of the frame can be determined, and the frame formed can be determined according to the coordinates of the four vertices. That is, the line connecting two adjacent vertices can form an edge in the frame; if there is a first target horizontal coordinate value, a second target horizontal coordinate value, and one of the first target vertical coordinate value and the second target vertical coordinate value, the coordinates of the two vertices of the frame can be determined, and the two vertices are two adjacent vertices. The frame formed is determined according to the coordinates of the two vertices, and the frame is a single frame. Similarly, if there is a first target vertical coordinate value, a second target vertical coordinate value, and one of the first target horizontal coordinate value and the second target horizontal coordinate value, the coordinates of the two vertices of the frame can also be determined, and the two vertices are two adjacent vertices. The frame formed is determined according to the coordinates of the two vertices, and the frame is a single frame. The above shows that the border of the edge image can be determined based on two longitudinal eigenvalues greater than or equal to the longitudinal threshold and at least one transverse eigenvalue greater than or equal to the transverse threshold. Alternatively, the border of the edge image can be determined based on two transverse eigenvalues greater than or equal to the transverse threshold and at least one longitudinal eigenvalue greater than or equal to the longitudinal threshold. It should be noted that the two longitudinal eigenvalues greater than or equal to the longitudinal threshold refer to the first target longitudinal eigenvalue and the second target longitudinal eigenvalue mentioned above, respectively, and the two transverse eigenvalues greater than or equal to the transverse threshold refer to the first target transverse eigenvalue and the second target transverse eigenvalue mentioned above, respectively. The at least one transverse eigenvalue greater than or equal to the transverse threshold refers to the first target transverse eigenvalue or the second target transverse eigenvalue mentioned above.The at least one longitudinal characteristic value that is greater than or equal to the longitudinal threshold value refers to the first target longitudinal characteristic value or the second target longitudinal characteristic value mentioned above.
[0062] Based on the above, after determining the border of the edge image, in order to further determine whether the border is a real border, it can be achieved in the following way: confirm whether the border satisfies at least one of the following conditions: condition one, the distance between the edge of the border and the margin in the corresponding edge image is greater than or equal to the distance threshold; condition two, the distance difference between the relative edge of the border and the central axis of the edge image is less than or equal to the error threshold. If one of the above two conditions is met, the border can be determined to be a real border. Based on this, the meaningless area of the original image can be determined according to the border of the edge image. That is, in the case of determining that the border is a real border, the border of the edge image is the border of the original image, and the border represents the meaningless area, thereby determining that the border of the edge image is the meaningless area of the original image. Correspondingly, if both of the above two conditions are not met, it can be determined that the border is not a real border. It is determined that there is no meaningless area in the original image. It should be noted that, according to the above, it can be known that the border in the edge image determined according to the longitudinal feature value greater than or equal to the longitudinal threshold and the lateral feature value greater than or equal to the lateral threshold may be a single border. At this time, it can be considered that the single border can meet the above condition two.
[0063] Based on the above, for the distance between the edge of the frame and the edge in the corresponding edge image is greater than or equal to the distance threshold, the above can be understood as follows: According to whether the frame is a single frame, it can be divided into the following two cases: Case 1, the frame is not a single frame, that is, there is a first target horizontal coordinate value, a second target horizontal coordinate value, a first target vertical coordinate value and a second target vertical coordinate value. In this case, the maximum horizontal coordinate value is the maximum horizontal coordinate value among the horizontal coordinate values of each pixel point corresponding to each vertical characteristic value, the first target horizontal coordinate value is the horizontal coordinate value of each pixel point corresponding to the first target vertical characteristic value as described above, and the second target horizontal coordinate value is the horizontal coordinate value of each pixel point corresponding to the second target vertical characteristic value as described above. And, the maximum vertical coordinate value is the maximum vertical coordinate value among the vertical coordinate values of each pixel point corresponding to each horizontal characteristic value, the first target vertical coordinate value is the vertical coordinate value of each pixel point corresponding to the first target horizontal characteristic value as described above, and the second target vertical coordinate value is the vertical coordinate value of each pixel point corresponding to the second target horizontal characteristic value as described above.
[0064] The distance between the edge of the border and the edge in the corresponding edge image is greater than or equal to the distance threshold, which can be understood as: the distance threshold can include a horizontal distance threshold and a vertical distance threshold. Among them, the numerical values of the horizontal distance threshold and the vertical distance threshold can be the same or different, and can be set according to actual conditions, and are not specifically limited here. The first target horizontal coordinate value is greater than or equal to the horizontal distance threshold, and the difference between the maximum horizontal coordinate value and the second target horizontal coordinate value is greater than or equal to the horizontal distance threshold, and / or the first target vertical coordinate value is greater than or equal to the vertical distance threshold, and the difference between the maximum vertical coordinate value and the second target vertical coordinate value is greater than or equal to the vertical distance threshold.
[0065] Case 2: The border is a single border. That is, there is a first target horizontal coordinate value, a second target horizontal coordinate value, and a first target vertical coordinate value. In this case, the distance between the edge of the border and the edge in the corresponding edge image is greater than or equal to the distance threshold, which can be understood as: the first target vertical coordinate value is greater than or equal to the longitudinal distance threshold. Alternatively, there is a first target horizontal coordinate value, a second target horizontal coordinate value, and a second target vertical coordinate value. In this case, the distance between the edge of the border and the edge in the corresponding edge image is greater than or equal to the distance threshold, which can be understood as: the difference between the maximum vertical coordinate value and the second target vertical coordinate value is greater than or equal to the longitudinal distance threshold. Alternatively, there is a first target horizontal coordinate value, a first target vertical coordinate value, and a second target vertical coordinate value. In this case, the distance between the edge of the border and the edge in the corresponding edge image is greater than or equal to the distance threshold, which can be understood as: the first target horizontal coordinate value is greater than or equal to the horizontal distance threshold. Alternatively, there is a second target horizontal coordinate value, a first target vertical coordinate value, and a second target vertical coordinate value. In this case, the distance between the edge of the border and the edge in the corresponding edge image is greater than or equal to the distance threshold, which can be understood as: the difference between the maximum horizontal coordinate value and the second target horizontal coordinate value is greater than or equal to the horizontal distance threshold.
[0066] The difference in distance from the relative edge of the frame to the central axis of the edge image is less than or equal to the error threshold, which can be understood as follows: According to whether the frame is a single frame, it can be divided into the following two cases: Case 1, the frame is not a single frame, that is, there are a first target horizontal coordinate value, a second target horizontal coordinate value, a first target vertical coordinate value and a second target vertical coordinate value. In this case, the absolute value of the difference between the sum of the first target horizontal coordinate value and the second target horizontal coordinate value and the maximum horizontal coordinate value is calculated to obtain the horizontal error value. And, the absolute value of the difference between the sum of the first target vertical coordinate value and the second target vertical coordinate value and the maximum vertical coordinate value is calculated to obtain the vertical error value. In this case, the difference in distance from the relative edge of the frame to the central axis of the edge image is less than or equal to the error threshold, which can be understood as: the error threshold may include a horizontal error threshold and a vertical error threshold. The numerical values of the horizontal error threshold and the vertical error threshold may be the same or different, and may be set according to actual conditions, and are not specifically limited here. The horizontal error value is less than or equal to the horizontal error threshold, and / or the vertical error value is less than or equal to the vertical error threshold.
[0067] Case 2: The border is a single border. That is, there is a first target horizontal coordinate value, a second target horizontal coordinate value, and one of the first target vertical coordinate value and the second target vertical coordinate value, or there is a first target vertical coordinate value and a second target vertical coordinate value, and one of the first target horizontal coordinate value and the second target horizontal coordinate value. The above can be considered to satisfy the difference in distance from the relative edge of the border to the central axis of the edge image is less than or equal to the error threshold.
[0068] The effect embodied by the above two conditions is that: condition one can be used to ensure that the distance between the edge of the frame and the edge of the corresponding edge image is not too small, that is, it is necessary to ensure that the distance between the edge of the frame and the edge in the corresponding edge image is greater than or equal to the distance threshold, otherwise the confirmed frame is likely not a frame. Condition two can be used to ensure the symmetry of the frame, that is, it is necessary to ensure that the symmetry of the frame is good. This is because the frame is usually symmetrical. The above two conditions can be met at the same time, or only one of them can be met. Compared with the frame that only satisfies the above two conditions at the same time, it is confirmed that the frame is indeed a real frame. Only the frame that satisfies one of the above two conditions can confirm that the frame is a real frame. The accuracy of the frame determined in this way is low. In other words, in order to maximize the accuracy of the determined frame, it can be set that only when the distance between the edge of the frame and the edge in the corresponding edge image is greater than or equal to the distance threshold and the distance difference between the relative edge of the frame and the central axis of the edge image is less than or equal to the distance threshold, the frame is determined to be a real frame.
[0069] In addition, in addition to the two conditions described above, another condition can be set, that is, the distance between the relative edges in the frame is greater than or equal to the range distance threshold. That is, the distance between the first target horizontal coordinate value and the second target horizontal coordinate value is greater than or equal to the horizontal range distance threshold, and the distance between the first target vertical coordinate value and the second target vertical coordinate value is greater than or equal to the vertical range distance threshold. Based on the above, if the frame satisfies at least one of the following conditions: the distance between the edge of the frame and the edge in the corresponding edge image is greater than or equal to the distance threshold, the distance difference between the relative edge of the frame and the central axis of the edge image is less than or equal to the error threshold, and the distance between the relative edges of the frame is greater than or equal to the range distance threshold, then the meaningless area of the original image can be determined based on the frame. Otherwise, it is determined that there is no meaningless area in the original image.
[0070] It can be understood that if there are meaningless areas in the original image, the unintentional areas can be single-frames and non-single-frames, wherein the non-single-frame can be a frame with two opposite sides or a frame with only one opposite side. The frame with two opposite sides needs to meet one of the following conditions: Condition 1: the horizontal coordinate value of the first target is greater than or equal to the horizontal distance threshold, the difference between the maximum horizontal coordinate value and the second target horizontal coordinate value is greater than or equal to the horizontal distance threshold, the vertical coordinate value of the first target is greater than or equal to the vertical distance threshold, and the difference between the maximum vertical coordinate value and the second target vertical coordinate value is greater than or equal to the vertical distance threshold; Condition 2: the horizontal error value is less than or equal to the horizontal error threshold, and the vertical error value is less than or equal to the vertical error threshold.
[0071] For example, Figure 3 As shown, a schematic diagram of an edge image is given. Figure 3 The size of the edge image in is 50×50, Figure 3 The middle frame is a non-single frame. There are a first target horizontal coordinate value, a second target horizontal coordinate value, a first target vertical coordinate value, and a second target vertical coordinate value. The first target horizontal coordinate value can be expressed as x l , the second target horizontal coordinate value can be expressed as x r , the maximum horizontal coordinate value can be expressed as x m ; The first target ordinate value can be expressed as y l , the second target ordinate value can be expressed as y r , the maximum ordinate value can be expressed as y m ; The lateral distance threshold can be expressed as l x,m , the vertical distance threshold can be expressed as l y,m ; The lateral error value can be expressed as r x =|x l -(x m -x r )|, the longitudinal error value can be expressed as r y =|y l -(ym -y r )|, the lateral error threshold can be expressed as r x,m , the longitudinal error threshold can be expressed as r y,m .like Figure 4 As shown, a schematic diagram for displaying characteristic values is given. Figure 4 In the example, the first target horizontal coordinate value x l Specifically, x,15 , the second target horizontal coordinate value x r Specifically, x,35 , the maximum horizontal coordinate value x m Specifically, x,50 ; The first target ordinate value y l Specifically, y,15 , the second target ordinate value y r Specifically, y,35 , the maximum ordinate value y m Specifically, y,50 According to the first target horizontal coordinate value x l , the second target horizontal coordinate value x r , the first target ordinate value y l , and the second target ordinate value y r , and get the coordinates of the four vertices, which are (x l ,y l )、(x l ,y r )、(x r ,y l ) and (x r ,y r ), connect two adjacent vertices among the four vertices respectively, and you will get the border of the edge image.
[0072] The border needs to satisfy the requirement that the distance between the border edge and the edge in the corresponding edge image is greater than or equal to the distance threshold and the distance difference between the relative edge of the border and the central axis of the edge image is less than or equal to the error threshold, then the border can be determined to be a real border. The distance between the border edge and the edge in the corresponding edge image is greater than or equal to the distance threshold, which can be understood as: l ≥l x,m and |x m -x r |≥l x,m , and / or, y l ≥l y,m and |y m -y r |≥l y,m The difference in distance from the opposite edge of the border to the central axis of the edge image is less than or equal to the error threshold, which can be understood as: r x ≤r x,m , and / or, ry ≤r y,m . Because x l ≥l x,m ,|x m -x r |≥l x,m ,y l ≥l y,m and |y m -y r |≥l y,m , and r x ≤r x,m and r y ≤r y,m , therefore, it can be determined that the border is a real border. Figure 5 As shown, a schematic diagram of displaying a meaningless area is given. Figure 5 The meaningless area of the original image is the border of the edge image.
[0073] like Figure 6 As shown, a schematic diagram of another edge image is given. Figure 6 The size of the edge image in is 50×50, Figure 6 The middle frame is a single frame. There are a first target horizontal coordinate value, a second target horizontal coordinate value, and a first target vertical coordinate value. The first target horizontal coordinate value can be expressed as x l , the second target horizontal coordinate value can be expressed as x r ; The first target ordinate value can be expressed as y l ; The vertical distance threshold can be expressed as l y,m .like Figure 7 As shown, another schematic diagram for displaying characteristic values is given. Figure 7 In the example, the first target horizontal coordinate value x l Specifically, x,15 , the second target horizontal coordinate value x r Specifically, x,35 , the maximum horizontal coordinate value x m Specifically, x,50 ; The first target ordinate value y l Specifically, y,15 According to the first target horizontal coordinate value x l , the second target horizontal coordinate value x r , and the first target ordinate value, get the two vertex coordinates, respectively (x l ,y l ) and (x r ,y l ), connect the two adjacent vertices of the two vertices respectively, and you will get the border of the edge image.
[0074] The border needs to satisfy the requirement that the distance between the border edge and the edge in the corresponding edge image is greater than or equal to the distance threshold and the distance difference between the relative edge of the border and the central axis of the edge image is less than or equal to the error threshold, then the border can be determined to be a real border. The distance between the border edge and the edge in the corresponding edge image is greater than or equal to the distance threshold, which can be understood as: l ≥l y,m The difference in distance from the relative edge of the frame to the center axis of the edge image is less than or equal to the error threshold, which can be understood as satisfying the above condition that the difference in distance from the relative edge of the frame to the center axis of the edge image is less than or equal to the error threshold. l ≥l y,m And the distance difference between the relative edge of the border and the central axis of the edge image is less than or equal to the error threshold, so the border can be determined to be a real border. Figure 8 As shown, another schematic diagram of displaying meaningless areas is given. Figure 8 The meaningless area of the original image is the border of the edge image.
[0075] Optionally, based on the above technical solution, edge extraction is performed on the original image to obtain an edge image, which may specifically include: based on an edge extraction algorithm, edge extraction is performed on the original image to obtain an edge image.
[0076] In an embodiment of the present invention, the edge extraction algorithm can be divided into a maximum gradient judgment algorithm based on a local image, an edge extraction algorithm based on a filter, and an edge analysis algorithm based on a global image. The filter-based edge extraction algorithm can include an edge extraction algorithm based on a Canny operator, an edge extraction algorithm based on a Sobel operator, an edge extraction algorithm based on a LoG operator, an edge extraction algorithm based on a Laplace operator, an edge extraction algorithm based on a Robert operator, and an edge extraction algorithm based on a Robinson operator. Since the edge extraction algorithm based on the Canny operator has the advantages of a large signal-to-noise ratio, high detection accuracy, and a small amount of calculation, the following is an example of using an edge extraction algorithm based on a Canny operator to extract the edge of an original image and obtain an edge image, specifically:
[0077] The Canny operator is actually the first-order derivative of the Gaussian function, which is an operator that combines the smoothing of the Gaussian function and the gradient operation of the image. Although the Canny operator does not have rotational symmetry, it is symmetric in the edge direction and antisymmetric along the gradient direction. The above makes the operator have a good compromise between anti-noise and precise edge positioning. The Canny operator extracts a segmentation algorithm that is still robust to changes in illumination, and accurately finds the edge of the target object as unaffected by illumination as possible. The implementation process of the edge extraction algorithm based on the Canny operator is as follows: Gaussian filtering is performed on the original image using a Gaussian filter to obtain a filtered image; the gradient value vector of the filtered image is calculated; non-maximum suppression is used to determine the local maximum value of each pixel in the filtered image according to the gradient value vector, that is, if the gradient amplitude of the pixel point is greater than or equal to the gradient amplitude of the two adjacent pixels along the gradient direction, it can be determined that the pixel point may be an edge point, and the remaining non-maximum pixel points are zeroed; the double threshold method is used to detect and connect the edge to obtain an edge image. That is, a high threshold and a low threshold are set. If the gray value of a pixel is greater than or equal to the high threshold, the pixel can be determined to be an edge point; if the gray value of a pixel is less than or equal to the low threshold, the pixel is determined not to be an edge point; if the gray value of a pixel is greater than the low threshold and less than the high threshold, then only when the pixel with the above characteristics is connected to the edge point determined previously according to a certain curve can it be determined to be an edge point. It should be noted that the setting of the above high threshold and low threshold is crucial. If the high threshold is set too high, the real edge point will be missed; if the high threshold is set too low, a large amount of noise will be mixed in the determined edge information. If the low threshold is set too high, edges with smaller mutations will be missed.
[0078] It should be noted that the specific edge extraction algorithm to be used can be selected according to actual conditions and is not specifically limited here.
[0079] Fig. 9 This is a flowchart of another image processing method provided by an embodiment of the present invention. This embodiment is a specific example of the above embodiment. This embodiment can be applied to the situation of accurately and quickly determining the meaningless area in the image. The method can be executed by an image processing device. The device can be implemented in software and / or hardware. The device can be configured in a device, such as a computer. Fig. 9 As shown, the method specifically comprises the following steps:
[0080] Step 210: Based on an edge extraction algorithm, edge extraction is performed on the original image to obtain an edge image.
[0081] Step 220: Determine a border in the edge image according to the longitudinal feature value greater than or equal to the longitudinal threshold and the transverse feature value greater than or equal to the transverse threshold.
[0082] Step 230: Whether the distance between the edge of the border and the edge in the corresponding edge image is greater than or equal to a distance threshold, and / or whether the distance difference between the relative edge of the border and the central axis of the edge image is less than or equal to an error threshold; if so, execute step 240; if not, execute step 250.
[0083] Step 240: Determine whether the original image has a meaningless area, and determine the meaningless area of the original image according to the frame.
[0084] Step 250: Determine whether the original image has any meaningless area.
[0085] The technical solution of this embodiment is to obtain an edge image by performing edge extraction on the original image, determine the border in the edge image according to the longitudinal feature value greater than or equal to the longitudinal threshold value and the lateral feature value greater than or equal to the lateral threshold value, and determine the result of determining the meaningless area of the original image according to the border of the edge image. As the edge image can highlight the important features in the original image, reduce the amount of data processing for subsequent image analysis, and make the understanding and recognition of the image easier, therefore, according to the feature value of the edge image, the meaningless area of the original image can be accurately and quickly determined, that is, the accuracy and timeliness of the determination of the meaningless area are taken into consideration.
[0086] Fig.10 The flowchart of a video processing method provided by an embodiment of the present invention is applicable to the case of accurately and quickly determining meaningless areas in a video. The method can be executed by a video processing device, which can be implemented in software and / or hardware, and can be configured in a device, such as a computer. Fig.10 As shown, the method specifically comprises the following steps:
[0087] Step 310: extract edges of each video frame in the video to obtain edge images of each video frame.
[0088] In an embodiment of the present invention, the video may be divided according to a preset time interval to obtain each video frame. For each video frame, edge extraction may be performed on the video frame based on an edge extraction algorithm to obtain an edge image of the video frame.
[0089] Step 320: Determine the result of determining the meaningless area of each video frame according to the longitudinal characteristic value and the transverse characteristic value of the edge image of each video frame.
[0090] In an embodiment of the present invention, for each video frame, the determination result of the meaningless area of the video frame can be determined according to the longitudinal eigenvalue and the transverse eigenvalue of the edge image of the video frame, and the above can be understood as follows: according to the longitudinal eigenvalue greater than or equal to the longitudinal threshold and the transverse eigenvalue greater than or equal to the transverse threshold, the border in the edge image of the video frame is determined. According to the longitudinal eigenvalue greater than or equal to the longitudinal threshold and the transverse eigenvalue greater than or equal to the transverse threshold, the border in the edge image is determined, which can be understood as follows: along the direction from small to large in the transverse coordinate value, the first longitudinal eigenvalue greater than or equal to the longitudinal threshold can be determined as the first target longitudinal eigenvalue, and according to the first target longitudinal eigenvalue, the first target transverse coordinate value of each pixel point corresponding to the first target longitudinal eigenvalue can be determined. Along the direction from large to small in the transverse coordinate value, the first longitudinal eigenvalue greater than or equal to the longitudinal threshold can be determined as the second target longitudinal eigenvalue, and according to the second target longitudinal eigenvalue, the second target transverse coordinate value of each pixel point corresponding to the second target longitudinal eigenvalue can be determined. Along the direction from small to large ordinate values, the first longitudinal feature value greater than or equal to the transverse threshold can be determined as the first target transverse feature value, and the first target longitudinal coordinate value of each pixel point corresponding to the first target transverse feature value can be determined according to the first target transverse feature value. Along the direction from large to small ordinate values, the first longitudinal feature value greater than or equal to the transverse threshold can be determined as the second target transverse feature value, and the second target transverse feature value can be determined according to the second target transverse feature value. The second target longitudinal coordinate value of each pixel point corresponding to the second target transverse feature value. If there are the first target transverse coordinate value, the second target transverse coordinate value, the first target longitudinal coordinate value, and the second target longitudinal coordinate value, the coordinates of the four vertices of the frame can be determined, and the formed frame can be determined according to the coordinates of the four vertices. That is, the line connecting two adjacent vertices can form an edge in the frame; if there are the first target transverse coordinate value, the second target transverse coordinate value, and one of the first target longitudinal coordinate value and the second target longitudinal coordinate value, the coordinates of the two vertices of the frame can be determined, and the two vertices are two adjacent vertices. The formed frame is determined according to the coordinates of the two vertices. At this time, the frame is a single frame. Similarly, if there is a first target ordinate value, a second target ordinate value, and one of the first target abscissa value and the second target abscissa value, then the above can also determine the coordinates of the two vertices of the border, the two vertices are two adjacent vertices, and the border formed is determined according to the coordinates of the two vertices, and the border is a single border. The above shows that the border of the edge image can be determined according to two longitudinal eigenvalues greater than or equal to the longitudinal threshold and at least one transverse eigenvalue greater than or equal to the transverse threshold. Alternatively, the border of the edge image can be determined according to two transverse eigenvalues greater than or equal to the transverse threshold and at least one longitudinal eigenvalue greater than or equal to the longitudinal threshold.It should be noted that the two longitudinal characteristic values greater than or equal to the longitudinal threshold value mentioned above refer to the first target longitudinal characteristic value and the second target longitudinal characteristic value mentioned above, respectively, and the two transverse characteristic values greater than or equal to the transverse threshold value refer to the first target transverse characteristic value and the second target transverse characteristic value mentioned above, respectively. The at least one transverse characteristic value greater than or equal to the transverse threshold value mentioned above refers to the first target transverse characteristic value or the second target transverse characteristic value mentioned above. The at least one longitudinal characteristic value greater than or equal to the longitudinal threshold value refers to the first target longitudinal characteristic value or the second target longitudinal characteristic value mentioned above.
[0091] Based on the above, after determining the border of the edge image, in order to further determine whether the border is a real border, it can be achieved in the following way: confirm whether the border satisfies at least one of the following conditions: condition one, the distance between the edge of the border and the margin in the corresponding edge image is greater than or equal to the distance threshold; condition two, the distance difference between the relative edge of the border and the central axis of the edge image is less than or equal to the error threshold. If one of the above two conditions is met, the border can be determined to be a real border. Based on this, the meaningless area of the original image can be determined according to the border of the edge image. That is, in the case of determining that the border is a real border, the border of the edge image is the border of the original image, and the border represents the meaningless area, thereby determining that the border of the edge image is the meaningless area of the original image. Correspondingly, if both of the above two conditions are not met, it can be determined that the border is not a real border. It is determined that there is no meaningless area in the original image. It should be noted that, according to the above, it can be known that the border in the edge image determined according to the longitudinal feature value greater than or equal to the longitudinal threshold and the lateral feature value greater than or equal to the lateral threshold may be a single border. At this time, it can be considered that the single border can meet the above condition two.
[0092] In addition, in addition to the two conditions described above, another condition can be set, that is, the distance between the relative edges in the frame is greater than or equal to the range distance threshold. That is, the distance between the first target horizontal coordinate value and the second target horizontal coordinate value is greater than or equal to the horizontal range distance threshold, and the distance between the first target vertical coordinate value and the second target vertical coordinate value is greater than or equal to the vertical range distance threshold. Based on the above, if the frame satisfies at least one of the following conditions: the distance between the edge of the frame and the edge in the corresponding edge image is greater than or equal to the distance threshold, the distance difference between the relative edge of the frame and the central axis of the edge image is less than or equal to the error threshold, and the distance between the relative edges of the frame is greater than or equal to the range distance threshold, then the meaningless area of the original image can be determined based on the frame. Otherwise, it is determined that there is no meaningless area in the original image.
[0093] It should be noted that for the understanding of Conditions 1 and 2, please refer to the above description and will not be elaborated here.
[0094] It should also be noted that the above method can be used for each video frame in the video to obtain the determination result of the meaningless area corresponding to the video frame.
[0095] Step 330: Determine the determination result of the meaningless area of the video according to the determination result of the meaningless area of each video frame.
[0096] In the embodiment of the present invention, the determination result of the meaningless region of the video frame is determined according to the determination result of the meaningless region of each video frame, which can be understood as follows: if there is a meaningless region in the video frame of the video, the meaningless region of the video frame is used as the meaningless region of the video. For example, if a video I includes N video frames, that is, I = {I1, I2, ..., I i ,......,I N-1 , I N}, where i=1, 2,..., N-1, N; N≥20. I1, I4, I 11 and I N There are meaningless areas in each of the video frames, and there are no meaningless areas in the other video frames. Based on the above, I1, I4, I 11 and I N The meaningless area of is taken as the meaningless area of the video I.
[0097] In addition, since for a certain video, it may only have meaningless areas at a certain time, that is, there may be meaningless areas only in one or several video frames, and the rest of the video frames are meaningless areas, therefore, in order to determine the existence of meaningless areas of the video, it is possible to consider using a weighted voting method to ensure the sensitivity of determining whether there are meaningless areas in the video while taking into account the overall situation. Specifically, weighted voting can be performed in the following manner: weighted summation is performed on the determination results of the meaningless areas of each video frame to obtain the determination results of the meaningless areas of the video. Corresponding weighting coefficients can be set for the existence of meaningless areas in the video frames and the absence of meaningless areas in the video frames, and weighted summation is performed on the weighted coefficients of each video frame. According to the weighted summation result, the determination results of the meaningless areas of the video are determined. The above-mentioned determination results of the meaningless areas of the video using the weighted voting method can further improve the accuracy of the determination of the meaningless areas.
[0098] It should be noted that the video containing the unintentional area can be marked as a low-quality video to facilitate subsequent other services.
[0099] The technical solution of this embodiment is to extract the edge of each video frame in the video respectively to obtain the edge image of each video frame, determine the determination result of the meaningless area of each video frame according to the longitudinal characteristic value and the transverse characteristic value of the edge image of each video frame, and determine the determination result of the meaningless area of the video according to the determination result of the meaningless area of each video frame. Because the edge image can highlight the important features in the original image, reduce the data processing amount of subsequent image analysis, and make the understanding and recognition of the image easier, therefore, according to the characteristic value of the edge image, it is possible to accurately and quickly determine the meaningless area of the original image, that is, to achieve the balance between accuracy and timeliness in the determination of the meaningless area.
[0100] Optionally, based on the above technical solution, the determination result of the meaningless area of the video is determined according to the determination result of the meaningless area of each video frame, which may specifically include: performing weighted summation on the determination result of the meaningless area of each video frame to obtain the determination result of the meaningless area of the video.
[0101] In an embodiment of the present invention, different weighting coefficients may be set for video frames with meaningless areas and video frames without meaningless areas. Specifically: for video frames with meaningless areas, the weighting coefficient corresponding to the video frame is a positive number. For video frames without meaningless areas, the weighting coefficient corresponding to the video frame is a negative number; or, for video frames with meaningless areas, the weighting coefficient corresponding to the video frame is a negative number. For video frames without meaningless areas, the weighting coefficient corresponding to the video frame is a positive number. Based on the determination result of the meaningless area of each video frame, the weighting coefficient corresponding to each video frame is determined, and the weighting coefficients corresponding to each video frame are summed to obtain a weighted summation result. Based on the weighted summation result, the determination result of the meaningless area of the video is determined.
[0102] It should be noted that, for video frames with meaningless areas, the weighting coefficients corresponding to different video frames can be the same or different, and can be set according to actual conditions, and are not specifically limited here. Similarly, for video frames without meaningless areas, the weighting coefficients corresponding to different video frames can be the same or different, and can be set according to actual conditions, and are not specifically limited here.
[0103] It should also be noted that the weighting coefficient corresponding to each video frame can be determined by each longitudinal eigenvalue and each transverse eigenvalue in the video frame. Specifically, it can be: calculate the average value of each longitudinal eigenvalue and each transverse eigenvalue in the video frame, and set the corresponding weighting coefficient according to the average value. Setting the corresponding weighting coefficient according to the average value can be understood as follows: the weighting coefficient is set based on the principle that the larger the average value, the larger the corresponding weighting coefficient. In addition, the weighting coefficient corresponding to each video frame can also be directly set by the user without considering the situation of each longitudinal eigenvalue and each transverse eigenvalue in the video frame. The setting method of the weighting coefficient corresponding to the video frame can be selected according to the actual situation, and is not specifically limited here.
[0104] Optionally, on the basis of the above technical solution, the determination results of the meaningless areas of each video frame are weighted and summed to obtain the determination results of the meaningless areas of the video, which may specifically include: summing the weighted coefficients corresponding to the video frames with the meaningless areas and the weighted coefficients of the video frames without the meaningless areas. If the weighted sum result is a non-negative number, the weighted coefficients corresponding to the video frames with the meaningless areas are positive numbers, and the weighted coefficients corresponding to the video frames without the meaningless areas are negative numbers, then it is determined that the video has the meaningless areas. Otherwise, it is determined that the video does not have the meaningless areas. Alternatively, if the weighted sum result is a non-positive number, the weighted coefficients corresponding to the video frames with the meaningless areas are negative numbers, and the weighted coefficients corresponding to the video frames without the meaningless areas are positive numbers, then it is determined that the video has the meaningless areas. Otherwise, it is determined that the video does not have the meaningless areas.
[0105] In an embodiment of the present invention, the weighted coefficients corresponding to the video frames with meaningless areas and the weighted coefficients of the video frames without meaningless areas are summed to obtain a weighted summation result. If the weighted coefficient corresponding to the video frames with meaningless areas is a positive number, and the weighted coefficient corresponding to the video frames without meaningless areas is a negative number, then if the weighted summation result is a non-negative number, it can be said that the video has a meaningless area, and the meaningless area of the video can be the meaningless area of the target video frame, and the target video frame can refer to the video frame with the meaningless area. On the contrary, if the weighted summation result is a negative number, it can be said that the video does not have a meaningless area. Or,
[0106] If the weighted coefficient corresponding to the video frame with the meaningless area is negative, and the weighted coefficient corresponding to the video frame without the meaningless area is positive, then if the weighted sum result is a non-positive number, it can be said that the video has a meaningless area, and the meaningless area of the video can be the meaningless area of the target video frame, and the target video frame can refer to the video frame with the meaningless area. On the contrary, if the weighted sum result is a positive number, it can be said that the video does not have a meaningless area.
[0107] For example, if a video I includes N video frames, that is, I = {I1, I2, ..., I i ,......,I N-1 , I N}, where i=1, 2,..., N-1, N; N≥20. I1, I4, I 11 and I N There are meaningless areas in each of the video frames, and there are no meaningless areas in the other video frames. Based on the above, the weighting coefficient corresponding to the video frame with meaningless areas is α i , i = 1, 4, 11, N, the weight coefficient corresponding to the video frame without meaningless area is β j , j = 2, 3, 5, ..., 10, 12, ..., N-1. Sum the weighted coefficients corresponding to each video frame, and the weighted summation result is T = ∑α i +∑β j .
[0108] Optionally, based on the above technical solution, the non-negative number is a value greater than or equal to the positive weighted sum threshold. The non-positive number is a value less than or equal to the negative weighted sum threshold.
[0109] In an embodiment of the present invention, in order to further improve the accuracy of the determination result of the meaningless area of the video, a positive weighted sum threshold and a negative weighted sum threshold can be set. If the weighted coefficient corresponding to the video frame with the meaningless area is a positive number, and the weighted coefficient corresponding to the video frame without the meaningless area is a negative number, then if the weighted sum result is a value greater than or equal to the positive weighted sum threshold, it can be explained that the video has a meaningless area, and the meaningless area of the video can be the meaningless area of the target video frame, and the target video frame can refer to the video frame with the meaningless area. On the contrary, if the weighted sum result is a value less than the positive weighted sum threshold, it can be explained that the video does not have a meaningless area. Alternatively, if the weighted coefficient corresponding to the video frame with the meaningless area is a negative number, and the weighted coefficient corresponding to the video frame without the meaningless area is a positive number, then if the weighted sum result is a value less than or equal to the negative weighted sum threshold, it can be explained that the video has a meaningless area, and the meaningless area of the video can be the meaningless area of the target video frame, and the target video frame can refer to the video frame with the meaningless area. On the contrary, if the weighted sum result is a value greater than the negative weighted sum threshold, it can be indicated that there is no meaningless area in the video.
[0110] Optionally, on the basis of the above technical solution, the determination result of the meaningless area of each video frame is determined according to the longitudinal eigenvalue and the transverse eigenvalue of the edge image of each video frame, which may specifically include: for each video frame, the border in the edge image of the video frame is determined according to the longitudinal eigenvalue greater than or equal to the longitudinal threshold and the transverse eigenvalue greater than or equal to the transverse threshold. If the border satisfies at least one of the following conditions: the distance between the edge of the border and the edge in the edge image of the corresponding video frame is greater than or equal to the distance threshold, and the distance difference between the relative edge of the border and the central axis of the edge image of the video frame is less than or equal to the error threshold, then the meaningless area of the video frame is determined according to the border. Otherwise, it is determined that there is no meaningless area in the video frame.
[0111] In the embodiment of the present invention, the specific description of the determination result of the meaningless area of the original image according to the longitudinal eigenvalue and the lateral eigenvalue of the edge image can be referred to above, and no further details will be given here. The original image mentioned above is each video frame mentioned here.
[0112] Optionally, based on the above technical solution, edge extraction is performed on each video frame to obtain an edge image of each video frame, which may specifically include: based on an edge extraction algorithm, edge extraction is performed on each video frame to obtain an edge image of each video frame.
[0113] In the embodiment of the present invention, the edge extraction algorithm based on the original image is referred to above, and the specific description of the edge image of the original image is obtained, which will not be repeated here. The original image mentioned above is each video frame mentioned here.
[0114] Fig.11 This is a flowchart of another video processing method provided by an embodiment of the present invention. This embodiment is a specific example of the above embodiment. This embodiment can be applied to the situation of accurately and quickly determining the meaningless area in the video. The method can be executed by a video processing device. The device can be implemented in software and / or hardware. The device can be configured in a device, such as a computer. Fig.11 As shown, the method specifically comprises the following steps:
[0115] Step 401: Based on an edge extraction algorithm, edge extraction is performed on each video frame in the video to obtain an edge image of each video frame.
[0116] Step 402: for each video frame, determine a border in an edge image of the video frame according to a longitudinal feature value greater than or equal to a longitudinal threshold and a transverse feature value greater than or equal to a transverse threshold.
[0117] Step 403: Whether the distance between the edge of the border and the edge in the edge image of the corresponding video frame is greater than or equal to a distance threshold, and / or whether the distance difference between the relative edge of the border and the central axis of the edge image of the video frame is less than or equal to an error threshold; if so, execute step 404; if not, execute step 405.
[0118] Step 404: Determine whether a meaningless area exists in the video frame, and determine the meaningless area of the video frame according to the border.
[0119] Step 405: Determine whether the video frame has no meaningless area.
[0120] Step 406: sum the weighted coefficients corresponding to the video frames with the meaningless regions and the weighted coefficients corresponding to the video frames without the meaningless regions to obtain a weighted summation result.
[0121] Step 407, whether the weighted sum result is a non-negative number, the weighted coefficient corresponding to the video frame with meaningless areas is a positive number, and the weighted coefficient corresponding to the video frame without meaningless areas is a negative number; if so, execute step 409; if not, execute step 410.
[0122] Step 408, whether the weighted sum result is a non-positive number, the weighted coefficient corresponding to the video frame with meaningless areas is a negative number, and the weighted coefficient corresponding to the video frame without meaningless areas is a positive number; if so, execute step 409; if not, execute step 410.
[0123] Step 409: Determine whether the video has a meaningless area, where the meaningless area of the video is determined by the meaningless area of each video frame.
[0124] Step 410: Determine whether the video has any meaningless area.
[0125] In the embodiments of the present invention, in order to better understand the technical solutions of the embodiments of the present invention, the following will be described through specific examples, specifically: Fig.12 As shown, a schematic diagram of the application of a video processing method is given. Fig.12 In the example, the video includes five video frames, namely, video frame 1, video frame 2, video frame 3, video frame 4 and video frame 5. The weight coefficients corresponding to the video frames with meaningless regions are set to be positive numbers, and the weight coefficients corresponding to the video frames without meaningless regions are set to be negative numbers.
[0126] For each video frame, the determination result of the meaningless area of the video frame is obtained according to each longitudinal eigenvalue and each transverse eigenvalue of the video frame. Among them, video frame 1, video frame 2 and video frame 3 are determined to have meaningless areas, and the meaningless areas are borders. Video frame 4 and video frame 5 do not have meaningless areas. The weighted coefficients corresponding to each video frame are summed to obtain a weighted summation result. If the weighted summation result is determined to be a positive number, it is determined that the video has a meaningless area, and the meaningless areas of the video are the meaningless areas of video frame 1, the meaningless areas of video frame 2 and the meaningless areas of video frame 3.
[0127] The technical solution of this embodiment is to extract the edge of each video frame in the video respectively to obtain the edge image of each video frame, determine the determination result of the meaningless area of each video frame according to the longitudinal characteristic value and the transverse characteristic value of the edge image of each video frame, and determine the determination result of the meaningless area of the video according to the determination result of the meaningless area of each video frame. Because the edge image can highlight the important features in the original image, reduce the data processing amount of subsequent image analysis, and make the understanding and recognition of the image easier, therefore, according to the characteristic value of the edge image, the meaningless area of the original image can be accurately and quickly determined, that is, the accuracy and timeliness of the determination of the meaningless area are taken into consideration. In addition, the determination result of the meaningless area of the video is determined based on the weighted voting method, which further improves the accuracy of the determination of the meaningless area.
[0128] Fig.13 This is a schematic diagram of the structure of an image processing device provided by an embodiment of the present invention. This embodiment is applicable to the situation of accurately and quickly determining meaningless areas in an image. The device can be implemented in software and / or hardware, and the device can be configured in a device, such as a computer. Fig.13 As shown, the device specifically includes:
[0129] The edge image acquisition module 510 is used to extract the edge of the original image to obtain an edge image.
[0130] The first determination result module 520 is used to determine the determination result of the meaningless area of the original image according to the longitudinal characteristic value and the transverse characteristic value of the edge image.
[0131] The technical solution of this embodiment is to obtain an edge image by performing edge extraction on the original image, and determine the determination result of the meaningless area of the original image according to the longitudinal eigenvalue and the transverse eigenvalue of the edge image. As the edge image can highlight the important features in the original image, reduce the amount of data processing for subsequent image analysis, and make the understanding and recognition of the image easier, therefore, according to the eigenvalue of the edge image, the meaningless area of the original image can be accurately and quickly determined, that is, the accuracy and timeliness of the determination of the meaningless area are taken into consideration.
[0132] Optionally, based on the above technical solution, the first determination result module 520 may specifically include:
[0133] The frame determination submodule is used to determine the frame in the edge image according to the longitudinal characteristic value greater than or equal to the longitudinal threshold and the transverse characteristic value greater than or equal to the transverse threshold.
[0134] The first determination result submodule is used to determine the meaningless area of the original image according to the border if the border satisfies at least one of the following conditions: the distance between the edge of the border and the edge in the corresponding edge image is greater than or equal to a distance threshold, and the distance difference between the relative edge of the border and the central axis of the edge image is less than or equal to an error threshold; otherwise, determine that there is no meaningless area in the original image.
[0135] Optionally, based on the above technical solution, the edge image acquisition module 510 may specifically include:
[0136] The edge image acquisition submodule is used to extract the edge of the original image based on the edge extraction algorithm to obtain the edge image.
[0137] The image processing apparatus configured on a device provided in an embodiment of the present invention can execute the image processing method applied to a device provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.
[0138] Fig.14 The schematic diagram of the structure of a video processing device provided by an embodiment of the present invention is applicable to the situation of accurately and quickly determining the meaningless area in the video. The device can be implemented in software and / or hardware, and the device can be configured in a device, such as a computer. Fig.14 As shown, the device specifically includes:
[0139] The video frame extraction module 610 is used to extract each video frame from the video as an original image.
[0140] The second determination result module 620 is used to execute the image processing method according to the embodiment of the present invention to determine the determination result of the meaningless area of each video frame.
[0141] The third determination result module 630 is used to determine the determination result of the meaningless area of the video according to the determination result of the meaningless area of each video frame.
[0142] The technical solution of this embodiment is to extract the edge of each video frame in the video respectively to obtain the edge image of each video frame, determine the determination result of the meaningless area of each video frame according to the longitudinal characteristic value and the transverse characteristic value of the edge image of each video frame, and determine the determination result of the meaningless area of the video according to the determination result of the meaningless area of each video frame. Because the edge image can highlight the important features in the original image, reduce the data processing amount of subsequent image analysis, and make the understanding and recognition of the image easier, therefore, according to the characteristic value of the edge image, it is possible to accurately and quickly determine the meaningless area of the original image, that is, to achieve the balance between accuracy and timeliness in the determination of the meaningless area.
[0143] Optionally, based on the above technical solution, the third determination result module 630 may specifically include:
[0144] The second determination result submodule is used to perform weighted summation on the determination results of the meaningless regions of each video frame to obtain the determination results of the meaningless regions of the video.
[0145] Optionally, based on the above technical solution, the second determination result submodule may specifically include:
[0146] A weighted coefficient sum result determination unit, used for summing the weighted coefficients corresponding to the video frames with the meaningless regions and the weighted coefficients of the video frames without the meaningless regions;
[0147] The first result determination unit is used to determine that the video has a meaningless region if the weighted sum result is a non-negative number, the weight coefficient corresponding to the video frame with the meaningless region is a positive number, and the weight coefficient corresponding to the video frame without the meaningless region is a negative number. Otherwise, it is determined that the video does not have a meaningless region. Or,
[0148] The second result determination unit is used to determine that the video has a meaningless region if the weighted sum result is a non-positive number, the weight coefficient corresponding to the video frame with the meaningless region is a negative number, and the weight coefficient corresponding to the video frame without the meaningless region is a positive number. Otherwise, determine that the video does not have a meaningless region.
[0149] The video processing device configured on the device provided by the embodiment of the present invention can execute the video processing method applied to the device provided by any embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method.
[0150] Fig.15 A schematic diagram of the structure of a device provided in an embodiment of the present invention. Fig.15 The device shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention. Fig.15As shown, the device provided by the embodiment of the present invention includes a processor 71, a memory 72, an input device 73 and an output device 74; the number of processors 71 in the device can be one or more. Fig.15 A processor 71 is taken as an example; the processor 71, memory 72, input device 73 and output device 74 in the device can be connected by a bus or other means. Fig.15 The example of connecting through bus is taken in the following.
[0151] The memory 72, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the device image processing method and video processing method in the embodiments of the present invention (for example, using the edge image acquisition module 510 and the first determination result module 530 in the image processing device, and using the video frame extraction module 610, the second determination result module 620 and the third determination result module 630 in the video processing device). The processor 71 executes various functional applications and data processing by running the software programs, instructions and modules stored in the memory 72, such as implementing the image processing method and video processing method applied to the device provided in the embodiments of the present invention.
[0152] The memory 72 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the device, etc. In addition, the memory 72 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 72 may further include a memory remotely arranged relative to the processor 71, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0153] The input device 73 may be used to receive digital or character information input by a user to generate key signal input related to user settings and function control of the device. The output device 74 may include a display device such as a display screen.
[0154] Of course, those skilled in the art can understand that the processor can also implement the technical solutions of the image processing method and video processing method applied to the device provided in any embodiment of the present invention. The hardware structure and function of the device can refer to the explanation of the content of the embodiment.
[0155] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, an image processing method provided by the embodiment of the present invention is implemented. The method includes:
[0156] Perform edge extraction on the original image to obtain an edge image.
[0157] The determination result of the meaningless area of the original image is determined according to the longitudinal eigenvalue and the transverse eigenvalue of the edge image.
[0158] The computer storage medium of the embodiment of the present invention may adopt any combination of one or more computer-readable media. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.
[0159] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0160] The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, radio frequency, etc., or any suitable combination of the foregoing.
[0161] The computer program code for performing the operation of the present invention may be written in one or more programming languages or a combination thereof, such as C language and Python, etc. The program code may be executed on a computer or server.
[0162] The embodiment of the present invention further provides another computer-readable storage medium, wherein the computer-executable instructions are used to execute a video processing method when executed by a computer processor, the method comprising:
[0163] Each video frame is extracted from the video as an original image.
[0164] The image processing method according to the embodiment of the present invention is executed to determine the determination result of the meaningless area of each video frame.
[0165] According to the determination result of the meaningless area of each video frame, the determination result of the meaningless area of the video is obtained.
[0166] Of course, the computer-readable storage medium provided in the embodiment of the present invention, whose computer executable instructions are not limited to the method operations described above, can also execute the related operations in the image processing method and video processing method of the device provided in any embodiment of the present invention. For the introduction of the storage medium, please refer to the content explanation in the embodiment.
[0167] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims
1. An image processing method, characterized in that: include: Extract the edge of the original image to obtain an edge image; Determining a determination result of a meaningless area of the original image according to the longitudinal eigenvalue and the transverse eigenvalue of the edge image; The determining result of the meaningless area of the original image according to the longitudinal characteristic value and the transverse characteristic value of the edge image includes: Determine the border in the edge image according to the longitudinal characteristic value greater than or equal to the longitudinal threshold and the transverse characteristic value greater than or equal to the transverse threshold; wherein the characteristic value of the edge image is determined by the pixel value of the pixel point, and the edge image is mapped to a plane rectangular coordinate system, and the longitudinal characteristic value of the edge image is determined in the following manner: along the transverse direction or X-axis direction of the edge image, according to the pixel value of each pixel point with the same transverse coordinate value, the longitudinal characteristic value of the edge image is determined, and the transverse characteristic value of the edge image is determined in the following manner: along the longitudinal direction or Y-axis direction of the edge image, according to the pixel value of each pixel point with the same longitudinal coordinate value, the transverse characteristic value of the edge image is determined; If the border satisfies the following conditions: the distance between the edge of the border and the edge in the corresponding edge image is greater than or equal to a distance threshold, the distance difference between the relative edge of the border and the central axis of the edge image is less than or equal to an error threshold, and the distance between the relative edges of the border is greater than or equal to a range distance threshold, then the meaningless area of the original image is determined based on the border; otherwise, it is determined that there is no meaningless area in the original image.
2. The method according to claim 1, characterized in that The step of extracting the edge of the original image to obtain the edge image comprises: Based on the edge extraction algorithm, the edge of the original image is extracted to obtain an edge image.
3. A video processing method, characterized in that: include: Extract each video frame from the video as an original image; Execute the image processing method as described in any one of claims 1-2 to determine the determination result of the meaningless area of each video frame; According to the determination result of the meaningless area of each video frame, the determination result of the meaningless area of the video is obtained.
4. The method according to claim 3, characterized in that The step of obtaining the determination result of the meaningless area of the video according to the determination result of the meaningless area of each video frame comprises: The determination results of the meaningless regions of each video frame are weighted and summed to obtain the determination results of the meaningless regions of the video.
5. The method according to claim 4, characterized in that The weighted summing of the determination results of the meaningless regions of each video frame to obtain the determination results of the meaningless regions of the video includes: The weight coefficients corresponding to the video frames with meaningless regions and the weight coefficients corresponding to the video frames without meaningless regions are summed; If the weighted sum result is a non-negative number, the weighted coefficient corresponding to the video frame with the meaningless region is a positive number, and the weighted coefficient corresponding to the video frame without the meaningless region is a negative number, then it is determined that the video has a meaningless region; otherwise, it is determined that the video does not have a meaningless region; or, If the weighted sum result is a non-positive number, the weighting coefficient corresponding to the video frame with the meaningless area is a negative number, and the weighting coefficient corresponding to the video frame without the meaningless area is a positive number, then it is determined that the video has a meaningless area; otherwise, it is determined that the video does not have a meaningless area.
6. An image processing device, characterized in that: include: The edge image acquisition module is used to extract the edge of the original image to obtain the edge image; A first determination result module, used to determine a determination result of a meaningless area of the original image according to a longitudinal characteristic value and a transverse characteristic value of the edge image; The first determination result module includes: A frame determination submodule, used for determining the frame in the edge image according to a longitudinal characteristic value greater than or equal to a longitudinal threshold and a transverse characteristic value greater than or equal to a transverse threshold; wherein the characteristic value of the edge image is determined by the pixel value of a pixel point, and the edge image is mapped into a plane rectangular coordinate system, and the longitudinal characteristic value of the edge image is determined in the following manner: along the transverse or X-axis direction of the edge image, the longitudinal characteristic value of the edge image is determined according to the pixel value of each pixel point having the same transverse coordinate value, and the transverse characteristic value of the edge image is determined in the following manner: along the longitudinal or Y-axis direction of the edge image, the transverse characteristic value of the edge image is determined according to the pixel value of each pixel point having the same longitudinal coordinate value; The first determination result submodule is used to determine the meaningless area of the original image according to the border if the border satisfies the following conditions: the distance between the edge of the border and the edge in the corresponding edge image is greater than or equal to a distance threshold, the distance difference between the relative edge of the border and the central axis of the edge image is less than or equal to an error threshold, and the distance between the relative edges of the border is greater than or equal to a range distance threshold; otherwise, determine that the original image does not have a meaningless area.
7. A video processing device, characterized in that: include: A video frame extraction module is used to extract each video frame from the video as an original image; A second determination result module, configured to execute the image processing method according to any one of claims 1 to 2, and determine a determination result of a meaningless area of each video frame; The third determination result module is used to determine the determination result of the meaningless area of the video according to the determination result of the meaningless area of each video frame.
8. An electronic device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: include: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.
