A detection method, device, electronic device and medium

Abstract

Embodiments of the present disclosure provide a detection method and device, electronic equipment and a medium. The method comprises: determining boundary candidate values of a video frame in a target direction; filtering each boundary candidate value based on a first region and a second region corresponding to the boundary candidate value, the first region and the second region being located on both sides of a corresponding candidate boundary, and the first region and the second region being two regions along a direction perpendicular to the target direction; and determining a video type of a video to which the video frame belongs and a boundary target value of the video frame based on the filtered boundary candidate values. The method filters each boundary candidate value based on the first region and the second region corresponding to the boundary candidate value, can accurately filter the boundary candidate value to obtain the filtered boundary candidate value, and can determine the boundary target value of the video frame while determining the video type of the video to which the video frame belongs based on the filtered boundary candidate value.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to video image technology, and in particular, to a detection method and device, electronic equipment and medium. BACKGROUND

[0002] Existing short video applications have cultivated user habits of playing vertical screen videos. When the source video is horizontal screen content, the source video is converted into vertical screen content through video editing, and at the same time, the peripheral area of the video content is filled through methods such as black edge filling and subtitle adding, so that the played video is displayed on the electronic device without distortion. Generally, the upper and lower filling parts are instrumental content, and the middle part is the core video content. Such a video with an upper-middle-lower structure can be referred to as a sandwich video.

[0003] In order to better improve the video quality and focus on the video content itself, it is necessary to detect the video type of the source video, and at the same time, after determining that the video type of the played video is a sandwich video, there is a need to segment the video content.

[0004] The existing detection method mainly trains a classification and recognition algorithm through a certain scale of labeled data set to distinguish whether the video type is a normal video or a sandwich video. However, the above detection method can only distinguish the video type and cannot accurately locate the boundary of the video frame. SUMMARY

[0005] The present disclosure provides a detection method, device, electronic equipment and medium to determine the boundary target value of the video frame while determining the video type of the video to which the video frame belongs.

[0006] In a first aspect, the embodiments of the present disclosure provide a detection method, comprising:

[0007] determining a boundary candidate value of a video frame in a target direction;

[0008] filtering each boundary candidate value based on a first region and a second region corresponding to the boundary candidate value, the first region and the second region being located on both sides of the corresponding candidate boundary, and the first region and the second region being two regions along a direction perpendicular to the target direction;

[0009] determining the video type of the video to which the video frame belongs and the boundary target value of the video frame based on the filtered boundary candidate value.

[0010] In a second aspect, the embodiments of the present disclosure also provide a detection device, comprising:

[0011] a first determination module configured to determine a boundary candidate value of a video frame in a target direction;

[0012] a filtering module configured to filter each of the boundary candidate values based on a first region and a second region corresponding to the boundary candidate values, the first region and the second region being located on two sides of a corresponding candidate boundary, and the first region and the second region being two regions along a direction perpendicular to the target direction;

[0013] a second determining module configured to determine a video type of a video to which the video frame belongs and a boundary target value of the video frame based on the filtered boundary candidate values.

[0014] In a third aspect, an electronic device is provided, including:

[0015] one or more processors;

[0016] a memory configured to store one or more programs,

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the detection method according to the embodiments of the present disclosure.

[0018] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the detection method according to the embodiments of the present disclosure is implemented.

[0019] The detection method, device, electronic device and medium provided by the embodiments of the present disclosure include: determining boundary candidate values of a video frame in a target direction; filtering each of the boundary candidate values based on a first region and a second region corresponding to the boundary candidate values, the first region and the second region being located on two sides of a corresponding candidate boundary, and the first region and the second region being two regions along a direction perpendicular to the target direction; and determining a video type of a video to which the video frame belongs and a boundary target value of the video frame based on the filtered boundary candidate values. With this technical solution, each boundary candidate value is filtered based on a first region and a second region corresponding to the boundary candidate values, so that the filtered boundary candidate values can be accurately filtered. Meanwhile, based on the filtered boundary candidate values, the video type of the video to which the video frame belongs is determined, and the boundary target value of the video frame is also determined. BRIEF DESCRIPTION OF DRAWINGS

[0020] The above and other features and advantages of the embodiments of the present disclosure will become more apparent by referring to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals are used to refer to the same or similar elements. It should be understood that the drawings are schematic, and the original and elements are not necessarily drawn according to the scale.

[0021] Figure 1This is a schematic flowchart of a detection method provided in an embodiment of the present disclosure;

[0022] Figure 2 This is a schematic flowchart of a detection method provided in an embodiment of the present disclosure;

[0023] Figure 3 A schematic diagram of a detection method provided in an embodiment of this disclosure;

[0024] Figure 4 This is a schematic diagram of the overall process of a detection method provided in an embodiment of the present disclosure;

[0025] Figure 5 This is a schematic diagram of a video frame provided in an embodiment of the present disclosure;

[0026] Figure 6 This is a schematic diagram of a pixel map provided in an embodiment of the present disclosure;

[0027] Figure 7 This is a schematic diagram of another pixel map provided in an embodiment of the present disclosure;

[0028] Figure 8 This is a schematic diagram of a percentage information provided in an embodiment of the present disclosure;

[0029] Figure 9 This is a schematic diagram illustrating another proportion information provided in an embodiment of the present disclosure;

[0030] Figure 10 This is a schematic diagram of the structure of a detection device provided in an embodiment of the present disclosure;

[0031] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0032] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0033] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0034] As used herein, the term "includes" and its variants are to be read to be analogous to "comprises," or "comprising." The term "based on" is to be read as "based, at least in part, on." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments." Related terms have corresponding meanings.

[0035] It should be noted that the terms "first", "second", etc. mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0036] It should be noted that the terms "one", "multiple" mentioned in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that "one or more" should be understood unless otherwise explicitly indicated in the context.

[0037] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0038] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type, use range, use scenario, etc. of the personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained through appropriate means according to relevant laws and regulations.

[0039] For example, in response to receiving the active request of the user, prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using the personal information of the user. Thus, the user can voluntarily choose whether to provide personal information to the software or hardware such as electronic device, application program, server or storage medium, etc. that performs the operation of the technical solutions of the present disclosure according to the prompt information.

[0040] As an optional but not limited implementation manner, in response to receiving the active request of the user, the manner of sending prompt information to the user may, for example, be the manner of pop-up window, and the prompt information may, for example, be presented in the form of text in the pop-up window. In addition, the pop-up window may, for example, carry selection controls for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0041] It can be understood that the above notification and user authorization process is only illustrative, and does not limit the implementation manner of the present disclosure, and other manners meeting the relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0042] It can be understood that the data involved in the technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the corresponding laws, regulations and relevant provisions.

[0043] Figure 1 A flowchart of a detection method provided by the embodiment of the disclosure is provided. The embodiment of the disclosure is applicable to the case of detecting a video to which a video frame belongs. The method can be executed by a detection device. The device can be implemented in the form of software and / or hardware. Optionally, the device is implemented by an electronic device, which can be a mobile terminal, a PC terminal or a server.

[0044] It can be considered that, for the purpose of high-definition playing and UI interaction, the existing short video application has cultivated the user habit of playing vertical screen videos. When the source video is a horizontal screen content, the user will convert it into a vertical screen content through video editing. At the same time, the user will fill the area outside the video content through the ways of filling black edges, adding subtitles, glass mask filling and taking pictures or moving pictures as backgrounds, so as to display the played video on the electronic device without distortion.

[0045] Generally, the upper and lower filling parts are instrumental content, and the middle part is the core video content. Such a video with an upper-middle-lower structure can be called a sandwich video. In the actual video processing link, in order to better improve the quality of the video and focus on the video content itself, it is necessary to distinguish the sandwich video from the ordinary video. At the same time, after determining that the played video is a sandwich video, there is a need to segment the video content. At present, the sandwich video content recognition and segmentation algorithm can serve different business scenarios such as video transcoding optimization, server enhancement processing algorithm, quality evaluation algorithm (inhibiting non-video content interference) and horizontal screen playing (full screen playing of pseudo-horizontal screen video content).

[0046] The existing detection method of the sandwich video is mainly based on the deep learning method. The ordinary video and the sandwich video are distinguished through artificial labeling, that is, the algorithm for classification and recognition is trained through obtaining a certain scale of labeled data set, so as to distinguish whether the video is an ordinary video or a sandwich video. However, the above detection method can only distinguish the video type and cannot accurately locate the boundary of the central dynamic content of the sandwich video. Moreover, the deep learning algorithm is a data-driven black box algorithm and has no interpretability, so it is impossible to further analyze the reasons for the misjudged video type for further adjustment and optimization.

[0047] Based on this, the embodiment of the disclosure provides a detection method, which comprises:

[0048] S110, determining a boundary candidate value of the video frame in a target direction.

[0049] The video frame can be considered as a frame image in the detected video, and the number of video frames can be one or more. The video frames can be obtained by frame extraction on the detected video, and the specific process of the frame extraction is not limited herein as long as the video frames can be obtained.

[0050] The target direction can be a preset direction for determining the boundary candidate value. Optionally, the target direction can be a horizontal direction, such as the direction of the x-axis in the screen coordinate system. The boundary candidate value can represent the position of the boundary preliminarily determined in the target direction, such as the position of the boundary in the direction perpendicular to the target direction.

[0051] Specifically, the boundary candidate value of the video frame in the target direction can be preliminarily determined, so as to determine the video type and the boundary target value of the video frame subsequently. The specific method for determining the boundary candidate value is not limited, such as determining according to each pixel in the video frame, or determining according to different determination methods corresponding to different contents in the video frame, which is not further expanded herein.

[0052] S120, filtering each boundary candidate value based on the first region and the second region corresponding to the boundary candidate value.

[0053] The first region and the second region are located on both sides of the corresponding candidate boundary, and the first region and the second region are two regions in the direction perpendicular to the target direction, that is, the first region and the second region can be considered as regions with the boundary candidate value as the side length, and the size and shape of the first region and the second region are not limited herein.

[0054] In one embodiment, the size and shape of the first region and the second region are the same, the side of the first region and the second region along the target direction is equal in length to the side of the video frame along the target direction, and the side of the first region and the second region in the height direction is smaller than the side of the video frame in the height direction, the height direction being the direction perpendicular to the target direction.

[0055] The size and shape of the first region and the second region can be the same, the side of the first region and the second region along the target direction can be equal in length to the side of the video frame along the target direction, and the side of the first region and the second region in the height direction can be smaller than the side of the video frame in the height direction, such as that the first region can be a rectangle with the boundary candidate value as the lower side length, the second region can be a rectangle with the boundary candidate value as the upper side length, and the height of the first region and the second region is smaller than the height of the video frame (i.e., the side in the height direction).

[0056] It can be considered that after obtaining the plurality of boundary candidate values through the above step, the boundary candidate values need to be filtered to obtain filtered boundary candidate values. For example, each boundary candidate value can be filtered based on the first region and the second region corresponding to the boundary candidate value to filter out the interfering boundary candidate value. The filtering means is not limited. For example, whether to filter the boundary candidate value can be determined according to the difference information of the pixel values of the first region and the second region corresponding to the boundary candidate value. Whether to filter the boundary candidate value can also be determined according to the main color difference value of the pixel values of the first region and the second region corresponding to the boundary candidate value. The filtered boundary candidate value can also be obtained by combining the above two methods. The embodiment is not limited in this regard.

[0057] In S130, the video type of the video to which the video frame belongs and the boundary target value of the video frame are determined based on the filtered boundary candidate value.

[0058] The video type can refer to the type of the video to which the video frame belongs, such as a first type (i.e., a video with the same display ratio of video content and display area and no padding area in the periphery of the video), such as a normal video or a second type (i.e., a video with a display ratio of video content smaller than a display ratio of a display area and a padding area in the periphery of the video), such as a sandwich video. The boundary target value can represent the position of the boundary of the final determined video content in the video frame. The boundary target value can be selected from the boundary candidate value.

[0059] Specifically, the video type of the video to which the video frame belongs and the boundary target value of the video frame can be determined based on the filtered boundary candidate value. For example, the video type of the video to which the video frame belongs and the boundary target value of the video frame can be directly determined based on the filtered boundary candidate value of a single video frame. The selection of the single video frame can be determined according to the actual situation. The video type of the video to which the video frame belongs and the boundary target value of the video frame can also be determined by comprehensively considering the filtered boundary candidate values corresponding to multiple video frames. This will not be further expanded here, as long as the video type of the video to which the video frame belongs and the boundary target value of the video frame can be obtained.

[0060] For example, the video frame type can be determined based on the number of boundary candidate values. For example, when the number of boundary candidate values is less than a set value (e.g., 1), it can be considered as a first type. When the number of boundary candidate values is greater than or equal to the set value, it can be considered as a second type. Alternatively, when the number of boundary candidate values is greater than or equal to the set value, the video type and the boundary target value can be determined in combination with the filtered boundary candidate values determined by multiple video frames.

[0061] The method provided by the embodiment of the present disclosure comprises: determining a boundary candidate value of a video frame in a target direction; filtering each boundary candidate value based on a first region and a second region corresponding to the boundary candidate value, wherein the first region and the second region are located on two sides of the corresponding candidate boundary, and the first region and the second region are two regions in a direction perpendicular to the target direction; and determining a video type of a video to which the video frame belongs and a boundary target value of the video frame based on the filtered boundary candidate value. By filtering each boundary candidate value based on the first region and the second region corresponding to the boundary candidate value, the filtered boundary candidate value can be accurately obtained. Meanwhile, based on the filtered boundary candidate value, the video type of the video to which the video frame belongs can be determined, and the boundary target value of the video frame can also be determined.

[0062] In one embodiment, the video frame is a plurality of video frames in the video, and the determination of the video type of the video to which the video frame belongs and the boundary target value of the video frame based on the filtered boundary candidate value comprises:

[0063] determining an average candidate value based on the filtered boundary candidate value corresponding to each video frame;

[0064] if the average candidate value is less than 1, determining the video type of the video to which the video frame belongs as a first type, wherein the display ratio of the video content of the first type is the same as the display ratio of the display region;

[0065] if the average candidate value is greater than or equal to 1, determining the video type of the video to which the video frame belongs and the boundary target value of the video frame based on each boundary candidate value.

[0066] The average candidate value can be considered as an average value of the number of the filtered boundary candidate values corresponding to the plurality of video frames. In this embodiment, the boundary candidate value included in each video frame can be determined, and then the average value of all the boundary candidate values is determined as the average candidate value. For example, the first frame includes two boundary candidate values, the second frame includes one boundary candidate value, and the third frame includes two boundary candidate values. The average candidate value can be obtained by dividing the sum of the boundary candidate values by the number of frames.

[0067] The first type and the second type can be considered as the type of the video to which the video frame belongs. For example, the first type can be a normal video, and the second type can be a sandwich video.

[0068] In one embodiment, when the video frame is a plurality of video frames in the video, the filtered boundary candidate value can be determined for each video frame first, then the average candidate value of the plurality of video frames can be determined based on the number of the filtered boundary candidate values corresponding to each video frame, the size of the average candidate value can be judged, and the video type of the video to which the video frame belongs can be determined according to the judgment result.

[0069] For example, when the average candidate value is less than 1, it can be considered that the number of boundaries in the video frame is less than 1, and it can be considered that the video type of the video to which the video frame belongs is the first type. The display ratio of the video content of the first type is the same as the display ratio of the display area, that is, the video content of the first type is displayed full screen on the display area of the electronic device; when the average candidate value is greater than or equal to 1, it can be considered that the number of boundaries in the video frame is greater than or equal to 1, and the video type of the video to which the video frame belongs and the boundary target value of the video frame can be determined based on the boundary candidate values. The process of determination is not further elaborated.

[0070] In an embodiment, when the video type and the boundary target value are determined based on the boundary candidate values, the concentration of the distribution of the boundary candidate values can be determined to determine the video type. If the concentration is greater than a set threshold, it can be considered that the video type is the second type, and the boundary target value is obtained by mathematical operation based on the boundary candidate values; otherwise, it is determined that the video type is the first type.

[0071] In an embodiment, the determination of the video type of the video to which the video frame belongs and the boundary target value of the video frame based on the boundary candidate values comprises:

[0072] quantizing each of the boundary candidate values at a second quantization interval;

[0073] determining a quantized boundary candidate value corresponding to a first boundary;

[0074] determining a quantized boundary candidate value corresponding to a second boundary, the first boundary and the second boundary being video content, and the first boundary and the second boundary being boundaries along the target direction respectively;

[0075] determining a first boundary candidate value corresponding to the first boundary before quantization;

[0076] determining a second boundary candidate value corresponding to the second boundary before quantization;

[0077] If the proportion of the first boundary candidate value and the second boundary candidate value is greater than a set threshold, it is determined that the type of the video to which the video frame belongs is the second type, and the mean of the first boundary candidate value and the mean of the second boundary candidate value are determined as the boundary target value of the video. The display ratio of the video content of the second type is less than the display ratio of the display area.

[0078] The second quantization interval can be considered as a preset quantization interval, and the interval for coarse quantization of the boundary candidate value. The first boundary and the second boundary can be considered as the upper boundary and the lower boundary of the video frame along the target direction. The quantized boundary candidate value corresponding to the first boundary is the value of the boundary candidate value corresponding to the first boundary after quantization. The quantized boundary candidate value corresponding to the second boundary is the value of the boundary candidate value corresponding to the second boundary after quantization. The first boundary candidate value can be considered as the boundary candidate value corresponding to the first boundary before quantization. The second boundary candidate value can be considered as the boundary candidate value corresponding to the second boundary before quantization.

[0079] In the process of determining the video type of the video to which the video frame belongs and the boundary target value of the video frame based on the boundary candidate values, the boundary candidate values can be first quantized at the second quantization interval. Then, the quantized boundary candidate value corresponding to the first boundary and the quantized boundary candidate value corresponding to the second boundary can be determined from the quantized boundary candidate values. For example, the quantized boundary candidate value corresponding to the first boundary with the highest frequency of occurrence and the quantized boundary candidate value corresponding to the second boundary with the highest frequency of occurrence can be selected in sequence. Then, the first boundary candidate value corresponding to the first boundary before quantization and the second boundary candidate value corresponding to the second boundary before quantization can be determined. Whether the type of the video belongs to the second type can be determined according to the specific size of the proportion of the first boundary candidate value and the proportion of the second boundary candidate value. For example, when the proportions of the first boundary candidate value and the second boundary candidate value are both greater than a set threshold, it can be considered that the first boundary candidate value and the second boundary candidate value are likely to be effective boundaries, and the type of the video belongs to the second type. The average value of the first boundary candidate value and the average value of the second boundary candidate value can be determined as the boundary target value of the video. The display ratio of the video content of the second type is less than the display ratio of the display area, that is, the video content is not displayed in full screen on the display area of the electronic device. When the proportions of the first boundary candidate value and the second boundary candidate value do not simultaneously exceed the set threshold, it can be considered that the first boundary candidate value and the second boundary candidate value are not likely to be effective boundaries. In this case, it can be considered that the type of the video to which the video frame belongs is the first type.

[0080] For example, the boundary candidate values can include all detected boundary candidate values in the video frame. After quantizing the boundary candidate values at the second quantization interval, the quantized boundary candidate value corresponding to the upper boundary, that is, the quantized boundary candidate value corresponding to the first boundary, and the quantized boundary candidate value corresponding to the lower boundary, that is, the quantized boundary candidate value corresponding to the second boundary, can be selected from the quantized boundary candidate values.

[0081] For example, the quantized boundary candidate values can be divided into quantized boundary candidate values corresponding to the first boundary and quantized boundary candidate values corresponding to the second boundary. For example, the quantized boundary candidate values can be divided based on a set value. The set value can be determined based on the middle line of the display area in the y direction. For example, the length of the display area in the y direction of the screen coordinate system is 600, and the unit can be determined based on the actual situation. The set value can be 30. The quantized boundary candidate values greater than 30 can be divided into quantized boundary candidate values corresponding to the first boundary, and the quantized boundary candidate values less than 30 can be divided into quantized boundary candidate values corresponding to the second boundary.

[0082] Taking the quantized boundary candidate values corresponding to the first boundary as an example, the boundary candidate values are 401, 400, 403, 405, and 420. The quantized boundary candidate values obtained by quantizing the boundary candidate values are 40, 40, 40, 40, and 42. The quantized boundary candidate value with the highest frequency in the quantized boundary candidate values corresponding to the first boundary is 40. The boundary candidate values before quantization are determined based on 40, that is, 401, 400, 403, and 405. Then, the proportion of the first boundary candidate value corresponding to the first boundary in the boundary candidate values corresponding to the first boundary is determined, that is, 4 (corresponding to 401, 400, 403, and 405) to 5 (corresponding to 401, 400, 403, 405, and 420).

[0083] Similarly, the second boundary performs the same operation as the first boundary. If the proportion of the first boundary candidate value and the proportion of the second boundary candidate value are both greater than a set threshold, it can be considered that the video type is the second type, and the boundary target value of the upper boundary of the video is determined based on the first boundary candidate value (that is, the average of 401, 400, 403, and 405), and the boundary target value of the lower boundary of the video is determined based on the second boundary candidate value. For example, the boundary represented by the average of the first boundary candidate value is taken as the upper boundary of the video, and the boundary represented by the average of the second boundary candidate value is taken as the lower boundary of the video.

[0084] In an embodiment, the proportion of the first boundary candidate value can be considered as the proportion of the first boundary candidate value in a plurality of boundary candidate values, or as the proportion of the first boundary candidate value in a candidate value range. The embodiment is not limited in this regard. The candidate value range can be determined according to the first boundary, such as the boundary candidate value corresponding to the first boundary.

[0085] In an embodiment, the method further includes:

[0086] The method further includes:

[0087] determining a boundary candidate value on the target boundary in the pixel map.

[0088] The target boundary can be considered as a boundary along a target direction, for example, when the target direction is a horizontal direction, the target boundary is a boundary along the horizontal direction.

[0089] In this embodiment, the pixel map can be used to display each pixel in the video frame, for example, the pixels on the target boundary and the pixels on the non-target boundary in the pixel map can be displayed differently, for example, the pixels on the target boundary can be displayed in white, and the pixels on the non-target boundary can be displayed in black.

[0090] In this embodiment, the pixel map of the video frame can be determined first, for example, the pixels on the horizontal boundary and the pixels at other positions in the video frame can be detected directly to determine the pixel map of the video frame, and the pixels on the horizontal boundary and the pixels at other positions in the video frame can be displayed differently to obtain the pixel map of the video frame; or the candidate pixels on the horizontal boundary in the video frame can be determined first, and then the candidate pixels on the horizontal boundary in the video frame are further screened to obtain the pixels on the horizontal boundary in the video frame, so as to determine the pixel map of the video frame, which is not limited in this embodiment.

[0091] Then, the boundary candidate value on the target boundary in the pixel map is determined, and the specific determination means is not limited, for example, the proportion information of the pixels on the target boundary in each row of the pixel map can be determined row by row, and then the boundary candidate value is determined according to each proportion information, for example, the boundary candidate value is determined according to the size of each proportion information.

[0092] In one embodiment, the pixel map of the video frame is determined, comprising:

[0093] determining the horizontal gradient and the vertical gradient of the video frame;

[0094] determining the candidate pixels on the target boundary based on the horizontal gradient and the vertical gradient;

[0095] filtering the interference pixels in the candidate pixels on the target boundary, the interference pixels including the detected pixels and the pixels within a set pixel distance from the detected pixels, and the detection threshold of the interference pixels is determined based on the interference;

[0096] determining the filtered pixels as the pixels on the target boundary;

[0097] determining the pixel map based on the pixels on the target boundary.

[0098] The horizontal gradient can be considered as the gradient value of the pixels in the video frame in the horizontal direction, and the vertical gradient can be considered as the gradient value of the pixels in the video frame in the vertical direction; the candidate pixels can be considered as the pixels on the target boundary preliminarily screened from each pixel in the video frame.

[0099] Interference pixels refer to pixels containing interfering objects, i.e., pixels that interfere with pixels on the defined target boundary. Interfering objects can be edited content such as text or emoticons. Interference pixels can include detected pixels and pixels within a set pixel range obtained from distance detection. The set pixel range can be a pre-defined pixel value, such as 7. Interference pixels can be detected by detection algorithms, such as the Canny edge detection algorithm, which can detect a subset of pixels. Then, the detected pixels and pixels within the set pixel range obtained from distance detection can be considered interference pixels. The detection threshold is used to detect interference pixels, and the specific value of the detection threshold can be determined based on the interfering object, which is not limited here.

[0100] Specifically, the horizontal and vertical gradients of each pixel in the video frame can be determined first. Then, based on these gradients, candidate pixels on the target boundary can be identified. The method for determining candidate pixels can be based on the horizontal and vertical gradients of the pixels. For example, if the horizontal and vertical gradients of a pixel meet the determination criteria, that pixel can be considered a candidate pixel; conversely, if the horizontal and vertical gradients of a pixel do not meet the criteria, that pixel is not a candidate pixel. This process continues until all pixels have been traversed. The determination criteria can be based on empirical values ​​or the video's usage scenario. For instance, in a short video application scenario, when detecting video type, the determination criteria could be an arctan(ratio) of the vertical gradient to the horizontal gradient of 90. The method for determining the horizontal and vertical gradients of a pixel is not limited; for example, the Sobel operator can be used to determine the corresponding gradient vector and normal vector, thereby determining the horizontal and vertical gradients of the pixel.

[0101] After determining the candidate pixels, interfering pixels can be filtered out, and the filtered pixels can be identified as the pixels on the target boundary. Finally, the pixel map of the video frame is determined based on the pixels on the target boundary.

[0102] In one embodiment, determining candidate boundary values ​​within the pixel map on the target boundary includes:

[0103] Along the target direction, determine the proportion of target pixels in each row of the pixel map, where the target pixels are pixels located on the target boundary;

[0104] Based on the proportion information, boundary candidate values ​​are selected from each row of the pixel map.

[0105] The proportion information can be considered as the proportion of the target pixel in a row in all pixels contained in the row, and the target pixel can be a pixel located on the target boundary, that is, the proportion information can be considered as the proportion of the pixel on the target boundary in a row in all pixels contained in the row.

[0106] In this step, the proportion information of the target pixel in each row can be determined along the target direction, for example, the proportion information of the target pixel in each row can be determined row by row along the horizontal direction, or the proportion information of the target pixel in each row can be determined every other row, which is not limited in the embodiment. Then, the boundary candidate value is selected from each row of the pixel graph according to the determined proportion information, and the means for selecting the boundary candidate value is not limited. For example, the non-maximum suppression method can be used to select the boundary candidate value from each row of the pixel graph. The specific process of selecting the boundary candidate value can be, for example, that a sliding window with a window size of N is taken from the direction of the upper and lower boundaries of the pixel graph to the center of the pixel graph, and the proportion information of the rows contained in the sliding window is judged in sequence, and the edge corresponding to the final preset condition is considered as the boundary candidate value. The preset condition can be determined by relevant personnel, for example, the proportion information is greater than a preset threshold, or the peak boundary in the sliding window is extracted based on the proportion information, and then it is judged whether the boundary candidate value corresponding to the peak boundary is greater than a set threshold. Optionally, the preset threshold can be 0.2.

[0107] Figure 2 A flowchart of a detection method provided by the embodiment of the present disclosure. The scheme in the embodiment can be combined with one or more optional schemes in the above-mentioned embodiments. As shown in the figure, the detection method provided by the embodiment of the present disclosure comprises the following steps: Figure 2

[0108] S210, determining a boundary candidate value of a video frame in a target direction.

[0109] S220, in the gray video frame, for each boundary candidate value, determining difference information of pixel values in a first region and a second region corresponding to the boundary candidate value.

[0110] The difference information can be considered as the difference of the pixel values in the first region and the second region corresponding to the boundary candidate value.

[0111] ​It can be understood that when the video frame is a gray video frame, the difference information of the pixel values in the first region and the second region corresponding to each boundary candidate value can be determined, such as directly determining the difference information according to the pixel values in the first region and the second region; or the first region and the second region can be divided into partitions, and the difference information can be determined according to the pixel values in the partitions of the first region and the second region; or the difference information can be determined by different operations on the pixel values in the first region and the second region. The specific process can be determined according to actual conditions, which is not limited further herein. The gray video frame can be converted from the video frame, for example, the red, green and blue video frames can be obtained by frame extraction of the played video, and then the video frame can be converted into a gray video frame.

[0112] In S230, whether to filter the boundary candidate value is determined based on the difference information.

[0113] After obtaining the difference information corresponding to each boundary candidate value, whether to filter the boundary candidate value can be determined according to the specific size of the difference information.

[0114] In an embodiment, the size of the difference information can be compared with a set value to obtain a comparison result, and whether to filter the boundary candidate value can be determined according to the comparison result. For example, when the size of the difference information is less than the set value, it can be considered that the pixel values in the first region and the second region are relatively small, and at this time, the boundary candidate value can be filtered. When the size of the difference information is greater than or equal to the set value, it can be considered that the pixel values in the first region and the second region are relatively large, and therefore, the boundary candidate value does not need to be filtered. The set value is determined by an empirical value, which is not limited herein.

[0115] In S240, the video type of the video to which the video frame belongs and the boundary target value of the video frame are determined based on the filtered boundary candidate value.

[0116] The detection method provided by the embodiments of the present disclosure can determine whether to filter the boundary candidate value based on the difference information of the pixel values in the first region and the second region corresponding to each boundary candidate value in the gray video frame, thereby providing a basis for obtaining the filtered boundary candidate value.

[0117] In an embodiment, the determination of the difference information of the pixel values in the first region and the second region corresponding to the boundary candidate value includes:

[0118] The first region and the second region are respectively divided into a plurality of block regions, and the sizes of the block regions in the first region and the second region are equal.

[0119] A target value of the pixel values in each block region is determined, and the target value includes a mean value and / or a variance.

[0120] determining a difference between the target value of the pixel value in the block region in the first region and the target value of the pixel value in the corresponding block region in the second region;

[0121] determining a mean value of the determined differences as the difference information.

[0122] It can be understood that in determining the difference information of the pixel values in the first region and the second region corresponding to the boundary candidate value, the first region and the second region can be divided into partitions, and the difference information can be determined according to the pixel values in each partition of the first region and the second region, that is, the first region and the second region can be divided into a plurality of block regions first, and the sizes of the block regions in the divided first region and the second region are equal, and the division manner and the number are not limited, for example, the first region and the second region can be divided into the same number of block regions in the target direction and the vertical direction, and the number of the block regions can be determined according to the width of the video frame.

[0123] Then the target value of the pixel value in each block region is determined, and the target value can include a mean value and / or a variance; then for each block region in the first region, the difference between the target value of the pixel value in the block region and the target value of the pixel value in the corresponding block region in the second region is determined to obtain each difference; finally, the mean value of the determined differences is determined as the difference information.

[0124] The midpoint of the block region in the first region and the corresponding block region in the second region can be considered as a block region symmetrical along the boundary represented by the corresponding boundary candidate value.

[0125] In an embodiment, when the target value includes a mean value, the difference information includes mean value difference information, and the mean value difference information is a mean value of the determined difference based on the mean value;

[0126] When the target value includes a variance, the difference information includes variance difference information, and the variance difference information is a mean value of the determined difference based on the variance;

[0127] Correspondingly, the determining whether to filter the boundary candidate value based on the difference information includes:

[0128] When the value represented by the mean value difference information corresponding to the boundary candidate value is less than a set mean value threshold, and the value represented by the variance difference information corresponding to the boundary candidate value is less than a set variance threshold, the boundary candidate value is filtered.

[0129] The mean difference information can be considered as a mean of the difference values determined based on the mean of the pixel values in the block region, and the variance difference information can be considered as a mean of the difference values determined based on the variance of the pixel values in the block region. For example, the mean of the pixels in each block region is determined, and for each block region in the first region, the difference between the mean of the block region and the mean of the corresponding block region (the block region below the block region in the second region) in the second region is determined.

[0130] In the embodiment, the target value can include the mean and / or the variance. When the target value includes the mean, the mean of the pixel values in each block region is determined, and then for each block region in the first region, the difference between the mean of the pixel values in the block region and the mean of the pixel values in the corresponding block region in the second region is determined to obtain each difference value. The mean of the determined difference values is determined as the mean difference information. When the target value includes the variance, the variance of the pixel values in each block region is determined, and then for each block region in the first region, the difference between the variance of the pixel values in the block region and the mean of the pixel values in the corresponding block region in the second region is determined to obtain each difference value. Finally, the mean of the determined difference values is determined as the variance difference information.

[0131] Correspondingly, whether to filter the boundary candidate value can be determined based on the mean difference information and the variance difference information. For example, the size of the value represented by the mean difference information and the variance difference information can be determined. When the value represented by the mean difference information is less than a set mean threshold value, and the value represented by the corresponding variance difference information is less than a set variance threshold value, it can be considered that the pixel values in the first region and the second region are less different, and in this case, the boundary candidate value can be filtered. Otherwise, it can be considered that the pixel values in the first region and the second region are more different, and in this case, the boundary candidate value does not need to be filtered.

[0132] In one embodiment, the filtering of each of the boundary candidate values based on the first region and the second region corresponding to the boundary candidate value includes:

[0133] For each boundary candidate value, a main color difference value of the first region and the second region corresponding to the boundary candidate value in the video frame in the red-green-blue color domain space is determined.

[0134] Based on the main color difference value, it is determined whether to filter the boundary candidate value.

[0135] The main color difference value can be used to represent the main color difference situation of the first region and the second region corresponding to the boundary candidate value in the video frame in the red-green-blue color domain space, such as the difference situation in the same color domain.

[0136] It should be noted that when filtering each boundary candidate value, whether the boundary candidate value is filtered can be determined according to the difference between the dominant colors of the first region and the second region corresponding to the boundary candidate value, that is, for each boundary candidate value, the difference value between the dominant colors of the first region and the second region corresponding to the boundary candidate value in the video frame in the red-green-blue color domain space can be determined, and the step of determining the difference value between the dominant colors is not limited, for example, the dominant colors of the first region and the second region can be respectively determined according to the frequency of the pixel values in the first region and the second region, and then the difference value between the dominant colors in the same color domain is determined. Then, based on the determined difference value between the dominant colors, whether to filter the boundary candidate value is determined, for example, whether to filter the boundary candidate value can be directly determined according to the size of the difference value between the dominant colors, or the difference value between the dominant colors can be compared with a difference threshold, and whether to filter the boundary candidate value is determined based on the comparison result, that is, when the difference value between the dominant colors is less than the difference threshold, it indicates that the difference between the dominant colors of the first region and the second region corresponding to the boundary candidate value is small, that is, the contents of the first region and the second region are similar, and then the boundary candidate value can be considered as an invalid boundary and filtered.

[0137] It can be considered that in the process of filtering each boundary candidate value, the difference information of the pixel values in the first region and the second region corresponding to each boundary candidate value in the gray video frame can be used to preliminarily determine whether to filter the boundary candidate value, and then based on the preliminarily filtered boundary candidate value, the difference value between the dominant colors of the first region and the second region corresponding to each boundary candidate value in the video frame in the red-green-blue color domain space is used to further determine whether to filter the boundary candidate value, so as to obtain the finally filtered boundary candidate value. Alternatively, the difference value between the dominant colors of the first region and the second region corresponding to each boundary candidate value in the video frame in the red-green-blue color domain space can be used to preliminarily determine whether to filter the boundary candidate value, to obtain the preliminarily filtered boundary candidate value, and then based on the preliminarily filtered boundary candidate value, the difference information of the pixel values in the first region and the second region corresponding to each boundary candidate value is used to further determine whether to filter the boundary candidate value, so as to obtain the finally filtered boundary candidate value. Alternatively, different processes can be started for the boundary candidate value, whether to filter the boundary candidate value is determined according to the difference information of the pixel values in the first region and the second region corresponding to the boundary candidate value and the difference value between the dominant colors of the first region and the second region corresponding to the boundary candidate value in the video frame in the red-green-blue color domain space, and the finally filtered boundary candidate value is obtained by summarizing.

[0138] In one embodiment, the determining the primary color difference value of the first region and the second region corresponding to the boundary candidate value in the video frame of the color gamut space comprises:

[0139] quantizing pixel values in the first region and the second region at the first quantization interval;

[0140] determining a first primary color with the highest frequency of occurrence in the first region based on the quantized pixel values in the first region;

[0141] determining a second primary color with the highest frequency of occurrence in the second region based on the quantized pixel values in the second region;

[0142] determining the difference value of the same color gamut between the first primary color and the second primary color;

[0143] determining the sum of the absolute values of the difference values as the primary color difference value.

[0144] The first quantization interval can be considered as a pre-set quantization interval for quantizing the pixel values in the first region and the second region. The first quantization interval can be set by relevant personnel, for example, the first quantization interval can be 4. The first primary color can be the color corresponding to the pixel value with the highest frequency of occurrence in the first region, and the second primary color can be the color corresponding to the pixel value with the highest frequency of occurrence in the second region. The first primary color and the second primary color are only used to distinguish different objects, and the present embodiment does not limit this.

[0145] In one embodiment, all pixel values in the first region and the second region can be quantized at the first quantization interval to obtain the quantized pixel values in the first region and the second region, for example, mapping each pixel value from the interval [0, 255] to the interval [0, 64]. Then, the first primary color with the highest frequency of occurrence in the first region is determined based on the quantized pixel values in the first region, and the second primary color with the highest frequency of occurrence in the second region is determined based on the quantized pixel values in the second region. Then, the difference value of the same color gamut between the first primary color and the second primary color is calculated, the absolute values of the difference values of the same color gamut are added, and the sum is determined as the primary color difference value.

[0146] Figure 3 A schematic diagram of a detection method provided by the present embodiment is shown in FIG. 1. Figure 3As shown, first, video frame extraction can be performed in a T-second time period at the beginning of the current video, and a red-green-blue (RGB) image (i.e., a video frame) can be obtained. The RGB image is converted into a grayscale image, and the grayscale image is subjected to horizontal boundary activated pixel detection, so that a pixel image a processed by horizontal boundary activated pixel processing (i.e., determining horizontal gradients and vertical gradients of the video frame; based on the horizontal gradients and the vertical gradients, determining candidate pixels in a target direction) can be output. The grayscale image is subjected to strong edge suppression, so that a pixel image b processed by strong edge suppression (i.e., filtering interference pixels in the candidate pixels in the target direction) can be output. The results of horizontal boundary activated pixel detection and strong edge suppression are summarized, that is, the activated pixels with a value of 1 obtained by horizontal boundary activated pixel detection are filtered out, and the activated pixels determined as 0 by strong edge suppression are filtered out, so that a pixel image c obtained by performing an AND operation on the pixel values of the pixel images a and b (i.e., determining filtered pixels as pixels in the target direction; determining the pixel image based on the pixels in the target direction) can be output. Based on the pixel image c, a boundary candidate value filtered after boundary filtering is output by using a non-maximum suppression method, that is, a height value of the candidate boundary value in the pixel image c. Subsequently, based on the candidate boundary value output by the non-maximum suppression and the grayscale image (i.e., determining proportion information of target pixels in each row of the pixel image along the target direction; selecting a boundary candidate value from each row of the pixel image based on the proportion information), a Banding suppression method is used to determine whether to filter the boundary candidate value based on the difference information of the pixel values in the first region and the second region corresponding to each boundary candidate value in the grayscale video frame, so that a preliminarily filtered boundary candidate value is obtained (i.e., for each boundary candidate value, determining difference information of pixel values in the first region and the second region corresponding to the boundary candidate value in the grayscale video frame; based on the difference information, determining whether to filter the boundary candidate value). Based on the preliminarily filtered boundary candidate value and the RGB image, a finally filtered boundary candidate value of the video frame can be obtained according to a primary color matching suppression method (i.e., for each boundary candidate value, determining a primary color difference value of the first region and the second region corresponding to the boundary candidate value in the video frame in a red-green-blue color domain space; based on the primary color difference value, determining whether to filter the boundary candidate value).

[0147] After the above steps are performed on each frame of image, a candidate boundary value corresponding to each frame of image is obtained, so that time domain multi-frame information fusion is performed (i.e., the video frame is a plurality of video frames in a video to which the video frame belongs, the video type of the video to which the video frame belongs and a boundary target value of the video frame are determined based on the filtered boundary candidate value), the mean value of the candidate boundary whose candidate boundary proportion is greater than 60% in the plurality of frames is determined as the corresponding boundary value (i.e., if the proportion of the first boundary candidate value and the second boundary candidate value is greater than a set threshold, it is determined that the type of the video to which the video frame belongs is a second type, and the mean value of the first boundary candidate value and the mean value of the second boundary candidate value are determined as the boundary target value of the video to which the video frame belongs), and finally, whether it is a sandwich (i.e., the video type of the video to which the video frame belongs) and the boundary target value (i.e., the boundary target value of the video frame) are output. It should be noted that the upper and lower boundaries are separately calculated when the time domain fusion is performed, and the upper and lower boundaries are not distinguished before the time domain fusion is performed. Moreover, the upper boundary can be considered as a candidate upper boundary above the middle line of the image height, and the lower boundary can be considered as a candidate lower boundary below the middle line.

[0148] Figure 4 The overall flowchart of the detection method provided by the embodiment of the present disclosure is shown in FIG. 1. Figure 4 As shown in FIG. 1, first, the video is frame-extracted, and the video frame is converted into a gray video frame with size information H*W. The horizontal boundary active pixel detection is performed on each pixel in the gray video frame, that is, the sobel operator is used to detect the horizontal gradient Grad_x and the vertical gradient Grad_y corresponding to the pixel (i.e., the horizontal gradient and the vertical gradient of the video frame are determined), and when the gradient information corresponding to the pixel point satisfies |Grad_x|-|Grad_y*K|>0, where K=40 (i.e., arctan(Grad_y / Grad_x)=90), it is determined that the pixel is a horizontal boundary active pixel (i.e., the candidate pixel in the target direction is determined), and the value is assigned as 1, otherwise, it is determined that the pixel is a non-horizontal boundary active pixel, and the value is assigned as 0. All pixels in the gray video frame are traversed to obtain a plurality of horizontal boundary active pixels with a value of 1 and a plurality of non-horizontal boundary active pixels with a value of 0.

[0149] Figure 5 The schematic diagram of a video frame provided by the embodiment of the present disclosure is shown in FIG. 2. Figure 5 As shown in FIG. 2, the video frame is converted into a gray video frame after the video is frame-extracted. Figure 6 The schematic diagram of a pixel graph provided by the embodiment of the present disclosure is shown in FIG. 3. Figure 6 As shown in FIG. 3, the pixel graph is obtained after the horizontal boundary active pixel detection is performed on each pixel in the gray video frame, and the white pixel point in the graph is a horizontal boundary active pixel with a value of 1, and the black pixel point in the graph is a non-horizontal boundary active pixel with a value of 0.

[0150] Referring toFigure 4 After detecting the activated pixels at horizontal boundaries, strong edge suppression is performed, which involves filtering the activated pixels at horizontal boundaries identified in the previous step (i.e., filtering out interfering pixels from the candidate pixels in the target direction). It can be assumed that short video scenes contain a large amount of edited content such as text and emojis, which can interfere with effective horizontal boundary detection. Therefore, the Canny edge detection algorithm can be used to filter pixels belonging to strong edges, and pixels with strong edges and content within M=7 pixels of the strong edge are classified as non-horizontal boundary activated pixels and assigned a value of 0. Interfering pixels include the detected pixels and pixels within a set number of pixels from the detected pixels, where the detection threshold can be 175 or 200. This detection threshold can be determined based on the size of the interfering object. The filtered pixels are then identified as pixels in the target direction. Figure 7 This is a schematic diagram of another pixel image provided in an embodiment of the present disclosure, such as... Figure 7 As shown, the horizontal edge activation pixels determined in the previous step are subjected to strong edge suppression to filter out interfering pixels in the candidate pixels along the target direction, and the filtered pixels are used to form a pixel map (i.e., Figure 7 ).

[0151] Next, horizontal boundary extraction based on non-maximum suppression (NMS) is performed. This involves averaging the activated pixels row by row along the horizontal direction based on the horizontally activated pixel map obtained after strong edge suppression. For example, for each pixel in a row, a value of 0 can be considered a non-horizontal boundary activated pixel, and a value of 1 can be considered a horizontal boundary activated pixel. The probability value `horizon_act` is calculated based on the number of pixels with a value of 1, representing the proportion of target pixels in each row of the pixel map (i.e., determining the proportion of target pixels in each row along the target direction). The closer `horizon_act` is to 1, the higher the probability that the boundary is a video boundary. After determining the proportion of pixels in each row of the pixel map that are in the target direction, peak boundary values ​​can be extracted from the one-dimensional probability distribution `horizon_act` from the top and bottom boundaries towards the center using non-maximum suppression (i.e., selecting candidate boundary values ​​from each row of the pixel map based on the proportion information). The sliding window size for NMS filtering is N = 10. When the peak boundary value is greater than the specified threshold (e.g., border_thres = 0.2), the boundary line is determined to be a valid border candidate value (i.e., border candidate value).

[0152] Figure 8 This is a schematic diagram of a proportion information provided in an embodiment of the present disclosure, such as... Figure 8 The image shows the percentage of pixels in each row of the pixel image that are aligned with the target direction. After horizontal boundary extraction based on NMS, Figure 9Another schematic diagram of the proportion information provided by the embodiments of the present disclosure can select the boundary candidate value from each row of the pixel diagram after using the non-maximum suppression method.

[0153] After the band interference suppression and the main color matching interference suppression, firstly, in order to filter out the interference items of the candidate boundary which is surrounded by smooth areas, the pixel value difference of the upper and lower band-shaped areas (i.e. the first area and the second area) of each boundary candidate value border_candiate can be compared, wherein the size of the band-shaped area can be L*W, generally L=16, the upper and lower band-shaped areas are divided into LxL block-shaped areas, and the mean value and variance difference of the contents of the two groups of blocks are compared. Specifically, the first area and the second area can be divided into a plurality of block-shaped areas, for each block-shaped area in the first area, the difference value between the target value of the pixel value in the block-shaped area and the target value of the pixel value in the corresponding block-shaped area in the second area is determined, and the mean value of the determined difference value is determined as the difference information, which is used to determine whether to filter the boundary candidate value.

[0154] For example, the first region and the second region are 16x960 band regions, the first region and the second region can be divided into 60 16x16 block regions respectively, the target value (such as mean and / or variance) of the pixel value in each block region is calculated, and for each block region in the first region, the difference between the target value of the pixel value in the block region and the target value of the pixel value in the corresponding block region in the second region is determined, and the mean of the determined difference is determined as the difference information, such as the mean difference information block_mean_diff = mean(|block_mean_upper_i-block_mean_lower_i|), i in [1, W / / L], the variance difference information block_std_diff = mean(|block_std_upper_i-block_std_lower_i|), i in [1, W / / L], block_mean_upper_i can be considered as the mean of the pixel value in the i-th block region of the first region, i.e., the upper band region, block_mean_lower_i can be considered as the mean of the pixel value in the i-th block region of the second region, i.e., the lower band region, block_std_upper_i can be considered as the variance of the pixel value in the i-th block region of the first region, i.e., the upper band region, and block_std_lower_i can be considered as the variance of the pixel value in the i-th block region of the second region, i.e., the lower band region; when block_mean_diff < 5.0 (i.e., a mean threshold is set) and block_std_diff < 5.0 (i.e., a variance threshold is set), it indicates that the contents on the candidate boundary are similar, i.e., the candidate boundary value is determined as invalid, and the preliminary filtered boundary candidate value is obtained (i.e., when the value represented by the mean difference information corresponding to the boundary candidate value is less than the set mean threshold, and the value represented by the variance difference information is less than the set variance threshold, the boundary candidate value is filtered).

[0155] On the basis of the obtained preliminary filtered boundary candidate value, the video frame can be converted to the RGB color domain space, and color similarity matching is performed on the upper and lower band-shaped regions of the candidate boundary value border_candiate, that is, the first quantization interval of Quantization_bin=4 is used to map all pixel values of the band-shaped region from the interval [0-255] to the interval [0-64] (that is, the first region and the second region are quantized by the first set quantization interval). Frequency statistics is performed on the quantized pixel values, and the highest frequency primary colors of the upper and lower band-shaped regions are [r2_upper, g2_upper, b2_upper] and [r2_lower, g2_lower, g2_lower] respectively (that is, the first region is determined to have the highest frequency of the first primary color; based on the quantized pixel values in the second region, the second region is determined to have the highest frequency of the second primary color). If the upper and lower primary color difference dominant_color_diff<5.0, it indicates that the content of the candidate boundary is similar, and it is determined to be an invalid candidate boundary value; if the upper and lower primary color difference dominant_color_diff≥5.0, it indicates that the content of the candidate boundary is different, and it is determined to be a valid candidate boundary value (that is, based on the primary color difference value, it is determined whether to filter the boundary candidate value), wherein the upper and lower primary color difference, that is, the primary color difference value dominant_color_diff=|r2_upper-r2_lower|+|g2_upper-g2_lower|+|b2_upper-b2_lower) (that is, among the first primary color and the second primary color, the difference value of the same color domain value is determined; the sum of the absolute values of each difference value is determined as the primary color difference value).

[0156] After the above steps (that is, Figure 4The middle dotted line part) can process the boundary information of each frame of video, provide a single frame effective horizontal boundary, that is, a plurality of filtered boundary candidate values corresponding to a plurality of single frame video frames can be obtained; and the video multi-frame results of the multi-frame effective horizontal boundary are summarized, that is, the detection results can be obtained after the time domain multi-frame information fusion, including whether it is a sandwich video and a boundary target value. Specifically, the video frames in the T second time period at the beginning of the video can be detected frame by frame, a total of T frames of video frames (that is, the video frames are a plurality of video frames in the video to which the video frames belong), the number of average border_candiates is calculated (that is, the average candidate value is determined), if the number of time domain average candidate window border_candiates is less than 1, it is directly determined as a normal video (that is, the video type of the video to which the video belongs is determined as the first type); otherwise, based on each boundary candidate value, the video type of the video to which the video frame belongs and the boundary target value of the video frame need to be determined, for example, the detection boundary information sequence border_candiates in the time domain is quantized in a coarse granularity (that is, each boundary candidate value is quantized in a second quantization interval) with a quantization interval of Quantization_bin=10 to obtain quant_border_candiates, so that the boundary detection is not disturbed by the boundary fluctuation of the smaller boundary.

[0157] Then, the quantization upper boundary quant_border_upper and the quantization lower boundary quant_border_lower with the highest occurrence frequency in the time domain are counted (i.e., the quantization after boundary candidate values corresponding to the first boundary and the quantization after boundary candidate values corresponding to the second boundary are determined). The border_candiates are reversely searched according to the quant_border_upper and the quant_border_lower (i.e., the first boundary candidate value before quantization corresponding to the first boundary is determined; the second boundary candidate value before quantization corresponding to the second boundary is determined). If the proportion of the candidate boundaries appearing in [quant_border_upper*Quantization_bin, (quant_border_upper+1)*Quantization_bin] and [quant_border_lower*Quantization_bin, (quant_border_lower+1)*Quantization_bin] is greater than 60% (i.e., if the proportion of the first boundary candidate value and the second boundary candidate value is greater than a set threshold), the mean value of the border_candiates meeting the condition is obtained as valid_boder_upper and valid_boder_lower, respectively, as the effective boundary of the sandwich video (i.e., the mean value of the first boundary candidate value and the mean value of the second boundary candidate value are determined as the boundary target value of the video), and it is determined that the video is a sandwich video (i.e., it is determined that the type of the video is the second type). If no stable boundary is detected, such as the proportion of the first boundary candidate value and the second boundary candidate value is not greater than the set threshold, it is determined that the video is a normal video.

[0158] Figure 10 A structure diagram of a detection device provided by an embodiment of the present disclosure is shown in FIG. 1. As shown in FIG. 1, the device includes a first determination module 310, a filtering module 320, and a second determination module 330. Figure 10

[0159] The first determination module 310 is configured to determine boundary candidate values of a video frame in a target direction.

[0160] The filtering module 320 is configured to filter each of the boundary candidate values based on a first region and a second region corresponding to the boundary candidate values, wherein the first region and the second region are located on both sides of the corresponding candidate boundary, and the first region and the second region are two regions along a direction perpendicular to the target direction.

[0161] ​The second determining module 330 is configured to determine the video type of the video to which the video frame belongs and the boundary target value of the video frame based on the filtered boundary candidate value.

[0162] The technical solution provided by the embodiments of the present disclosure determines the boundary candidate value of the video frame in the target direction through the first determining module 310; filters each boundary candidate value based on the first region and the second region corresponding to the boundary candidate value through the filtering module 320, the first region and the second region are located on both sides of the corresponding candidate boundary, and the first region and the second region are two regions along the direction perpendicular to the target direction; determines the video type of the video to which the video frame belongs and the boundary target value of the video frame based on the filtered boundary candidate value through the second determining module 330. By using the device, each boundary candidate value is filtered based on the first region and the second region corresponding to the boundary candidate value, so that the filtered boundary candidate value can be accurately filtered; and the video type of the video to which the video frame belongs is determined, and the boundary target value of the video frame can be determined at the same time based on the filtered boundary candidate value.

[0163] Optionally, the video frame is a grayscale video frame, and the filtering module 320 includes:

[0164] The first determining unit is configured to determine, in the grayscale video frame, difference information of pixel values in the first region and the second region corresponding to each boundary candidate value;

[0165] The second determining unit is configured to determine whether to filter the boundary candidate value based on the difference information.

[0166] Optionally, the first determining unit is specifically configured to:

[0167] divide the first region and the second region into a plurality of block regions respectively, and the sizes of the block regions in the first region and the second region are equal;

[0168] determine a target value of pixel values in each block region, and the target value includes a mean value and / or a variance value;

[0169] determine, for each block region in the first region, a difference value between the target value of pixel values in the block region and the target value of pixel values in the corresponding block region in the second region;

[0170] determine the mean value of the determined difference value as the difference information.

[0171] Optionally, when the target value includes the mean value, the difference information includes mean value difference information, and the mean value difference information is the mean value of the determined difference value based on the mean value;

[0172] When the target value comprises a variance, the difference information comprises variance difference information, and the variance difference information is a mean value of the difference value determined based on the variance;

[0173] Correspondingly, the second determining unit comprises:

[0174] When the value represented by the mean difference information corresponding to the boundary candidate value is less than a set mean threshold value, and the value represented by the variance difference information corresponding to the boundary candidate value is less than a set variance threshold value, the boundary candidate value is filtered.

[0175] Optionally, the filtering module 320 comprises:

[0176] A third determining unit is configured to determine, for each boundary candidate value, a main color difference value of a first region and a second region corresponding to the boundary candidate value in a video frame in a red-green-blue color gamut space;

[0177] A fourth determining unit is configured to determine, based on the main color difference value, whether to filter the boundary candidate value.

[0178] Optionally, the third determining unit is specifically configured to:

[0179] quantize pixel values in the first region and the second region at a first quantization interval;

[0180] determine a first main color with the highest occurrence frequency in the first region based on the quantized pixel values in the first region;

[0181] determine a second main color with the highest occurrence frequency in the second region based on the quantized pixel values in the second region;

[0182] determine a difference value of the same color gamut value between the first main color and the second main color;

[0183] determine a sum of absolute values of the difference values as the main color difference value.

[0184] Optionally, the video frame is a plurality of video frames in a video, and the second determining module 330 comprises:

[0185] A fifth determining unit is configured to determine an average candidate value based on the filtered boundary candidate values corresponding to each video frame;

[0186] A sixth determining unit is configured to determine a video type of the video as a first type if the average candidate value is less than 1, and the display ratio of the video content of the first type is the same as the display ratio of the display region.

[0187] The seventh determining unit is configured to determine a video type of the video to which the video frame belongs and a boundary target value of the video frame based on the boundary candidate values, if the average candidate value is greater than or equal to 1.

[0188] Optionally, the seventh determining unit is specifically configured to:

[0189] quantize each of the boundary candidate values at a second quantization interval;

[0190] determine a quantized boundary candidate value corresponding to a first boundary;

[0191] determine a quantized boundary candidate value corresponding to a second boundary, the first boundary and the second boundary being boundaries along the target direction, and a video content being between the first boundary and the second boundary;

[0192] determine a first boundary candidate value corresponding to the first boundary before quantization;

[0193] determine a second boundary candidate value corresponding to the second boundary before quantization;

[0194] if a proportion of the first boundary candidate value and the second boundary candidate value is greater than a set threshold, determine that the type of the video to which the video frame belongs is a second type, and determine an average of the first boundary candidate value and an average of the second boundary candidate value as a boundary target value of the video, a display proportion of the video content of the second type being less than a display proportion of a display area.

[0195] Optionally, the first region and the second region have the same size and shape, edges of the first region and the second region along the target direction being equal in length to an edge of the video frame along the target direction, and edges of the first region and the second region in a height direction being smaller than an edge of the video frame in the height direction, the height direction being perpendicular to the target direction.

[0196] Optionally, the first determining module 310 includes:

[0197] an eighth determining unit configured to determine a pixel map of a video frame, pixels on a target boundary and pixels on a non-target boundary in the pixel map being displayed differently, the target boundary being a boundary along a target direction;

[0198] a ninth determining unit configured to determine a boundary candidate value on the target boundary in the pixel map.

[0199] Optionally, the eighth determining unit is specifically configured to:

[0200] determine a horizontal gradient and a vertical gradient of the video frame;

[0201] Based on the horizontal gradient and the vertical gradient, candidate pixels on the target boundary are determined;

[0202] Filter out interfering pixels among candidate pixels on the target boundary. The interfering pixels include detected pixels and pixels within a set pixel distance from the detected pixels. The detection threshold of the interfering pixels is determined based on the interfering object.

[0203] The filtered pixels are identified as pixels on the target boundary;

[0204] The pixel map is determined based on the pixels on the target boundary.

[0205] Optionally, the ninth determining unit is specifically used for:

[0206] Along the target direction, determine the proportion of target pixels in each row of the pixel map, where the target pixels are pixels located on the target boundary;

[0207] Based on the proportion information, boundary candidate values ​​are selected from each row of the pixel map.

[0208] The detection device provided in this disclosure can execute the detection method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the method.

[0209] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0210] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Reference is made below. Figure 11 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 11 The diagram below shows the structure of the terminal device or server 500. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 11 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0211] like Figure 11As shown, the electronic device 500 may include a processor (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processor 501, ROM 502, and RAM 503 are interconnected via a bus 504. An edit / output (I / O) interface 505 is also connected to the bus 504.

[0212] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 11 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0213] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processor 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0214] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0215] The electronic device provided in this embodiment and the detection method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0216] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the detection method provided in the above embodiments.

[0217] Note that the computer readable medium described above in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination thereof. The computer readable storage medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination thereof. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program used by or in connection with an instruction execution system, apparatus or device. In the present disclosure, the computer readable signal medium can include a data signal propagated in baseband or propagated as a carrier wave in a propagated data signal, in which the computer readable program code is contained. Such a propagated data signal can take many forms, including but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer readable signal medium can also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained in the computer readable medium can be transmitted by any suitable medium, including but not limited to, wire, cable, RF (radio frequency), etc., or any suitable combination thereof.

[0218] In some embodiments, the client, server, or both can communicate using any current known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet, and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any current known or future developed networks.

[0219] The computer readable medium described above can be included in the electronic device; or can exist separately from the electronic device, and can be accessed via the electronic device.

[0220] The computer readable medium described above carries one or more programs, which when executed by the electronic device, cause the electronic device to:

[0221] The computer readable medium described above carries one or more programs, when the one or more programs are executed by the electronic device, cause the electronic device to: determine boundary candidate values of a video frame in a target direction; filter each of the boundary candidate values based on a first region and a second region corresponding to the boundary candidate values, the first region and the second region being located on two sides of a corresponding candidate boundary, and the first region and the second region being two regions in a direction perpendicular to the target direction; and determine a video type of a video to which the video frame belongs and a boundary target value of the video frame based on the filtered boundary candidate values.

[0222] Computer program code for carrying out operations of the present disclosure can be written in any of one or more programming languages or combinations of languages including object or visual programming languages such as Java, Smalltalk, C++ or conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0223] The computer program instructions can also be loaded onto a computer or other programmable information processing apparatus to cause a series of operations to be performed on the computer or other programmable information processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable information processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0224] The units described in the embodiments of the present disclosure can be implemented in the form of software, or can be implemented in the form of hardware. In some cases, the names of the units do not constitute a limitation on the units themselves. For example, the first obtaining unit can also be described as a unit that obtains at least two Internet protocol addresses.

[0225] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, example types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (ASICs), Application-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.

[0226] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0227] According to one or more embodiments of the present disclosure, example 1 provides a detection method, comprising:

[0228] determining boundary candidate values of the video frame in a target direction;

[0229] filtering each of the boundary candidate values based on first and second regions corresponding to the boundary candidate values, the first and second regions being located on two sides of a corresponding candidate boundary, and the first and second regions being two regions along a direction perpendicular to the target direction;

[0230] determining a video type of a video to which the video frame belongs and a boundary target value of the video frame based on the filtered boundary candidate values.

[0231] According to one or more embodiments of the present disclosure, example 2 provides the method of example 1, wherein the video frame is a grayscale video frame, and the filtering each of the boundary candidate values based on first and second regions corresponding to the boundary candidate values comprises:

[0232] In the gray-scale video frame, for each boundary candidate value, difference information of pixel values in the first region and the second region corresponding to the boundary candidate value is determined;

[0233] Whether to filter the boundary candidate value is determined based on the difference information.

[0234] According to one or more embodiments of the present disclosure, example 3, the method according to example 2, the determination of the difference information of the pixel values in the first region and the second region corresponding to the boundary candidate value comprises:

[0235] The first region and the second region are respectively divided into a plurality of block regions, and the sizes of the block regions in the first region and the second region are equal;

[0236] A target value of the pixel values in each block region is determined, and the target value comprises a mean value and / or a variance value;

[0237] For each block region in the first region, a difference value between the target value of the pixel values in the block region and the target value of the pixel values in the corresponding block region in the second region is determined;

[0238] A mean value of the determined difference value is determined as the difference information.

[0239] According to one or more embodiments of the present disclosure, example 4, the method according to example 3,

[0240] When the target value comprises a mean value, the difference information comprises mean difference information, and the mean difference information is a mean value of the difference value determined based on the mean value;

[0241] When the target value comprises a variance value, the difference information comprises variance difference information, and the variance difference information is a mean value of the difference value determined based on the variance value;

[0242] Correspondingly, the determination of whether to filter the boundary candidate value based on the difference information comprises:

[0243] When the value represented by the mean difference information corresponding to the boundary candidate value is less than a set mean value threshold, and the value represented by the variance difference information corresponding to the boundary candidate value is less than a set variance threshold, the boundary candidate value is filtered.

[0244] According to one or more embodiments of the present disclosure, example 5, the method according to example 1, the filtering of each boundary candidate value based on the first region and the second region corresponding to the boundary candidate value comprises:

[0245] For each boundary candidate value, a main color difference value of the first region and the second region corresponding to the boundary candidate value in the video frame in the red-green-blue color domain space is determined;

[0246] determining whether to filter the boundary candidate value based on the primary color difference value.

[0247] According to one or more embodiments of the present disclosure, example 6, the method according to example 5, the determining of the primary color difference value of the first region and the second region in the video frame of the color gamut space corresponding to the boundary candidate value comprises:

[0248] quantizing pixel values in the first region and the second region at a first quantization interval;

[0249] determining a first primary color with the highest frequency of occurrence in the first region based on the quantized pixel values in the first region;

[0250] determining a second primary color with the highest frequency of occurrence in the second region based on the quantized pixel values in the second region;

[0251] determining a difference value of the same color gamut value between the first primary color and the second primary color;

[0252] determining a sum of absolute values of the difference values as the primary color difference value.

[0253] According to one or more embodiments of the present disclosure, example 7, the method according to example 1, the video frame being a plurality of video frames in a video, the determining of the video type of the video to which the video frame belongs and the boundary target value of the video frame based on the filtered boundary candidate value comprises:

[0254] determining an average candidate value based on the filtered boundary candidate value corresponding to each video frame;

[0255] if the average candidate value is less than 1, determining the video type of the video as a first type, the display ratio of the video content of the first type being the same as the display ratio of the display area;

[0256] if the average candidate value is greater than or equal to 1, determining the video type of the video to which the video frame belongs and the boundary target value of the video frame based on each boundary candidate value.

[0257] According to one or more embodiments of the present disclosure, example 8, the method according to example 7, the determining of the video type of the video to which the video frame belongs and the boundary target value of the video frame based on each boundary candidate value comprises:

[0258] quantizing each boundary candidate value at a second quantization interval;

[0259] determining a quantized boundary candidate value corresponding to a first boundary;

[0260] determining a quantized first boundary candidate value corresponding to the first boundary;

[0261] determining a first boundary candidate value corresponding to the first boundary before quantization;

[0262] determining a second boundary candidate value corresponding to the second boundary before quantization;

[0263] if a proportion of the first boundary candidate value and the second boundary candidate value is greater than a set threshold, determining that the type of the video is a second type, and determining a mean value of the first boundary candidate value and a mean value of the second boundary candidate value as a boundary target value of the video, a display proportion of the video content of the second type being less than a display proportion of a display area.

[0264] According to one or more embodiments of the present disclosure, example 9, the method of example 1,

[0265] the first region and the second region have the same size and shape, a side of the first region and the second region along the target direction is equal in length to a side of the video frame along the target direction, and a side of the first region and the second region in a height direction is smaller than a side of the video frame in the height direction, the height direction being perpendicular to the target direction.

[0266] According to one or more embodiments of the present disclosure, example 10, the method of example 1, the determining of the boundary candidate value of the video frame in the target direction comprises:

[0267] determining a pixel map of the video frame, pixels on a target boundary and pixels on a non-target boundary in the pixel map being displayed differently, the target boundary being a boundary along the target direction;

[0268] determining a boundary candidate value on the target boundary in the pixel map.

[0269] According to one or more embodiments of the present disclosure, example 11, the method of example 10, the determining of the pixel map of the video frame comprises:

[0270] determining a horizontal gradient and a vertical gradient of the video frame;

[0271] based on the horizontal gradient and the vertical gradient, determining a candidate pixel on the target boundary;

[0272] filtering an interference pixel in the candidate pixel on the target boundary, the interference pixel including a detected pixel and a pixel within a set pixel distance from the detected pixel, a detection threshold of the interference pixel being determined based on an interference object;

[0273] determining a pixel on the target boundary as a filtered pixel;

[0274] determining the pixel map based on the pixel on the target boundary.

[0275] According to one or more embodiments of the present disclosure, example 12 provides a method according to example 10, determining the boundary candidate value in the target direction within the pixel map, comprising:

[0276] determining, along the target direction, proportion information of target pixels in each row of the pixel map, the target pixels being pixels on the target boundary;

[0277] selecting the boundary candidate value from each row of the pixel map based on the proportion information.

[0278] According to one or more embodiments of the present disclosure, example 13 provides a detection device, comprising:

[0279] a first determining module configured to determine a boundary candidate value of a video frame in a target direction;

[0280] a filtering module configured to filter each of the boundary candidate values based on a first region and a second region corresponding to the boundary candidate value, the first region and the second region being located on both sides of the corresponding candidate boundary, and the first region and the second region being two regions along a direction perpendicular to the target direction;

[0281] a second determining module configured to determine a video type of a video to which the video frame belongs and a boundary target value of the video frame based on the filtered boundary candidate value.

[0282] According to one or more embodiments of the present disclosure, example 14 provides an electronic device, comprising:

[0283] one or more processors;

[0284] a memory configured to store one or more programs,

[0285] when the one or more programs are executed by the one or more processors, the one or more processors implement the detection method according to any one of examples 1-12.

[0286] According to one or more embodiments of the present disclosure, example 15 provides a computer-readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the detection method according to any one of examples 1-12.

[0287] The above description merely illustrates the preferred embodiments of the disclosure and a principle for applying the technologies. It is understood by those skilled in the art that the disclosed scope of the disclosure is not limited to the technical solutions formed by the specific combinations of the technical features described above, and should also cover other technical solutions formed by the combinations of the technical features described above or their equivalent features without departing from the disclosed concept. For example, the technical solutions formed by the mutual replacement of the above-described features and the technical features with similar functions disclosed in the disclosure (but not limited to) can be formed.

[0288] Further, although operations are depicted in a particular, sequential order, this should not be understood as requiring or implying that the operations are performed in the order illustrated or sequentially. In certain circumstances, multitasking and parallel processing can be advantageous. Likewise, although specific implementation details are contained in the above discussion, these should not be construed as limiting the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable sub-combination.

[0289] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A method of detection, characterized in that, The method comprises: determining boundary candidate values of a video frame in a target direction, wherein the boundary candidate values represent positions of preliminarily determined boundaries in the target direction; filtering each of the boundary candidate values based on difference information or a main color difference value of pixel values of a first region and a second region corresponding to the boundary candidate value, wherein the first region and the second region are located on two sides of the corresponding boundary candidate value and are two regions in a direction perpendicular to the target direction; determining a video type of a video to which the video frame belongs and a boundary target value of the video frame based on the filtered boundary candidate values, wherein the boundary target value represents a position of a finally determined boundary of video content in the video frame, and the boundary target value is selected from the boundary candidate values.

2. The method of claim 1, wherein, The video frame is a grayscale video frame, and the filtering of each of the boundary candidate values based on the first region and the second region corresponding to the boundary candidate value comprises: determining, in the grayscale video frame, difference information between pixel values in the first region and the second region corresponding to each boundary candidate value; determining whether to filter the boundary candidate value based on a comparison between the difference information and a set value.

3. The method of claim 2, wherein, The determination of the difference information between the pixel values in the first region and the second region corresponding to the boundary candidate value comprises: dividing the first region and the second region into a plurality of block regions respectively, wherein the block regions in the first region and the second region are equal in size; determining a target value of pixel values in each block region, wherein the target value comprises a mean value and / or a variance value; determining a difference value between the target value of pixel values in each block region in the first region and the target value of pixel values in a corresponding block region in the second region; determining a mean value of the determined difference values as the difference information.

4. The method of claim 3, wherein: when the target value comprises the mean value, the difference information comprises mean difference information, and the mean difference information is a mean value of the determined difference values based on the mean value; when the target value comprises the variance value, the difference information comprises variance difference information, and the variance difference information is a mean value of the determined difference values based on the variance value; correspondingly, the determination of whether to filter the boundary candidate value based on the difference information comprises: filtering the boundary candidate value when a value represented by the mean difference information corresponding to the boundary candidate value is less than a set mean value threshold and a value represented by the variance difference information corresponding to the boundary candidate value is less than a set variance threshold.

5. The method of claim 1, wherein, The filtering of each of the boundary candidate values based on the first region and the second region corresponding to the boundary candidate value comprises: determining, for each boundary candidate value, a main color difference value of the first region and the second region corresponding to the boundary candidate value in the video frame in a red-green-blue color domain space; determining whether to filter the boundary candidate value based on a comparison between the main color difference value and a difference threshold value; wherein the main color difference value is used to represent a main color difference condition of the first region and the second region corresponding to the boundary candidate value in the video frame in the red-green-blue color domain space.

6. The method of claim 5, wherein, The method for determining the main color difference value of the first region and the second region corresponding to the boundary candidate value in the video frame in the red-green-blue color domain space comprises: quantizing pixel values in the first region and the second region at a first quantization interval; determining a first main color with the highest frequency in the first region based on the quantized pixel values in the first region; determining a second main color with the highest frequency in the second region based on the quantized pixel values in the second region; determining the difference value of the same color domain value between the first main color and the second main color; determining the sum of the absolute values of the difference values as the main color difference value.

7. The method of claim 1, wherein, The video frame is a plurality of video frames in the video, and the method for determining the video type of the video and the boundary target value of the video frame based on the filtered boundary candidate value comprises: determining an average candidate value based on the filtered boundary candidate value corresponding to each video frame, wherein the average candidate value is the average of the number of the filtered boundary candidate values corresponding to the plurality of video frames; if the average candidate value is less than 1, determining the video type of the video as a first type, wherein the display ratio of the video content of the first type is the same as the display ratio of the display area; if the average candidate value is greater than or equal to 1, determining the video type of the video and the boundary target value of the video frame based on each boundary candidate value.

8. The method of claim 7, wherein, The method for determining the video type of the video and the boundary target value of the video frame based on each boundary candidate value comprises: quantizing each boundary candidate value at a second quantization interval; determining a quantized boundary candidate value corresponding to a first boundary; determining a quantized boundary candidate value corresponding to a second boundary, wherein the video content is between the first boundary and the second boundary, and the first boundary and the second boundary are boundaries along the target direction; determining a first boundary candidate value before quantization corresponding to the first boundary; determining a second boundary candidate value before quantization corresponding to the second boundary; if the proportion of the first boundary candidate value in the boundary candidate value before quantization corresponding to the first boundary and the proportion of the second boundary candidate value in the boundary candidate value before quantization corresponding to the second boundary are both greater than a set threshold, determining the type of the video as a second type, and determining the average of the first boundary candidate value and the average of the second boundary candidate value as the boundary target value of the video, wherein the display ratio of the video content of the second type is less than the display ratio of the display area.

9. The method of claim 1, wherein: the size and shape of the first region and the second region are the same, the edges of the first region and the second region along the target direction are equal in length to the edges of the video frame along the target direction, the edges of the first region and the second region in the height direction are smaller than the edges of the video frame in the height direction, and the height direction is perpendicular to the target direction.

10. The method of claim 1, wherein, The method for determining the boundary candidate value of the video frame in the target direction comprises: determining a pixel map of a video frame, the pixel map having pixels on a target boundary and pixels on a non-target boundary distinguished from each other, the target boundary being a boundary in a target direction; determining boundary candidate values on the target boundary in the pixel map.

11. The method of claim 10, wherein, The determining a pixel map of a video frame includes: determining horizontal gradients and vertical gradients of the video frame; determining candidate pixels on the target boundary based on the horizontal gradients and the vertical gradients; filtering out interfering pixels from the candidate pixels on the target boundary, the interfering pixels including detected pixels and pixels within a set distance from the detected pixels, the detected pixels being determined based on an interference threshold; determining the filtered pixels as pixels on the target boundary; determining the pixel map based on the pixels on the target boundary.

12. The method of claim 10, wherein, The determining boundary candidate values on the target boundary in the pixel map includes: determining target pixel proportion information of each row in the pixel map in the target direction, the target pixels being pixels on the target boundary; selecting boundary candidate values from the rows in the pixel map based on the target pixel proportion information.

13. A detection device, characterized in that The method includes: a first determining module configured to determine boundary candidate values of a video frame in a target direction, the boundary candidate values representing positions of boundaries preliminarily determined in the target direction; a filtering module configured to filter the boundary candidate values based on difference information or primary color difference values of pixel values of first and second regions corresponding to the boundary candidate values, the first and second regions being on two sides of the corresponding candidate boundaries and being two regions in a direction perpendicular to the target direction; a second determining module configured to determine a video type of a video to which the video frame belongs and a boundary target value of the video frame based on the filtered boundary candidate values, the boundary target value representing a position of a boundary of video content finally determined in the video frame, the boundary target value being selected from the boundary candidate values.

14. An electronic device, comprising: The electronic device includes: one or more processors; a storage device configured to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the detection method as claimed in any one of claims 1-12.

15. A storage medium containing computer-executable instructions for performing the detection method as claimed in any one of claims 1-12 when executed by a computer processor.

Images