Identification method, device, electronic device and storage medium

Abstract

A recognition method, device, electronic device and storage medium. The method comprises extracting multiple frames of original images from a video to be recognized, obtaining an edge gradient image (101) of each frame of the original image; determining the median of pixel values ​​of pixels at the same position in the multiple frames of edge images (102); generating an intermediate feature image (103) based on each median and the position of each median pixel; and determining an object composed of pixels with non-zero pixel values ​​in the intermediate feature image as a stationary object (104).

Description

Technical Field

[0001] The embodiments of the present application relate to computer technology, and in particular to an identification method, device, electronic device, and storage medium. Background Art

[0002] With the development of science and technology, various video resources are becoming more and more abundant. Each video contains many objects. How electronic devices can identify objects with the same characteristics in the video has become an important research topic.

[0003] Currently, when recognizing a video, electronic devices usually extract multiple frames of images from the video and use the multiple frames of images to represent the video as the recognition subject, thereby recognizing a stationary object in the video. Summary of the Invention

[0004] The present application provides an identification method, device, electronic device and storage medium, which can improve the recognition accuracy of stationary objects in videos.

[0005] In a first aspect, an embodiment of the present application provides an identification method, including:

[0006] Extract multiple frames of original images from the video to be recognized, obtain the edge gradient image corresponding to each frame of the original image, and obtain multiple frames of edge gradient images;

[0007] Obtaining a pixel value of each pixel point in each frame of the edge gradient image, and determining a median of the pixel values ​​of the pixel points located at the same position in the plurality of frames of the edge image;

[0008] Generate an intermediate feature image based on each median and the pixel position corresponding to each median;

[0009] An object formed by pixel points whose pixel values ​​in the intermediate feature image are not equal to zero is determined as a stationary object in the video to be identified.

[0010] In a second aspect, an embodiment of the present application further provides an identification device, including:

[0011] The first acquisition module is used to extract multiple frames of original images from the video to be identified, obtain the edge gradient image corresponding to each frame of the original image, and obtain multiple frames of edge gradient images;

[0012] a first determining module, configured to obtain a pixel value of each pixel point in each frame of the edge gradient image, and determine a median of the pixel values ​​of the pixels located at the same position in the plurality of frames of the edge image;

[0013] A generation module is used to generate an intermediate feature image based on each median and the pixel position corresponding to each median;

[0014] The second determination module is configured to determine an object formed by pixels having non-zero pixel values ​​in the intermediate feature image as a stationary object in the video to be identified.

[0015] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements an identification method when executing the computer program:

[0016] Extract multiple frames of original images from the video to be recognized, obtain the edge gradient image corresponding to each frame of the original image, and obtain multiple frames of edge gradient images;

[0017] Obtaining a pixel value of each pixel point in each frame of the edge gradient image, and determining a median of the pixel values ​​of the pixel points located at the same position in the plurality of frames of the edge image;

[0018] Generate an intermediate feature image based on each median and the pixel position corresponding to each median;

[0019] An object formed by pixel points whose pixel values ​​in the intermediate feature image are not equal to zero is determined as a stationary object in the video to be identified.

[0020] In a fourth aspect, an embodiment of the present application further provides a storage medium containing electronic device executable instructions, which, when executed by an electronic device processor, are used to execute the identification method described in the embodiment of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Other features, objects and advantages of the present application will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings.

[0022] Figure 1 This is a first flow chart of the identification method provided in an embodiment of the present application.

[0023] Figure 2 It is the original image a and the edge gradient map A corresponding to the original image a provided in the embodiment of the present application.

[0024] Figure 3 This is a scenario diagram of the identification method provided in an embodiment of the present application.

[0025] Figure 4 This is a second flow chart of the identification method provided in an embodiment of the present application.

[0026] Figure 5 This is a third flow chart of the identification method provided in an embodiment of the present application.

[0027] Figure 6This is a schematic diagram of the region division of the intermediate feature image provided in an embodiment of the present application.

[0028] Figure 7 It is a structural diagram of the identification device provided in an embodiment of the present application.

[0029] Figure 8 This is a first structural diagram of an electronic device provided in an embodiment of the present application.

[0030] Figure 9 This is a second structural diagram of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended to explain the present application, not to limit the present application. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present application, not all structures.

[0032] The present application provides an identification method, which is applied to an electronic device. The identification method may be performed by an identification device provided in the present application, or an electronic device incorporating the identification device. The identification device may be implemented in hardware or software, and the electronic device may be a smartphone, tablet computer, PDA, laptop computer, desktop computer, or other device equipped with a processor and having processing capabilities.

[0033] See also Figure 1 , Figure 1 This is a schematic diagram of the first flow chart of the identification method provided in the embodiment of the present application. The identification method is applied to the electronic device provided in the embodiment of the present application, such as Figure 1 As shown, the process of the identification method provided in the embodiment of the present application can be as follows:

[0034] 101. Extract multiple frames of original images from the video to be recognized, obtain an edge gradient image corresponding to each frame of the original image, and obtain multiple frames of edge gradient images.

[0035] For example, the electronic device obtains a video to be identified, extracts multiple frames of original images from the video to be identified, obtains an edge gradient image corresponding to each frame of the original image, and obtains multiple frames of edge gradient images.

[0036] The edge gradient image corresponding to each frame of the original image is the image obtained by edge extraction from that frame of the original image. An edge is where a region's attributes suddenly change; it's the intersection of an image region and another region with another attribute. Edges include step edges and roof edges. A step edge has distinctly different pixel values ​​on either side. A roof edge is a turning point where pixel values ​​change from small to large and then back to small.

[0037] For example, see Figure 2 , Figure 2 The original image a and the edge gradient image A corresponding to the original image a provided in the embodiments of the present application are shown. The edge gradient image A corresponding to the original image a is obtained by performing edge extraction on the original image a. Compared to the original image a, in the edge gradient image A, the pixel values ​​of all pixels except those constituting the edge are 0.

[0038] The embodiments of the present application do not impose any specific restrictions on the method for obtaining the edge gradient image corresponding to each frame of the original image. For example, the edge gradient image corresponding to each frame of the original image can be obtained by using a Laplacian edge detection operator. For example, the edge gradient image corresponding to each frame of the original image can be obtained by using a Roberts edge detection operator. For example, the edge gradient image corresponding to each frame of the original image can be obtained by using a Sobel edge detection operator. For example, the edge gradient image corresponding to each frame of the original image can be obtained by using a Kirsch edge detection operator, etc.

[0039] It should be noted that the edge gradient image corresponding to the original image is obtained by using an edge detection operator. Different edge detection operators are used to obtain different edge gradient images.

[0040] 102. Obtain a pixel value of each pixel point in each frame of edge gradient image, and determine a median of the pixel values ​​of pixels located at the same position in the plurality of frames of edge image.

[0041] For example, after obtaining the edge gradient image corresponding to each frame of the original image and obtaining multiple frames of edge gradient images, the electronic device obtains the pixel value of each pixel point in each frame of the edge gradient image and determines the median of the pixel values ​​of the pixel points located at the same position in the multiple frames of edge images.

[0042] For example, see Figure 3 , Figure 3 Schematic diagram of a scenario for the recognition method provided in an embodiment of the present application. Assume that the multi-frame edge image is three edge gradient images, labeled edge gradient image B1, edge gradient image B2, and edge gradient image B3. Obtain the pixel value of each pixel in edge gradient image B1, edge gradient image B2, and edge gradient image B3.

[0043] The following description of the acquisition of the median is based on the center position of three edge gradient images. After the electronic device acquires the pixel value of each pixel in edge gradient image B1, it is known that the pixel value of the pixel at the center position of edge gradient image B1 is P1. Similarly, it is known that the pixel value of the pixel at the center position of edge gradient image B2 is P2, and the pixel value of the pixel at the center position of edge gradient image B3 is P3. According to the principle that in an edge gradient image, except for the pixels constituting the edge, the pixel values ​​of other pixels are 0, P1 = 0 (because the pixel at the center position of edge gradient image B1 does not constitute an edge), P2 ≠ 0 (because the pixel at the center position of edge gradient image B2 does constitute an edge), and P3 = 0 (because the pixel at the center position of edge gradient image B3 does not constitute an edge). The median of pixel values ​​P1, P2, and P3 is obtained, and the median of the pixel values ​​of the pixels at the center position in the three edge images is P1 or P3, that is, the median is 0.

[0044] It is understandable that, because the three edge gradient images are obtained from the same video, the three edge gradient images have the same size, that is, the edge gradient images obtained from one video in the embodiment of the present application have the same size.

[0045] 103. Generate an intermediate feature image based on each median and the pixel position corresponding to each median.

[0046] For example, after determining the median of the pixel values ​​of the pixels at the same position in the multiple edge images, the electronic device can generate an intermediate feature image based on each median and the pixel position corresponding to each median. The size of the intermediate feature image is the same as the size of each edge gradient image.

[0047] For example, following the example above where "the median of the pixel values ​​of the pixel at the center is P1 or P3," the electronic device obtains the median of the pixel values ​​of the pixels at the same position in the three edge images in the above manner and then generates an intermediate feature image based on the median and the pixel position corresponding to the median. For example, the pixel value of the pixel at the center of the intermediate feature image is the median P1 or P3.

[0048] It should be noted that the intermediate feature image generated according to each median and the pixel position corresponding to each median can eliminate objects whose positions have changed in the multi-frame edge images and retain objects whose positions have not changed in the multi-frame edge images.

[0049] 104. Determine an object formed by pixels in the intermediate feature image whose pixel values ​​are not equal to zero as a stationary object in the video to be identified.

[0050] For example, after generating the intermediate feature image, the electronic device may determine that the objects formed by the pixels in the intermediate feature image whose pixel values ​​are not equal to zero are stationary objects in the video to be identified. There may be one or more stationary objects.

[0051] From the above, it can be seen that in the embodiment of the present application, when identifying a stationary object in a video to be identified, the stationary object is not determined directly based on the object changes in multiple frames of images captured from the video to be identified, but is determined based on the object changes in multiple frames of edge gradient images. Because the edge gradient image only retains the edge of each object, there are fewer interference factors, which is conducive to improving the recognition accuracy of stationary objects in the video to be identified.

[0052] See also Figure 4 , Figure 4 A second flow chart of the identification method provided in an embodiment of the present application.

[0053] 201. Extract multiple frames of original images from the video to be recognized, obtain an edge gradient image corresponding to each frame of the original image, and obtain multiple frames of edge gradient images.

[0054] For example, an electronic device obtains a video to be identified, extracts multiple frames of original images from the video to be identified, obtains an edge gradient image corresponding to each frame of the original image, and obtains multiple frames of edge gradient images. The edge gradient image corresponding to each frame of the original image is an image obtained after edge extraction of the frame of the original image. An edge is a location where a region's attributes suddenly change, and is the intersection of an image region and another attribute region. Edges include step edges and roof edges. The pixel values ​​of pixels on both sides of a step edge are significantly different. A roof edge is at a turning point where the pixel value changes from small to large and then to small.

[0055] In another example, multiple frames of original images are extracted from a video to be recognized, and edge gradient images corresponding to each frame of the original image are obtained using multiple different methods, resulting in multiple edge gradient images. Each edge gradient image includes multiple frames of edge gradient images. For the same frame of the original image, the edge gradient image obtained using the first method is different from the edge gradient image obtained using the second method.

[0056] For example, extract S frames of original images from a video to be recognized. A first edge gradient image corresponding to each frame of the original image is obtained using a first method, resulting in a first edge gradient image. The first edge gradient image includes the S frame edge gradient images. A second edge gradient image corresponding to each frame of the original image is obtained using a second method, resulting in a second edge gradient image. The second edge gradient image includes the S frame edge gradient images. The first method is different from the second method. For the same frame of the original image, the edge gradient image obtained using the first method is different from the edge gradient image obtained using the second method.

[0057] Next, each edge gradient image is processed, and the pixel value of each pixel in each frame of the edge gradient image is obtained. The median of the pixel values ​​of pixels located at the same position in the multiple edge images is determined. Based on each median and the pixel position corresponding to each median, a frame of intermediate feature image is generated. After processing the multiple edge gradient images in the above manner, multiple frames of intermediate feature images are obtained. Objects consisting of pixels with non-zero pixel values ​​in each frame of the intermediate feature image are determined as candidate stationary objects in the video to be identified.

[0058] It should be noted that the number of ways to obtain the edge gradient image corresponding to each frame of the original image is equal to the number of intermediate feature images.

[0059] Next, the recognition rate of the same object among the candidate stationary objects is calculated, where the object recognition rate = the number of times the object is recognized / the number of intermediate feature images. When the object recognition rate reaches a preset ratio, the object is determined to be a stationary object in the video to be recognized.

[0060] In some embodiments, when extracting multiple frames of original images from the video to be identified, the electronic device may continuously extract multiple frames of original images from the video to be identified.

[0061] For example, the electronic device intercepts a short video segment with a playback time of 20 minutes to 30 minutes from the video to be identified, and uses all original images in the short video segment as the multiple frames of original images extracted from the video to be identified.

[0062] In some embodiments, when extracting multiple frames of original images from a video to be identified, the electronic device may extract multiple frames of original images from the video to be identified at intervals according to a time axis of the video to be identified.

[0063] For example, the electronic device obtains the original image played when the playback time is 1 minute, the original image played when the playback time is 21 minutes, the original image played when the playback time is 41 minutes, and the original image played when the playback time is 61 minutes based on the timeline of the video to be identified as multiple frames of original images extracted from the video to be identified.

[0064] 202. Obtain the pixel value of each pixel point in each frame of the edge gradient image, and determine the median of the pixel values ​​of the pixels located at the same position in the multiple frames of the edge image.

[0065] For example, after obtaining the edge gradient image corresponding to each frame of the original image and obtaining multiple frames of edge gradient images, the electronic device obtains the pixel value of each pixel point in each frame of the edge gradient image and determines the median of the pixel values ​​of the pixel points located at the same position in the multiple frames of edge images.

[0066] It can be understood that in the embodiment of the present application, the edge gradient images of multiple frames obtained from one video have the same size.

[0067] 203. Generate an intermediate feature image according to each median and the pixel position corresponding to each median.

[0068] For example, after determining the median of the pixel values ​​of the pixels at the same position in the multiple edge images, the electronic device can generate an intermediate feature image based on each median and the pixel position corresponding to each median. The size of the intermediate feature image is the same as the size of each edge gradient image.

[0069] It should be noted that the intermediate feature image generated according to each median and the pixel position corresponding to each median can eliminate objects whose positions have changed in the multi-frame edge images and retain objects whose positions have not changed in the multi-frame edge images.

[0070] 204. Determine an object formed by pixels in the intermediate feature image whose pixel values ​​are not equal to zero as a stationary object in the video to be identified.

[0071] For example, after generating the intermediate feature image, the electronic device may determine that the objects formed by the pixels in the intermediate feature image whose pixel values ​​are not equal to zero are stationary objects in the video to be identified. There may be one or more stationary objects.

[0072] 205. Match the stationary object with a plurality of preset identifiers, and determine the number of preset identifiers that successfully match the stationary object.

[0073] For example, after identifying a stationary object in a video to be recognized, if the type of the stationary object is still unclear, the electronic device can match each stationary object with multiple preset identifiers and determine the number of preset identifiers that successfully match each stationary object. It should be noted that if an identifier exists in the video, it generally belongs to a type of stationary object.

[0074] For example, suppose there are two stationary objects in the video to be identified, which are respectively denoted as stationary object R1 and stationary object R2. The electronic device matches the stationary object R1 with multiple preset identifiers to obtain the similarity between the stationary object R1 and each preset identifier, and uses the preset identifier corresponding to the similarity that meets the preset conditions as the candidate identifier of the stationary object R1, and uses the identifier with the highest similarity among the candidate identifiers as the preset identifier that successfully matches the stationary object R1. Similarly, the preset identifier that successfully matches the stationary object R2 is determined, and finally the number of preset identifiers that successfully match the two stationary objects is determined. It should be noted that the number of preset identifiers that successfully match each stationary object can only be 0 or 1.

[0075] The electronic device may store multiple preset identifiers in advance, and the multiple preset identifiers may be added or removed by the user. For example, when the electronic device plays a video on a display interface, if a preset identifier storage instruction is received from the user, the new preset identifier may be stored in the memory according to the preset identifier storage instruction.

[0076] It should be noted that the user can trigger the shooting instruction in a preset way. For example, when watching a video, the user slides down on the display screen with three fingers to trigger the preset identification storage instruction. When the electronic device receives the preset identification storage instruction, it obtains the display image when the preset identification storage instruction is triggered, identifies the identification in the display image, and saves the identification as a new preset identification to the memory. For another example, when watching a video, the user performs a circle operation on the display screen to trigger the preset identification storage instruction. When the electronic device receives the preset identification storage instruction, it obtains the circled area of ​​the circle operation, saves the object in the circled area as a new preset identification to the memory, etc.

[0077] 206. Determine a recommendation priority of the to-be-identified video according to the preset number of identifiers, wherein the preset number of identifiers is inversely proportional to the recommendation priority.

[0078] For example, after determining the number of preset identifiers that successfully match a stationary object, the electronic device can determine a recommendation priority for the video to be identified based on the number of preset identifiers. The number of preset identifiers is inversely proportional to the recommendation priority. That is, the greater the number of preset identifiers, the lower the recommendation priority, and the less likely the electronic device is to recommend the video to be identified; the fewer the number of preset identifiers, the higher the recommendation priority, and the more likely the electronic device is to recommend the video to be identified.

[0079] In some embodiments, when a need for video recommendation is detected, the electronic device obtains the user's historical browsing history, determines the target type of video to be recommended from the historical browsing history, searches for the video corresponding to the target type in the first preset video library, and uses the video corresponding to the target type as a candidate video. According to the recommendation priority of the candidate video, the video is displayed on the display interface from high to low in descending order of recommendation priority.

[0080] In some embodiments, when it is detected that a video recommendation is needed, the electronic device obtains the user's historical browsing history, determines a target type of video that most needs to be recommended from the historical browsing history, searches for videos corresponding to the target type in the first sub-video library of the second preset video library (storing videos with the highest recommendation priority), and uses the videos corresponding to the target type as alternative videos, and displays the alternative videos on the display interface; when it is detected that the user has viewed a certain proportion (such as 50%), searches for videos corresponding to the target type in the second sub-video library of the second preset video library (storing videos with the second highest recommendation priority), uses the videos corresponding to the target type as alternative videos, and displays the alternative videos on the display interface, and so on.

[0081] See also Figure 5 , Figure 5 This is a schematic diagram of the third flow chart of the identification method provided in an embodiment of the present application.

[0082] 301. Extract multiple frames of original images from the video to be recognized, obtain an edge gradient image corresponding to each frame of the original image, and obtain multiple frames of edge gradient images.

[0083] For example, an electronic device obtains a video to be identified, extracts multiple frames of original images from the video to be identified, obtains an edge gradient image corresponding to each frame of the original image, and obtains multiple frames of edge gradient images. The edge gradient image corresponding to each frame of the original image is an image obtained after edge extraction of the frame of the original image. An edge is a location where a region's attributes suddenly change, and is the intersection of an image region and another attribute region. Edges include step edges and roof edges. The pixel values ​​of pixels on both sides of a step edge are significantly different. A roof edge is at a turning point where the pixel value changes from small to large and then to small.

[0084] 302 : Obtain the pixel value of each pixel point in each frame of edge gradient image, and determine the median of the pixel values ​​of the pixels located at the same position in the multiple frames of edge image.

[0085] For example, after obtaining the edge gradient image corresponding to each frame of the original image and obtaining multiple frames of edge gradient images, the electronic device obtains the pixel value of each pixel point in each frame of the edge gradient image and determines the median of the pixel values ​​of the pixel points located at the same position in the multiple frames of edge images.

[0086] It can be understood that in the embodiment of the present application, the edge gradient images of multiple frames obtained from one video have the same size.

[0087] 303. Generate an intermediate feature image based on each median and the pixel position corresponding to each median.

[0088] For example, after determining the median of the pixel values ​​of the pixels at the same position in the multiple edge images, the electronic device can generate an intermediate feature image based on each median and the pixel position corresponding to each median. The size of the intermediate feature image is the same as the size of each edge gradient image.

[0089] It should be noted that the intermediate feature image generated according to each median and the pixel position corresponding to each median can eliminate objects whose positions have changed in the multi-frame edge images and retain objects whose positions have not changed in the multi-frame edge images.

[0090] 304. Determine an object formed by pixels in the intermediate feature image whose pixel values ​​are not equal to zero as a stationary object in the video to be identified.

[0091] For example, after generating the intermediate feature image, the electronic device may determine that the objects formed by the pixels in the intermediate feature image whose pixel values ​​are not equal to zero are stationary objects in the video to be identified. There may be one or more stationary objects.

[0092] 305. Determine whether the intermediate feature image includes the identifier of the video to be identified based on the pre-trained recognition model.

[0093] For example, after determining the presence of a stationary object in the video to be identified, the pre-trained recognition model is used to determine whether the intermediate feature image contains the identifier of the video to be identified. It should be noted that in this solution, the identifier of the video to be identified may be one or more. For example, the identifier of the video to be identified may include "XXTV" and "XX Theater".

[0094] For example, assume that based on the intermediate feature image, it is determined that there are three stationary objects in the video to be identified, which are recorded as stationary object Y1, stationary object Y2, and stationary object Y3. The electronic device inputs the intermediate feature image into a pre-trained recognition model and obtains an output result of 0 or 1. When the output result is 0, it is determined that the intermediate feature image does not include the identifier of the video to be identified. When the output result is 1, it is determined that the intermediate feature image includes the identifier of the video to be identified. Among them, 0 indicates that the intermediate feature image does not include the identifier of the video to be identified, that is, the stationary objects Y1, stationary object Y2, and stationary object Y3 are objects other than the identifier; 1 indicates that the intermediate feature image includes the identifier of the video to be identified.

[0095] For example, suppose the intermediate feature image determines that there are three stationary objects in the video to be identified. These objects are designated as stationary objects Y4, Y5, and Y6. The electronic device inputs the intermediate feature image into a pre-trained recognition model, which outputs the object type of each stationary object. For example, the object type of stationary object Y4 is "building," the object type of stationary object Y5 is "sign," and the object type of stationary object Y6 is "person." Based on the object type of each stationary object, the electronic device determines whether the intermediate feature image contains the sign of the video to be identified.

[0096] It should be noted that the input of the recognition model in this solution is the intermediate feature image obtained from the original image, which is beneficial to improving the accuracy of the recognition model's identification judgment compared to directly inputting the original image into the recognition model.

[0097] In some embodiments, before determining whether the intermediate feature image contains the identifier of the video to be recognized based on a pre-trained recognition model, the electronic device may obtain multiple frames of intermediate feature images from multiple training videos to form a training set. This training set is used to train a pre-set convolutional neural network model, and the trained convolutional neural network model is used as the recognition model. In this solution, using intermediate feature images to train the recognition model can improve the model's recognition accuracy.

[0098] 306. If the intermediate feature image does not include the identifier of the video to be identified, determine the highest level among multiple preset levels as the recommended level for the video to be identified.

[0099] For example, after determining that the intermediate feature image does not include the identifier of the video to be identified, the electronic device may determine the highest level among multiple preset levels as the recommendation level for the video to be identified. It should be noted that the higher the recommendation level of the video to be identified, the more likely the electronic device is to recommend it. Compared to videos without identifiers, videos with identifiers may obscure the video content, resulting in poor viewing experience for users. Therefore, in this solution, videos without identifiers have the highest recommendation level.

[0100] 307. If the intermediate feature image includes the identifier of the video to be identified, determine the area ratio of the identifier in the video to be identified.

[0101] For example, after determining that the intermediate feature image includes a logo in the video to be identified, the electronic device can determine the area occupied by the logo and calculate the area ratio of the logo in the video to be identified according to the formula: area ratio = area occupied by the logo / area occupied by the intermediate feature image.

[0102] For another example, after determining that the intermediate feature image includes multiple identifiers of the video to be identified, the electronic device can determine the area ratios of the multiple identifiers in the video to be identified. The electronic device can determine the largest area ratio among multiple candidate area ratios (the area ratios of each identifier in the video to be identified are the candidate area ratios) as the area ratios of the multiple identifiers in the video to be identified. The electronic device can also calculate the area ratios of the multiple identifiers in the video to be identified, etc.

[0103] For example, assuming that a video to be identified includes a first identifier and a second identifier, the electronic device may determine a first area ratio of the first identifier in the video to be identified, determine a second area ratio of the second identifier in the video to be identified, and use the largest of the first area ratio and the second area ratio as the area ratio of the video to be identified. If the first area ratio is greater than the second area ratio, the area ratio of the multiple identifiers in the video to be identified is determined to be the first area ratio.

[0104] For example, assuming that the video to be identified includes a first identifier and a second identifier, the electronic device can determine the area occupied by the first identifier and the area occupied by the second identifier according to the formula: Area ratio = (Area occupied by the first identifier + Area occupied by the second identifier) ​​ / Area occupied by the intermediate feature image, calculate the area ratio, and use it as the area ratio of multiple identifiers in the video to be identified.

[0105] 308. Determine a recommendation level for the video to be identified from preset levels other than the highest level based on the area ratio.

[0106] For example, if a video to be identified only includes one identifier, after determining the area ratio of the identifier in the video to be identified, the electronic device can determine a recommendation level for the video to be identified from a preset level other than the highest level based on the area ratio of the identifier in the video to be identified. The larger the area ratio, the lower the recommendation level, and the lower the probability that the electronic device will recommend the video to be identified; the smaller the area ratio, the higher the recommendation level, and the higher the probability that the electronic device will recommend the video to be identified.

[0107] In some embodiments, after determining that the intermediate feature image includes the identifier of the video to be identified, the electronic device may further determine the position of the identifier in the video to be identified and determine a recommendation level for the video to be identified from preset levels other than the highest level based on the position.

[0108] Among them, when determining the recommended level of the video to be identified from the preset levels other than the highest level according to the position, the electronic device can determine the number of divisions V of the intermediate feature image according to the difference V between the number of preset levels and 1, and divide the intermediate feature image into V areas using rectangles from the center of the intermediate feature image, each area corresponds to a preset level, and the closer the area is to the edge, the higher the preset level.

[0109] like Figure 6 As shown, Figure 6 Schematic diagram of the regional division of the intermediate feature image provided in the embodiment of the present application. Assume that the number of preset levels is 6, the preset levels are recorded as D1, D2, D3, D4, D5 and D6, and the levels are D1>D2>D3>D4>D5>D6. When determining the recommended level of the video to be identified from the preset levels other than the highest level (from D2, D3, D4, D5 and D6) according to the position, the electronic device can determine that the number of divisions of the intermediate feature image is 5, and divide the intermediate feature image into 5 areas with a rectangle from the center of the intermediate feature image, recorded as area Q1, area Q2, area Q3, area Q4 and area Q5. Each area corresponds to a preset level, and the closer the area is to the edge, the higher the preset level, such as area Q1 corresponds to preset level D6, area Q2 corresponds to preset level D5, area Q3 corresponds to preset level D4, area Q4 corresponds to preset level D3, and area Q5 corresponds to preset level D1.

[0110] Next, based on the location of the video's identifier, the recommended level for the video is determined from the preset levels (D2, D3, D4, D5, and D6) other than the highest level. For example, if the video's identifier is located in area Q3, the recommended level for the video is determined to be preset level D4.

[0111] It should be noted that when the intermediate feature image includes multiple identifiers of the video to be identified, multiple levels can be obtained according to the positions of the multiple identifiers, and the lowest level among the multiple levels is used as the recommended level of the video to be identified.

[0112] Figure 7 This is a schematic diagram of the structure of the identification device provided in the embodiment of the present application. The device is used to execute the identification method provided in the above embodiment and has the corresponding functional modules and beneficial effects of the execution method. Figure 7 As shown, the identification device 400 specifically includes: a first acquisition module 401, a first determination module 402, a generation module 403 and a second determination module 404, wherein:

[0113] The first acquisition module 401 is used to extract multiple frames of original images from the video to be identified, obtain the edge gradient image corresponding to each frame of the original image, and obtain multiple frames of edge gradient images;

[0114] A first determining module 402 is configured to obtain a pixel value of each pixel in each edge gradient image frame and determine a median of pixel values ​​of pixels located at the same position in the edge images of the plurality of frames;

[0115] A generating module 403 is configured to generate an intermediate feature image based on each median and the pixel position corresponding to each median;

[0116] The second determining module 404 is configured to determine an object formed by pixels having non-zero pixel values ​​in the intermediate feature image as a stationary object in the video to be identified.

[0117] In some embodiments, when extracting multiple frames of original images from a video to be identified, the first acquisition module 401 is configured to extract multiple frames of original images from the video to be identified at intervals according to a time axis of the video to be identified.

[0118] In some embodiments, after determining the object composed of pixels having non-zero pixel values ​​in the intermediate feature image as a stationary object in the video to be identified, the identification device 400 further includes a matching module and a third determination module;

[0119] The matching module is configured to match the stationary object with a plurality of preset identifiers and determine the number of preset identifiers that successfully match the stationary object;

[0120] The third determination module is configured to determine a recommendation priority of the video to be identified based on the number of preset identifiers, wherein the number of preset identifiers is inversely proportional to the recommendation priority.

[0121] In some embodiments, after determining the object composed of pixels having non-zero pixel values ​​in the intermediate feature image as a stationary object in the video to be identified, the identification device 400 further includes a judgment module and a fourth determination module;

[0122] The judgment module is used to judge whether the intermediate feature image includes the identifier of the video to be recognized based on a pre-trained recognition model;

[0123] The fourth determining module is configured to determine the highest level among a plurality of preset levels as the recommended level for the video to be identified if the intermediate feature image does not include the identifier of the video to be identified.

[0124] In some embodiments, after determining whether the intermediate feature image includes the identifier of the video to be identified, the identification device 400 further includes a fifth determination module and a sixth determination module;

[0125] The fifth determining module is configured to determine a position of the identifier of the video to be identified if the intermediate feature image includes the identifier of the video to be identified;

[0126] The sixth determining module is configured to determine, based on the position, a recommendation level of the video to be identified from preset levels other than the highest level.

[0127] In some embodiments, after determining whether the intermediate feature image includes the identifier of the video to be identified, the fifth determination module is configured to determine an area ratio of the identifier in the video to be identified if the intermediate feature image includes the identifier of the video to be identified;

[0128] The sixth determination module is configured to determine, based on the area ratio, a recommendation level of the video to be identified from preset levels other than the highest level.

[0129] In some embodiments, before determining whether the intermediate feature image includes the identifier of the video to be identified, the identification device 400 further includes a second acquisition module and a training module;

[0130] The second acquisition module is used to obtain multiple frames of intermediate feature images obtained from multiple training videos to form a training set;

[0131] The training module is used to train a preset convolutional neural network model using the training set, and use the trained convolutional neural network model as a recognition model.

[0132] It should be noted that the identification device provided in the embodiment of the present application belongs to the same concept as the identification method in the above embodiment. Any method provided in the identification method embodiment can be run on the identification device. The specific implementation process is detailed in the identification method embodiment and will not be repeated here.

[0133] The present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed on a computer, the computer performs the steps of the identification method provided in the present invention. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0134] The present application also provides an electronic device. Figure 8 , the electronic device 500 includes a processor 501 and a memory 502. The processor 501 is electrically connected to the memory 502.

[0135] The processor 501 is the control center of the electronic device 500. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device 500 and processes data by running or loading computer programs stored in the memory 502 and calling data stored in the memory 502.

[0136] The memory 502 can be used to store software programs and modules. The processor 501 executes various functional applications and data processing by running the computer programs and modules stored in the memory 502. The memory 502 can mainly include a program storage area and a data storage area. The program storage area can store an operating system and computer programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created based on the use of the electronic device.

[0137] In addition, the memory 502 may include a high-speed random access memory and a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 502 may also include a memory controller to provide the processor 501 with access to the memory 502.

[0138] In the embodiment of the present application, the processor 501 in the electronic device 500 loads instructions corresponding to one or more computer program processes into the memory 502 according to the following steps, and the processor 501 runs the computer program stored in the memory 502 to implement various functions as follows:

[0139] Extract multiple frames of original images from the video to be recognized, obtain the edge gradient image corresponding to each frame of the original image, and obtain multiple frames of edge gradient images;

[0140] Obtaining a pixel value of each pixel point in each frame of the edge gradient image, and determining a median of the pixel values ​​of the pixel points located at the same position in the plurality of frames of the edge image;

[0141] Generate an intermediate feature image based on each median and the pixel position corresponding to each median;

[0142] An object formed by pixel points whose pixel values ​​in the intermediate feature image are not equal to zero is determined as a stationary object in the video to be identified.

[0143] Please refer to Figure 9 , Figure 9 A second structural diagram of an electronic device provided in an embodiment of the present application, Figure 8 The difference between the electronic device shown is that the electronic device further includes: a camera component 603, a display component 604, an audio circuit 605, a radio frequency circuit 606, and a power supply 607. Among them, the camera component 603, the display component 604, the audio circuit 605, the radio frequency circuit 606, and the power supply 607 are electrically connected to the processor 601 respectively.

[0144] The camera component 603 may include an image processing circuit, which may be implemented using hardware and / or software components and may include various processing units that define an image signal processing (Image Signal Processing) pipeline. The image processing circuit may include at least: multiple cameras, an image signal processor (ISP processor), a control logic, an image memory, and a display. Each camera may include at least one or more lenses and an image sensor. The image sensor may include a color filter array (such as a Bayer filter). The image sensor may obtain light intensity and wavelength information captured by each imaging pixel of the image sensor and provide a set of raw image data that can be processed by the image signal processor.

[0145] The display component 604 can be used to display information input by the user or information provided to the user and various graphical user interfaces. These graphical user interfaces can be composed of graphics, texts, icons, videos and any combination thereof.

[0146] The audio circuit 605 may be configured to provide an audio interface between a user and the electronic device through a speaker and a microphone.

[0147] The radio frequency circuit 606 can be used to transmit and receive radio frequency signals, so as to establish wireless communication with a network device or other electronic devices through wireless communication, and to transmit and receive signals with the network device or other electronic devices.

[0148] The power supply 607 can be used to supply power to various components of the electronic device 600. In some embodiments, the power supply 607 can be logically connected to the processor 601 through a power management system, thereby managing charging, discharging, and power consumption through the power management system.

[0149] In the embodiment of the present application, the processor 601 in the electronic device 600 loads instructions corresponding to one or more computer program processes into the memory 602 according to the following steps, and the processor 601 runs the computer program stored in the memory 602 to implement various functions as follows:

[0150] Extract multiple frames of original images from the video to be recognized, obtain the edge gradient image corresponding to each frame of the original image, and obtain multiple frames of edge gradient images;

[0151] Obtaining a pixel value of each pixel point in each frame of the edge gradient image, and determining a median of the pixel values ​​of the pixel points located at the same position in the plurality of frames of the edge image;

[0152] Generate an intermediate feature image based on each median and the pixel position corresponding to each median;

[0153] An object formed by pixel points whose pixel values ​​in the intermediate feature image are not equal to zero is determined as a stationary object in the video to be identified.

[0154] In some embodiments, when extracting multiple frames of original images from the video to be identified, the processor 601 may execute:

[0155] According to the time axis of the video to be identified, multiple frames of original images are extracted from the video to be identified at intervals.

[0156] In some embodiments, after determining that an object formed by pixels having non-zero pixel values ​​in the intermediate feature image is a stationary object in the video to be identified, the processor 601 may execute:

[0157] Matching the stationary object with a plurality of preset identifiers, and determining the number of preset identifiers that successfully match the stationary object;

[0158] The recommendation priority of the video to be identified is determined according to the number of preset identifiers, wherein the number of preset identifiers is inversely proportional to the recommendation priority.

[0159] In some embodiments, after determining that an object formed by pixels having non-zero pixel values ​​in the intermediate feature image is a stationary object in the video to be identified, the processor 601 may execute:

[0160] Determining, based on a pre-trained recognition model, whether the intermediate feature image includes an identifier of the video to be recognized;

[0161] If the intermediate feature image does not include the identifier of the video to be identified, the highest level among multiple preset levels is determined as the recommended level of the video to be identified.

[0162] In some embodiments, after determining whether the intermediate feature image includes the identifier of the video to be identified, the processor 601 may execute:

[0163] If the intermediate feature image includes the identifier of the video to be identified, determining a position of the identifier in the video to be identified;

[0164] According to the position, a recommendation level of the video to be identified is determined from preset levels except the highest level.

[0165] In some embodiments, after determining whether the intermediate feature image includes the identifier of the video to be identified, the processor 601 may execute:

[0166] If the intermediate feature image includes the identifier of the video to be identified, determining the area ratio of the identifier in the video to be identified;

[0167] According to the area proportion, a recommendation level of the to-be-identified video is determined from preset levels except the highest level.

[0168] In some embodiments, before determining whether the intermediate feature image includes the identifier of the video to be identified, the processor 601 may execute:

[0169] Obtain multiple frames of intermediate feature images from multiple training videos to form a training set;

[0170] The training set is used to train a preset convolutional neural network model, and the trained convolutional neural network model is used as a recognition model.

[0171] From the above, it can be seen that the electronic device provided in this embodiment, after extracting multiple frames of original images from the video to be identified, obtains the edge gradient image corresponding to each frame of the original image to obtain multiple frames of edge gradient images, and then determines the median of the pixel values ​​of the pixel points located at the same position in the multiple frames of edge images. Then, based on each median and the pixel point position corresponding to each median, an intermediate feature image is generated. Finally, the object composed of pixel points with pixel values ​​not equal to zero in the intermediate feature image is determined as a stationary object in the video to be identified, which can improve the recognition accuracy of stationary objects in the video to be identified.

[0172] An embodiment of the present application also provides a storage medium, which stores a computer program. When the computer program runs on a computer, the computer executes the recognition method in any of the above embodiments, such as: extracting multiple frames of original images from the video to be identified, obtaining an edge gradient image corresponding to each frame of the original image, and obtaining multiple frames of edge gradient images; obtaining the pixel value of each pixel point in each frame of the edge gradient image, and determining the median of the pixel values ​​of the pixels located at the same position in the multiple frames of edge images; generating an intermediate feature image based on each median and the pixel point position corresponding to each median; and determining an object composed of pixels whose pixel values ​​in the intermediate feature image are not equal to zero as a stationary object in the video to be identified.

[0173] In the embodiment of the present application, the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0174] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0175] It should be noted that, with respect to the identification method of the embodiment of the present application, a person skilled in the art will understand that all or part of the process of implementing the identification method of the embodiment of the present application can be accomplished by controlling the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, such as a memory of an electronic device, and executed by at least one processor in the electronic device. During execution, the process may include the process of the embodiment of the identification method. The storage medium may be a magnetic disk, an optical disk, a read-only memory, a random access memory, or the like.

[0176] For the identification device of the embodiment of the present application, its various functional modules can be integrated into a processing chip, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated modules can be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk or an optical disk.

[0177] The above is a detailed introduction to an identification method, device, storage medium and electronic device provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A recognition method, wherein: include: Extract multiple frames of original images from the video to be recognized, obtain the edge gradient image corresponding to each frame of the original image, and obtain multiple frames of edge gradient images; Obtaining a pixel value of each pixel point in each frame of the edge gradient image, and determining a median of the pixel values ​​of the pixel points located at the same position in the plurality of frames of the edge gradient image; Generate an intermediate feature image based on each median and the pixel position corresponding to each median; An object formed by pixel points whose pixel values ​​in the intermediate feature image are not equal to zero is determined as a stationary object in the video to be identified.

2. The identification method according to claim 1, wherein: The step of extracting multiple frames of original images from the video to be identified includes: According to the time axis of the video to be identified, multiple frames of original images are extracted from the video to be identified at intervals.

3. The identification method according to claim 1, wherein: After determining the object formed by the pixels having non-zero pixel values ​​in the intermediate feature image as a stationary object in the video to be identified, the method further includes: Matching the stationary object with a plurality of preset identifiers, and determining the number of preset identifiers that successfully match the stationary object; The recommendation priority of the video to be identified is determined according to the number of preset identifiers, wherein the number of preset identifiers is inversely proportional to the recommendation priority.

4. The identification method according to claim 1, wherein: After determining the object formed by the pixels having non-zero pixel values ​​in the intermediate feature image as a stationary object in the video to be identified, the method further includes: Determining, based on a pre-trained recognition model, whether the intermediate feature image includes an identifier of the video to be recognized; If the intermediate feature image does not include the identifier of the video to be identified, the highest level among multiple preset levels is determined as the recommended level of the video to be identified.

5. The identification method according to claim 4, wherein: After determining whether the intermediate feature image includes the identifier of the video to be identified, the method further includes: If the intermediate feature image includes the identifier of the video to be identified, determining a position of the identifier in the video to be identified; According to the position, a recommendation level of the video to be identified is determined from preset levels except the highest level.

6. The identification method according to claim 4, wherein: After determining whether the intermediate feature image includes the identifier of the video to be identified, the method further includes: If the intermediate feature image includes the identifier of the video to be identified, determining the area ratio of the identifier in the video to be identified; According to the area proportion, a recommendation level of the to-be-identified video is determined from preset levels except the highest level.

7. The identification method according to claim 4, wherein: Before determining whether the intermediate feature image includes the identifier of the video to be identified, the method further includes: Obtain multiple frames of intermediate feature images from multiple training videos to form a training set; The training set is used to train a preset convolutional neural network model, and the trained convolutional neural network model is used as a recognition model.

8. An identification device, wherein: include: The first acquisition module is used to extract multiple frames of original images from the video to be identified, obtain the edge gradient image corresponding to each frame of the original image, and obtain multiple frames of edge gradient images; a first determining module, configured to obtain a pixel value of each pixel point in each frame of the edge gradient image, and determine a median of the pixel values ​​of the pixels located at the same position in the plurality of frames of the edge gradient image; A generation module is used to generate an intermediate feature image based on each median and the pixel position corresponding to each median; The second determination module is configured to determine an object formed by pixels having non-zero pixel values ​​in the intermediate feature image as a stationary object in the video to be identified.

9. The identification device according to claim 8, wherein: The first acquisition module is used to extract multiple frames of original images from the video to be identified at intervals according to the time axis of the video to be identified.

10. The identification device according to claim 8, wherein: The identification device further comprises: A matching module, configured to match the stationary object with a plurality of preset identifiers, and determine the number of preset identifiers that successfully match the stationary object; The third determining module is configured to determine a recommendation priority of the video to be identified according to the number of preset identifiers, wherein the number of preset identifiers is inversely proportional to the recommendation priority.

11. The identification device according to claim 8, wherein: The identification device further comprises: A judgment module, configured to judge whether the intermediate feature image includes the identifier of the video to be recognized based on a pre-trained recognition model; The fourth determining module is configured to determine the highest level among a plurality of preset levels as the recommended level for the video to be identified if the intermediate feature image does not include the identifier of the video to be identified.

12. The identification device according to claim 11, wherein: The identification device further comprises: The second acquisition module is used to obtain multiple frames of intermediate feature images obtained from multiple training videos to form a training set; The training module is used to train a preset convolutional neural network model using the training set, and use the trained convolutional neural network model as a recognition model.

13. An electronic device comprising: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, an identification method is implemented: Extract multiple frames of original images from the video to be recognized, obtain the edge gradient image corresponding to each frame of the original image, and obtain multiple frames of edge gradient images; Obtaining a pixel value of each pixel point in each frame of the edge gradient image, and determining a median of the pixel values ​​of the pixel points located at the same position in the plurality of frames of the edge gradient image; Generate an intermediate feature image based on each median and the pixel position corresponding to each median; An object formed by pixel points whose pixel values ​​in the intermediate feature image are not equal to zero is determined as a stationary object in the video to be identified.

14. The electronic device according to claim 13, wherein: When extracting multiple frames of original images from the video to be identified, the processor is configured to execute: According to the time axis of the video to be identified, multiple frames of original images are extracted from the video to be identified at intervals.

15. The electronic device according to claim 13, wherein After determining the object formed by the pixels having non-zero pixel values ​​in the intermediate feature image as a stationary object in the video to be identified, the processor is configured to execute: Matching the stationary object with a plurality of preset identifiers, and determining the number of preset identifiers that successfully match the stationary object; The recommendation priority of the video to be identified is determined according to the number of preset identifiers, wherein the number of preset identifiers is inversely proportional to the recommendation priority.

16. The electronic device according to claim 13, wherein: After determining the object formed by the pixels having non-zero pixel values ​​in the intermediate feature image as a stationary object in the video to be identified, the processor is configured to execute: Determining, based on a pre-trained recognition model, whether the intermediate feature image includes an identifier of the video to be recognized; If the intermediate feature image does not include the identifier of the video to be identified, the highest level among multiple preset levels is determined as the recommended level of the video to be identified.

17. The electronic device according to claim 16, wherein: After determining whether the intermediate feature image includes the identifier of the video to be identified, the processor is configured to execute: If the intermediate feature image includes the identifier of the video to be identified, determining a position of the identifier in the video to be identified; According to the position, a recommendation level of the video to be identified is determined from preset levels except the highest level.

18. The electronic device according to claim 16, wherein: After determining whether the intermediate feature image includes the identifier of the video to be identified, the processor is configured to execute: If the intermediate feature image includes the identifier of the video to be identified, determining the area ratio of the identifier in the video to be identified; According to the area proportion, a recommendation level of the to-be-identified video is determined from preset levels except the highest level.

19. The electronic device according to claim 16, wherein Before determining whether the intermediate feature image includes the identifier of the video to be identified, the processor is configured to execute: Obtain multiple frames of intermediate feature images from multiple training videos to form a training set; The training set is used to train a preset convolutional neural network model, and the trained convolutional neural network model is used as a recognition model.

20. A storage medium containing instructions executable by an electronic device, wherein: The electronic device executable instructions are used to perform the identification method according to any one of claims 1 to 7 when executed by a processor of the electronic device.

Images