Video file word segmentation method, device and electronic equipment

Abstract

The present disclosure relates to a video file segmentation method, device and electronic device, the method comprising: determining a video file to be processed, extracting multiple image frames of the video file to be processed; performing text detection processing on the multiple image frames to obtain a first feature map and a second feature map; the first feature map includes global text features that are in the same position and have the same text content in the multiple image frames and a first area position corresponding to the global text feature; the second feature map includes basic text features corresponding to any text content in the multiple image frames; based on the global text features in the first feature map, video information is enhanced on the basic text features to obtain local text features of the multiple image frames; according to the local text features corresponding to the multiple image frames and the relationship between the area position corresponding to the local text features and the first area position, determining whether to perform word segmentation on the word segmentation text corresponding to the local text features in combination with the first feature map to obtain a word segmentation result.

Description

Technical Field

[0001] The present disclosure relates to the field of information processing technology, and in particular to a video file word segmentation method, device and electronic device. Background Art

[0002] With the advancement of technology and the popularization of the Internet, more and more people are searching for short videos to obtain news, knowledge, life dynamics and other information, and can share information by publishing short videos. When obtaining information through short videos, you can search for the required short videos based on your understanding of short videos, or the recommendation system can make personalized recommendations based on its understanding of short videos.

[0003] Specifically, a basic technology for understanding short videos is to perform Chinese word segmentation on the text in the short video. In related technologies, when performing word segmentation on any natural language sentence in a short video, the word segmentation result of the short video is obtained based on a sequence labeling model.

[0004] In the above-mentioned related technologies, the sequence annotation model has limitations in terms of the number of words and templates, and because the sequence annotation template is a template with fixed syntax, the annotated words may have different meanings from the text in the short video. Therefore, the word segmentation results determined by the word segmentation method based on the sequence annotation model may not conform to the true intention of the original text information, resulting in semantic flaws, resulting in short video recommendations or search results that do not meet actual usage needs, reducing the user experience. Summary of the invention

[0005] The present disclosure provides a video file word segmentation method, device and electronic device, which are used to solve the technical problem of low accuracy of word segmentation results corresponding to video files in related technologies. The technical solution of the present disclosure is as follows:

[0006] In a first aspect of an embodiment of the present disclosure, a video file word segmentation method is provided, the method comprising:

[0007] Determine a video file to be processed, and extract multiple image frames of the video file to be processed;

[0008] Performing text detection processing on the multiple image frames to obtain a first feature map and a second feature map; wherein the first feature map includes a global text feature that is in the same position and has the same text content in the multiple image frames and a first area position corresponding to the global text feature; and the second feature map includes a basic text feature corresponding to any text content in the multiple image frames;

[0009] Based on the global text features in the first feature map, the basic text features are enhanced with video information to obtain local text features of the multiple image frames;

[0010] According to the local text features corresponding to the multiple image frames and the relationship between the area position corresponding to the local text features and the first area position, determine whether to perform word segmentation on the word segmentation text corresponding to the local text features in combination with the first feature map to obtain a word segmentation result.

[0011] In a possible implementation, performing text detection processing on the multiple image frames to obtain a first feature map and a second feature map includes:

[0012] Performing text detection processing on the multiple image frames to obtain a text feature atlas; the text feature atlas includes feature maps including text intercepted from the multiple image frames;

[0013] Based on the text feature atlas, two feature maps corresponding to the same text in any two image frames are screened from the multiple image frames, and it is determined that the regions corresponding to the two feature maps meet a first preset condition, and it is determined that the areas corresponding to the two feature maps are the largest among the feature maps that meet the first preset condition, and the two feature maps are used as the first feature maps;

[0014] The feature maps in the text feature map except the first feature map are used as the second feature map.

[0015] In a possible implementation, the first preset condition is: the areas corresponding to the two feature maps are mapped to the same image frame, the distance between the areas corresponding to the two feature maps belongs to a first preset range; and / or the area where the two feature maps are located belongs to a preset position area.

[0016] In a possible implementation, determining whether to perform word segmentation on the word segmentation text corresponding to the local text feature in combination with the first feature map to obtain a word segmentation result, according to the local text features corresponding to the multiple image frames and the relationship between the region position corresponding to the local text feature and the first region position, includes:

[0017] Determine a third region position corresponding to any of the local text features;

[0018] Determine a first distance between the third area position and the corresponding first area position, and determine whether the first distance belongs to a second preset range;

[0019] When it is determined that the first distance belongs to the second preset range, the same characters in the segmented text corresponding to the local text feature and in the text corresponding to the first feature map are taken as a phrase;

[0020] Segment the text corresponding to the local text feature, except for the phrase, based on the natural language segmentation rule to obtain a first segmentation result, and use the phrase and the first segmentation result as the segmentation result;

[0021] When it is determined that the first distance does not belong to the second preset range, segmentation is performed on the segmented text corresponding to the local text feature based on the natural language segmentation rule to obtain a segmentation result.

[0022] According to a second aspect of the present disclosure, a video file word segmentation device is provided, the device comprising:

[0023] An extraction unit is configured to determine a video file to be processed and extract a plurality of image frames of the video file to be processed;

[0024] An obtaining unit is configured to perform text detection processing on the multiple image frames to obtain a first feature map and a second feature map; wherein the first feature map includes a global text feature that is in the same position and has the same text content in the multiple image frames and a first area position corresponding to the global text feature; and the second feature map includes a basic text feature corresponding to any text content in the multiple image frames;

[0025] A processing unit is configured to perform video information enhancement on the basic text features based on the global text features in the first feature map to obtain local text features of the multiple image frames;

[0026] The word segmentation unit is configured to determine whether to perform word segmentation on the word segmentation text corresponding to the local text features in combination with the first feature map based on the local text features corresponding to the multiple image frames and the relationship between the area position corresponding to the local text features and the first area position to obtain a word segmentation result.

[0027] In a possible implementation, the obtaining unit is configured to execute:

[0028] Performing text detection processing on the multiple image frames to obtain a text feature atlas; the text feature atlas includes feature maps including text intercepted from the multiple image frames;

[0029] Based on the text feature atlas, two feature maps corresponding to the same text in any two image frames are screened from the multiple image frames, and it is determined that the regions corresponding to the two feature maps meet a first preset condition, and it is determined that the areas corresponding to the two feature maps are the largest among the feature maps that meet the first preset condition, and the two feature maps are used as the first feature maps;

[0030] The feature maps in the text feature map except the first feature map are used as the second feature map.

[0031] In a possible implementation, the first preset condition is: the areas corresponding to the two feature maps are mapped to the same image frame, the distance between the areas corresponding to the two feature maps belongs to a first preset range; and / or the area where the two feature maps are located belongs to a preset position area.

[0032] In a possible implementation, the word segmentation unit is configured to perform:

[0033] Determine a third region position corresponding to any of the local text features;

[0034] Determine a first distance between the third area position and the corresponding first area position, and determine whether the first distance belongs to a second preset range;

[0035] When it is determined that the first distance belongs to the second preset range, the same characters in the segmented text corresponding to the local text feature and in the text corresponding to the first feature map are taken as a phrase;

[0036] Segment the text corresponding to the local text feature, except for the phrase, based on the natural language segmentation rule to obtain a first segmentation result, and use the phrase and the first segmentation result as the segmentation result;

[0037] When it is determined that the first distance does not belong to the second preset range, segmentation is performed on the segmented text corresponding to the local text feature based on the natural language segmentation rule to obtain a segmentation result.

[0038] According to a third aspect of the present disclosure, an electronic device is provided, the electronic device comprising:

[0039] processor;

[0040] a memory for storing processor-executable instructions;

[0041] The processor is configured to execute instructions to implement the first aspect of the embodiment of the present disclosure and any possible method involved in the first aspect.

[0042] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the above-mentioned first aspect of the embodiment of the present disclosure and any possible method involved in the first aspect.

[0043] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the first aspect of the embodiment of the present disclosure and any method that may be involved in the first aspect are implemented.

[0044] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0045] In the disclosed embodiment, a video file to be processed can be determined, and multiple image frames of the processed video file can be extracted, that is, the image frames corresponding to the video file to be processed are screened. Then, text detection processing is performed on the multiple image frames to obtain a first feature map and a second feature map, that is, the feature maps corresponding to the text features in the multiple image frames are divided to obtain a first feature map corresponding to the global text features that are in the same position and have the same text content in the multiple image frames, and a second feature map corresponding to the basic text features corresponding to any text content. Then, based on the global text features in the first feature map, the basic text features can be enhanced for video information, thereby determining the local text features corresponding to the segmented text that is finally segmented, thereby providing a good implementation basis for subsequent accurate segmentation.

[0046] Further, according to the local text features corresponding to the multiple image frames and the relationship between the region position corresponding to the local text features and the first region position, it is determined whether to perform word segmentation on the word segmentation text corresponding to the local text features in combination with the first feature map to obtain the word segmentation result. It can be seen that when performing word segmentation, it is also considered whether the local text features are associated with the corresponding first feature map that affects the understanding of the meaning of the video file, so that the word segmentation text corresponding to the local text features is performed in combination with the first feature map, so that the obtained word segmentation result is more consistent with the true meaning of the video file, and the accuracy of determining the word segmentation result is improved.

[0047] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0049] Figure 1 is a schematic diagram showing an application scenario according to an exemplary embodiment;

[0050] Figure 2 is a flow chart of a video file word segmentation method according to an exemplary embodiment;

[0051] Figure 3is a schematic diagram showing a method of determining a distance between two feature maps according to an exemplary embodiment;

[0052] Figure 4 is a schematic diagram showing how to obtain a first characteristic map and a second characteristic map according to an exemplary embodiment;

[0053] Figure 5 is a structural block diagram of a video file word segmentation method according to an exemplary embodiment;

[0054] Figure 6 is a schematic structural diagram of an electronic device according to an exemplary embodiment;

[0055] Figure 7 It is another structural schematic diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0056] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0057] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the images used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0058] The following is a brief introduction to the design concept of the embodiment of the present disclosure:

[0059] As mentioned above, the word segmentation results determined by the word segmentation method based on the sequence labeling model in the related art may not conform to the true intention of the original text information, resulting in semantic flaws, causing the recommendation of short videos or search results to not meet actual usage needs, thereby reducing the user experience.

[0060] In view of this, the present disclosure provides a video file segmentation method, through which the word segmentation text corresponding to the video file can be segmented in combination with the position of the text in the image frame corresponding to the video, so that the word segmentation result is more consistent with the true meaning of the original text information, thereby improving the accuracy of video file segmentation.

[0061] After introducing the design concept of the embodiment of the present disclosure, the following is a brief introduction to the application scenarios to which the video file segmentation technical solution in the embodiment of the present disclosure is applicable. It should be noted that the application scenarios described in the embodiment of the present disclosure are for the purpose of more clearly illustrating the technical solution of the embodiment of the present disclosure, and do not constitute a limitation on the technical solution provided by the embodiment of the present disclosure. Ordinary technicians in this field can know that with the emergence of new application scenarios, the technical solution provided by the embodiment of the present disclosure is also applicable to similar technical problems.

[0062] refer to Figure 1 , which is a schematic diagram of an application scenario of the video file segmentation method provided by an embodiment of the present disclosure. The application scenario includes multiple terminal devices 101 (including terminal device 101-1, terminal device 101-2, ... terminal device 101-n) and a server 102. Among them, each terminal device 101 is connected to the server 102 via a wireless or wired network.

[0063] The server 102 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The terminal device 101 may be a smart phone, a tablet computer, a laptop, a desktop computer, a smart TV, a smart wearable device, etc., but is not limited thereto.

[0064] In this scenario, the user can log in to the application that can publish videos based on the terminal device 101, so that the terminal device 101 can send the video published by the user to the server 102, and the server 102 can determine the video file to be processed based on the received video, so as to perform word segmentation processing on the video file and obtain the word segmentation result. After obtaining the word segmentation result, the word segmentation result can be stored for subsequent word library construction; or the video can be classified based on the word segmentation result and fed back to the terminal device 101 video keywords, etc., which is not limited in the embodiment of the present disclosure.

[0065] To further illustrate the scheme of the video file segmentation method provided by the embodiment of the present disclosure, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present disclosure provides the method operation steps as shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiment of the present disclosure. The method can be executed in the actual processing process or when the device is executed, it can be executed in the method sequence shown in the embodiment or drawings or in parallel (for example, an application environment of a parallel processor or multi-threaded processing).

[0066] The following combination Figure 2 The method flow chart shown in the figure illustrates the video file word segmentation method in the embodiment of the present disclosure. Figure 2 The steps shown can be performed as follows Figure 1 The electronic device shown performs.

[0067] Step 201: Determine a video file to be processed, and extract multiple image frames of the video file to be processed.

[0068] In the embodiments of the present disclosure, the electronic device can determine the video file to be processed from the video sent by the terminal device, and can also determine the video file to be processed from the database. That is to say, the video file to be processed can be a video file uploaded in real time or a pre-stored video file, which is not limited in the embodiments of the present disclosure.

[0069] Exemplarily, after determining the video file to be processed, the video file to be processed can be deframed to obtain all image frames determined in a deframe order, and key image frames including complete text are extracted from all image frames to determine multiple image frames of the video file to be processed.

[0070] For example, the video file A to be processed is deframed to obtain 120 image frames. When it is determined that 50 of the 120 image frames include complete text content, the 50 image frames can be determined as the image frames of the video file to be processed.

[0071] Step 202: Perform text detection processing on multiple image frames to obtain a first feature map and a second feature map; wherein the first feature map includes global text features that are located at the same position and have the same text content in multiple image frames and a first area position corresponding to the global text feature; and the second feature map includes basic text features corresponding to any text content in the multiple image frames.

[0072] Exemplarily, text detection can be performed on multiple image frames based on a pre-trained network model to obtain a first feature map and a second feature map. The pre-trained network model includes a Faster RCNN subnetwork for global information enhancement and an LSTM decoder and a Transformer encoder for global information guidance. The Faster RCNN subnetwork can perform text detection processing on multiple image frames to obtain a text feature map set, that is, a feature map including text intercepted from multiple image frames, and then analyze and judge the text features included in the feature map and the regional position corresponding to the text features, so as to determine the first feature map and the second feature map. The first feature map includes global text features that exist in the same position and have the same text content in multiple image frames, and the first regional position corresponding to the global text feature. The global text feature corresponding to the first feature map can be understood as a global feature that has an impact on the entire video; the second feature map includes basic text features corresponding to any text content in multiple image frames, and the second regional position corresponding to the basic text feature. It should be noted that the first regional position and the second regional position can be represented based on the coordinate position x and the coordinate position y.

[0073] Exemplarily, text detection processing can be performed on multiple image frames to obtain a text feature atlas; the text feature atlas includes feature maps including text captured from the multiple image frames; then based on the text feature atlas, two feature maps corresponding to the same text in any two image frames are screened from the multiple image frames, and it is determined that the areas corresponding to the two feature maps meet the first preset condition, and it is determined that the areas corresponding to the two feature maps are the largest among the feature maps that meet the first preset condition, then the two feature maps are used as the first feature map, and the feature maps in the text feature map except the first feature map are used as the second feature map.

[0074] Based on the above method, the feature map corresponding to the same text existing in multiple frame images and the text that meets the first preset condition and has the largest area can be determined as the first feature map, and the feature maps except the first feature map in the text feature map set can be used as the second feature map. In this way, it is possible to more accurately determine the global content that can characterize the theme or overall direction of the video file to be processed, such as determining the indexing information corresponding to the video file, that is, the information named by the publisher himself, or the corresponding platform of the video publisher included in the background part of the image frame, thereby providing a more practical segmentation basis for the subsequent word splitting, thereby improving the accuracy of the segmentation results of the video file to be processed.

[0075] Exemplarily, the first preset condition is that the areas corresponding to the two feature maps are mapped to the same image frame, and the distance between the areas corresponding to the two feature maps belongs to the first preset range. The first preset range can be determined based on actual implementation or historical experience, and is not limited in the embodiments of the present disclosure.

[0076] For example, see the attached Figure 3 , Figure 3 The schematic diagram of determining the distance between the regions corresponding to two feature maps is exemplified, wherein the region corresponding to the feature map S is region 1 and the region corresponding to the feature map N is region 2, and then region 1 and region 2 are mapped to the image frame F, and the distance between region 1 and region 2 can be determined. When region 1 and region 2 completely overlap, it is considered to belong to the first preset range.

[0077] In the above method, feature maps corresponding to multiple image frames with the same text and similar area sizes can be determined.

[0078] Exemplarily, the first preset condition is that the area where the two feature maps are located belongs to a preset position area.

[0079] In the embodiments of the present disclosure, a preset location area can also be determined in advance. For example, the preset location area can be the location area corresponding to the indexing information corresponding to the video file when the video file is uploaded, or it can be the location area corresponding to the logo information of the platform to which the publisher of the video file belongs determined based on historical data. Of course, it can also be a location area determined based on other methods, which is not limited in the embodiments of the present disclosure.

[0080] In addition, the first preset condition may also be that the areas corresponding to the two feature maps are mapped to the same image frame, the distance between the areas corresponding to the two feature maps belongs to the first preset range, and the areas where the two feature maps are located belong to the preset position area.

[0081] Based on the above method, it is possible to ensure that the feature graph that has an impact on the overall word segmentation of the video file is screened out from the text feature graph set to obtain the first feature graph, thereby providing a segmentation basis for subsequent word segmentation based on the first feature graph.

[0082] For example, taking the case of performing text detection on two image frames, such as image frame A and image frame B, the process of obtaining the first feature map and the second feature map is described. Figure 4 , Figure 4 The first feature map and the second feature map are exemplarily shown. The electronic device can perform text detection processing on the image frame A to obtain a text feature map set 1 corresponding to the image frame A, that is, Figure 4, the feature maps corresponding to regions 1, 2, 3, and 4 shown in FIG. 1 ; similarly, the electronic device can perform text detection processing on the image frame B to obtain a text feature atlas 2 corresponding to the image frame B, that is, Figure 4 The feature maps corresponding to regions 7, 8, 9, 10, and 11 shown in the figure are determined, and it is determined that regions 2, 4 corresponding to image frame A and regions 8, 10 corresponding to image frame B include the same text, then it is determined whether regions 4, 2, 10, and 8 corresponding to image frame A belong to the preset area positions. When the area positions corresponding to regions 4, 2, 10, and 8 belong to the preset area positions, it can be determined that the first preset condition is met, and the areas of region 2 and the corresponding region 8 and the areas of region 4 and the corresponding region 10 are determined, and the area with the largest area is determined. When it is determined that the areas of region 2 and the corresponding region 8 are the largest, the feature map corresponding to region 2 and the corresponding region 8 is determined as the first feature map, and the feature maps corresponding to regions 1, 3, 4, 7, 10, 9, and 11 are determined as the second feature map.

[0083] Step 203: Based on the global text features in the first feature map, the basic text features are enhanced with video information to obtain local text features of multiple image frames.

[0084] Exemplarily, the LSTM decoder used for global information guidance in the pre-trained network model can decode the first feature map and the second feature map to obtain the decoded data corresponding to the first feature map and the decoded data corresponding to the second feature map, and then input the decoded data corresponding to the first feature map and the decoded data corresponding to the second feature map into the Transformer encoder, so that the multi-head attention mechanism in the Transformer encoder analyzes the text to be segmented corresponding to multiple image frames, and performs adaptive fusion processing on the decoded data corresponding to the first feature map and the decoded data corresponding to the second feature map, that is, based on the global text features in the first feature map, the basic text features are enhanced with video information, and the adaptive fusion processing results are fed back to the feedforward network, and the decoded data corresponding to the first feature map and the decoded data corresponding to the second feature map are adaptively fused through the feedforward network, thereby realizing video information enhancement processing of the basic text features, and determining the features corresponding to the segmented texts corresponding to the image frames after the video information enhancement as local text features of multiple image frames.

[0085] It can be seen that in the embodiment of the present disclosure, the global text features in the first feature map are combined with the basic text features through a multi-level fusion method to obtain local text features corresponding to multiple image frames.

[0086] Step 204: Based on the local text features corresponding to the multiple image frames and the relationship between the region position corresponding to the local text features and the first region position, determine whether to perform word segmentation on the word segmentation text corresponding to the local text features in combination with the first feature map to obtain a word segmentation result.

[0087] Exemplarily, the third area position corresponding to any local text feature is determined, and then the first distance between the third area position and the corresponding first area position is determined, and it is determined whether the first distance belongs to the second preset range; when it is determined that the first distance belongs to the second preset range, the characters in the word segmentation text corresponding to the local text feature that are the same as the characters corresponding to the corresponding first feature map are taken as a phrase; the characters other than the phrase in the word segmentation text corresponding to the local text feature are segmented based on natural language segmentation rules to obtain a first segmentation result, and the phrase and the first segmentation result are taken as the segmentation result.

[0088] It can be seen that in the embodiment of the present disclosure, when it is determined that the third area position corresponding to the local text feature is within the second preset range from the first area position in the same image frame where the local text feature is located, that is, the first distance corresponding to the corresponding first area position, then it can be determined that the segmented text corresponding to the local text feature is related to the text corresponding to the first feature map, and then the text in the segmented text corresponding to the local text feature that is the same as the text corresponding to the corresponding first feature map can be used as a phrase, and then the text in the segmented text corresponding to the local text feature except the phrase is segmented based on the natural language segmentation rules to obtain the first segmentation result, so that the phrase and the first segmentation result are used as the segmentation result. In other words, in the embodiment of the present disclosure, considering the user-defined words or novel special names in the text to be segmented, the segmentation can be performed directly based on its indexing information or background text information, so that the segmentation result is more consistent with the meaning that the video file hopes to represent, and the segmentation result obtained is more accurate.

[0089] Exemplarily, when it is determined that the first distance does not fall within the second preset range, the segmentation text corresponding to the local text feature is segmented based on the natural language segmentation rule to obtain the segmentation result. That is, when it is determined that the difference between the first distance between the regional position corresponding to the local text feature and the first position region does not fall within the second preset range, it can be considered that the user-defined words or novel special names in the text to be segmented have little effect on the segmentation result of the local text feature, and the speed of segmenting the segmentation text corresponding to the local text feature can be directly accelerated based on the natural language segmentation rule to obtain the segmentation result accurately and quickly.

[0090] It should be noted that in the embodiment of the present disclosure, the second preset range can be determined based on the actual implementation situation or based on historical experience. This is not limited in the embodiment of the present disclosure, and the second preset range can be updated based on actual needs.

[0091] Based on the same inventive concept, the embodiment of the present disclosure provides a video file segmentation device, which can realize the functions corresponding to the aforementioned video file segmentation method. The video file segmentation device can be a hardware structure, a software module, or a hardware structure plus a software module. The video file segmentation device can be implemented by a chip system, which can be composed of a chip or include a chip and other discrete devices. See Figure 5 As shown, the video file word segmentation device includes an extraction unit 501, an acquisition unit 502, a processing unit 503 and a word segmentation unit 504. Among them:

[0092] An extraction unit 501 is configured to determine a video file to be processed and extract a plurality of image frames of the video file to be processed;

[0093] The obtaining unit 502 is configured to perform text detection processing on the multiple image frames to obtain a first feature map and a second feature map; wherein the first feature map includes a global text feature that is in the same position and has the same text content in the multiple image frames and a first region position corresponding to the global text feature; and the second feature map includes a basic text feature corresponding to any text content in the multiple image frames;

[0094] The processing unit 503 is configured to perform video information enhancement on the basic text features based on the global text features in the first feature map to obtain local text features of the multiple image frames;

[0095] The word segmentation unit 504 is configured to determine whether to perform word segmentation on the word segmentation text corresponding to the local text features in combination with the first feature map based on the local text features corresponding to the multiple image frames and the relationship between the area position corresponding to the local text features and the first area position to obtain a word segmentation result.

[0096] In a possible implementation, the obtaining unit 502 is configured to execute:

[0097] Performing text detection processing on the multiple image frames to obtain a text feature atlas; the text feature atlas includes feature maps including text intercepted from the multiple image frames;

[0098] Based on the text feature atlas, two feature maps corresponding to the same text in any two image frames are screened from the multiple image frames, and it is determined that the regions corresponding to the two feature maps meet a first preset condition, and it is determined that the areas corresponding to the two feature maps are the largest among the feature maps that meet the first preset condition, and the two feature maps are used as the first feature maps;

[0099] The feature maps in the text feature map except the first feature map are used as the second feature map.

[0100] In a possible implementation, the first preset condition is: the areas corresponding to the two feature maps are mapped to the same image frame, the distance between the areas corresponding to the two feature maps belongs to a first preset range; and / or the area where the two feature maps are located belongs to a preset position area.

[0101] In a possible implementation, the word segmentation unit 504 is configured to perform:

[0102] Determine a third region position corresponding to any of the local text features;

[0103] Determine a first distance between the third area position and the corresponding first area position, and determine whether the first distance belongs to a second preset range;

[0104] When it is determined that the first distance belongs to the second preset range, the characters in the segmented text corresponding to the local text feature that are the same as the characters corresponding to the corresponding first feature graph are taken as a phrase;

[0105] Segment the text corresponding to the local text feature, except for the phrase, based on the natural language segmentation rule to obtain a first segmentation result, and use the phrase and the first segmentation result as the segmentation result;

[0106] When it is determined that the first distance does not belong to the second preset range, segmentation is performed on the segmented text corresponding to the local text feature based on the natural language segmentation rule to obtain a segmentation result.

[0107] As mentioned above Figure 2 All relevant contents of each step involved in the embodiment of the video file segmentation method can be referred to the functional description of the functional unit corresponding to the video file segmentation device in the embodiment of the present disclosure, and will not be repeated here.

[0108] The division of units in the embodiments of the present disclosure is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processor, or may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0109] Based on the same inventive concept, the present disclosure also provides an electronic device, such as Figure 6 As shown, the electronic device in the embodiment of the present disclosure includes at least one processor 601, and a memory 602 and a communication interface 603 connected to the at least one processor 601. The specific connection medium between the processor 601 and the memory 602 is not limited in the embodiment of the present disclosure. Figure 6 In the example, the processor 601 and the memory 602 are connected via a bus 600. Figure 6 The connection between other components is shown by bold lines, and is not intended to be limiting. The bus 600 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0110] In the embodiment of the present disclosure, the memory 602 stores instructions that can be executed by at least one processor 601. The at least one processor 601 can execute the steps included in the aforementioned video file segmentation method by executing the instructions stored in the memory 602.

[0111] Among them, the processor 601 is the control center of the electronic device, and can use various interfaces and lines to connect various parts of the entire fault detection device, and through running or executing instructions stored in the memory 602 and calling data stored in the memory 602, various functions of the computing device and processing data, so as to monitor the computing device as a whole. Optionally, the processor 601 may include one or more processing units, and the processor 601 may integrate an application processor and a modem processor, wherein the processor 601 mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 601. In some embodiments, the processor 601 and the memory 602 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on independent chips.

[0112] The processor 601 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present disclosure. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present disclosure may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0113] The memory 602 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 602 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 602 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 602 in the embodiment of the present disclosure can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.

[0114] The communication interface 603 is a transmission interface that can be used for communication, and data can be received or sent through the communication interface 603 .

[0115] See also Figure 7 A further structural diagram of the electronic device shown is shown, which also includes a basic input / output system (I / O system) 701 for helping to transmit information between various devices within the electronic device, and a large-capacity storage device 705 for storing an operating system 702, application programs 703 and other program modules 704.

[0116] The basic input / output system 701 includes a display 706 for displaying information and an input device 707 such as a mouse and a keyboard for user inputting information. The display 706 and the input device 707 are connected to the processor 601 through the basic input / output system 701 connected to the system bus 600. The basic input / output system 701 may also include an input / output controller for receiving and processing inputs from a plurality of other devices such as a keyboard, a mouse, or an electronic stylus. Similarly, the input / output controller also provides output to a display screen, a printer, or other types of output devices.

[0117] The mass storage device 705 is connected to the processor 601 through a mass storage controller (not shown) connected to the system bus 600. The mass storage device 705 and its associated computer readable medium provide non-volatile storage for the server package. That is, the mass storage device 705 may include a computer readable medium (not shown) such as a hard disk or a CD-ROM drive.

[0118] According to various embodiments of the present disclosure, the electronic device package can also be connected to a remote computer on the network through a network such as the Internet to run. That is, the electronic device can be connected to the network 708 through the communication interface 603 connected to the system bus 600, or in other words, the communication interface 603 can also be used to connect to other types of networks or remote computer systems (not shown).

[0119] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of the device to complete the above video file segmentation method. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a tape, a floppy disk, and an optical image storage device, etc.

[0120] In an exemplary embodiment, a computer program product is also provided, which includes a program code. When the program product is run on a server, the program code is used to enable the server to execute the steps of the video file segmentation method according to various exemplary embodiments of the present disclosure described above in this specification.

[0121] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present disclosure may take the form of a computer program product implemented in one or more computer-usable storage media containing computer-usable program codes, and the computer-usable storage media include but are not limited to disk storage and optical storage, etc.

[0122] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, servers, computer-readable storage media, and computer program products according to embodiments of the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable image processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable image processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0123] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable image processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0124] These computer program instructions may also be loaded onto a computer or other programmable image processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0125] Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A video file word segmentation method, characterized in that: The method comprises: Determine a video file to be processed, and extract multiple image frames of the video file to be processed; Performing text detection processing on the multiple image frames to obtain a first feature map and a second feature map; wherein the first feature map includes a global text feature that is in the same position and has the same text content in the multiple image frames and a first area position corresponding to the global text feature; and the second feature map includes a basic text feature corresponding to any text content in the multiple image frames; Based on the global text features in the first feature map, the basic text features are enhanced with video information to obtain local text features of the multiple image frames; Determine a third region position corresponding to any of the local text features; Determine a first distance between the third area position and the corresponding first area position, and determine whether the first distance belongs to a second preset range; When it is determined that the first distance belongs to the second preset range, the same characters in the segmented text corresponding to the local text feature and in the text corresponding to the first feature map are taken as a phrase; Segment the text corresponding to the local text feature, except for the phrase, based on the natural language segmentation rule to obtain a first segmentation result, and use the phrase and the first segmentation result as the segmentation result; When it is determined that the first distance does not belong to the second preset range, segmentation is performed on the segmented text corresponding to the local text feature based on the natural language segmentation rule to obtain a segmentation result.

2. The method according to claim 1, characterized in that Performing text detection processing on the multiple image frames to obtain a first feature map and a second feature map includes: Performing text detection processing on the multiple image frames to obtain a text feature atlas; the text feature atlas includes feature maps including text intercepted from the multiple image frames; Based on the text feature atlas, two feature maps corresponding to the same text in any two image frames are screened from the multiple image frames, and it is determined that the regions corresponding to the two feature maps meet a first preset condition, and it is determined that the areas corresponding to the two feature maps are the largest among the feature maps that meet the first preset condition, and the two feature maps are used as the first feature maps; The feature maps in the text feature map except the first feature map are used as the second feature map.

3. The method according to claim 2, characterized in that The first preset condition is that the areas corresponding to the two feature maps are mapped to the same image frame, and the distance between the areas corresponding to the two feature maps belongs to a first preset range; And / or, the area where the two feature maps are located belongs to a preset location area.

4. A video file word segmentation device, characterized in that: The device comprises: An extraction unit is configured to determine a video file to be processed and extract a plurality of image frames of the video file to be processed; An obtaining unit is configured to perform text detection processing on the multiple image frames to obtain a first feature map and a second feature map; wherein the first feature map includes a global text feature that is in the same position and has the same text content in the multiple image frames and a first area position corresponding to the global text feature; and the second feature map includes a basic text feature corresponding to any text content in the multiple image frames; A processing unit is configured to perform video information enhancement on the basic text features based on the global text features in the first feature map to obtain local text features of the multiple image frames; A word segmentation unit is configured to determine a third region position corresponding to any of the local text features; Determine a first distance between the third area position and the corresponding first area position, and determine whether the first distance belongs to a second preset range; when it is determined that the first distance belongs to the second preset range, take the same characters in the segmented text corresponding to the local text feature as the text corresponding to the corresponding first feature map as a phrase; segment the characters other than the phrase in the segmented text corresponding to the local text feature based on a natural language segmentation rule to obtain a first segmentation result, and take the phrase and the first segmentation result as the segmentation result; when it is determined that the first distance does not belong to the second preset range, segment the segmented text corresponding to the local text feature based on the natural language segmentation rule to obtain a segmentation result.

5. The device according to claim 4, characterized in that The obtaining unit is configured to execute: Performing text detection processing on the multiple image frames to obtain a text feature atlas; the text feature atlas includes feature maps including text intercepted from the multiple image frames; Based on the text feature atlas, two feature maps corresponding to the same text in any two image frames are screened from the multiple image frames, and it is determined that the regions corresponding to the two feature maps meet a first preset condition, and it is determined that the areas corresponding to the two feature maps are the largest among the feature maps that meet the first preset condition, and the two feature maps are used as the first feature maps; The feature maps in the text feature map except the first feature map are used as the second feature map.

6. The device according to claim 5, characterized in that The first preset condition is that the areas corresponding to the two feature maps are mapped to the same image frame, and the distance between the areas corresponding to the two feature maps belongs to a first preset range; And / or, the area where the two feature maps are located belongs to a preset location area.

7. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the video file segmentation method according to any one of claims 1 to 3.

8. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the video file segmentation method as described in any one of claims 1 to 3.

9. A computer program product, comprising a computer program, characterized in that The computer program is executed by a processor to implement the video file segmentation method according to any one of claims 1 to 3.

Images