Text tracking model training method, text tracking method, device and electronic equipment

Abstract

The text tracking model training method, the text tracking method, the device and the electronic equipment provided by the embodiment of the present application comprise: obtaining multiple videos; performing frame extraction processing on each video to obtain a video frame image sequence, dividing the video frame image sequence into multiple video frame image subsequences, determining a target video frame image subsequence from the multiple video frame image subsequences; inputting the multiple target video frame image subsequences into a neural network model for training to obtain a text tracking model. In the present application, the target video frame image subsequence is a sequence of video frame images with the same text and a large degree of image background change. Therefore, the neural network model is trained based on the target video frame image subsequence, so that the neural network model pays more attention to the text than to the image background during the training process, so as to obtain the text tracking model. The text tracking model can accurately track the text, and greatly improves the text tracking performance.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of text tracking, in particular to a text tracking model training method, a text tracking method, a device and an electronic device. BACKGROUND

[0002] With the development of computer technology and the wide application of computer vision principles, the use of computer image processing technology to track targets in real time is becoming more and more popular. Dynamic real-time tracking of targets has wide application value in intelligent traffic systems, intelligent monitoring systems, military target detection, etc.

[0003] Currently, video operators can track topics through text tracking for popular videos. Since text tracking has many commonalities with natural language processing technologies such as information retrieval, information mining and information extraction, and faces corpus with suddenness and continuity, it has gradually become a research hotspot in the current information processing field. Text tracking takes large-scale video streams as the research object, discovers new information of interest to users through the topics of video text description, thereby attracting new users to watch videos and improving video viewership.

[0004] Unlike traditional object tracking, the text tracking area has a situation where the proportion of background pixels is greater than the proportion of text pixels. If the data is not screened, a large number of training data with consistent background will cause the text tracking model to pay too much attention to the changes of background pixels, thereby causing the text tracking model to be poorly trained and the text tracking performance to be too low. SUMMARY

[0005] In view of this, embodiments of the present application provide a text tracking model training method, a text tracking method, a device and an electronic device, which can train a neural network model based on sequence video frame images with large image background changes and the same text, so that the neural network model does not pay too much attention to the image background but pays attention to the text during the training process to obtain a text tracking model. The use of the text tracking model can accurately track the text, greatly improving the text tracking performance.

[0006] In a first aspect, embodiments of the present application provide a text tracking model training method, wherein the method comprises:

[0007] obtaining a plurality of videos;

[0008] The following operations are performed for each video: frame extraction is performed on the video to obtain a video frame image sequence, the video frame image sequence is divided into a plurality of video frame image subsequences, and a target video frame image subsequence is determined from the plurality of video frame image subsequences; wherein the target video frame image subsequence is a sequence video frame image with the same text and a large image background change degree, and the text is labeled with a text box.

[0009] The plurality of target video frame image subsequences are input into a neural network model for training until a preset training period is reached or a model loss value reaches a preset loss threshold, thereby obtaining a text tracking model.

[0010] In one possible implementation, dividing the video frame image sequence into a plurality of video frame image subsequences includes:

[0011] Performing text recognition detection on the video frame image sequence.

[0012] Determining whether the video frame image in the video frame image sequence includes text.

[0013] In a case where it is determined that the video frame image in the video frame image sequence includes text, merging video frame images with the same text to obtain a plurality of video frame image subsequences.

[0014] In a case where it is determined that the video frame image in the video frame image sequence does not include text, dividing the video frame image sequence according to a preset frame number to obtain a plurality of video frame image subsequences.

[0015] In one possible implementation, determining a target video frame image subsequence from the plurality of video frame image subsequences includes:

[0016] For each video frame image subsequence, a preset number of first video frame images are selected from the video frame image subsequence, and a background change degree value of the video frame image subsequence is determined based on the plurality of first video frame images.

[0017] The target video frame image subsequence is determined from the plurality of video frame image subsequences based on a plurality of background change degree values.

[0018] In one possible implementation, determining a background change degree value of a video frame image subsequence based on a plurality of first video frame images includes:

[0019] The similarity of any two adjacent first video frame images in the plurality of first video frame images is determined using a similarity algorithm.

[0020] The background change degree value is determined based on a plurality of similarities.

[0021] In a possible implementation, determining the target video frame image sub-sequence from the plurality of video frame image sub-sequences based on the plurality of background change degree values comprises:

[0022] The video frame image sub-sequence corresponding to the background change degree value less than the preset degree value is determined as the target video frame image sub-sequence.

[0023] In a possible implementation, the method further comprises:

[0024] In a case where the target video frame image in the target video frame image sub-sequence does not include text, adding the preset text to a preset position in the target video frame image.

[0025] In a possible implementation, the text recognition detection on the video frame image sequence comprises:

[0026] The text recognition detection on the video frame image sequence is performed by using the pre-trained text recognition detection model.

[0027] In a second aspect, an embodiment of the present application provides a text tracking method, and the method comprises:

[0028] Obtaining a video to be tracked;

[0029] Frame extraction is performed on the video to be tracked to obtain a video frame image sequence to be tracked;

[0030] Text labeling of to-be-tracked text is performed on the video frame images to be tracked at a preset video frame interval from the video frame image sequence to be tracked;

[0031] The video frame image sequence to be tracked after the text labeling is input into a text tracking model, and the text tracking model outputs the video frame image sequence to be tracked in which each frame is labeled with a text box of to-be-tracked text; wherein the text tracking model is obtained by training the text tracking model training method.

[0032] In a third aspect, an embodiment of the present application provides a text tracking model training device, and the device comprises:

[0033] A first obtaining module is configured to obtain a plurality of videos;

[0034] An executing module is configured to perform the following operations on each video: frame extraction is performed on the video to obtain a video frame image sequence, the video frame image sequence is divided into a plurality of video frame image sub-sequences, and a target video frame image sub-sequence is determined from the plurality of video frame image sub-sequences; wherein the target video frame image sub-sequence is a sequence video frame image with the same text and a large image background change degree, and the text is labeled by using a text box.

[0035] The training module is configured to input the plurality of target video frame image subsequences into the neural network model for training until a preset training period is reached or a preset loss threshold is reached, and obtain the text tracking model.

[0036] In a fourth aspect, an embodiment of the present application provides a text tracking device, and the device comprises:

[0037] The second obtaining module is configured to obtain a video to be tracked.

[0038] The frame extraction processing module is configured to perform frame extraction processing on the video to be tracked to obtain a sequence of video frame images to be tracked.

[0039] The text labeling module is configured to perform text labeling on the video frame images to be tracked at a preset video frame interval in the sequence of video frame images to be tracked.

[0040] The text tracking module is configured to input the sequence of video frame images to be tracked after text labeling into a text tracking model, and the text tracking model outputs a sequence of video frame images to be tracked, each of which has a text box of text to be tracked labeled. The text tracking model is trained by using the text tracking model training method.

[0041] In a fifth aspect, an embodiment of the present application provides an electronic device, and the device comprises: a processor and a memory, and the processor is configured to execute a text tracking model training program and a text tracking program stored in the memory, so as to implement the text tracking model training method and the text tracking method.

[0042] In a sixth aspect, an embodiment of the present application provides a storage medium, and the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the text tracking model training method and the text tracking method.

[0043] The text tracking model training method, the text tracking method, the device and the electronic equipment provided by the embodiment of the present application comprise the following steps: obtaining a plurality of videos; for each video, performing the following operations: performing frame extraction processing on the video to obtain a video frame image sequence, dividing the video frame image sequence into a plurality of video frame image subsequences, and determining a target video frame image subsequence from the plurality of video frame image subsequences; inputting the plurality of target video frame image subsequences into a neural network model for training until a preset training period is reached or a model loss value reaches a preset loss threshold, thereby obtaining a text tracking model. In the present application, the target video frame image subsequence is a sequence of video frame images with the same text and a large degree of image background change. Therefore, the neural network model is trained based on the target video frame image subsequence, so that the neural network model pays more attention to the text than the image background during the training process, thereby obtaining the text tracking model. The text tracking model can accurately track the text, thereby greatly improving the text tracking performance. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 An embodiment flowchart of a text tracking model training method provided by the embodiment of the present application is provided.

[0045] Figure 2 An embodiment flowchart of a text tracking method provided by the embodiment of the present application is provided.

[0046] Figure 3 An embodiment block diagram of a text tracking model training device provided by the embodiment of the present application is provided.

[0047] Figure 4 An embodiment block diagram of a text tracking device provided by the embodiment of the present application is provided.

[0048] Figure 5 A structural schematic diagram of an electronic equipment provided by the embodiment of the present application is provided. DETAILED DESCRIPTION

[0049] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0050] To facilitate the understanding of the embodiments of the present application, the following will be further explained and described with specific embodiments in combination with the drawings, and the embodiments do not constitute a limitation on the embodiments of the present application.

[0051] The embodiment of the present application provides a text tracking model training method, see Figure 1 An embodiment flowchart of the text tracking model training method provided by the embodiment of the present application. Figure 1 The flowchart can include the following steps:

[0052] Step 101, obtaining a plurality of videos;

[0053] The plurality of videos are short videos and / or long videos of different contents, and the specific videos can be captured from a specific video website through a crawler technology. The specific number of videos can be set according to actual needs, and is not limited herein.

[0054] Step 102, for each video, the following operations are performed: frame extraction processing is performed on the video to obtain a video frame image sequence, the video frame image sequence is divided into a plurality of video frame image subsequences, and a target video frame image subsequence is determined from the plurality of video frame image subsequences.

[0055] The target video frame image in the target video frame image subsequence is a video frame image with a text box and a large image background change degree.

[0056] The video is composed of a frame of video frame image. Since the text tracking model is trained based on the video frame image, the video needs to be frame extracted. The specific video frame extraction process is the existing video frame extraction process, which is not described in detail herein.

[0057] For a long video, the number of video frame images obtained by frame extraction is too large to be processed. Therefore, the video frame image sequence obtained by frame extraction needs to be divided into a plurality of video frame image subsequences including a small number of video frame images. Usually, the number of video frame images included in the plurality of video frame image subsequences is the same or slightly different.

[0058] In order to make the trained text tracking model track the text accurately, in the embodiment, the sequence video frame image with a large image background change degree and the same text needs to be used for model training. In the model training process, the neural network model cannot pay too much attention to the image background, but pay attention to the text labeled with the text box for model training, so as to obtain the text tracking model that tracks the text accurately. Therefore, the target video frame image subsequence with the same text and a large image background change degree needs to be determined from the plurality of video frame image subsequences, so as to train the neural network model by using the target video frame image subsequence.

[0059] Step 103, inputting the plurality of target video frame image subsequences into the neural network model for training until a preset training period is reached or a model loss value reaches a preset loss threshold, to obtain the text tracking model.

[0060] The model loss value can be cross-entropy loss or mean square error, and is not limited herein; the preset training period or the preset loss threshold can be set according to actual needs, and is not limited herein.

[0061] The text tracking model training method provided by the embodiment of the application comprises: acquiring a plurality of videos; for each video, the following operations are performed: frame extraction processing is performed on the video to obtain a video frame image sequence, the video frame image sequence is divided into a plurality of video frame image subsequences, and a target video frame image subsequence is determined from the plurality of video frame image subsequences; the plurality of target video frame image subsequences are input into a neural network model for training until a preset training period is reached or a model loss value reaches a preset loss threshold, thereby obtaining a text tracking model. In the application, the target video frame image subsequence is a sequence of video frame images with the same text and a large degree of image background change. Therefore, the neural network model is trained based on the target video frame image subsequence, so that the neural network model pays more attention to the text than the image background during the training process, thereby obtaining the text tracking model. The text tracking model can accurately track the text, thereby greatly improving the text tracking performance.

[0062] In an embodiment, the step of dividing the video frame image sequence into a plurality of video frame image subsequences in step 102 can be implemented by steps A1 to A4.

[0063] Step A1: text recognition detection is performed on the video frame image sequence.

[0064] Specifically, the text recognition detection is performed on the video frame image sequence by using a pre-trained text recognition detection model. The text recognition detection model is obtained by training a neural network model using a text image training set with calibrated text.

[0065] Step A2: it is determined whether the video frame image in the video frame image sequence includes text.

[0066] Step A3 is performed when it is determined that the video frame image in the video frame image sequence includes text; and step A4 is performed when it is determined that the video frame image in the video frame image sequence does not include text.

[0067] Step A3: video frame images with the same text are merged to obtain a plurality of video frame image subsequences.

[0068] For example, the video frame image sequence includes 60 video frame images, the video frame images with frame numbers 0-19 include the text "Hello", so the 20 video frame images are taken as a video frame image sub-sequence, the video frame images with frame numbers 20-39 include the text "The sun is so sufficient", so the 20 video frame images are taken as a video frame image sub-sequence, and the video frame images with frame numbers 40-59 include the text "Let's go to Dali", so the 20 video frame images are taken as a video frame image sub-sequence.

[0069] Step A4, dividing the video frame image sequence according to the preset frame number to obtain a plurality of video frame image sub-sequences.

[0070] The preset frame number can be set according to actual needs, which is not limited here. Continuing the previous example, the video frame image sequence includes 60 video frame images, and the preset frame number is 20, so the 60 video frame images are divided into 3 video frame image sub-sequences, each of which includes 20 video frame images; if the preset frame number is 10, then the 60 video frame images are divided into 6 video frame image sub-sequences, each of which includes 10 video frame images.

[0071] In an embodiment, the step 102 of determining the target video frame image sub-sequence from the plurality of video frame image sub-sequences can be implemented by steps B1-B2:

[0072] Step B1, for each video frame image sub-sequence, randomly selecting a preset number of first video frame images from the video frame image sub-sequence, and determining the background change degree value of the video frame image sub-sequence based on the plurality of first video frame images.

[0073] The preset number can be set according to actual needs, or can be set according to the number of video frame images included in the video frame image sub-sequence, and the preset number should include the first and last frames.

[0074] For example, the video frame image sub-sequence includes 20 video frame images with frame numbers 0-19, and the preset number is 8 frames, so 8 first video frame images including the first and last frames are randomly selected from the video frame image sub-sequence, wherein the frame numbers of the 8 first video frame images are 0, 3, 7, 10, 13, 15, 17 and 19 respectively.

[0075] If the video frame image sub-sequence includes 10 video frame images with frame numbers 50-60, the preset number is 4 frames, so 4 first video frame images including the first and last frames are selected from the video frame image sub-sequence, wherein the frame numbers of the 4 first video frame images are 50, 53, 56, 58 and 60 respectively.

[0076] The process of specifically determining the background change degree value of the video frame image subsequence is: determining the similarity of any two adjacent first video frame images in the plurality of first video frame images by using a similarity algorithm; and determining the background change degree value based on the plurality of similarities.

[0077] The similarity algorithm is cosine similarity, and the background change degree value is obtained by performing mean value calculation after obtaining a plurality of similarities.

[0078] Taking the frame numbers of the four first video frame images as 50, 53, 56, 58, and 60 as an example, the cosine similarity of the two first video frame images with the frame numbers of 50 and 53 is calculated to obtain a first similarity value, the cosine similarity of the two first video frame images with the frame numbers of 53 and 56 is calculated to obtain a second similarity value, the cosine similarity of the two first video frame images with the frame numbers of 56 and 58 is calculated to obtain a third similarity value, and the cosine similarity of the two first video frame images with the frame numbers of 58 and 60 is calculated to obtain a fourth similarity value, and then the first similarity value, the second similarity value, the third similarity value, and the fourth similarity value are subjected to mean value calculation to obtain the background change degree value.

[0079] Step B2, determining a target video frame image subsequence from the plurality of video frame image subsequences based on the plurality of background change degree values.

[0080] Specifically, the video frame image subsequence corresponding to the background change degree value lower than the preset degree value is determined as the target video frame image subsequence.

[0081] In this embodiment, the greater the background change degree value, the higher the similarity, and the smaller the background change degree, and vice versa. In order to obtain the target video frame subsequence with large image background change, the background change degree value corresponding to the video frame image subsequence needs to be lower than the preset degree value.

[0082] The preset degree value can be 0.8, 0.7, or 0.6, and can be limited according to actual needs, which is not limited here.

[0083] If the target video frame image in the target video frame image subsequence does not include text, the preset text needs to be added to the preset position in the target video frame image, so that the target video frame image with text can be used to train the neural network model to obtain the text tracking model.

[0084] The embodiment of the present application provides a text tracking method, which is shown in Figure 2 The embodiment of the text tracking method provided by the embodiment of the present application is shown in the flowchart. Figure 2 The flowchart can include the following steps:

[0085] Step 201, obtaining a video to be text tracking;

[0086] Step 202, performing frame extraction processing on the video to be text tracked to obtain a sequence of frame images of the video to be text tracked;

[0087] Step 203, performing text labeling of the text to be tracked on the frame images of the video to be text tracked in a preset frame number interval from the sequence of frame images of the video to be text tracked;

[0088] According to the above method, a plurality of frame images of the video to be text tracked are obtained from the sequence of frame images of the video to be text tracked in a preset frame number interval, and text labeling of the text to be tracked in each frame image of the video to be text tracked is performed, which can specifically be text box framing of the text to be tracked.

[0089] Step 204, inputting the frame images of the video to be text tracked after text labeling into a text tracking model, and outputting, by the text tracking model, the frame images of the video to be text tracked with text boxes labeled with the text to be tracked;

[0090] Since the text tracking model is trained by the above text tracking model training method, the text tracking model can accurately track the text, and the text tracking model can accurately output the text in the frame images of the video to be text tracked without text labeling in the sequence of frame images of the video to be text tracked. Compared with manually labeling and tracking the text in each frame image of the video to be text tracked, the text tracking method using the text tracking model can greatly save the text tracking time and improve the accuracy.

[0091] An embodiment of a text tracking model training device is provided in the present application. Figure 3 An embodiment block diagram of the text tracking model training device is provided in the present application. Figure 3 As shown in the figure, the device can include:

[0092] The first acquisition module 301 is configured to acquire a plurality of videos.

[0093] The execution module 302 is configured to perform the following operations on each video: performing frame extraction processing on the video to obtain a sequence of frame images, dividing the sequence of frame images into a plurality of video frame image subsequences, and determining a target video frame image subsequence from the plurality of video frame image subsequences; wherein the target video frame image subsequence is a sequence of video frame images with the same text and large image background change degree, and the text is labeled by a text box.

[0094] The training module 303 is configured to input the plurality of target video frame image subsequences into a neural network model for training until a training period reaches a preset training period or a model loss value reaches a preset loss threshold, thereby obtaining a text tracking model.

[0095] The embodiment of the present application provides a text tracking device, referring to Figure 4 An embodiment block diagram of the text tracking device provided by the embodiment of the present application is shown in the figure. Figure 4 The device can include:

[0096] The second acquisition module 401 is used for acquiring a video to be tracked.

[0097] The frame extraction processing module 402 is used for performing frame extraction processing on the video to be tracked to obtain a sequence of frame images of the video to be tracked.

[0098] The text labeling module 403 is used for performing text labeling on the frame images of the video to be tracked in a preset frame number interval of the sequence of frame images of the video to be tracked.

[0099] The text tracking module 404 is used for inputting the frame images of the video to be tracked after text labeling into a text tracking model, and the text tracking model outputs the sequence of frame images of the video to be tracked, in which each frame is labeled with a text box of text to be tracked; wherein the text tracking model is obtained by training the text tracking model training method.

[0100] Figure 5 The structure schematic diagram of the electronic device provided by the embodiment of the present application is shown in the figure. Figure 5 The electronic device 500 shown in the figure includes at least one processor 501, a memory 502, at least one network interface 504 and other user interfaces 503. Each component in the electronic device 500 is coupled together through a bus system 505. It can be understood that the bus system 505 is used to realize the connection and communication between the components. In addition to the data bus, the bus system 505 also includes a power bus, a control bus and a status signal bus. However, in order to clearly illustrate, all kinds of buses are marked as the bus system 505 in the figure. Figure 5

[0101] The user interface 503 can include a display, a keyboard or a clicking device (for example, a mouse, a trackball, a touchpad or a touch screen, etc.).

[0102] ​It is to be appreciated that the memory 502 in the embodiments of the present application can be a volatile memory or a nonvolatile memory, or can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM can be used, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synch link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 502 described herein is intended to include, without being limited to, these and any other suitable types of memory.

[0103] In some embodiments, the memory 502 stores the following elements, executable units or data structures, or a subset of them, or an extended set of them: an operating system 5021 and application programs 5022.

[0104] Among them, the operating system 5021 contains various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application programs 5022 contain various application programs, such as a Media Player, a Browser, etc., for implementing various application services. The programs for implementing the method embodiments of the present application can be contained in the application programs 5022.

[0105] In the embodiments of the present application, the processor 501 is used to execute the method steps provided by each method embodiment by invoking the programs or instructions stored in the memory 502, specifically, the programs or instructions stored in the application programs 5022.

[0106] The method disclosed by the embodiments of the present application can be applied to the processor 501 or implemented by the processor 501. The processor 501 can be an integrated circuit chip having a signal processing capability. In the implementation process, the steps of the method can be completed by an integrated logic circuit or an instruction in the form of software in the processor 501. The processor 501 described above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The disclosed methods, steps and logic block diagrams in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software units in the code processor for execution. The software unit can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502 and combines the hardware to complete the steps of the above method.

[0107] It can be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described in the present application, or a combination thereof.

[0108] For software implementation, the techniques described herein can be implemented with a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0109] The electronic device provided by the embodiment can be an electronic device as shown in Figure 5 The electronic device provided by the embodiment can be an electronic device as shown in Figures 1-2 The electronic device provided by the embodiment can be an electronic device as shown in Figures 1-2 The electronic device provided by the embodiment can be an electronic device as shown in Figures 1-2 The electronic device provided by the embodiment can be an electronic device as shown in

[0110] The embodiment of the present application further provides a storage medium (computer readable storage medium). The storage medium stores one or more programs. The storage medium can include a volatile memory such as a random access memory, and the memory can also include a non-volatile memory such as a read-only memory, a flash memory, a hard disk, or a solid state disk, and the memory can also include a combination of the above-mentioned memories.

[0111] When the one or more programs in the storage medium are executed by one or more processors, the above-mentioned method can be implemented.

[0112] The processor is configured to execute the program stored in the memory to implement the steps of the text tracking model training method and the text tracking method. It should also be further realized by those skilled in the art that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in the above description in general terms. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0113] The steps of the method or algorithm described in combination with the embodiments disclosed herein can be implemented in hardware, software modules executed by a processor, or a combination of both. The software modules can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0114] The above specific embodiments further illustrate the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for training a text tracking model, characterized in that, The method comprises: acquiring a plurality of videos; for each of the videos, performing the following operations: frame extraction processing on the video to obtain a video frame image sequence, dividing the video frame image sequence into a plurality of video frame image subsequences, and determining a target video frame image subsequence from the plurality of video frame image subsequences; wherein the target video frame image subsequence is a sequence of video frame images with the same text and a large degree of image background change, the text is labeled with a text box, and the background change degree value corresponding to the target video frame image subsequence is lower than a preset degree value; inputting the plurality of target video frame image subsequences into a neural network model for training until a preset training period is reached or a model loss value reaches a preset loss threshold, thereby obtaining a text tracking model.

2. The method of claim 1, wherein, The dividing of the video frame image sequence into a plurality of video frame image subsequences comprises: text recognition detection on the video frame image sequence; determining whether the video frame images in the video frame image sequence include text; in a case where it is determined that the video frame images in the video frame image sequence include text, merging the video frame images with the same text to obtain a plurality of video frame image subsequences; in a case where it is determined that the video frame images in the video frame image sequence do not include text, dividing the video frame image sequence according to a preset frame number to obtain a plurality of video frame image subsequences.

3. The method of claim 1, wherein, The determining of the target video frame image subsequence from the plurality of video frame image subsequences comprises: for each of the video frame image subsequences, drawing a preset number of first video frame images from the video frame image subsequence, and determining a background change degree value of the video frame image subsequence based on the plurality of first video frame images; determining a target video frame image subsequence from the plurality of video frame image subsequences based on the plurality of background change degree values.

4. The method of claim 3, wherein, The determining of the background change degree value of the video frame image subsequence based on the plurality of first video frame images comprises: determining the similarity between any two adjacent first video frame images in the plurality of first video frame images using a similarity algorithm; determining a background change degree value based on the plurality of similarities.

5. The method of claim 3, wherein, The determining of the target video frame image subsequence from the plurality of video frame image subsequences based on the plurality of background change degree values comprises: determining the video frame image subsequence corresponding to the background change degree value lower than the preset degree value as the target video frame image subsequence.

6. The method of claim 2, wherein, The method further comprises: in a case where the target video frame image in the target video frame image subsequence does not include text, adding a preset text to a preset position in the target video frame image.

7. The method of claim 2, wherein, The text recognition detection on the video frame image sequence comprises: performing text recognition detection on the video frame image sequence by using a pre-trained text recognition detection model.

8. A text tracking method characterized by, The method comprises: acquiring a video to be tracked for text; performing frame extraction processing on the video to be tracked for text to obtain a video frame image sequence to be tracked for text; text labeling of the to-be-tracked text on a preset video frame interval of to-be-text-tracked video frame images in the to-be-text-tracked video frame image sequence; inputting the to-be-text-tracked video frame image sequence after text labeling into a text tracking model, the text tracking model outputting a to-be-text-tracked video frame image sequence in which each frame is labeled with a text box of the to-be-tracked text; wherein the text tracking model is trained by the text tracking model training method in claims 1-7. 9.A text tracking model training apparatus, characterized by comprising: The apparatus comprises: The first obtaining module is configured to obtain a plurality of videos. The execution module is configured to perform the following operations on each of the videos: frame extraction processing on the video to obtain a video frame image sequence, division of the video frame image sequence into a plurality of video frame image subsequences, and determination of a target video frame image subsequence from the plurality of video frame image subsequences; wherein the target video frame image subsequence is a sequence of video frame images with the same text and a large degree of image background change, the text is labeled with a text box, and the background change degree value corresponding to the target video frame image subsequence is lower than a preset degree value. The training module is configured to input the plurality of target video frame image subsequences into a neural network model for training until a preset training period is reached or a model loss value reaches a preset loss threshold, thereby obtaining a text tracking model.

10. A text tracking apparatus, characterized by comprising: The apparatus comprises: The second obtaining module is configured to obtain a to-be-text-tracked video. The frame extraction processing module is configured to perform frame extraction processing on the to-be-text-tracked video to obtain a to-be-text-tracked video frame image sequence. The text labeling module is configured to perform text labeling of the to-be-tracked text on a preset video frame interval of to-be-text-tracked video frame images in the to-be-text-tracked video frame image sequence. The text tracking module is configured to input the to-be-text-tracked video frame image sequence after text labeling into a text tracking model, the text tracking model outputting a to-be-text-tracked video frame image sequence in which each frame is labeled with a text box of the to-be-tracked text; wherein the text tracking model is trained by the text tracking model training method in claims 1-7.

11. An electronic device, comprising: The processor is configured to execute a text tracking model training program and a text tracking program stored in the memory to implement the text tracking model training method in any one of claims 1-7 and the text tracking method in claim 8. The storage medium stores one or more programs that can be executed by one or more processors to implement the text tracking model training method in any one of claims 1-7 and the text tracking method in claim 8.

12. A storage medium, characterized by ​

Images