Video processing method and apparatus, and sports video processing method and apparatus

Abstract

Embodiments of the present disclosure relate to the technical field of multimedia processing, and provide a video processing method and apparatus, and a sports video processing method and apparatus. The video processing method comprises: determining a target black-and-white video, and determining a reference black-and-white video frame from a black-and-white video frame sequence of the target black-and-white video; coloring the reference black-and-white video frame to obtain a reference color video frame; on the basis of the reference color video frame, the target black-and-white video frame, and a previous color video frame, using an image processing model to color the target black-and-white video frame, so as to obtain a target color video frame; and on the basis of the target color video frame, generating a target color video corresponding to the target black-and-white video. The method can improve the spatial accuracy of video coloring and video temporal consistency, and improve video processing efficiency.

Description

Video processing methods and apparatus, sports video processing methods and apparatus Technical Field

[0001] This disclosure relates to the field of multimedia processing technology, and in particular to a video processing method. One or more embodiments of this disclosure also relate to a sports video processing method, a video processing apparatus, and a sports video processing device. Background Technology

[0002] There are many historical videos with high historical value. These videos record historical changes, cultural heritage, technological development, and so on. However, due to limitations in technology, improper preservation, or the passage of time, some historical videos are only black and white. Colorizing these black and white videos is of great significance for protecting and inheriting historical culture, enhancing the viewing experience and research value of historical videos, and so on.

[0003] Some feasible solutions achieve colorization of black and white videos by colorizing each frame of the black and white video. However, the resulting color videos usually have low spatial accuracy and poor temporal sequence, which reduces the viewing experience of the resulting color videos.

[0004] Therefore, there is an urgent need for a technical solution to improve the spatial accuracy of video colorization and the temporal sequence of videos, in order to solve the above-mentioned technical problems. Summary of the Invention

[0005] In view of this, the present disclosure provides a video processing method. One or more embodiments of the present disclosure also relate to a sports video processing method, a video processing apparatus, a sports video processing device, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies in existing technologies such as low color fidelity and poor video temporal sequence in video colorization.

[0006] According to a first aspect of the present disclosure, a video processing method is provided, comprising:

[0007] Determine the target black and white video, and determine the reference black and white video frame from the black and white video frame sequence of the target black and white video;

[0008] Colorize the reference black and white video frame to obtain a reference color video frame;

[0009] Based on the reference color video frame, the target black and white video frame, and the previous color video frame, the target black and white video frame is colorized using an image processing model to obtain a target color video frame. The target black and white video frame is any black and white video frame in the sequence of black and white video frames, and the previous color video frame is the color video frame corresponding to the previous black and white video frame of the target black and white video frame. The image processing model is a machine learning model.

[0010] Based on the target color video frame, generate a target color video corresponding to the target black and white video.

[0011] According to a second aspect of the present disclosure, a sports video processing method is provided, comprising:

[0012] Determine the target sports black and white video, and determine the reference sports black and white video frame from the sports black and white video frame sequence of the target sports black and white video;

[0013] Colorize the reference sports black and white video frame to obtain a reference sports color video frame;

[0014] Based on the reference sports color video frame, the target sports black and white video frame, and the previous sports color video frame, the target sports black and white video frame is colorized using an image processing model to obtain the target sports color video frame. The target sports black and white video frame is any sports black and white video frame in the sports black and white video frame sequence, and the previous sports color video frame is the sports color video frame corresponding to the previous sports black and white video frame of the target sports black and white video frame. The image processing model is a machine learning model.

[0015] Based on the target sports color video frame, generate the target sports color video corresponding to the target sports black and white video.

[0016] According to a third aspect of the present disclosure, a video processing apparatus is provided, comprising:

[0017] The first frame determination module is configured to determine the target black and white video and determine the reference black and white video frame from the black and white video frame sequence of the target black and white video;

[0018] The first frame colorization module is configured to colorize the reference black and white video frame to obtain a reference color video frame;

[0019] The second frame colorization module is configured to colorize the target black-and-white video frame based on the reference color video frame, the target black-and-white video frame, and the previous color video frame using an image processing model to obtain a target color video frame. The target black-and-white video frame is any black-and-white video frame in the sequence of black-and-white video frames, and the previous color video frame is the color video frame corresponding to the previous black-and-white video frame of the target black-and-white video frame. The image processing model is a machine learning model.

[0020] The color video determination module is configured to generate a target color video corresponding to the target black-and-white video based on the target color video frame.

[0021] According to a fourth aspect of the present disclosure, a sports video processing apparatus is provided, comprising:

[0022] The first sports frame determination module is configured to determine a target sports black and white video and determine a reference sports black and white video frame from the sports black and white video frame sequence of the target sports black and white video;

[0023] The first sports frame colorization module is configured to colorize the reference sports black and white video frame to obtain the reference sports color video frame.

[0024] The second sports frame determination module is configured to sequentially determine the sports black and white video frames in the sports black and white video frame sequence as target sports black and white video frames.

[0025] The second sports frame colorization module is configured to colorize the target sports black-and-white video frame using an image processing model based on the reference sports color video frame, the target sports black-and-white video frame, and the previous sports color video frame, to obtain the target sports color video frame. The target sports black-and-white video frame is any sports black-and-white video frame in the sports black-and-white video frame sequence, and the previous sports color video frame is the sports color video frame corresponding to the previous sports black-and-white video frame of the target sports black-and-white video frame. The image processing model is a machine learning model.

[0026] The sports color video determination module is configured to generate a target sports color video corresponding to the target sports black and white video based on the target sports color video frame.

[0027] According to a fifth aspect of the present disclosure, a computing device is provided, comprising:

[0028] Memory and processor;

[0029] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, they implement the steps of the above-mentioned video processing method or sports video processing method.

[0030] According to a sixth aspect of the present disclosure, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the video processing method or sports video processing method described above.

[0031] According to a seventh aspect of the present disclosure, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the video processing method or sports video processing method described above.

[0032] This disclosure provides a video processing method that, through colorizing a reference black-and-white video frame in a target black-and-white video, obtains a reference color video frame with more accurate colors. This reference color frame serves as a color reference for subsequent colorizing of the target black-and-white video frame, improving the spatial accuracy of colorizing the target black-and-white video frame. Furthermore, based on the reference color video frame, the target black-and-white video frame, and the previous color video frame, an image processing model is used to colorize the target black-and-white video frame. In addition to improving the color spatial accuracy of the target black-and-white video frame, the reference to the previous color video frame makes the color transition between the target black-and-white video frame and the previous color video frame more natural, improving the temporal sequence of the subsequent target color video obtained from the target black-and-white video frame. Moreover, the image processing model is used to automatically colorize the target black-and-white video frame, improving video processing efficiency. In summary, this method improves the spatial accuracy and temporal sequence of video colorization, and also improves video processing efficiency. Attached Figure Description

[0033] Figure 1 is a specific application scenario diagram of a video processing method provided in an embodiment of this disclosure;

[0034] Figure 2 is a flowchart of a video processing method provided in an embodiment of this disclosure;

[0035] Figure 3 is a flowchart of a video processing method provided in an embodiment of this disclosure;

[0036] Figure 4 is a reference colorization process flowchart of a video processing method provided in an embodiment of this disclosure;

[0037] Figure 5 is a schematic diagram of the interactive interface of a video processing method provided in an embodiment of this disclosure;

[0038] Figure 6 is a flowchart of a sports video processing method provided in an embodiment of this disclosure;

[0039] Figure 7 is a schematic diagram of the structure of a video processing apparatus provided in an embodiment of the present disclosure;

[0040] Figure 8 is a schematic diagram of the structure of a sports video processing device provided in an embodiment of the present disclosure;

[0041] Figure 9 is a structural block diagram of a computing device provided in an embodiment of this disclosure. Detailed Implementation

[0042] Numerous specific details are set forth in the following description to provide a full understanding of this disclosure. However, this disclosure can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this disclosure. Therefore, this disclosure is not limited to the specific implementations disclosed below.

[0043] The terminology used in one or more embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this disclosure. The singular forms “a,” “the,” and “the” as used in one or more embodiments of this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this disclosure refers to and includes any or all possible combinations of one or more associated listed items.

[0044] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this disclosure, and similarly, second may also be referred to as first. Depending on the context, the word “if” as used herein may be interpreted as “when”, “in response to a determination”, or “when…”.

[0045] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0046] In one or more embodiments of this disclosure, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0047] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as natural language processing tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0048] With the development of digital media and technology, more and more high-quality video content is being produced. However, due to limitations in technology, improper preservation, or the passage of time, many historical videos of high value suffer from problems such as low quality, partial damage, and / or black and white images. Examples include historical Olympic videos, historical films, black and white game videos, and black and white animations. These historical videos urgently need to be restored and colorized.

[0049] This disclosure provides a video processing method in one embodiment, which automatically processes video end-to-end by applying image / video processing technology. For example, it converts video colorization into image processing of independent video frames, and achieves video frame colorization by colorizing all frames of the video.

[0050] However, because each video frame is independently colored, the sequential nature of the color video obtained by splicing together the independently colored video frames is poor. Furthermore, since the coloring of the video frames is automatically implemented by image processing, the colored colors may differ from the colors actually desired by the user, resulting in distortion. Therefore, the color video obtained by this video processing method has poor viewing quality and a poor user experience.

[0051] To address the aforementioned technical problems, this disclosure provides a video processing method, and also relates to a sports video processing method, a video processing apparatus, a sports video processing apparatus, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0052] Considering the large number of model parameters in large language models and the limited computing resources of mobile terminals, the video processing method provided in this application embodiment can be applied to the application scenario shown in Figure 1, but is not limited thereto.

[0053] Referring to Figure 1, which illustrates a specific application scenario of a video processing method according to an embodiment of this disclosure, in the application scenario shown in Figure 1, an image processing model is deployed in a video processing server 104. The video processing server 104 and one or more clients 102 can be connected via a local area network (LAN), a wide area network (WAN), the Internet, or other types of data networks. The client 102 may include, but is not limited to, smartphones, tablets, laptops, PDAs, personal computers, smart home devices, in-vehicle devices, etc. The video processing server 104 can interact with the user through the user interface of the client 102 to implement the video processing method provided in this embodiment of the disclosure.

[0054] Specifically, the video processing method includes:

[0055] Determine the reference black and white video frame from the black and white video frame sequence of the target black and white video;

[0056] Colorize the reference black and white video frame to obtain a reference color video frame;

[0057] Based on the reference color video frame, the target black and white video frame, and the previous color video frame, the target black and white video frame is colorized using an image processing model to obtain a target color video frame. The target black and white video frame is any black and white video frame in the sequence of black and white video frames, and the previous color video frame is the color video frame corresponding to the previous black and white video frame of the target black and white video frame. The image processing model is a machine learning model.

[0058] Based on the target color video frame, generate a target color video corresponding to the target black and white video.

[0059] In this embodiment of the disclosure, the system consisting of client 102 and video processing server 104 can perform the following steps: the video processing server 104 determines the target black and white video that the user needs to process through the user's touch operation on the user interface of client 102, and determines a reference black and white video frame from the black and white video frame sequence of the target black and white video; then, the reference black and white video frame is colorized to obtain a reference color video frame for subsequent colorization reference of the black and white video frames in the target black and white video frame sequence.

[0060] Given a reference color video frame, the black and white video frames in the black and white video frame sequence are sequentially identified as target black and white video frames, that is, the black and white video frames in the black and white video frame sequence are colored frame by frame.

[0061] Specifically, based on the reference color video frame, black and white video frame, and the color video frame corresponding to the previous black and white video frame, the target black and white video frame is colored using an image processing model to obtain the target color video frame, thus completing the process of colorizing the target black and white video frame.

[0062] Then, based on the target color video frame corresponding to the target black and white video frame, an initial color video corresponding to the target black and white video can be generated. Based on this initial color video, the target color video corresponding to the target black and white video can be determined, and the target black and white video can be colorized. The target color video is then returned to the client 102 for the user to view.

[0063] It should be noted that, provided that the operating resources of the client 102 can meet the deployment and operating conditions of the image processing model, the embodiments of this disclosure can be carried out on the client 102; in some embodiments, the image processing model can also be deployed on a third-party server, and the third-party server and video processing server 104 include, but are not limited to, physical servers, cloud servers, physical server groups, cloud server clusters, etc.

[0064] This disclosure provides a video processing method in one embodiment. The video processing server determines the target black-and-white video to be processed through user-triggered operations in the client's user interface. It then colorizes a reference black-and-white video frame within the target black-and-white video to obtain a more accurate reference color video frame. This reference color frame is used to provide color reference for subsequent colorization of the target black-and-white video frame, improving the color space accuracy of the colorization. Finally, based on the reference color video frame, the target black-and-white video frame, and the previous color video frame, an image processing model is used to colorize the target black-and-white video frame, further improving the color space accuracy of the target black-and-white video frame. In this case, by referencing the previous color video frame of the target black-and-white video frame, the color transition between the target black-and-white video frame and the previous color video frame is made more natural, improving the temporality of the subsequent target color video obtained from the target black-and-white video frame. Furthermore, by utilizing an image processing model, automatic colorization of the target black-and-white video frame is achieved, improving video processing efficiency. In summary, this method improves the color realism and temporality of video colorization, and also improves video processing efficiency. Furthermore, the target color video obtained by this method has more realistic colors, better meets user needs, and has stronger temporality, greatly enhancing the user experience.

[0065] Referring to Figure 2, Figure 2 shows a flowchart of a video processing method according to an embodiment of the present disclosure, which specifically includes the following steps.

[0066] Step 202: Determine the target black and white video, and determine the reference black and white video frame from the black and white video frame sequence of the target black and white video.

[0067] The target black and white video can be understood as a black and white video clip to be processed. For example, the target black and white video can be understood as a sports-type target black and white video, a movie-type target black and white video, a TV series-type target black and white video, a game-type target black and white video, an animation-type target black and white video, etc. Video processing includes, but is not limited to, video colorization, video quality restoration, video segmentation, video frame synthesis, etc. For example, black and white videos include, but are not limited to, black and white video clips of sports, movies, TV series, games, and black and white animation clips, etc.

[0068] The black and white video frame sequence of the target black and white video can be understood as an image sequence composed of black and white video frames contained in the target black and white video, and the target black and white video is synthesized from the target black and white video frames.

[0069] A reference black and white video frame can be understood as one or more black and white video frames determined from the target black and white video frame sequence.

[0070] In practical applications, when the black and white video to be processed is long, the camera transitions can cause significant color errors. In such cases, using this black and white video as the target black and white video for video processing, and still referencing the reference black and white video frames of the target black and white video, will result in significant color errors.

[0071] Based on this, the initial black-and-white video to be processed can be segmented, and then the shorter segmented black-and-white video can be used for subsequent video processing. The specific implementation method is as follows:

[0072] The method further includes:

[0073] Determine an initial black and white video, and segment the initial black and white video to obtain at least two segmented black and white video segments;

[0074] Each segment of the at least two black-and-white video segments is sequentially identified as the target black-and-white video.

[0075] The initial black and white video can be understood as a black and white video to be processed. For example, the initial black and white video can be understood as a black and white sports event video, a black and white movie, a black and white TV series, a black and white game video, a black and white animation, etc.

[0076] Video segmentation can be achieved through various technical means, including but not limited to scene detection-based video segmentation (detecting changes in the scene within the video to automatically identify the boundaries of video segments, thereby achieving video segmentation), timeline-based automatic segmentation (setting a fixed duration to automatically segment the video, such as dividing the video into segments every 5 minutes), content understanding-based segmentation (understanding the content of the video through deep learning and computer vision technology, identifying specific events, people, or actions, and then automatically segmenting the video based on these key points), manual operation-based segmentation (segmenting the video based on manually selected video frames), and video segmentation using machine learning models (inputting the video into a video segmentation machine learning model, which then segments the video), etc.

[0077] Specifically, after determining the initial black and white video to be processed, the initial black and white video can be divided into multiple shorter segmented black and white videos using video segmentation technology. Each segmented black and white video is then designated as the target black and white video, so that the initial black and white video can be colorized by performing a subsequent video colorization step on the target black and white video.

[0078] This disclosure provides a video processing method in one or more embodiments, which segments an initial black and white video to make the black and white video frames in the target black and white video obtained after segmentation more similar, thereby improving the spatial accuracy of subsequent video processing based on the reference black and white video frames of the target black and white video.

[0079] In practical applications, due to limitations in technological development and improper storage, some videos may suffer from poor image quality or video corruption. Therefore, it is possible to perform image quality restoration on original black and white videos with poor image quality or video corruption to obtain an initial black and white video. The specific implementation method is as follows:

[0080] The determination of the initial black and white video includes:

[0081] The original black and white video is identified, and its image quality is restored to obtain the initial black and white video.

[0082] Among them, original black and white videos can be understood as black and white videos with poor image quality or video damage, such as black and white videos of historical events with video noise, black and white videos of events with video scratches due to improper storage, black and white videos of events with field patterns, black and white videos of movies with low resolution, black and white animations, black and white game videos, etc.

[0083] Image restoration includes, but is not limited to, video denoising, video scratch removal, video field pattern removal, etc. In practical applications, the appropriate image restoration method can be selected based on the image quality damage problems existing in the original black and white video, and there are no restrictions on this.

[0084] In practice, the original black and white video is determined, and image quality restoration is performed on the original black and white video to obtain the initial black and white video. This can be understood as determining the original black and white video, identifying a suitable image quality restoration plan based on the image quality damage issues existing in the original black and white video, and automatically performing image quality restoration on the original black and white video according to the image quality restoration plan to obtain the restored original black and white video, which is the initial black and white video. Subsequent video processing can be performed on this initial black and white video in conjunction with the above-described embodiments, which will not be elaborated here.

[0085] This disclosure provides a video processing method in one or more embodiments. By performing image quality restoration on an original black and white video with image quality degradation, the video quality of the obtained initial black and white video is improved, enhancing the viewing experience of the initial black and white video. Furthermore, subsequent video processing based on the initial black and white video results in a target color video with higher video quality, higher clarity, and better viewing experience, thus improving the user's viewing experience.

[0086] In practical applications, the video processing method provided in one or more embodiments of this disclosure can also process the initial black and white video requested by the user according to the user's needs, so as to improve the user experience. Specifically, the initial black and white video can be selected and determined by the user, or it can be obtained by the user's upload. The specific implementation method is as follows:

[0087] The determination of the initial black and white video includes:

[0088] In response to a selection instruction sent by the client for the initial black-and-white video, the initial black-and-white video is determined based on the identification information of the initial black-and-white video carried in the selection instruction, wherein the selection instruction is generated by the client in response to a selection operation triggered by the user on the client's user interface; or,

[0089] The client receives the initial black and white video sent by the client, wherein the initial black and white video is sent by the client in response to an interactive operation triggered by the user on the client's user interface.

[0090] Specifically, the client here can be understood as the aforementioned client 102.

[0091] User interface can be understood as the interface through which users interact with the client, including but not limited to dialog box interfaces, graphical interfaces, etc.; selection operations include but are not limited to drop-down list selection operations, radio button selection operations, check box selection operations, list or grid selection operations, slider selection operations, voice command selection operations, keyboard input selection operations, interface area selection operations, tag or category selection operations, etc.

[0092] The selection command can be understood as a command generated based on the user's selection operation in the client's user interface, used to determine the initial black and white video corresponding to the selection operation.

[0093] The identification information includes, but is not limited to, video encoding information, video address, video sequence number, etc. In practical applications, appropriate identification information can be selected according to actual development and design requirements to identify black and white videos. This disclosure does not limit this. The identification information of the initial black and white video can be understood as the video encoding information, video address, video sequence number, etc. of the initial black and white video, used to identify the initial black and white video.

[0094] In practice, the client can display multiple black and white videos to the user through a user interface. The user selects a black and white video as the initial black and white video in the user interface and triggers a selection operation. The client generates a selection instruction based on the user's selection operation, carrying the identification information of the initial black and white video selected by the user. The selection instruction carrying the identification information of the initial black and white video is sent to the video processing server. The video processing server can determine the initial black and white video based on the selection instruction carrying the identification information of the initial black and white video.

[0095] Interactive operations can be understood as upload operations, including but not limited to uploading compressed files, uploading video files, uploading database data, uploading encrypted files, and so on.

[0096] In practice, the client can also determine the initial black and white video uploaded by the user by responding to the interactive operation triggered by the user interface, and send the initial black and white video to the video processing server. The video processing server can also determine the initial black and white video by receiving the initial black and white video sent by the client.

[0097] This disclosure provides a video processing method through one or more embodiments, which allows interaction between a client and a user, enabling the user to more intuitively determine the video that needs video processing and to process the initial black and white video requested by the user. This improves user interactivity, makes video processing more in line with user needs, and thus enhances the user experience.

[0098] In practical applications, after determining the initial video, video segmentation can also be achieved based on the differences between the individual black and white video frames of the initial black and white video, in order to reduce the differences between the black and white video frames in the segmented black and white video. The specific implementation method is as follows:

[0099] The process of segmenting the initial black-and-white video to obtain at least two segmented black-and-white video segments includes:

[0100] The difference between the detected black and white video frame and the next initial black and white video frame is calculated to obtain the degree of difference between the detected black and white video frame and the next initial black and white video frame, wherein the detected black and white video frame is any initial black and white video frame in the sequence of initial black and white video frames of the initial black and white video.

[0101] If the degree of difference is determined to be greater than or equal to a preset difference threshold, the initial black and white video is segmented according to the detected black and white video frames to obtain at least two segmented black and white video segments.

[0102] The initial black and white video frame sequence can be understood as an image sequence composed of initial black and white video frames contained in the initial black and white video, which is synthesized from the initial black and white video frames.

[0103] Specifically, after determining the initial black and white video, the initial black and white video frames for synthesizing the initial black and white video, as well as the initial black and white video frame sequence composed of the initial black and white video frames in sequence, can be determined from the initial black and white video frame sequence. The detection black and white video frames can be determined from the initial black and white video frame sequence. By detecting the detection black and white video frames, the video segmentation position of the initial black and white video can be determined.

[0104] The degree of difference can be understood as the degree of difference in image content, including but not limited to the degree of color difference, brightness difference, contrast difference, texture difference, etc. In practical applications, the movement of objects, changes in lighting conditions, camera movement, scene changes, etc. in the initial black and white video will cause differences between two adjacent initial black and white video frames. Based on the degree of difference between the detected black and white video frame and the next initial black and white video frame, the degree of change in object movement, camera movement, etc. between the detected black and white video frame and the next initial black and white video frame can be determined. The greater the degree of difference between the detected black and white video frame and the next initial black and white video frame, the higher the probability that the detected black and white video frame has undergone a shot change or shot movement.

[0105] Based on this, a difference threshold can be preset to identify the initial black and white video frames with more obvious video changes.

[0106] The difference between the detected black-and-white video frame and the next initial black-and-white video frame is calculated to obtain the degree of difference between the detected black-and-white video frame and the next initial black-and-white video frame.

[0107] In specific implementation, after determining the black and white video frame to be detected, the difference between the detected black and white video frame and the next initial black and white video frame is calculated to obtain the degree of difference between the detected black and white video frame and the next initial black and white video frame. If the degree of difference is greater than or equal to a preset difference threshold, the initial black and white video can be segmented based on the detected black and white video to obtain at least two segmented black and white video segments. Then, the next step of determining the initial black and white video frame in the initial black and white video frame sequence as the detected black and white video frame continues.

[0108] In practical applications, when the degree of difference is greater than or equal to a preset difference threshold, the detected black and white video frame can be marked as a segmented video frame. After all the initial black and white video frames in the initial black and white video frame have been identified as detected black and white video frames and the difference calculation has been performed with the next initial black and white video frame of the detected black and white video frame, the initial black and white video is segmented according to all the segmented video frames obtained by marking, and at least two segmented black and white video segments are obtained.

[0109] This disclosure provides a video processing method in one or more embodiments. By determining an initial black-and-white video frame as a detection black-and-white video frame, and calculating the difference between the detection black-and-white video frame and the next initial black-and-white video frame, the method calculates the degree of difference between the detection black-and-white video frame and the next initial black-and-white video frame. By quantitatively determining the video segmentation boundary, the method separates initial black-and-white video frames with large differences, making video segmentation more reasonable and improving the rationality of video segmentation.

[0110] In practical applications, the degree of difference between the detected black-and-white video frame and the next initial black-and-white video frame can be determined by the difference in the boundary regions of the black-and-white video frames. The specific implementation method is as follows:

[0111] The step of calculating the difference between the detected black-and-white video frame and the next initial black-and-white video frame to obtain the degree of difference between the detected black-and-white video frame and the next initial black-and-white video frame includes:

[0112] Boundary detection is performed on the detected black-and-white video frame and the next initial black-and-white video frame to determine the first boundary region of the detected black-and-white video frame and the second boundary region of the next initial black-and-white video frame.

[0113] Calculate the color difference between the first boundary region and the second boundary region, and determine the color difference as the degree of difference between the detected black and white video frame and the next initial black and white video frame of the detected black and white video frame.

[0114] Boundary detection can be understood as a technical means of identifying and locating the boundaries or edges of objects contained in an image. Boundary detection includes, but is not limited to, boundary detection based on classical edge detection operators, boundary detection based on deep learning, etc. The embodiments of this disclosure do not limit the selection of boundary detection techniques and parameter settings, which can be set according to application requirements in actual applications.

[0115] A boundary region can be understood as the boundary portion of an object contained in an image. For example, an image may contain an athlete, the athlete's shirt, and the boundary region of the image can be understood as the boundary portion or edge portion of the athlete, the boundary portion or edge portion of the athlete's shirt, and so on.

[0116] Detecting the first boundary region of a black and white video frame can be understood as detecting the boundary region of an object in the black and white video frame, and detecting the second boundary region of the next initial black and white video frame can be understood as detecting the boundary region of an object in the next initial black and white video frame.

[0117] Based on this, after determining the black and white video frame to be detected, boundary detection can be performed on the black and white video frame to obtain the first boundary region of the black and white video frame to be detected, and boundary detection can be performed on the next initial black and white video frame to obtain the second boundary region of the next initial black and white video frame to be detected.

[0118] Then, the color space difference (i.e., color difference) between the first boundary region and the second boundary region can be calculated, and this color space difference can be determined as the degree of difference between the detected black and white video frame and the next initial black and white video frame after the detected black and white video frame.

[0119] This disclosure provides a video processing method that reduces the impact of static image regions such as ground and sky on the analysis of shot transitions by calculating the color difference between a first boundary region of a detected black-and-white video frame and a second boundary region of the next initial black-and-white video frame. This improves the spatial accuracy of the degree of difference between the detected black-and-white video frame and the next initial black-and-white video frame.

[0120] Optionally, after obtaining the first boundary region and the second boundary region, the pixel difference, contrast difference, or brightness difference between the first boundary region and the second boundary region can be calculated, and the pixel difference, contrast difference, or brightness difference can be determined as the degree of difference between the detected black and white video frame and the next initial black and white video frame. In specific implementation, the appropriate object for calculating the difference can be determined according to actual needs.

[0121] In practical applications, the degree of difference between the detected black-and-white video frame and the next initial black-and-white video frame can also be determined by the difference between the main regions in the black-and-white video frames. The specific implementation method is as follows:

[0122] The step of calculating the difference between the detected black-and-white video frame and the next initial black-and-white video frame to obtain the degree of difference between the detected black-and-white video frame and the next initial black-and-white video frame includes:

[0123] Subject detection is performed on the detected black and white video frame and the next initial black and white video frame to determine the first subject region of the detected black and white video frame and the second subject region of the next initial black and white video frame.

[0124] Calculate the color difference between the first main area and the second main area, and determine the color difference as the degree of difference between the detected black and white video frame and the next initial black and white video frame of the detected black and white video frame.

[0125] Subject detection can be understood as a technical means of identifying and locating objects contained in an image. Subject detection includes, but is not limited to, subject detection based on color and texture, subject detection based on shape and contour, and using saliency mapping technology to automatically identify the more attractive parts of an image as the subject, etc. The embodiments of this disclosure do not limit the selection of subject detection techniques and parameter settings, which can be set according to application requirements in actual applications.

[0126] The main region can be understood as the image of the object contained in an image. For example, an image may contain an athlete, the athlete's shirt, and the main region of the image can be understood as the image of the athlete, the image of the athlete's shirt, and so on.

[0127] Detecting the first main region of a black and white video frame can be understood as detecting the main region of an object in the black and white video frame, and detecting the second main region of the next initial black and white video frame can be understood as detecting the main region of an object in the next initial black and white video frame.

[0128] Specifically, the specific implementation of calculating the color difference between the first main region and the second main region, and determining the color difference as the degree of difference between the detected black and white video frame and the next initial black and white video frame of the detected black and white video frame, can be found in the embodiments of the above specification, and will not be repeated here.

[0129] This disclosure provides a video processing method that improves the spatial accuracy of object change analysis by calculating the color difference between a first main body region of a detected black-and-white video frame and a second main body region of the next initial black-and-white video frame. This improves the spatial accuracy of the degree of difference between the detected black-and-white video frame and the next initial black-and-white video frame.

[0130] In practical applications, the degree of difference between the detected black-and-white video frame and the next initial black-and-white video frame can also be determined by the differences between the boundary regions and the main body regions in the black-and-white video frame. The specific implementation method is as follows:

[0131] The step of calculating the difference between the detected black-and-white video frame and the next initial black-and-white video frame to obtain the degree of difference between the detected black-and-white video frame and the next initial black-and-white video frame includes:

[0132] Boundary detection is performed on the detected black and white video frame and the next initial black and white video frame to determine the first boundary region of the detected black and white video frame and the second boundary region of the next initial black and white video frame, and the first color difference between the first boundary region and the second boundary region is calculated.

[0133] Subject detection is performed on the detected black and white video frame and the next initial black and white video frame, to determine the first subject region of the detected black and white video frame and the second subject region of the next initial black and white video frame, and to calculate the second color difference between the first subject region and the second subject region.

[0134] Based on the first color difference and the second color difference, the degree of difference between the detected black and white video frame and the next initial black and white video frame is determined.

[0135] Specifically, the specific implementation of obtaining the first color difference and the second color difference in this embodiment can be found in the above-described embodiment, and will not be repeated here.

[0136] After obtaining the first color difference and the second color difference, the degree of difference between the detected black and white video frame and the next initial black and white video frame can be determined based on the first color difference and the second color difference. For example, the average value of the first color difference and the second color difference can be calculated, or the first color difference and the second color difference can be weighted and summed, etc. This disclosure does not limit this.

[0137] This disclosure provides a video processing method that improves the accuracy of calculating the degree of difference between a detected black-and-white video frame and the next initial black-and-white video frame by calculating a first color difference between a first boundary region of the detected black-and-white video frame and a second boundary region of the next initial black-and-white video frame, and by combining this with a second color difference between a first main body region of the detected black-and-white video frame and a second main body region of the next initial black-and-white video frame. This improves the accuracy of video segmentation.

[0138] In practical applications, after determining the target black-and-white video through the above-described embodiments, a reference black-and-white video frame is determined from the black-and-white video frame sequence of the target black-and-white video. This can be determined by randomly selecting or specifying the image quality of a particular frame or black-and-white video frame. The specific implementation method is as follows:

[0139] Determining a reference black-and-white video frame from the black-and-white video frame sequence of the target black-and-white video includes:

[0140] From the sequence of black and white video frames in the target black and white video, one black and white video frame is randomly selected as the reference black and white video frame; or...

[0141] From the sequence of black and white video frames of the target black and white video, select a black and white video frame at a preset sequence position and determine it as the reference black and white video frame; or...

[0142] Determine the image quality of the black and white video frames in the black and white video frame sequence of the target black and white video, and determine a reference black and white video frame from the black and white video frame sequence of the target black and white video based on the image quality.

[0143] The preset sequence position can be understood as the preset sequence position number of the black and white video frames, such as the second frame, the third frame, the middle frame, the first frame, the last frame, etc. The specific implementation can set the preset sequence position according to actual needs; image quality includes but is not limited to image clarity, the completeness of objects contained in the image, etc.

[0144] In specific implementation, after the target black and white video is determined as described above, a black and white video frame can be randomly selected from the black and white video frame sequence of the target black and white video as a reference black and white video frame. Alternatively, a black and white video frame at a preset sequence position can be selected from the black and white video frame sequence of the target black and white video as a reference black and white video frame. Furthermore, the image quality of the black and white video frames in the black and white video frame sequence of the target black and white video can be determined, and the black and white video frame with higher image quality can be selected as the reference black and white video frame.

[0145] This disclosure provides a video processing method in one or more embodiments, which selects a reference black-and-white video frame from a sequence of black-and-white video frames of a target black-and-white video. This reference black-and-white video frame can then be colorized and used as a reference image for subsequent colorization of the target black-and-white video frames, thereby improving the color space accuracy of subsequent colorization of the target black-and-white video frames.

[0146] Step 204: Colorize the reference black and white video frame to obtain a reference color video frame.

[0147] The reference color video frame can be understood as the color version of the reference black and white video frame.

[0148] In practice, after obtaining the reference black and white video frame, the reference black and white video frame can be colored to obtain a reference color video frame, which can then be used as a reference for colorizing the subsequent target black and white video frame.

[0149] In practical applications, because purely automatic colorization can easily lead to distorted colorization results, meaning that the colorization of the reference black and white video frame may not meet the user's actual needs, we can interact with the client to colorize the reference black and white video frame according to the user's requirements. This makes the colorized reference video frame more in line with the user's needs, and thus makes the subsequent colorization of the target black and white video more in line with the user's requirements.

[0150] Based on this, the reference black-and-white video frame can be sent to the client to adapt to user needs. The specific implementation method is as follows:

[0151] The step of colorizing the reference black-and-white video frame to obtain a reference color video frame includes:

[0152] The reference black and white video frame is sent to the client, and the color reference data for the reference black and white video frame returned by the client is received.

[0153] The reference black and white video frame is colored according to the color reference data to obtain the reference color video frame.

[0154] Here, the client can be understood as the aforementioned client 102.

[0155] In practice, the video processing server sends the reference black and white video frame to the client, and the client can display the reference black and white video frame in the user coloring interface.

[0156] The user coloring interface can be understood as the interface through which users interact with the client to color. Users can use this user coloring interface to color the reference black and white video frame in the user coloring interface.

[0157] Furthermore, in response to the user's color selection and smearing operations on the reference black and white video frame, the client obtains the coordinates of the smearing position on the reference black and white video frame and the color of the smearing position, determines the coordinates of the smearing position and the color of the smearing position as color reference data, and sends the color reference data to the video processing server.

[0158] After receiving the color reference data, the video processing server can colorize the reference black and white video frame based on the coordinates of the smeared position and the color of the smeared position to obtain the reference color video frame.

[0159] This disclosure provides a video processing method in one or more embodiments. By sending a reference black-and-white video frame to a client, and through user interaction with the client, obtaining user requirements, i.e., color reference data, the reference black-and-white video frame is colorized based on the color reference data. This results in a reference color video frame that better meets the user's requirements, thereby making subsequent colorization of the target black-and-white video more in line with the user's needs.

[0160] In practical applications, image colorization models can also be used to automatically colorize reference black and white video frames based on color reference data, thereby improving spatial accuracy and colorization efficiency. The specific implementation method is as follows:

[0161] The step of colorizing the reference black-and-white video frame according to the color reference data to obtain the reference color video frame includes:

[0162] Based on the color reference data, determine the colored video frame and the masked black and white video frame corresponding to the reference black and white video frame;

[0163] The reference black-and-white video frame, the colored video frame, and the masked black-and-white video frame are input into the image colorization model to obtain the reference color video frame, wherein the image colorization model is a machine learning model.

[0164] Among them, the coloring video frame can be understood as a video frame that includes a reference black and white video frame, coordinate information of the coloring position, and color information for the coloring position. That is, based on the reference black and white video frame, a coloring image is added for the coordinates of the coloring position.

[0165] A masked black and white video frame can be understood as a black and white video frame obtained by masking the coordinates corresponding to the smeared position based on a reference black and white video frame.

[0166] In specific implementation, after obtaining the color reference data, the reference black and white video frame can be colored according to the coordinates of the smearing position and the color of the smearing position contained in the color reference data to obtain a smeared color image. The coordinates corresponding to the smearing position in the reference black and white video frame are then masked to obtain a masked black and white video frame. Then, the reference black and white video frame, the smeared video frame, and the masked black and white video frame are input into the image coloring model to obtain the reference color video frame output by the image coloring model that corresponds to the reference black and white video frame.

[0167] This disclosure provides a video processing method in one or more embodiments, which inputs a reference black-and-white video frame, a colored video frame, and a masked black-and-white video frame into an image coloring model to obtain a reference color video frame output by the image coloring model that corresponds to the reference black-and-white video frame. This makes the color of the reference color video frame more in line with user needs, and makes the subsequent target color video frame that references the reference color video frame more in line with user needs.

[0168] In one or more embodiments of this disclosure, the image colorization model can be trained by using black and white sample images, smeared images of black and white sample images, and masked black and white images of black and white sample images as training samples, and using color sample images corresponding to the black and white sample images as training labels. The training process of the image colorization model will not be described in detail in the embodiments of this disclosure.

[0169] In one or more embodiments of this disclosure, the aforementioned color reference data may also be prompt text input by the user. That is, after the client displays a reference black and white video frame to the user, it obtains the prompt text input by the user for the reference black and white video frame and sends the prompt text to the video processing server. The video processing server uses a video processing model to colorize the reference black and white video frame based on the prompt text and the reference black and white video frame to obtain a colorized reference video frame.

[0170] Step 206: Based on the reference color video frame, the target black-and-white video frame, and the previous color video frame, use an image processing model to colorize the target black-and-white video frame to obtain the target color video frame.

[0171] Wherein, the target black-and-white video frame is any black-and-white video frame in the sequence of black-and-white video frames, the previous color video frame is the color video frame corresponding to the previous black-and-white video frame of the target black-and-white video frame, and the image processing model is a machine learning model.

[0172] Specifically, an image processing model can be understood as a model that can perform image similarity calculation and image coloring. An image processing model at least includes a similarity calculation network and a coloring network. For example, an image processing model can be understood as a deep exemplar-based video colorization model, a convolutional neural network model with reference examples and temporal sequences, and so on.

[0173] In practice, the video processing server can input a reference color video frame, a target black-and-white video frame, and the previous color video frame into the image processing model. The image processing model refers to the reference color video frame and the previous color video frame, colors the target black-and-white video frame, and outputs the target color video frame. The video processing server can then obtain the target color video frame output by the video processing model.

[0174] In practical applications, the image processing model references a previous color video frame and then colors the target black-and-white video frame as follows:

[0175] The image processing model includes a similarity calculation network and a coloring network;

[0176] The step of colorizing the target black-and-white video frame using an image processing model based on the reference color video frame, the target black-and-white video frame, and the previous color video frame to obtain the target color video frame includes:

[0177] Based on the similarity calculation network, similarity calculation is performed on the reference color video frame and the target black and white video frame to obtain the similarity between the reference color video frame and the target black and white video frame;

[0178] Based on the similarity, the target black-and-white video frame, and the previous color video frame, the colorization network is used to colorize the target black-and-white video frame to obtain the target color video frame.

[0179] Specifically, after obtaining the reference color video frame, the target black-and-white video frame, and the previous color video frame, the reference color video frame and the target black-and-white video frame can be input into the similarity calculation network in the image processing model to obtain the similarity between the reference color video frame and the target black-and-white video frame (similarity includes, but is not limited to, image content similarity, image pixel similarity, image color similarity, etc., which can be set according to actual needs in specific implementation). Then, the similarity, the target black-and-white video frame, and the previous color video frame are input into the colorization network to obtain the target color video frame output by the colorization network.

[0180] In one or more embodiments of this disclosure, a similarity calculation network is used to calculate the similarity between a reference color video frame and a target black-and-white video frame. This allows the target black-and-white video frame to reference the colors of the reference color video frame that meet the user's needs. Then, the similarity, the target black-and-white video frame, and the previous color video frame are input into a coloring network to obtain the target color video frame output by the coloring network. This allows the obtained target color video frame to be combined with the colors of the reference color video frame, making the target color video frame meet the user's needs. This does not require the user to expend a lot of manpower and resources, and by referencing the previous color video frame, the temporal sequence of the subsequently obtained target color video is guaranteed.

[0181] In practical applications, the image processing model can be deployed on a video processing server or on a third-party server, namely the model server 106 mentioned above. The specific implementation method for the application of the image processing model is as follows:

[0182] The step of colorizing the target black-and-white video frame using an image processing model based on the reference color video frame, the target black-and-white video frame, and the previous color video frame to obtain the target color video frame includes:

[0183] The image processing model is determined, and the reference color video frame, the target black-and-white video frame, and the previous color video frame are input into the image processing model. In the image processing model, the previous color video frame is colorized based on the reference color video frame and the previous color video frame to obtain the target color video frame; or...

[0184] According to the model call interface, the image processing model is called, and the reference color video frame, the target black and white video frame, and the previous color video frame are input into the image processing model. In the image processing model, the previous color video frame is colored according to the reference color video frame and the previous color video frame to obtain the target color video frame.

[0185] Specifically, when the image processing model is deployed on the video processing server, an image processing model is determined to process the target black and white video frame. The reference color video frame, the target black and white video frame, and the previous color video frame are input into the image processing model. The image processing model colorizes the target black and white video frame based on the reference color video frame and the previous color video frame, and outputs the colorized target color video frame. The video processing server can then obtain the target color video frame.

[0186] When the image processing model is deployed on a third-party server, the video processing server can call the image processing model by calling the model call interface, and input the reference color video frame, the target black and white video frame, and the previous color video frame into the image processing model. The image processing model colorizes the target black and white video based on the reference color video frame and the previous color video frame, and outputs the colorized target color video frame. The third-party server returns the target color video frame to the video processing server, and the video processing server can then obtain the target color video frame.

[0187] This disclosure provides a video processing method in one or more embodiments that can call an image processing model through a model call interface to colorize a target black and white video frame. This eliminates the need for local deployment, reducing local resource consumption and improving local data processing efficiency. Alternatively, the image processing model can be deployed locally to further improve the efficiency of colorizing the target black and white video frame.

[0188] Step 208: Generate a target color video corresponding to the target black and white video based on the target color video frame.

[0189] The initial color video can be understood as the color video that corresponds to the target black and white video after it has been colored.

[0190] Specifically, there can be one or more target black-and-white videos. When there is only one target black-and-white video, the specific implementation methods for generating the target color video corresponding to the target black-and-white video based on the target color video frame include, but are not limited to, image-to-video conversion based on a neural network model, that is, inputting the target color video frame into the image-to-video neural network model to obtain the target color video corresponding to the target color video frame output by the image-to-video neural network model, or frame-by-frame synthesis, that is, combining the target color video frames to obtain the target color video, etc.

[0191] When there are multiple target black-and-white videos, the initial color video can be determined based on the target color video frames. That is, the initial color video corresponding to each target black-and-white video is determined based on the initial color video. The specific implementation method is as follows:

[0192] The step of determining the target color video based on the initial color video includes:

[0193] Based on the target color video frames, generate initial color videos corresponding to each target black-and-white video;

[0194] The initial color videos corresponding to each target black-and-white video are stitched together to obtain the target color video.

[0195] The splicing can be performed according to the time sequence of the target black and white video or according to the naming of the target black and white video. This embodiment does not limit the splicing method.

[0196] The initial color video can be understood as the color video corresponding to the black and white video of each target after colorization.

[0197] Specifically, the initial color video corresponding to each target black-and-white video can be obtained through the above implementation method. Then, the initial color videos corresponding to each target black-and-white video can be stitched together to obtain the stitched target color video.

[0198] For example, if the initial color videos corresponding to each target black-and-white video are A, B, and C, then the stitched target color video can be understood as ABC.

[0199] This disclosure provides a video processing method in one or more embodiments. By stitching together the initial color videos corresponding to each target black-and-white video to obtain a target color video, and by segmenting the initial black-and-white videos, the black-and-white video frames in the segmented target black-and-white videos are made more similar, thereby improving the color space accuracy of the initial color videos corresponding to each target black-and-white video. This improves the video quality of the stitched target color video and enhances the user's viewing experience.

[0200] In practical applications, the specific implementation method for generating the initial color video corresponding to the target black-and-white video from the target color video frames through frame-by-frame synthesis is as follows:

[0201] The step of generating an initial color video corresponding to the target black-and-white video based on the target color video frame includes:

[0202] The target color video frames corresponding to the target black-and-white video frames are combined to generate the initial color video corresponding to the target black-and-white video.

[0203] Specifically, splicing can be performed according to the time sequence of the target color video frames or according to the naming of the target color video frames. This embodiment does not limit the splicing method.

[0204] Optionally, parameters such as the display time of the target color video frame and the conversion speed of the target color video frame can also be set. The specific settings can be made according to actual needs, and this embodiment does not limit them.

[0205] Specifically, after obtaining the target color video frames as described above, the target color video frames can be combined to obtain the stitched target color video.

[0206] For example, if the target color video frames are a, b, and c, then the stitched target color video can be understood as abc.

[0207] This disclosure provides a video processing method in one or more embodiments, which uses target color video frames with colors that better meet user needs and stronger temporal sequence to stitch together an initial color video, making the initial color video better meet user needs and improving the video coherence of the initial color video.

[0208] In practical applications, to improve the quality of the subsequently obtained target color video, the initial color video can be parsed after it is obtained to ensure its quality. The specific implementation method is as follows:

[0209] After generating the initial color video corresponding to the target black-and-white video based on the target color video frame, the method further includes:

[0210] The initial color video is analyzed. If it is determined that there are target color video frames in the initial color video that do not meet the preset video frame retention conditions, the target black and white video is segmented to obtain the segmented target black and white sub-videos.

[0211] Perform video processing on the target black-and-white sub-video to generate an initial color sub-video corresponding to the target black-and-white sub-video;

[0212] Update the initial color video corresponding to the target black-and-white video based on the initial color sub-video.

[0213] The preset video frame retention conditions can be understood as the color of the target color video frame being less than the color of the initial color video, the number of target color video frames being less than a preset number threshold, etc. This embodiment does not limit these conditions, and the preset video frame retention conditions can be set according to actual needs in practical applications.

[0214] Specifically, the specific implementation method for segmenting the target black and white video to obtain the segmented target black and white sub-videos can be found in the above-mentioned implementation method for segmenting the initial black and white video to obtain at least two segmented black and white video segments, which will not be repeated here.

[0215] In practice, the specific implementation method for processing the target black-and-white sub-video to generate the corresponding initial color sub-video is as follows:

[0216] The step of performing video processing on the target black-and-white sub-video to generate an initial color sub-video corresponding to the target black-and-white sub-video includes:

[0217] From the sequence of black and white sub-video frames of the target black and white sub-video, determine the reference black and white sub-video frame;

[0218] Colorize the reference black-and-white sub-video frame to obtain a reference color sub-video frame;

[0219] The black and white sub-video frames in the black and white sub-video frame sequence are sequentially identified as the target black and white sub-video frames;

[0220] Based on the reference color sub-video frame, the target black and white sub-video frame, and the previous color sub-video frame, the target black and white sub-video frame is colored using an image processing model to obtain the target color sub-video frame. The previous color sub-video frame is the color sub-video frame corresponding to the previous black and white sub-video frame of the target black and white sub-video frame.

[0221] Based on the target color sub-video frame, an initial color sub-video corresponding to the target black-and-white sub-video is generated.

[0222] Specifically, the detailed technical content of video processing of the target black-and-white sub-video to generate the initial color sub-video corresponding to the target black-and-white sub-video in the embodiments of this disclosure can be found in the specific implementation of video processing of the target black-and-white video in the embodiments of the above specification. The difference is that the object of video processing in the embodiments of this disclosure is the target black-and-white sub-video, thereby generating the initial color sub-video corresponding to the target black-and-white sub-video, which will not be repeated here.

[0223] It should be noted that the reference black and white sub-video frame determined from the black and white sub-video frame sequence of the target black and white sub-video can be the same as the reference black and white video frame determined from the black and white video frame sequence of the target black and white video, or the reference black and white sub-video can be re-determined from the black and white sub-video frame sequence of the target black and white sub-video. This disclosure does not limit this.

[0224] In specific implementation, after obtaining the initial color sub-video, the step of parsing the initial color sub-video can be continued until there are no target color video frames in the initial color video that do not meet the preset video frame retention conditions. For specific implementation methods, please refer to the above-described embodiment, which will not be repeated here.

[0225] This disclosure provides a video processing method in one or more embodiments, which improves the spatial accuracy of the initial color sub-video corresponding to the target black-and-white sub-video by performing video processing on the target black-and-white sub-video, thereby improving the accuracy of the initial color video subsequently updated based on the initial color sub-video.

[0226] After obtaining the initial color sub-video, the initial color video corresponding to the target black-and-white video obtained above can be updated based on the initial color sub-video. Specifically, the initial color sub-videos can be spliced ​​together to obtain the initial color video corresponding to the target black-and-white video, or the initial color video segment in the initial color video corresponding to the target black-and-white sub-video in the target black-and-white video can be replaced with the initial color video segment in the initial color video at the corresponding position of the target black-and-white sub-video, and so on.

[0227] This disclosure provides a video processing method in one or more embodiments. By parsing the initial color video after it has been generated, the method adjusts the reference frame and performs secondary colorization during the colorization process of the target black and white video in the initial color video, thereby further improving the spatial accuracy of the subsequently obtained target color video.

[0228] In practical applications, after obtaining the target color video, it can be displayed to the user for viewing, video adjustment, and other processing. The specific implementation method is as follows:

[0229] After determining the target color video, the process also includes:

[0230] The target color video is sent to the client so that the client can display the target color video through the client's video display interface.

[0231] The video display interface can be understood as an interactive interface that can display and play video content.

[0232] Specifically, after obtaining the target color video, the video processing server can send the target color video to the client. After receiving the target color video, the client displays the target color video in the client's video display interface so that the user can view the target color video and make it easier for the user to perform subsequent adjustments or save the target color video.

[0233] This disclosure provides a video processing method through one or more embodiments, which displays a target color video to a user through a client, allowing the user to view the colorized target color video more intuitively for subsequent processing, thereby improving the user experience.

[0234] This disclosure provides a video processing method that, through colorizing a reference black-and-white video frame in a target black-and-white video, obtains a more accurate reference color video frame to provide a color reference for subsequent colorizing of the target black-and-white video frame, improving the color space accuracy of the colorizing. Based on the reference color video frame, the target black-and-white video frame, and the previous color video frame, an image processing model is used to colorize the target black-and-white video frame. In addition to improving the color space accuracy of the target black-and-white video frame, the reference to the previous color video frame makes the color transition between the target black-and-white video frame and the previous color video frame more natural, improving the temporal sequence of the subsequent target color video obtained from the target black-and-white video frame. Furthermore, the image processing model is used to automatically colorize the target black-and-white video frame, improving video processing efficiency. In summary, this method improves the color realism and temporal sequence of video colorization and enhances video processing efficiency.

[0235] The following description, in conjunction with Figure 3, uses the application of the video processing method provided in this disclosure in sports video restoration as an example to further illustrate the video processing method. Figure 3 shows a flowchart of the processing procedure of a video processing method provided in an embodiment of this disclosure, specifically including the following steps.

[0236] Step 302: Image quality restoration.

[0237] Image quality restoration can be understood as restoring the image quality of the original black and white video.

[0238] Specifically, the video processing server processes the original black and white video. Due to limitations in technological development and improper storage, the original black and white video may have image quality defects such as noise, scratches, and field patterns. Based on this, the original black and white video can be restored in terms of image quality. Image quality restoration includes, but is not limited to, noise reduction, scratch removal, and field pattern removal. After image quality restoration, the initial black and white video is obtained.

[0239] Step 304: Video segmentation.

[0240] Video segmentation can be understood as segmenting an initial black and white video.

[0241] Specifically, after obtaining the initial black and white video, there are usually many transitions, camera changes, and other scene changes in the initial black and white video, which is not conducive to the subsequent extraction of keyframes as references. Or the initial black and white video may be too long, and processing a long initial black and white video is inefficient and of poor quality. Based on this, the initial black and white video can be segmented to obtain segmented video segments 1-N (the video segments are the target black and white video mentioned above), where N is a natural number greater than 1. For example, if N is 2, then video segments 1-N can be understood as video segment 1 and video segment 2. For another example, if N is 3, then video segments 1-N can be understood as video segment 1, video segment 2, and video segment 3.

[0242] In practice, all initial black and white video frames of the initial black and white video are extracted, the content difference value between each initial black and white video frame and the previous initial black and white video frame is calculated, and the initial black and white video frames are segmented based on the content difference value being greater than or equal to the preset content difference value threshold.

[0243] Step 306: Refer to the coloring process.

[0244] Reference coloring can be understood as coloring all the initial black and white video frames of the initial black and white video with reference.

[0245] The reference coloring will be explained in detail below with reference to Figure 4. Referring to Figure 4, Figure 4 shows a flowchart of the reference coloring process of a video processing method provided in an embodiment of the present disclosure, which specifically includes the following steps.

[0246] Step 402: Identify the target video segment.

[0247] The target video segment can be understood as the aforementioned target reference black and white video frame. Determining the target video segment can be understood as sequentially determining the aforementioned video segments 1-N as target video segments for subsequent reference colorization of the target video segments.

[0248] Step 404: Extract keyframes.

[0249] Extracting keyframes can be understood as extracting keyframes from the target video segment, and keyframes can be understood as the aforementioned reference black and white video frames.

[0250] Specifically, a video frame can be selected as the keyframe of the target video segment from the video frames of the target video segment (i.e., the aforementioned target black and white video frames) through random selection, specified selection, or quality screening.

[0251] Step 406: Coloring the keyframe.

[0252] Among them, keyframe coloring can be understood as coloring the aforementioned keyframes.

[0253] Specifically, the smeared image of the keyframe can be obtained first through human-computer interaction.

[0254] The above-mentioned human-computer interaction will be explained in detail below with reference to Figure 5. Referring to Figure 5, Figure 5 shows a schematic diagram of the interactive interface of a video processing method provided in an embodiment of the present disclosure.

[0255] As shown in Figure 5, the client displays the user interface to the user. The keyframes mentioned above are displayed in this user interface (as shown in Figure 5). In response to the user's triggering operation of the "Start Coloring" control, the client provides the user with color editing functions. Through the color editing functions, the user can perform smearing operations and color selection operations on the keyframe. The client generates smearing data based on the user's smearing operations and, in response to the user's triggering operation of the "Upload Image" control, sends the smearing data to the video processing server. After the data is sent, the client displays the upload status "Image uploaded successfully" to the user.

[0256] After receiving the smear data, the video processing server generates a smear image based on the smear data, and obtains the mask image of the keyframe based on the smear image.

[0257] Subsequently, the video processing server inputs the keyframes, smeared images, and mask images into the pre-trained interactive intelligent image coloring model (i.e., the image coloring model mentioned above) to obtain the reference frame (i.e., the reference color video frame mentioned above) output by the interactive intelligent image coloring model.

[0258] Step 408: Apply color as needed.

[0259] Among them, reference-based coloring can be understood as determining the video frames of the target video segment as the current frame in sequence, and then performing coloring processing on the current frame in sequence based on the reference frame and the coloring frame corresponding to the previous frame of the current frame (i.e., the color video frame corresponding to the previous black and white video frame of the target black and white video frame).

[0260] In practice, the current frame, the aforementioned reference frame, and the colorized frame corresponding to the previous frame of the current frame are input into the video reference colorization model (i.e., the aforementioned image processing model) to obtain the colorized frame (i.e., the target color video frame) output by the video reference colorization model.

[0261] Step 410: Frame-by-frame synthesis.

[0262] Frame-by-frame synthesis can be understood as synthesizing the colorized frames corresponding to the video frames of the target video segment using frame-by-frame synthesis technology.

[0263] Specifically, after obtaining the colorized frames corresponding to the video frames of the target video segment, the colorized frames corresponding to the video frames of the target video segment are synthesized in sequence to obtain the colorized target video segment (i.e., the target color video).

[0264] Furthermore, after obtaining the target video segment for colorization, the color video frames contained in the target video segment can be analyzed. If it is determined that there are color video frames in the target video segment that do not meet the preset video frame retention conditions, the target black and white video is segmented to obtain sub-black and white videos of the target black and white video. According to the steps 304-306 above, the sub-black and white videos are processed to obtain the sub-colorization videos corresponding to the sub-black and white videos. Thus, the target video segment for colorization is updated according to the sub-colorization videos.

[0265] It should be noted that the preset video frame retention condition can be understood as the number of color video frames being less than the preset number of video frames (i.e., the target black and white video is considered to be longer), or the color of the color video frame meeting the expected color requirements, or the color of the color video frame being less than the preset color difference threshold compared to other color video frames in the target video segment, etc. The embodiments of this disclosure do not limit the preset video frame retention condition, and the preset video frame retention condition can be set according to actual needs in practical applications.

[0266] By following the steps above, we can obtain the target video segments corresponding to each video segment of the initial black and white video. Furthermore, we can obtain the color video corresponding to the initial black and white video by merging the video segments.

[0267] Step 308: Merge video clips.

[0268] In this context, video clip merging can be understood as merging the various colored target video clips.

[0269] Specifically, the color target video segments corresponding to each video segment of the initial black and white video obtained above are spliced ​​and processed in sequence to obtain the color video corresponding to the initial black and white video, thus completing the video processing of the initial black and white video.

[0270] This disclosure provides a video processing method that, based on keyframes in a target black-and-white video, colorizes the video, making it more aligned with user needs, improving its spatial accuracy, and eliminating the need for manual guidance for each black-and-white video frame, thus reducing manpower consumption and improving video processing efficiency. Furthermore, during colorization, the colorization frame of the previous black-and-white video frame is referenced, improving the temporal sequence of the target black-and-white video, thereby enhancing both its spatial accuracy and temporal sequence. In addition, adjustments to reference frames and secondary colorization during the colorization process further enhance the spatial accuracy of the target black-and-white video.

[0271] The above is an illustrative scheme of a video processing method according to this embodiment. It should be noted that the technical solution of this video processing method belongs to the same concept as the technical solution of the video processing method described above. For details not described in detail in the technical solution of this video processing method, please refer to the description of the technical solution of the video processing method described above.

[0272] Referring to Figure 6, Figure 6 shows a flowchart of a sports video processing method according to an embodiment of the present disclosure, which specifically includes the following steps.

[0273] Step 602: Determine the target sports black and white video, and determine the reference sports black and white video frame from the sports black and white video frame sequence of the target sports black and white video.

[0274] Step 604: Colorize the reference sports black and white video frame to obtain a reference sports color video frame.

[0275] Step 606: Based on the reference sports color video frame, the target sports black and white video frame, and the previous sports color video frame, use an image processing model to colorize the target sports black and white video frame to obtain the target sports color video frame.

[0276] Wherein, the target black-and-white sports video frame is any one of the black-and-white sports video frames in the sequence of black-and-white sports video frames, the previous color sports video frame is the color sports video frame corresponding to the previous black-and-white sports video frame of the target black-and-white sports video frame, and the image processing model is a machine learning model.

[0277] Step 608: Generate a target sports color video corresponding to the target sports black and white video based on the target sports color video frame.

[0278] Specifically, the specific implementation of the embodiments of this disclosure can be found in the embodiments of the above specification. The difference is that the object of video processing in the embodiments of this disclosure is a target sports black and white video, and the corresponding video processing result of the target sports black and white video is obtained accordingly.

[0279] This disclosure provides a sports video processing method in one embodiment. By colorizing a reference black-and-white sports video frame within a target black-and-white sports video, a more accurate reference color sports video frame is obtained. This reference frame provides color reference for subsequent colorization of the target black-and-white sports video frame, improving the color space accuracy of the colorization. Based on the reference color sports video frame, the target black-and-white sports video frame, and the previous color sports video frame, an image processing model is used to colorize the target black-and-white sports video frame. In addition to improving the color space accuracy of the target black-and-white sports video frame, the reference to the previous color sports video frame makes the color transition between the target black-and-white sports video frame and the previous color sports video frame more natural, improving the temporal sequence of the subsequent target color sports video obtained from the target black-and-white sports video frame. Furthermore, the image processing model is used to automatically colorize the target black-and-white sports video frame, improving the efficiency of sports video processing. In summary, this method improves the color realism and temporal sequence of sports video colorization and enhances the efficiency of sports video processing.

[0280] The above is an illustrative scheme of a sports video processing method according to this embodiment. It should be noted that the technical solution of this sports video processing method belongs to the same concept as the technical solution of the video processing method described above. For details not described in detail in the technical solution of the sports video processing method, please refer to the description of the technical solution of the video processing method described above.

[0281] Corresponding to the above method embodiments, this disclosure also provides a video processing apparatus embodiment. FIG7 shows a schematic structural diagram of a video processing apparatus provided in one embodiment of this disclosure. As shown in FIG7, the apparatus includes:

[0282] The first frame determination module 702 is configured to determine the target black and white video and determine the reference black and white video frame from the black and white video frame sequence of the target black and white video.

[0283] The first frame colorization module 704 is configured to colorize the reference black and white video frame to obtain a reference color video frame.

[0284] The second frame colorization module 706 is configured to colorize the target black and white video frame using an image processing model based on the reference color video frame, the target black and white video frame, and the previous color video frame to obtain a target color video frame. The previous color video frame is the color video frame corresponding to the previous black and white video frame of the target black and white video frame, and the image processing model is a machine learning model.

[0285] The color video determination module 708 is configured to generate an initial color video corresponding to the target black and white video based on the target color video frame, and to determine the target color video based on the initial color video.

[0286] Optionally, the device further includes a video segmentation module configured to:

[0287] Determine an initial black and white video, and segment the initial black and white video to obtain at least two segmented black and white video segments;

[0288] Each segment of the at least two segmented black-and-white videos is sequentially identified as the target black-and-white video;

[0289] The color video determination module 708 is further configured to:

[0290] Based on the target color video frames, generate initial color videos corresponding to each target black-and-white video;

[0291] The initial color videos corresponding to each target black-and-white video are stitched together to obtain the target color video.

[0292] Optionally, the video segmentation module is further configured to:

[0293] Determine the initial black and white video frame sequence of the initial black and white video, and sequentially determine the initial black and white video frames in the initial black and white video frame sequence as detection black and white video frames;

[0294] The difference between the detected black-and-white video frame and the next initial black-and-white video frame is calculated to obtain the degree of difference between the detected black-and-white video frame and the next initial black-and-white video frame.

[0295] If the degree of difference is determined to be greater than or equal to a preset difference threshold, the initial black and white video is segmented according to the detected black and white video frames to obtain at least two segmented black and white video segments.

[0296] Optionally, the video segmentation module is further configured to:

[0297] Boundary detection is performed on the detected black-and-white video frame and the next initial black-and-white video frame to determine the first boundary region of the detected black-and-white video frame and the second boundary region of the next initial black-and-white video frame.

[0298] Calculate the color difference between the first boundary region and the second boundary region, and determine the color difference as the degree of difference between the detected black and white video frame and the next initial black and white video frame of the detected black and white video frame.

[0299] Optionally, the video segmentation module is further configured to:

[0300] Subject detection is performed on the detected black and white video frame and the next initial black and white video frame to determine the first subject region of the detected black and white video frame and the second subject region of the next initial black and white video frame.

[0301] Calculate the color difference between the first main area and the second main area, and determine the color difference as the degree of difference between the detected black and white video frame and the next initial black and white video frame of the detected black and white video frame.

[0302] Optionally, the video segmentation module is further configured to:

[0303] Boundary detection is performed on the detected black and white video frame and the next initial black and white video frame to determine the first boundary region of the detected black and white video frame and the second boundary region of the next initial black and white video frame, and the first color difference between the first boundary region and the second boundary region is calculated.

[0304] Subject detection is performed on the detected black and white video frame and the next initial black and white video frame, to determine the first subject region of the detected black and white video frame and the second subject region of the next initial black and white video frame, and to calculate the second color difference between the first subject region and the second subject region.

[0305] Based on the first color difference and the second color difference, the degree of difference between the detected black and white video frame and the next initial black and white video frame is determined.

[0306] Optionally, the video segmentation module is further configured to:

[0307] The original black and white video is identified, and its image quality is restored to obtain the initial black and white video.

[0308] Optionally, the first frame determination module 702 is further configured to:

[0309] From the sequence of black and white video frames in the target black and white video, one black and white video frame is randomly selected as the reference black and white video frame; or...

[0310] From the sequence of black and white video frames of the target black and white video, select a black and white video frame at a preset sequence position and determine it as the reference black and white video frame; or...

[0311] Determine the image quality of the black and white video frames in the black and white video frame sequence of the target black and white video, and determine a reference black and white video frame from the black and white video frame sequence of the target black and white video based on the image quality.

[0312] Optionally, the first frame coloring module 704 is also configured as follows:

[0313] The reference black and white video frame is sent to the client, and the color reference data for the reference black and white video frame returned by the client is received.

[0314] The reference black and white video frame is colored according to the color reference data to obtain the reference color video frame.

[0315] Optionally, the first frame coloring module 704 is also configured as follows:

[0316] Based on the color reference data, determine the colored video frame and the masked black and white video frame corresponding to the reference black and white video frame;

[0317] The reference black-and-white video frame, the colored video frame, and the masked black-and-white video frame are input into the image colorization model to obtain the reference color video frame, wherein the image colorization model is a machine learning model.

[0318] Optionally, the image processing model includes a similarity calculation network and a coloring network;

[0319] The second frame coloring module 706 is also configured to:

[0320] Based on the similarity calculation network, similarity calculation is performed on the reference color video frame and the target black and white video frame to obtain the similarity between the reference color video frame and the target black and white video frame;

[0321] Based on the similarity, the target black-and-white video frame, and the previous color video frame, the colorization network is used to colorize the target black-and-white video frame to obtain the target color video frame.

[0322] Optionally, the color video determination module 708 is further configured to:

[0323] The target color video frames corresponding to the target black-and-white video frames are combined to generate the initial color video corresponding to the target black-and-white video.

[0324] Optionally, the apparatus further includes an initial color video processing module, configured to:

[0325] The initial color video is analyzed. If it is determined that there are target color video frames in the initial color video that do not meet the preset video frame retention conditions, the target black and white video is segmented to obtain the segmented target black and white sub-videos.

[0326] Perform video processing on the target black-and-white sub-video to generate an initial color sub-video corresponding to the target black-and-white sub-video;

[0327] Update the initial color video corresponding to the target black-and-white video based on the initial color sub-video.

[0328] Optionally, the initial color video processing module is further configured to:

[0329] From the sequence of black and white sub-video frames of the target black and white sub-video, determine the reference black and white sub-video frame;

[0330] Colorize the reference black-and-white sub-video frame to obtain a reference color sub-video frame;

[0331] The black and white sub-video frames in the black and white sub-video frame sequence are sequentially identified as the target black and white sub-video frames;

[0332] Based on the reference color sub-video frame, the target black and white sub-video frame, and the previous color sub-video frame, the target black and white sub-video frame is colored using an image processing model to obtain the target color sub-video frame. The previous color sub-video frame is the color sub-video frame corresponding to the previous black and white sub-video frame of the target black and white sub-video frame.

[0333] Based on the target color sub-video frame, an initial color sub-video corresponding to the target black-and-white sub-video is generated.

[0334] Optionally, the color video determination module 708 is further configured to:

[0335] The image processing model is determined, and the reference color video frame, the target black-and-white video frame, and the previous color video frame are input into the image processing model. In the image processing model, the previous color video frame is colorized based on the reference color video frame and the previous color video frame to obtain the target color video frame; or...

[0336] According to the model call interface, the image processing model is called, and the reference color video frame, the target black and white video frame, and the previous color video frame are input into the image processing model. In the image processing model, the previous color video frame is colored according to the reference color video frame and the previous color video frame to obtain the target color video frame.

[0337] Optionally, the target black-and-white video is a sports-themed target black-and-white video.

[0338] Optionally, the first frame determination module 702 is further configured to:

[0339] In response to a selection instruction sent by the client for the initial black-and-white video, the initial black-and-white video is determined based on the identification information of the initial black-and-white video carried in the selection instruction, wherein the selection instruction is generated by the client in response to a selection operation triggered by the user on the client's user interface; or,

[0340] The client receives the initial black and white video sent by the client, wherein the initial black and white video is sent by the client in response to an interactive operation triggered by the user on the client's user interface.

[0341] Optionally, the device further includes a target color video transmission module, configured to:

[0342] The target color video is sent to the client so that the client can display the target color video through the client's video display interface.

[0343] This disclosure provides a video processing apparatus in one embodiment. The apparatus obtains a more accurate reference color video frame by colorizing a reference black-and-white video frame within a target black-and-white video. This reference color frame serves as a color reference for subsequent colorization of the target black-and-white video frame, improving the color space accuracy of the colorization. Furthermore, based on the reference color video frame, the target black-and-white video frame, and the previous color video frame, an image processing model is used to colorize the target black-and-white video frame. In addition to improving the color space accuracy of the target black-and-white video frame, the reference to the previous color video frame makes the color transition between the target black-and-white video frame and the previous color video frame more natural, improving the temporal sequence of the subsequent target color video obtained from the target black-and-white video frame. Moreover, the image processing model enables automatic colorization of the target black-and-white video frame, improving video processing efficiency. In summary, this method improves the color realism and temporal sequence of video colorization and enhances video processing efficiency.

[0344] The above is an illustrative scheme of a video processing apparatus according to this embodiment. It should be noted that the technical solution of this video processing apparatus and the technical solution of the video processing method described above belong to the same concept. For details not described in detail in the technical solution of the video processing apparatus, please refer to the description of the technical solution of the video processing method described above.

[0345] Corresponding to the above method embodiments, this disclosure also provides an embodiment of a sports video processing apparatus. Figure 8 shows a schematic diagram of the structure of a sports video processing apparatus provided in one embodiment of this disclosure. As shown in Figure 8, the apparatus includes:

[0346] The first sports frame determination module 802 is configured to determine a target sports black and white video and determine a reference sports black and white video frame from the sports black and white video frame sequence of the target sports black and white video;

[0347] The first sports frame colorization module 804 is configured to colorize the reference sports black and white video frame to obtain a reference sports color video frame.

[0348] The second sports frame colorization module 806 is configured to colorize the target sports black and white video frame using an image processing model based on the reference sports color video frame, the target sports black and white video frame, and the previous sports color video frame, to obtain the target sports color video frame. The target sports black and white video frame is any sports black and white video frame in the sports black and white video frame sequence, and the previous sports color video frame is the sports color video frame corresponding to the previous sports black and white video frame of the target sports black and white video frame. The image processing model is a machine learning model.

[0349] The sports color video determination module 808 is configured to generate a target sports color video corresponding to the target sports black and white video based on the target sports color video frame.

[0350] This disclosure provides a sports video processing apparatus in one embodiment. The apparatus obtains a more accurate reference sports color video frame by colorizing a reference black-and-white video frame within a target black-and-white sports video. This reference provides a color reference for subsequent colorization of the target black-and-white sports video frame, improving the color space accuracy of the colorization. Based on the reference color video frame, the target black-and-white sports video frame, and the previous color video frame, an image processing model is used to colorize the target black-and-white sports video frame. In addition to improving the color space accuracy of the target black-and-white sports video frame, the reference to the previous color video frame makes the color transition between the target black-and-white sports video frame and the previous color video frame more natural, improving the temporal sequence of the subsequent target sports color video obtained from the target black-and-white sports video frame. Furthermore, the image processing model is used to automatically colorize the target black-and-white sports video frame, improving the efficiency of sports video processing. In summary, this method improves the color realism and temporal sequence of sports video colorization and enhances the efficiency of sports video processing.

[0351] The above is an illustrative scheme of a sports video processing device according to this embodiment. It should be noted that the technical solution of this sports video processing device and the technical solution of the sports video processing method described above belong to the same concept. For details not described in detail in the technical solution of the sports video processing device, please refer to the description of the technical solution of the sports video processing method described above.

[0352] Figure 9 shows a structural block diagram of a computing device 900 according to an embodiment of the present disclosure. The components of the computing device 900 include, but are not limited to, a memory 910 and a processor 920. The processor 920 is connected to the memory 910 via a bus 930, and a database 950 is used to store data.

[0353] The computing device 900 also includes an access device 940, which enables the computing device 900 to communicate via one or more networks 960. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 940 may include one or more of any type of wired or wireless network interface (e.g., a network interface controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0354] In one embodiment of this disclosure, the aforementioned components of the computing device 900, as well as other components not shown in FIG. 9, may also be connected to each other, for example, via a bus. It should be understood that the computing device structural block diagram shown in FIG. 9 is merely for illustrative purposes and is not intended to limit the scope of this disclosure. Those skilled in the art can add or replace other components as needed.

[0355] The computing device 900 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 900 can also be a mobile or stationary server.

[0356] The memory 910 is used to store computer programs / instructions, and the processor 920 is used to execute the computer programs / instructions stored in the memory 910. When the computer programs / instructions are executed by the processor, they implement the steps of the above-mentioned video processing method or sports video processing method.

[0357] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device belongs to the same concept as the technical solution of the video processing method or sports video processing method described above. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the video processing method or sports video processing method described above.

[0358] An embodiment of this disclosure also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the video processing method or sports video processing method described above.

[0359] The above is an illustrative embodiment of a computer-readable storage medium. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solutions of the aforementioned video processing method or sports video processing method. Details not described in detail in the technical solution of the storage medium can be found in the descriptions of the technical solutions of the aforementioned video processing method or sports video processing method.

[0360] An embodiment of this disclosure also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described video processing method or sports video processing method.

[0361] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the technical solution of the video processing method or sports video processing method described above. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the video processing method or sports video processing method described above.

[0362] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0363] The computer program / instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0364] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this disclosure are not limited to the described order of actions, because according to the embodiments of this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this disclosure.

[0365] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0366] The preferred embodiments disclosed above are merely illustrative of this disclosure. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments of this disclosure. These embodiments are selected and specifically described in this disclosure to better explain the principles and practical applications of the embodiments of this disclosure, thereby enabling those skilled in the art to better understand and utilize this disclosure. This disclosure is limited only by the claims and their full scope and equivalents.

Claims

1. A video processing method, comprising: determining a reference black-and-white video frame from a sequence of black-and-white video frames of a target black-and-white video; colorizing the reference black-and-white video frame to obtain a reference color video frame; colorizing a target black-and-white video frame according to the reference color video frame, the target black-and-white video frame, and a previous color video frame by using an image processing model, wherein the target black-and-white video frame is any one of the sequence of black-and-white video frames, the previous color video frame is a color video frame corresponding to a previous black-and-white video frame of the target black-and-white video frame, and the image processing model is a machine learning model; generating a target color video corresponding to the target black-and-white video according to the target color video frame.

2. The video processing method of claim 1, further comprising: determining an initial black-and-white video, and performing video segmentation on the initial black-and-white video to obtain at least two segmented black-and-white videos after segmentation; determining each of the at least two segmented black-and-white videos as a target black-and-white video in sequence; the determining of the target color video according to the initial color video comprises: generating an initial color video corresponding to each of the target black-and-white videos according to the target color video frame; and splicing the initial color videos corresponding to the target black-and-white videos to obtain the target color video.

3. The video processing method of claim 2, wherein the performing of the video segmentation on the initial black-and-white video to obtain the at least two segmented black-and-white videos after segmentation comprises: performing difference calculation on a detection black-and-white video frame and a next initial black-and-white video frame of the detection black-and-white video frame to obtain a difference degree between the detection black-and-white video frame and the next initial black-and-white video frame of the detection black-and-white video frame, wherein the detection black-and-white video frame is any one of a sequence of initial black-and-white video frames of the initial black-and-white video; in a case where it is determined that the difference degree is greater than or equal to a preset difference threshold, performing video segmentation on the initial black-and-white video according to the detection black-and-white video frame to obtain the at least two segmented black-and-white videos after segmentation.

4. The video processing method of claim 3, wherein the performing of the difference calculation on the detection black-and-white video frame and the next initial black-and-white video frame of the detection black-and-white video frame to obtain the difference degree between the detection black-and-white video frame and the next initial black-and-white video frame of the detection black-and-white video frame comprises: performing boundary detection on the detection black-and-white video frame and the next initial black-and-white video frame of the detection black-and-white video frame to determine a first boundary region of the detection black-and-white video frame and a second boundary region of the next initial black-and-white video frame of the detection black-and-white video frame; calculating a color difference value of the first boundary region and the second boundary region, and determining the color difference value as the difference degree between the detection black-and-white video frame and the next initial black-and-white video frame of the detection black-and-white video frame.

5. The video processing method of claim 3, wherein the calculating the difference between the detected black-and-white video frame and the next initial black-and-white video frame of the detected black-and-white video frame comprises: performing subject detection on the detected black-and-white video frame and the next initial black-and-white video frame of the detected black-and-white video frame to determine a first subject region of the detected black-and-white video frame and a second subject region of the next initial black-and-white video frame of the detected black-and-white video frame; and calculating a color difference value of the first subject region and the second subject region as the difference between the detected black-and-white video frame and the next initial black-and-white video frame of the detected black-and-white video frame.

6. The video processing method of claim 3, wherein the calculating the difference between the detected black-and-white video frame and the next initial black-and-white video frame of the detected black-and-white video frame comprises: performing boundary detection on the detected black-and-white video frame and the next initial black-and-white video frame of the detected black-and-white video frame to determine a first boundary region of the detected black-and-white video frame and a second boundary region of the next initial black-and-white video frame of the detected black-and-white video frame, and calculating a first color difference value of the first boundary region and the second boundary region; performing subject detection on the detected black-and-white video frame and the next initial black-and-white video frame of the detected black-and-white video frame to determine a first subject region of the detected black-and-white video frame and a second subject region of the next initial black-and-white video frame of the detected black-and-white video frame, and calculating a second color difference value of the first subject region and the second subject region; and determining the difference between the detected black-and-white video frame and the next initial black-and-white video frame of the detected black-and-white video frame according to the first color difference value and the second color difference value.

7. The video processing method of claim 2, wherein the determining the initial black-and-white video comprises: determining an original black-and-white video, and performing quality restoration on the original black-and-white video to obtain the initial black-and-white video.

8. The video processing method of any of claims 1-7, wherein the determining the reference black-and-white video frame from the sequence of black-and-white video frames of the target black-and-white video comprises: randomly selecting a black-and-white video frame from the sequence of black-and-white video frames of the target black-and-white video as the reference black-and-white video frame; or selecting a black-and-white video frame at a preset sequence position from the sequence of black-and-white video frames of the target black-and-white video as the reference black-and-white video frame; or determining an image quality of the black-and-white video frames in the sequence of black-and-white video frames of the target black-and-white video, and determining the reference black-and-white video frame from the sequence of black-and-white video frames of the target black-and-white video according to the image quality.

9. The video processing method of claim 1, wherein the coloring the reference black-and-white video frame to obtain a reference color video frame comprises: ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ sending the reference black-and-white video frame to a client and receiving color reference data returned by the client for the reference black-and-white video frame; coloring the reference black-and-white video frame according to the color reference data to obtain the reference color video frame.

10. The video processing method of claim 9, wherein coloring the reference black-and-white video frame according to the color reference data to obtain the reference color video frame comprises: determining a colored video frame and a mask black-and-white video frame corresponding to the reference black-and-white video frame according to the color reference data; and inputting the reference black-and-white video frame, the colored video frame, and the mask black-and-white video frame into an image coloring model to obtain the reference color video frame, wherein the image coloring model is a machine learning model.

11. The video processing method of claim 1, wherein the image processing model comprises a similarity calculation network and a coloring network, and wherein coloring the target black-and-white video frame according to the reference color video frame, the target black-and-white video frame, and a previous color video frame using the image processing model to obtain a target color video frame comprises: performing similarity calculation on the reference color video frame and the target black-and-white video frame according to the similarity calculation network to obtain a similarity between the reference color video frame and the target black-and-white video frame; and coloring the target black-and-white video frame according to the similarity, the target black-and-white video frame, and the previous color video frame using the coloring network to obtain the target color video frame.

12. The video processing method of claim 2, further comprising, after generating the initial color video corresponding to the target black-and-white video according to the target color video frame: analyzing the initial color video, and in a case where it is determined that there is a target color video frame in the initial color video that does not meet a preset video frame retention condition, splitting the target black-and-white video to obtain a split target black-and-white sub-video; performing video processing on the target black-and-white sub-video to generate an initial color sub-video corresponding to the target black-and-white sub-video; and updating the initial color video corresponding to the target black-and-white video according to the initial color sub-video.

13. A sports video processing method, comprising: determining a target sports black-and-white video and determining a reference sports black-and-white video frame from a sequence of sports black-and-white video frames of the target sports black-and-white video; coloring the reference sports black-and-white video frame to obtain a reference sports color video frame; coloring a target sports black-and-white video frame according to the reference sports color video frame, the target sports black-and-white video frame, and a previous sports color video frame using an image processing model to obtain a target sports color video frame, wherein the target sports black-and-white video frame is any one of the sequence of sports black-and-white video frames, the previous sports color video frame is a sports color video frame corresponding to a previous sports black-and-white video frame of the target sports black-and-white video frame, and the image processing model is a machine learning model. ​ ​ ​ According to the target sports color video frame, a target sports color video corresponding to the target sports black-and-white video is generated.

14. A video processing apparatus, comprising: a first frame determination module configured to determine a target black-and-white video, and determine a reference black-and-white video frame from a sequence of black-and-white video frames of the target black-and-white video; a first frame coloring module configured to color the reference black-and-white video frame to obtain a reference color video frame; a second frame coloring module configured to color a target black-and-white video frame according to the reference color video frame, the target black-and-white video frame, and a previous color video frame by using an image processing model, to obtain a target color video frame, wherein the target black-and-white video frame is any one of the sequence of black-and-white video frames, the previous color video frame is a color video frame corresponding to a previous black-and-white video frame of the target black-and-white video frame, and the image processing model is a machine learning model; a color video determination module configured to generate a target color video corresponding to the target black-and-white video according to the target color video frame.

15. A sports video processing apparatus, comprising: a first sports frame determination module configured to determine a target sports black-and-white video, and determine a reference sports black-and-white video frame from a sequence of sports black-and-white video frames of the target sports black-and-white video; a first sports frame coloring module configured to color the reference sports black-and-white video frame to obtain a reference sports color video frame; a second sports frame coloring module configured to color a target sports black-and-white video frame according to the reference sports color video frame, the target sports black-and-white video frame, and a previous sports color video frame by using an image processing model, to obtain a target sports color video frame, wherein the target sports black-and-white video frame is any one of the sequence of sports black-and-white video frames, the previous sports color video frame is a sports color video frame corresponding to a previous sports black-and-white video frame of the target sports black-and-white video frame, and the image processing model is a machine learning model; a sports color video determination module configured to generate a target sports color video corresponding to the target sports black-and-white video according to the target sports color video frame.

16. A computing device, comprising: a memory and a processor; the memory is configured to store computer programs / instructions, and the processor is configured to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method of any one of claims 1 to 13.

17. A computer-readable storage medium storing computer programs / instructions, which, when executed by a processor, implement the steps of the method of any one of claims 1 to 13.

18. A computer program product comprising computer programs / instructions, which, when executed by a processor, implement the steps of the method of any one of claims 1 to 13.

Images