Video detection and processing method and device, electronic equipment, medium and program product

Abstract

The invention provides a video detection and processing method and device, electronic equipment, a medium and a program product, and relates to the technical field of video processing, in particular to the fields of target detection, video synthesis, generative artificial intelligence and the like. According to the implementation scheme, a current to-be-detected target video frame in video data is acquired; determining whether positioning information associated with the face of the target object is detected in the target video frame; in response to determining that the positioning information is not detected in the target video frame, the positioning information is predicted based on the current state of a pre-constructed motion prediction model to obtain predicted positioning information, the motion prediction model is used for predicting the motion of the face of the target object, and the predicted positioning information is obtained; the current state of the motion prediction model is determined based on face positioning information detected in a previous video frame before the target video frame; and based on the predicted positioning information, determining a face region corresponding to the face of the target object in the target video frame.

Description

Technical Field

[0001] The present disclosure relates to the field of video processing technology, particularly to the fields of object detection, video synthesis, generative artificial intelligence, and more particularly to a video detection and processing method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Art

[0002] Video processing technology, a branch of digital image processing, is widely used in diverse fields such as security, entertainment, and education. With the advancement of computing power and the development of algorithms, video processing technology has significantly improved in terms of real-time performance and accuracy. Object detection is a key technology in video processing, aiming to locate and identify desired objects within a video frame, such as facial recognition and location. With the rapid development of artificial intelligence (AI), particularly generative AI, its application scenarios are expanding, raising new expectations and requirements for the use of technologies such as video processing, object detection, and video synthesis in these emerging scenarios.

[0003] The approaches described in this section are not necessarily approaches that have been previously conceived or employed. Unless otherwise indicated, it should not be assumed that any approach described in this section is prior art simply by virtue of its inclusion in this section. Similarly, unless otherwise indicated, the issues raised in this section should not be considered as having been recognized in any prior art. Summary of the Invention

[0004] The present disclosure provides a video detection and processing method, apparatus, electronic device, computer-readable storage medium, and computer program product.

[0005] According to one aspect of the present disclosure, a video detection method is provided, comprising: obtaining a target video frame to be detected in video data; determining whether positioning information associated with a face of a target object is detected in the target video frame; in response to determining that the positioning information is not detected in the target video frame, predicting the positioning information based on a current state of a pre-constructed motion prediction model to obtain predicted positioning information, wherein the motion prediction model is used to predict the movement of the face of the target object, and the current state of the motion prediction model is determined based on the positioning information of the face detected in a previous video frame before the target video frame; and determining a facial area corresponding to the face of the target object in the target video frame based on the predicted positioning information.

[0006] According to one aspect of the present disclosure, a video processing method is provided, comprising: obtaining video data to be processed, the video data comprising a plurality of video frames; determining a facial area corresponding to the face of a target object in at least one designated video frame among the plurality of video frames, wherein, in response to no facial area being detected in any designated video frame, determining the facial area according to the video detection method as described above; and modifying an image of the facial area in each designated video frame to a preset image to obtain updated video data.

[0007] According to one aspect of the present disclosure, a video detection device is provided, comprising: a video frame acquisition module configured to acquire a target video frame currently to be detected in video data; a face detection module configured to determine whether positioning information associated with a face of a target object is detected in the target video frame; a face prediction module configured to, in response to determining that no positioning information is detected in the target video frame, predict the positioning information based on a current state of a pre-constructed motion prediction model to obtain predicted positioning information, wherein the motion prediction model is used to predict the movement of the face of the target object, and the current state of the motion prediction model is determined based on the positioning information of a face detected in a previous video frame before the target video frame; and a face positioning module configured to determine a facial area corresponding to the face of the target object in the target video frame based on the predicted positioning information.

[0008] According to one aspect of the present disclosure, a video processing device is provided, comprising: a video acquisition module configured to acquire video data to be processed, the video data comprising a plurality of video frames; a video detection module configured to determine a facial area corresponding to the face of a target object in at least one designated video frame among the plurality of video frames, wherein, in response to no facial area being detected in any designated video frame, the facial area is determined according to the video detection device as described above; and a video update module configured to modify an image of the facial area in each designated video frame to a preset image to obtain updated video data.

[0009] According to another aspect of the present disclosure, an electronic device is provided, comprising at least one processor; and a memory communicatively connected to the at least one processor; the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of the present disclosure as described above.

[0010] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, where the computer instructions are used to cause a computer to execute the method described above in the present disclosure.

[0011] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, which implements the method of the present disclosure when executed by a processor.

[0012] According to one or more embodiments of the present disclosure, the robustness and accuracy of face detection of a target object in a video frame can be improved.

[0013] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings illustrate exemplary embodiments and constitute a part of the specification. Together with the description of the specification, they serve to explain exemplary implementation of the embodiments. The illustrated embodiments are for illustrative purposes only and do not limit the scope of the claims. Throughout the drawings, the same reference numerals designate similar, but not necessarily identical, elements.

[0015] Figure 1 A schematic diagram illustrating an exemplary system in which the various methods described herein may be implemented according to an embodiment of the present disclosure is shown.

[0016] Figure 2 A flowchart of a video detection method according to an embodiment of the present disclosure is shown.

[0017] Figure 3 A flowchart of a video detection method according to another embodiment of the present disclosure is shown.

[0018] Figure 4 A flowchart of a method for determining a storyboard according to an embodiment of the present disclosure is shown.

[0019] Figure 5 A schematic diagram of a backfill prediction strategy according to an embodiment of the present disclosure is shown.

[0020] Figure 6 A schematic diagram of a local search compensation strategy according to an embodiment of the present disclosure is shown.

[0021] Figure 7 A flowchart of a video processing method according to an embodiment of the present disclosure is shown.

[0022] Figure 8 The figure shows a structural block diagram of a video detection device according to an embodiment of the present disclosure.

[0023] Figure 9 A structural block diagram of a video detection device according to another embodiment of the present disclosure is shown.

[0024] Figure 10A structural block diagram of a video processing device according to an embodiment of the present disclosure is shown.

[0025] Figure 11 A structural block diagram of a video processing device according to another embodiment of the present disclosure is shown.

[0026] Figure 12 A structural block diagram of an exemplary electronic device that can be used to implement the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0027] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0028] In this disclosure, unless otherwise specified, the use of terms such as "first" and "second" to describe various elements is not intended to limit the positional relationship, temporal relationship, or importance relationship of these elements. Such terms are only used to distinguish one element from another. In some examples, the first element and the second element may refer to the same instance of the element, while in some cases, based on the context of the description, they may also refer to different instances.

[0029] The terms used in the descriptions of the various examples described in this disclosure are for the purpose of describing specific examples only and are not intended to be limiting. Unless the context clearly indicates otherwise, if the number of elements is not specifically limited, the element may be one or more. In addition, the term "and / or" used in this disclosure encompasses any one and all possible combinations of the listed items.

[0030] With the emergence of emerging video applications such as digital human live streaming and video content generation, specific facial processing of video characters may be required, such as face synthesis or replacing the current face with a specified facial style. Traditional methods are mainly based on deep learning, such as replacement methods based on generative adversarial networks (GANs). By learning the mapping relationship between the current face and the target face, they can achieve face replacement in single-frame static images or short video sequences.

[0031] However, these traditional methods are primarily suitable for situations where faces appear stable and the scene changes minimally. With the advancement of video technology, the use of long-sequence videos (hereinafter referred to as long videos) is increasing. These long videos often feature frequent camera cuts, moving characters, and dramatic zoom changes, especially in long-range shots where the facial area accounts for a small portion of the scene. These situations can make traditional methods difficult to handle, preventing them from achieving effective face detection and processing.

[0032] To this end, the embodiments of the present disclosure provide a video detection method and a video processing method that can more effectively cope with the common characteristics of long videos such as multiple shots, long views, and complex camera movements, and accurately implement facial detection and processing.

[0033] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0034] Figure 1 FIG2 is a schematic diagram of an exemplary system 100 in which the various methods and apparatuses described herein may be implemented according to an embodiment of the present disclosure. Figure 1 , the system 100 includes one or more client devices 101, 102, 103, 104, 105, and 106, a server 120, and one or more communication networks 110 coupling the one or more client devices to the server 120. The client devices 101, 102, 103, 104, 105, and 106 can be configured to execute one or more applications.

[0035] In an embodiment of the present disclosure, the server 120 may run one or more services or software applications that enable execution of the methods described in the embodiment of the present disclosure.

[0036] In some embodiments, server 120 may also provide other services or software applications, which may include non-virtualized environments and virtualized environments. In some embodiments, these services may be provided as web-based services or cloud services, such as provided to users of client devices 101, 102, 103, 104, 105, and / or 106 under a software as a service (SaaS) model.

[0037] exist Figure 1 In the configuration shown, the server 120 may include one or more components that implement the functions performed by the server 120. These components may include software components, hardware components, or a combination thereof that can be executed by one or more processors. Users operating client devices 101, 102, 103, 104, 105, and / or 106 may, in turn, utilize one or more client applications to interact with the server 120 to utilize the services provided by these components. It should be understood that a variety of different system configurations are possible, which may differ from the system 100. Therefore, Figure 1 is one example of a system for implementing the various methods described herein and is not intended to be limiting.

[0038] A user may use client devices 101, 102, 103, 104, 105, and / or 106 to provide video data to be processed or receive processed video data. The client device may provide an interface that enables a user of the client device to interact with the client device. The client device may also output information to the user via the interface. Figure 1 Only six client devices are depicted, but one skilled in the art will appreciate that the present disclosure can support any number of client devices.

[0039] Client devices 101, 102, 103, 104, 105, and / or 106 may include various types of computer devices, such as portable handheld devices, general-purpose computers (such as personal computers and laptops), workstation computers, wearable devices, smart screen devices, self-service terminal devices, service robots, gaming systems, thin clients, various messaging devices, sensors or other sensing devices, etc. These computer devices may run various types and versions of software applications and operating systems, such as Microsoft Windows, Apple iOS, UNIX-like operating systems, Linux, or Linux-like operating systems (such as Google Chrome OS); or include various mobile operating systems, such as Microsoft Windows Mobile OS, iOS, Windows Phone, and Android. Portable handheld devices may include cellular phones, smartphones, tablet computers, personal digital assistants (PDAs), etc. Wearable devices may include head-mounted displays (such as smart glasses) and other devices. Gaming systems may include various handheld gaming devices, internet-enabled gaming devices, etc. Client devices are capable of executing a variety of different applications, such as various internet-related applications, communication applications (such as email applications), and short message service (SMS) applications, and may use various communication protocols.

[0040] The network 110 may be any type of network known to those skilled in the art that can support data communications using any of a variety of available protocols, including but not limited to TCP / IP, SNA, IPX, etc. By way of example only, the one or more networks 110 may be a local area network (LAN), an Ethernet-based network, a token ring, a wide area network (WAN), the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a blockchain network, a public switched telephone network (PSTN), an infrared network, a wireless network (e.g., Bluetooth, WIFI), and / or any combination of these and / or other networks.

[0041] Server 120 may include one or more general-purpose computers, specialized server computers (e.g., PC (personal computer) servers, UNIX servers, mid-range servers), blade servers, mainframe computers, server clusters, or any other suitable arrangement and / or combination. Server 120 may include one or more virtual machines running virtual operating systems, or other computing architectures involving virtualization (e.g., one or more flexible pools of logical storage devices that may be virtualized to maintain a server's virtual storage device). In various embodiments, server 120 may run one or more services or software applications that provide the functionality described below.

[0042] The computing units in the server 120 may run one or more operating systems including any of the operating systems described above as well as any commercially available server operating systems. The server 120 may also run any of a variety of additional server applications and / or middle-tier applications, including HTTP servers, FTP servers, CGI servers, JAVA servers, database servers, and the like.

[0043] In some implementations, server 120 may include one or more applications to analyze and consolidate data feeds and / or event updates received from users of client devices 101, 102, 103, 104, 105, and / or 106. Server 120 may also include one or more applications to display the data feeds and / or real-time events via one or more display devices of client devices 101, 102, 103, 104, 105, and / or 106.

[0044] In some embodiments, server 120 may be a distributed system server or a server integrated with blockchain. Server 120 may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host equipped with artificial intelligence technology. A cloud server is a host product within the cloud computing service system that addresses the management difficulties and poor scalability of traditional physical hosts and virtual private servers (VPS) services.

[0045] The system 100 may also include one or more databases 130. In some embodiments, these databases may be used to store data and other information. For example, one or more of the databases 130 may be used to store information such as audio files and video files. The databases 130 may reside in a variety of locations. For example, the database used by the server 120 may be local to the server 120, or may be remote from the server 120 and communicate with the server 120 via a network-based or dedicated connection. The databases 130 may be of different types. In some embodiments, the databases used by the server 120 may be, for example, relational databases. One or more of these databases may store, update, and retrieve data to and from the databases in response to commands.

[0046] In some embodiments, one or more of the databases 130 may also be used by applications to store application data. The databases used by the applications may be different types of databases, such as a key-value store, an object store, or a conventional store backed by a file system.

[0047] Figure 1 The system 100 may be configured and operated in various ways to enable application of the various methods and apparatuses described in accordance with the present disclosure.

[0048] Various aspects of the video detection method and the video processing method according to the embodiments of the present disclosure will be described in detail below.

[0049] Figure 2 FIG. 2 shows a flow chart of a video detection method 200 according to an embodiment of the present disclosure.

[0050] like Figure 2 As shown, method 200 includes steps S201 , S202 , S203 and S204 .

[0051] In step S201, a target video frame to be detected in the current video data is obtained.

[0052] In this example, the video data can be a long video sequence, i.e., a long video. Long videos typically feature multiple shots, long shots, and complex camera movements. Detection can be performed sequentially on each frame in the video data, or on one or more specified frames.

[0053] In step S202 , it is determined whether positioning information associated with the face of the target object is detected in the target video frame.

[0054] In this example, the target object can be a person or digital human character in video data, so detecting the positioning information associated with its face can refer to identifying and locating its face in the target video frame. Facial detection models such as RetinaFace and YOLO-Face can be used to obtain the facial positioning information.

[0055] In an example, the positioning information may include static position information and dynamic motion information of the face, wherein the static position information may include the center position (c x ,c y ) and the size of the face (w,h), dynamic motion information can include the speed of facial movement (v x ,v y ) and facial size change rate (v w ,v h ).

[0056] In this example, if positioning information is detected, a bounding box and confidence score corresponding to the face can be obtained. The bounding box can indicate the position coordinates of the face, such as the horizontal and vertical coordinates of the upper left corner and the horizontal and vertical coordinates of the lower right corner. The confidence score can be expressed as a score to indicate the reliability of the detection result. In this example, if positioning information is not detected, it may be due to various reasons, such as a small face size, facial occlusion, or blur making detection difficult.

[0057] In step S203, in response to determining that no positioning information is detected in the target video frame, the positioning information is predicted based on the current state of a pre-built motion prediction model to obtain predicted positioning information. The motion prediction model is used to predict the motion of the face of the target object, and the current state of the motion prediction model is determined based on the positioning information of the face detected in a previous video frame before the target video frame.

[0058] In this example, the motion prediction model can be constructed based on a Kalman filter. The Kalman filter is a recursive estimation algorithm. In embodiments of the present disclosure, the characteristics of the Kalman filter and the continuity of motion can be utilized to predict the motion of the target object's face. Since facial positioning information can be one dimension of information used to characterize facial motion, facial positioning information detected in a previous video frame preceding the target video frame can be used in the prediction. Accordingly, the current state of the motion prediction model can refer to the current state of the Kalman filter-based system, reflecting the system's predictive capabilities and also referred to as the Kalman state. If no observation data is available at this time, the system makes a prediction based on the existing current state, which depends on the facial positioning information detected in the previous video frame. In other words, for the previous video frame, if facial detection is successful, the detected positioning information can be used as an observation or observation data to update the state of the motion prediction model, thereby facilitating its use in the next video frame. From this perspective, the motion prediction model of embodiments of the present disclosure can obtain timely state updates based on the observations of successful detection.

[0059] In this example, if facial positioning information is not detected in the current target video frame, but facial positioning information can be detected in the previous video frame before the target video frame and the subsequent video frame after the target video frame, linear interpolation can also be used to predict the facial positioning information in the target video frame. Therefore, if detection fails in the subsequent video frame, the method of using the motion prediction model for prediction in the embodiments of the present disclosure is particularly beneficial and can provide a sufficient compensation mechanism.

[0060] In step S204, based on the predicted positioning information, a facial region corresponding to the face of the target object in the target video frame is determined.

[0061] Therefore, the embodiments of the present disclosure provide an effective compensation mechanism that can effectively deal with the situation where face detection in a video fails, wherein a pre-built motion prediction model is used to predict the positioning information of the face based on the physical characteristics of the face's movement patterns, thereby predicting the corresponding positioning information of the face in the target video frame based on the positioning information of the face detected in the previous video frame based on the continuity of the motion. Thanks to the mechanism of the video detection method of the embodiments of the present disclosure, it can effectively deal with the common characteristics of long videos such as multiple shots, long shots, and complex camera movements. Even in cases where the face accounts for a small proportion or the face is blocked or blurred, making detection difficult, it can still effectively perform face detection, thereby achieving adaptive prediction of the face and improving the robustness of video detection.

[0062] Figure 3 FIG. 3 is a flowchart of a video detection method 300 according to another embodiment of the present disclosure.

[0063] In some embodiments, as Figure 3 As shown, in step S301, it is determined whether the positioning information associated with the face of the target object is detected in the target video frame (such as the positioning information associated with the face of the target object is detected in the target video frame). Figure 2 After the step S202), the method 300 may further include steps S302 and S303.

[0064] In step S302, in response to determining that no positioning information is detected in the target video frame, it can be determined whether motion change information of the target object's face can be estimated. The motion change information can include at least one of first information related to the speed of movement of the face or second information related to the speed of size change of the face.

[0065] In this example, the motion change information of the target object's face can be estimated by an optical flow tracking method. The optical flow tracking method can calculate the motion vector of the pixel point, i.e., the optical flow, by analyzing the brightness change of the pixel point in consecutive video frames. When the optical flow tracking method is applied to the embodiment of the present disclosure, the required motion change information can be the facial movement speed (v) directly related to the face. x ,v y ) is the first information and the facial size change speed (v w ,v h ) is at least one of the second information. Such information will be used as an observation value or observation data to update the state of the motion prediction model.

[0066] In step S303, in response to determining that the motion change information can be estimated, a prediction is performed based on the current state of the motion prediction model (e.g., in combination with Figure 2 Before the step S203), the current state of the motion prediction model is updated based on the motion change information, so that the prediction of the motion prediction model is performed based on the updated current state.

[0067] In this example, when face detection fails in the current target video frame, the optical flow tracking method can be used to track the facial key points in the previous video frame and calculate the pixel displacement to estimate the facial movement speed (v x ,v y ) and facial size change rate (v w ,v h ), so that the estimated corresponding information is input into the motion prediction model to be used as an observation quantity for state update. In other words, being able to estimate motion change information means being able to calculate at least one of the above two types of information.

[0068] In other words, if combined Figure 2The method 200 described above may involve a scenario where optical flow calculation fails, and thus, in this scenario, prediction is performed based on the existing current state of the motion prediction model without any observations to update the state. In contrast, the method 300 of this embodiment may involve a scenario where optical flow calculation succeeds, in which case additional observations can be obtained to update the state of the motion prediction model and perform prediction based on the updated state.

[0069] Therefore, the video detection method of the disclosed embodiment can dynamically switch state prediction mechanisms based on the type of observation within the frame. In the event that detection fails but optical flow calculation succeeds, the data obtained from the optical flow calculation can be further utilized to update the state of the motion prediction model and perform prediction based on the updated state, thereby further improving detection reliability.

[0070] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0071] In some embodiments, the video detection method of the present disclosure, such as combining Figure 2 The method 200 or Figure 3 The method 300 may include: after acquiring the target video frame, determining whether the target video frame belongs to a new frame. Predicting the positioning information based on the current state of the motion prediction model (e.g., combining the current state of the motion prediction model with the current state of the motion prediction model) Figure 2 The step S203) may be performed based on determining that the target video frame does not belong to a new storyboard.

[0072] In the example, shot splitting is also called shot cutting, and shot splitting may appear frequently in long videos due to the semantics of the content. When a new shot appears, the facial position of the target object in the new shot may be quite different from that in the previous shot. Therefore, if the two frames do not belong to the same shot, the prediction and update logic of the motion prediction model may also need to be changed accordingly. Therefore, the prediction step in the embodiment of the present disclosure can be performed on the premise that the target video frame and the previous video frame belong to the same shot. In this case, the position of the face in the target video frame can be recorded normally without clearing the historical frame record.

[0073] Therefore, considering the fact that split shots often appear in long videos, the video detection method of the embodiment of the present disclosure can further optimize the adaptation to long videos through split shot determination, thereby accurately making predictions.

[0074] In some embodiments, the above-mentioned step of determining whether the target video frame belongs to a new storyboard may include: determining whether the center offset rate and area change ratio of a predetermined control object of the target video frame compared with the previous video frame reach a predetermined threshold; and in response to determining that both the center offset rate and the area change ratio do not reach the predetermined threshold, determining that the target video frame does not belong to a new storyboard.

[0075] In the example, dynamic comparison between frames can be used to determine whether a new shot or a dramatic camera movement is currently being taken. To this end, a specific reference object can be selected for this dynamic comparison. In the embodiment of the present disclosure, in the case where facial detection fails, the face itself can be used as a reference object instead of the face itself. Instead, other objects that are more suitable and easier to detect can be selected, such as other parts of the target object or other objects that appear together with the target object. Such reference objects can have as simple an outline or shape as possible to facilitate comparison. In the case where facial detection is successful, the face of the target object can be directly used as a reference object.

[0076] In the example, the center offset rate of the reference object refers to the ratio of the Euclidean distance of the center point between the target video frame and the previous video frame to the diagonal length, which can be expressed as:

[0077]

[0078] Where shift_ratio represents the center shift ratio, x1, y1 and x2, y2 represent the center coordinates of the reference object in the target video frame and the previous video frame, respectively, and w, h represent the width and height of the reference object. The center shift ratio can be used to measure whether the reference object's position has shifted significantly. The threshold for the center shift ratio can be set, for example, as 0.2. When the center shift ratio reaches this threshold, for example, when it is greater than or equal to this threshold, it can be considered that the reference object's position has shifted significantly, indicating that a split shot has occurred.

[0079] In the example, the area change ratio of the reference object refers to the area change ratio between the target video frame and the previous video frame, which can be expressed as:

[0080]

[0081] Where area_ratio represents the area change ratio, and w1, h1, w2, and h2 represent the width and height of the reference object in the target video frame and the previous video frame, respectively. The area change ratio can be used to indicate a zoom or a sudden change in perspective. A threshold for the area change ratio can be set, for example, as 0.5. When the area change ratio reaches this threshold, for example, when it is greater than or equal to this threshold, it is considered that a zoom or a sudden change in perspective has occurred, indicating that a split shot has occurred.

[0082] In the example, the detection of the control object may also generally involve a confidence level, and the confidence level may be set such that the detection of the control object is successful only when a certain threshold is reached. The confidence level threshold may be selected as 0.4, for example.

[0083] Therefore, the frame determination method used in the disclosed embodiments differs from traditional methods such as color histograms or edge gradients. Instead, it uses the motion of characters as the basis for judgment, segmenting shot boundaries from a semantic perspective. This effectively improves the accuracy of frame determination in video footage primarily featuring characters, helping to ensure the accuracy of motion-based predictions.

[0084] Figure 4 A flowchart of a storyboard determination method 400 according to an embodiment of the present disclosure is shown.

[0085] like Figure 4 As shown, after obtaining the detection results for the predetermined reference object in consecutive frames, the process proceeds to step S401A, which calculates the center shift rate, and step S401B, which calculates the area change ratio. Following these calculations, step S402 determines whether the current target video frame and the previous video frame belong to the same frame by comparing the calculated results with a corresponding threshold.

[0086] If at least one of the center deviation rate and area change ratio reaches the threshold, it means that there may be shot push-pull, sudden change or drastic deviation (such as Figure 4 If the center offset rate and area change ratio do not reach the threshold, it means that no shot switching has occurred (e.g., Figure 4 "Yes" is shown), so the process can enter step S403B, at which time the current lens state can be maintained, the positioning information of the face of the target object in the current target video frame can be predicted, or the detected positioning information can be recorded for use as an observation quantity to update the state of the motion prediction model.

[0087] In some embodiments, in the case of considering the split mirror, the video detection method of the embodiment of the present disclosure, such as combining Figure 2 The method 200 or Figure 3Method 300 may include: in response to determining that the target video frame does not belong to a new frame, and that positioning information was not detected in a previous video frame belonging to the current frame, but positioning information was detected for the first time in the target video frame, predicting positioning information of a face in the previous video frame from back to front based on a current state of the motion prediction model to obtain predicted positioning information of the face in the previous video frame. In this case, the current state of the motion prediction model is determined based on the positioning information of the face detected in the target video frame.

[0088] In this example, if a face is first detected in a target video frame after several previous video frames within a given frame, this indicates a possible missed detection in the previous video frames. In this case, the prediction mechanism described above can be applied in reverse, attempting to compensate for the previous video frames, predicting facial location information from back to front.

[0089] In this way, a back-fill prediction strategy can be provided to attempt to compensate for previous video frames that may have been missed through prediction, thereby detecting each frame in the video data as accurately and without omission as possible, further improving the robustness of detection.

[0090] Figure 5 A schematic diagram of a backfill prediction strategy according to an embodiment of the present disclosure is shown.

[0091] like Figure 5 As shown, in the current frame 500, no facial positioning information is detected in the previous video frame 502 preceding the target video frame 501, while positioning information is detected for the first time in the target video frame 501. Therefore, backward prediction can be applied to predict the facial positioning information in the previous video frame 502 from backward to forward based on the current state of the motion prediction model.

[0092] For ease of explanation, this description uses a previous video frame 502 as an example. However, it will be appreciated that when there are multiple previous video frames 502, prediction can also be performed sequentially from back to front. In this case, the current state of the motion prediction model can be based on the positioning information of the previously predicted video frame.

[0093] In some embodiments, the predicted positioning information obtained through the motion prediction model may include a predicted center position and a predicted size of the face. Accordingly, determining whether the target video frame belongs to a new cut may include: after obtaining the predicted positioning information, determining whether a face is detected in a predetermined area of ​​the target video frame. The center of the predetermined area may be located at the predicted center position, and the size of the predetermined area may be in a predetermined proportional relationship to the predicted size; and in response to determining that no face is detected, determining that the target video frame belongs to a new cut.

[0094] In this example, after the face position and size in the current target video frame are predicted, adaptive local search detection can be initiated. Assume that the predicted positioning information of the face includes the center position and size, for example, represented as (x t ,y t , w t ,h t ), then you can use (x t ,y t ) as the center, crop the image with a width and height of 2×max(w t ,h t ) to determine whether a face is detected. If the detection fails, it can be gradually reduced to a width and height of max(w t ,h t ) to determine whether a face is detected. This continues until a face is detected successfully or the range is reduced to a minimum. If no face is detected, the target video frame can be determined to belong to a new shot.

[0095] Therefore, this embodiment provides a local search compensation strategy, which can not only confirm the predicted positioning information to accurately fine-tune the positioning information within a relatively small range, but also be used to verify whether a new shot has occurred in the current target video frame.

[0096] Figure 6 A schematic diagram of a local search compensation strategy according to an embodiment of the present disclosure is shown.

[0097] like Figure 6 As shown, the coordinates of the facial center position 601 predicted by the motion prediction model can be expressed as (x t ,y t ), the size 602 can be expressed as width and height w t ,h t In this case, a first verification rectangle 610 may be created with the facial center position 601 as the center. The first verification rectangle 610 may be a square, and its size 612 may be expressed as 2w. t ,2w t or 2 hours t ,2h t , Figure 6 Then, it can be confirmed whether the outline of the first verification rectangle 610 can cover the face. If so, it can be determined that the face is detected. If not, the first verification rectangle 610 can be further reduced to a second verification rectangle 620. The second verification rectangle 620 can also be a square, and its size 622 can be expressed as w t ,w t or h t ,ht , Figure 6 As mentioned above, if no face is detected in the end, it can be determined that the target video frame belongs to a new shot.

[0098] In some embodiments, the target video frame and the previous video frame may be consecutive adjacent video frames in the video data.

[0099] The video detection method of the disclosed embodiment can be analyzed frame by frame. This is more advantageous for long videos with many shot changes, as it can capture shot changes in a finer granularity, thereby further improving the robustness of video detection.

[0100] Figure 7 FIG. 7 is a flowchart of a video processing method 700 according to an embodiment of the present disclosure.

[0101] like Figure 7 As shown, method 700 includes steps S701 , S702 and S703 .

[0102] In step S701 , video data to be processed is obtained, where the video data includes a plurality of video frames.

[0103] In step S702, a facial region corresponding to the face of the target object is determined in at least one designated video frame among the multiple video frames, wherein, in response to not detecting the facial region in any designated video frame, the facial region is determined according to the video detection method as described above.

[0104] In the example, it may be specified to perform face detection on all of the multiple video frames, or to perform face detection on only a portion of the video frames.

[0105] In step S703, the image of the facial region in each designated video frame is modified to a preset image to obtain updated video data.

[0106] In the example, the updated video data can retain the resolution, frame rate, and bit rate of the original video data, and automatically adjust details such as boundary processing and brightness matching.

[0107] Therefore, thanks to the detection robustness provided by the video detection method of the embodiment of the present disclosure, the video processing method applied on this basis can better cope with the common characteristics of long videos such as multiple shots, long views and complex camera movements, which is conducive to outputting smooth, natural and structurally complete video processing results.

[0108] In some embodiments, as Figure 7Step S703 shown may include: cropping a facial region from a designated video frame; extracting a first feature of the facial region; modifying the first feature into a second feature of a preset image; and generating a modified facial region based on the second feature.

[0109] In this example, the cropped facial region can be fed into a pre-trained image synthesis model for feature replacement and fusion. The modified facial region can then be smoothly transitioned and fused back into the original video frame, achieving a natural and unobtrusive modification effect.

[0110] Therefore, by performing the modification of the facial region in the form of local cropping and feature replacement, the computational cost can be reduced, which is especially beneficial for long videos with large data processing volume.

[0111] The embodiments of the present disclosure also provide a video detection device and a video processing device.

[0112] Figure 8 FIG. 8 is a structural block diagram of a video detection device 800 according to an embodiment of the present disclosure.

[0113] like Figure 8 As shown, the apparatus 800 includes a video frame acquisition module 801 , a face detection module 802 , a face prediction module 803 and a face positioning module 804 .

[0114] The video frame acquisition module 801 is configured to acquire a target video frame to be detected in the video data.

[0115] The face detection module 802 is configured to determine whether positioning information associated with a face of a target object is detected in a target video frame.

[0116] The face prediction module 803 is configured to, in response to determining that no positioning information is detected in the target video frame, predict the positioning information based on the current state of a pre-built motion prediction model to obtain predicted positioning information. The motion prediction model is used to predict the motion of the face of the target object, and the current state of the motion prediction model is determined based on the positioning information of the face detected in a previous video frame before the target video frame.

[0117] The face positioning module 804 is configured to determine a facial region corresponding to the face of the target object in the target video frame based on the predicted positioning information.

[0118] The operations of the above-mentioned video frame acquisition module 801, face detection module 802, face prediction module 803 and face positioning module 804 can be combined with Figure 2 The operations of steps S201, S202, S203 and S204 are the same, so the details of each aspect are not repeated here.

[0119] Figure 9 FIG. 4 shows a structural block diagram of a video detection device 900 according to an embodiment of the present disclosure.

[0120] like Figure 9 As shown, the apparatus 900 may include a video frame acquisition module 901, a face detection module 902, a face prediction module 903, and a face positioning module 904. The operations of these modules may be similar to those described above. Figure 8 The operations of the video frame acquisition module 801, the face detection module 802, the face prediction module 803 and the face positioning module 804 are the same. In addition, these modules and the apparatus 900 may further include further submodules.

[0121] In some embodiments, the device 900 may include: an optical flow estimation module 905, configured to determine whether it is possible to estimate motion change information of the face of the target object in response to determining that no positioning information is detected in the target video frame, wherein the motion change information includes at least one of first information related to the movement speed of the face or second information related to the size change speed of the face; and a state update module 906, configured to update the current state of the motion prediction model based on the motion change information before making a prediction based on the current state of the motion prediction model in response to determining that the motion change information can be estimated, so that the prediction of the motion prediction model is based on the updated current state.

[0122] In some embodiments, the device 900 may include: a storyboard determination module 907, which is configured to determine whether the target video frame belongs to a new storyboard after obtaining the target video frame, wherein the operation of the face prediction module 903, that is, predicting the positioning information based on the current state of the motion prediction model, is performed based on determining that the target video frame does not belong to a new storyboard.

[0123] In some embodiments, the storyboard determination module 907 may include: a threshold comparison module 9071, configured to determine whether the center offset rate and area change ratio of a predetermined control object in the target video frame compared with the previous video frame reach a predetermined threshold; and a first determination execution module 9072, configured to determine that the target video frame does not belong to a new storyboard in response to determining that both the center offset rate and the area change ratio do not reach the predetermined threshold.

[0124] In some embodiments, the device 900 may include: a backfill prediction module 908, which is configured to, in response to determining that the target video frame does not belong to a new storyboard, and that no positioning information is detected in the previous video frame belonging to the current storyboard but the positioning information is detected for the first time in the target video frame, predict the positioning information of the face in the previous video frame from back to front based on the current state of the motion prediction model to obtain the predicted positioning information of the face in the previous video frame, wherein the current state of the motion prediction model is determined based on the positioning information of the face detected in the target video frame.

[0125] In some embodiments, the predicted positioning information may include a predicted center position of the face and a predicted size of the face. The frame determination module 907 may include a detection verification module 9073 configured to, after obtaining the predicted positioning information, determine whether a face is detected in a preset area in the target video frame, wherein the center of the preset area is located at the predicted center position and the size of the preset area has a predetermined proportional relationship to the predicted size; and a second determination execution module 9074 configured to, in response to determining that no face is detected, determine that the target video frame belongs to a new frame.

[0126] Figure 10 FIG. 1 shows a structural block diagram of a video processing apparatus 1000 according to an embodiment of the present disclosure.

[0127] like Figure 10 As shown, the apparatus 1000 includes a video acquisition module 1001 , a video detection module 1002 and a video update module 1003 .

[0128] The video acquisition module 1001 is configured to acquire video data to be processed, where the video data includes multiple video frames.

[0129] The video detection module 1002 is configured to determine a facial area corresponding to the face of the target object in at least one specified video frame among the multiple video frames, wherein, in response to not detecting the facial area in any specified video frame, the facial area is determined according to the video detection device as described above.

[0130] The video updating module 1003 is configured to modify the image of the facial region in each designated video frame into a preset image to obtain updated video data.

[0131] The operation and combination of the above video acquisition module 1001, video detection module 1002 and video update module 1003 Figure 7 The operations of steps S701, S702 and S703 are the same, so the details of each aspect are not repeated here.

[0132] Figure 11 FIG. 1 shows a structural block diagram of a video processing device 1100 according to another embodiment of the present disclosure.

[0133] like Figure 11 As shown, the video processing device 1100 may include a video acquisition module 1101, a video detection module 1102, and a video update module 1103. The operations of these modules may be similar to those in FIG. Figure 10 The operations of the video acquisition module 1001, the video detection module 1002 and the video update module 1003 are the same. In addition, these modules may also include further submodules.

[0134] In some embodiments, the video update module 1103 may include: a region extraction module 1103a, configured to crop a facial region from a specified video frame; a feature extraction module 1103b, configured to extract a first feature of the facial region; a feature modification module 1103c, configured to modify the first feature into a second feature of a preset image; and an image modification module 1103d, configured to generate a modified facial region based on the second feature.

[0135] According to an embodiment of the present disclosure, an electronic device is also provided, comprising at least one processor; and a memory communicatively connected to the at least one processor, wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described above.

[0136] According to an embodiment of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is further provided, wherein the computer instructions are used to enable a computer to execute the method described above.

[0137] According to an embodiment of the present disclosure, a computer program product is further provided, including a computer program, wherein the computer program implements the method described above when executed by a processor.

[0138] refer to Figure 12 , a block diagram of an electronic device 1200 that can serve as a server or client of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0139] like Figure 12As shown, the electronic device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. Various programs and data required for the operation of the electronic device 1200 can also be stored in the RAM 1203. The computing unit 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0140] Multiple components in the electronic device 1200 are connected to the I / O interface 1205, including: an input unit 1206, an output unit 1207, a storage unit 1208, and a communication unit 1209. The input unit 1206 can be any type of device that can input information to the electronic device 1200. The input unit 1206 can receive input digital or character information and generate key signal input related to user settings and / or function control of the electronic device, and can include but is not limited to a mouse, a keyboard, a touch screen, a trackpad, a trackball, a joystick, a microphone, and / or a remote control. The output unit 1207 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1208 can include but is not limited to a magnetic disk and an optical disk. The communication unit 1209 allows the electronic device 1200 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver and / or a chipset, such as a Bluetooth device, an 802.11 device, a WiFi device, a WiMax device, a cellular communication device and / or the like.

[0141] The computing unit 1201 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1201 performs the various methods and processes described above. For example, in some embodiments, the method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 1208. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1200 via the ROM 1202 and / or the communication unit 1209. When the computer program is loaded into the RAM 1203 and executed by the computing unit 1201, one or more steps of the method described above can be performed. Alternatively, in other embodiments, the computing unit 1201 can be configured to perform the method in any other appropriate manner (e.g., by means of firmware).

[0142] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0143] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0144] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0145] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0146] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0147] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0148] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0149] Although the embodiments or examples of the present disclosure have been described with reference to the accompanying drawings, it should be understood that the above-mentioned methods, systems and devices are merely exemplary embodiments or examples, and the scope of the present invention is not limited by these embodiments or examples, but is only limited by the claims after authorization and their equivalents. Various elements in the embodiments or examples may be omitted or replaced by their equivalents. In addition, the steps may be performed in an order different from that described in this disclosure. Further, the various elements in the embodiments or examples may be combined in various ways. It is important that as technology evolves, many of the elements described herein may be replaced by equivalent elements that appear after this disclosure.

Claims

1. A video detection method, comprising: Obtain the target video frame to be detected in the current video data; determining whether positioning information associated with a face of a target object is detected in the target video frame; In response to determining that the positioning information is not detected in the target video frame, predicting the positioning information based on a current state of a pre-built motion prediction model to obtain predicted positioning information, wherein the motion prediction model is used to predict motion of the face of the target object, and the current state of the motion prediction model is determined based on the positioning information of the face detected in a previous video frame before the target video frame; and Based on the predicted positioning information, a facial area corresponding to the face of the target object in the target video frame is determined.

2. The method according to claim 1, wherein The method comprises: In response to determining that the positioning information is not detected in the target video frame, determining whether motion change information of the face of the target object can be estimated, wherein the motion change information includes at least one of first information related to a moving speed of the face or second information related to a size change speed of the face; and In response to determining that the motion change information can be estimated, before performing prediction based on the current state of the motion prediction model, the current state of the motion prediction model is updated based on the motion change information so that prediction of the motion prediction model is performed based on the updated current state.

3. The method according to claim 1 or 2, wherein: The method comprises: After acquiring the target video frame, determining whether the target video frame belongs to a new shot, wherein the predicting of the positioning information based on the current state of the motion prediction model is performed based on determining that the target video frame does not belong to a new shot.

4. The method according to claim 3, wherein: Determining whether the target video frame belongs to a new storyboard includes: determining whether a center shift rate and an area change ratio of a predetermined reference object in the target video frame compared with the previous video frame reach a predetermined threshold; and In response to determining that both the center offset rate and the area change ratio do not reach the predetermined threshold, it is determined that the target video frame does not belong to a new storyboard.

5. The method according to claim 3 or 4, wherein: The method comprises: In response to determining that the target video frame does not belong to a new storyboard, and that the positioning information was not detected in the previous video frame belonging to the current storyboard but was detected for the first time in the target video frame, based on the current state of the motion prediction model, the positioning information of the face in the previous video frame is predicted from back to front to obtain the predicted positioning information of the face in the previous video frame, wherein the current state of the motion prediction model is determined based on the positioning information of the face detected in the target video frame.

6. The method according to any one of claims 3 to 5, wherein The predicted positioning information includes a predicted center position of the face and a predicted size of the face, and determining whether the target video frame belongs to a new shot includes: After obtaining the predicted positioning information, determining whether the face is detected in a preset area in the target video frame, wherein a center of the preset area is located at the predicted center position, and a size of the preset area has a predetermined proportional relationship with the predicted size; and In response to determining that the face is not detected, determining that the target video frame belongs to a new storyboard.

7. The method according to any one of claims 1 to 6, wherein The target video frame and the previous video frame are consecutive and adjacent video frames in the video data.

8. A video processing method, comprising: Acquire video data to be processed, where the video data includes multiple video frames; determining a facial region corresponding to a face of a target object in at least one designated video frame among the plurality of video frames, wherein, in response to not detecting the facial region in any designated video frame, determining the facial region according to the video detection method according to any one of claims 1 to 7; and The image of the facial area in each designated video frame is modified to a preset image to obtain updated video data.

9. The method according to claim 8, wherein The step of modifying the image of the facial region in each designated video frame into a preset image includes: Cropping the facial region from the designated video frame; extracting a first feature of the facial region; Modifying the first feature into a second feature of the preset image; and A modified facial region is generated based on the second feature.

10. A video detection device comprising: A video frame acquisition module is configured to acquire a target video frame to be detected in the video data; a face detection module configured to determine whether positioning information associated with a face of a target object is detected in the target video frame; a face prediction module configured to, in response to determining that the positioning information is not detected in the target video frame, predict the positioning information based on a current state of a pre-built motion prediction model to obtain predicted positioning information, wherein the motion prediction model is used to predict motion of the face of the target object, and the current state of the motion prediction model is determined based on the positioning information of the face detected in a previous video frame before the target video frame; as well as The face positioning module is configured to determine a face area corresponding to the face of the target object in the target video frame based on the predicted positioning information.

11. A video processing device, comprising: A video acquisition module is configured to acquire video data to be processed, wherein the video data includes a plurality of video frames; a video detection module configured to determine a facial region corresponding to a face of a target object in at least one designated video frame among the plurality of video frames, wherein the video detection apparatus according to claim 10 determines the facial region in response to the facial region not being detected in any designated video frame; and The video updating module is configured to modify the image of the facial area in each designated video frame into a preset image to obtain updated video data.

12. An electronic device comprising: at least one processor; as well as a memory communicatively coupled to the at least one processor; in The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 9.

14. A computer program product comprising a computer program, wherein The computer program implements the method according to any one of claims 1 to 9 when executed by a processor.

Images