Broadcast production method, device, and system

Abstract

The present disclosure relates to a broadcast directing method, an apparatus, a system, and a computer storage medium, which are related to the technical field of computers. The broadcast directing method includes the steps of: acquiring a reference video stream from a reference camera position; performing event recognition on the reference video stream to obtain at least one reference event frame interval, where each reference event frame interval corresponds to a unique event and each reference event frame interval includes frame identifiers of multiple consecutive images in which the same event occurs; determining a local sequence for each reference event frame interval according to a correspondence between the event and the camera position identifier, where the local sequence includes a camera position identifier for each frame image of a video to be broadcast corresponding to the reference event frame interval and a frame identifier corresponding to the camera position identifier; generating a broadcast directing sequence according to the local sequence; and generating a broadcast directing video according to the broadcast directing sequence and the video stream of the corresponding camera position. This disclosure reduces labor costs and improves real-time performance and accuracy of broadcast directing.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is based on and claims the benefit of Chinese Patent Application No. 202010477406.4, filed on May 29, 2020, the disclosure of which is incorporated herein in its entirety.

[0002] The present disclosure relates to the field of computer technology, and in particular to a broadcast directing method, apparatus, and system, and a computer-readable storage medium. [Background technology]

[0003] In television program production, simultaneous filming using multiple camera positions is often used for programs with fixed scenes to improve the program's comprehensiveness and enjoyment, and then the video streams from the multiple camera positions are edited and blended according to specific narrative rules to form a multi-angle, multi-shot scale broadcast-directed video.

[0004] For film and television shoots, there is ample time for post-processing and editing after filming is complete. However, in television live streaming scenes, video streams from various camera positions are directly transmitted to a video switcher, and then, under the coordination of the team, a broadcast-stage video is synthesized according to the on-site director's instructions to meet the live streaming latency requirements. During this process, the on-site director needs to select the appropriate camera position video stream for output based on the live streaming scene conditions. In addition, some live streaming scenes also require selecting the appropriate clip from the multi-path video stream for playback.

[0005] Typically, a complete live streaming broadcast production team includes a cameraman, an editor, and an on-site director.

[0006] Cameramen are deployed at multiple locations on the live streaming scene and use cameras of different specifications to provide different types of scene pictures. The cameramen's work has a certain autonomy, i.e., autonomous filming at the live streaming scene is performed according to a certain principle. In some specific cases, the cameramen are also controlled by the instructions of an on-site editor. The editor is deployed in a broadcast production vehicle and is responsible for selecting valuable clips from the multi-pass video stream from the cameramen for playback. In most cases, the editor needs to solely undertake the editing of the multi-pass video. The on-site director is deployed in a broadcast production vehicle and views the multi-pass real-time video stream and the playback clips provided by the editor, and selects appropriate material therefrom to generate the broadcast production video. The on-site director also needs to instruct the cameramen and editors to obtain effective raw video material and beautifully edited clips.

[0007] In the related art, the on-site director manually selects suitable video materials in a short time according to the acquired video stream to synthesize the broadcasting production video. Summary of the Invention [Means for solving the problem]

[0008] According to a first aspect of the present disclosure, there is provided a broadcast directing method including the steps of: acquiring a reference video stream from a reference camera position; performing event recognition on the reference video stream to obtain at least one reference event frame interval, wherein each reference event frame interval corresponds to a unique event and each reference event frame interval includes frame identifiers of multiple consecutive images in which the same event occurs; determining a local sequence for each reference event frame interval according to a correspondence between the event and the camera position identifier, wherein the local sequence includes a camera position identifier of each frame image of a video to be played corresponding to the reference event frame interval and a frame identifier corresponding to the camera position identifier; generating a broadcast directing sequence according to the local sequence; and generating a broadcast directing video according to the broadcast directing sequence and a video stream of a camera position corresponding to the camera position identifier of the broadcast directing sequence.

[0009] In some embodiments, the at least one reference event frame interval includes an i-th reference event frame interval, where i is a positive integer, and the step of determining a local sequence for each reference event frame interval according to a correspondence between events and camera position identifiers includes the steps of determining an initial local sequence for the i-th reference event frame interval according to a correspondence between events and camera position identifiers, wherein a start frame identifier and an end frame identifier of the initial local sequence are the start frame identifier and end event identifier of the i-th reference event frame interval, respectively; acquiring a video stream from at least one first auxiliary camera position; and extending the initial local sequence for the i-th reference event frame interval by using the video stream from the at least one first auxiliary camera position to obtain a local sequence for the i-th reference event frame interval.

[0010] In some embodiments, the at least one reference event frame interval further includes an (i+1)th reference event frame interval, and the start frame identifier and the end frame identifier of the i-th reference event frame interval are respectively s i and e i and the start frame identifier of the (i+1)th reference event frame interval is s i+1 and the step of extending the initial local sequence of the i-th reference event frame interval is, for i equal to 1, .... i and 1 or non-adjacent e i and s i+1 from at least one first auxiliary camera position, s i Video stream between 1 and e i and s i+1 and acquiring at least one of the video streams between as an extended video stream, and extending the initial local sequence of the i-th reference event frame interval by using the extended video stream to acquire a local sequence of the i-th reference event frame interval.

[0011] In some embodiments, the at least one reference frame interval further includes an (i-1)th reference event frame interval, and the end frame identifier of the (i-1)th reference event frame interval is E i-1 and the step of extending the initial local sequence of the i-th reference event frame interval is, for i greater than 1, .... i and E i-1 or non-adjacent e i and s i+1 from at least one first auxiliary camera position, s i and E i-1 Video stream between or e i and s i+1and acquiring at least one of the video streams between as an extended video stream, and extending the initial local sequence of the i-th reference event frame interval by using the extended video stream to acquire a local sequence of the i-th reference event frame interval.

[0012] In some embodiments, the augmented video stream is a multi-pass augmented video stream, the multi-pass augmented video stream being from a plurality of first auxiliary camera positions, and augmenting the initial local sequence of the ith reference event frame interval includes performing face recognition on each pass of the multi-pass augmented video stream to obtain at least one face frame interval corresponding to a pass of the multi-pass augmented video stream, each face frame interval corresponding to a unique face recognition result, each face frame interval including frame identifiers of a plurality of consecutive images having the same face recognition result; and generating at least one augmented frame interval according to the face frame interval of each pass of the multi-pass augmented video stream. The method includes the steps of: obtaining an extended sequence according to an extended frame interval having the largest number of corresponding first auxiliary camera positions and the largest total number of frames in at least one extended frame interval, wherein each extended frame interval includes at least a portion of a plurality of face frame intervals corresponding to different first auxiliary camera positions that can be concatenated; and extending an initial local sequence of the i-th reference event frame interval according to the extended sequence to obtain a local sequence of the i-th reference event frame interval.

[0013] In some embodiments, generating at least one extended frame interval according to a face frame interval of a path of the multi-path extended video stream includes determining, for the multi-path extended video stream of each first auxiliary camera position, a face frame interval adjacent to the i-th reference event frame interval as an initial extended frame interval; updating the initial extended frame interval by starting from the face frame interval adjacent to the i-th reference event frame interval and linking at least a portion of one face frame interval of a first auxiliary camera position other than the first auxiliary camera position that can be linked to the initial extended frame interval along a direction of decreasing or increasing frame identifiers; cyclically updating the initial extended frame interval until there are no more face frame intervals of first auxiliary camera positions other than the first auxiliary camera position corresponding to the initial extended frame interval that can be linked to the initial extended frame interval; and determining the updated initial extended frame interval as the extended frame interval.

[0014] In some embodiments, the at least one reference event frame interval includes an i-th reference event frame interval and an (i+1)-th reference event frame interval, where i is an integer equal to or greater than 1, and the start frame identifier and the end frame identifier of the i-th reference event frame interval are respectively set to s i and e i The start frame identifier of the (i+1)th reference event frame interval is s i+1 and determining a local sequence for each reference event frame interval according to the correspondence between the events and the camera position identifiers includes determining an initial local sequence for the i-th reference event frame interval according to the correspondence between the events and the camera position identifiers, wherein the start frame identifier and the end frame identifier of the initial local sequence are respectively set to s i and e i Step, and,e i and s i+1are not adjacent, the method includes determining a playback type according to an event corresponding to the i-th reference event frame interval; obtaining at least one path of the playback video stream corresponding to the playback type; and extending the initial local sequence according to the at least one path of the playback video stream to obtain a local sequence of the i-th reference event frame interval.

[0015] In some embodiments, extending the initial local sequence comprises generating at least one playback sequence according to at least one path of the playback video stream, each playback sequence comprising: i and s i+1 and a frame identifier corresponding to the camera position identifier; and extending the initial local sequence by using at least one playback sequence.

[0016] In some embodiments, the playback type includes a first playback type, and the step of generating at least one playback sequence according to at least one path of the playback video stream includes, under the condition that the playback type is the first playback type, performing event recognition on the at least one path of the playback video stream to obtain at least one auxiliary event frame interval, wherein the auxiliary event frame interval includes frame identifiers of multiple consecutive images in which an event corresponding to the i-th reference event frame interval occurs; and generating at least one playback sequence according to the at least one auxiliary event frame interval.

[0017] In some embodiments, generating at least one playback sequence according to at least one auxiliary event frame interval includes ranking the at least one auxiliary event frame interval according to a total number of frames and a weight of each auxiliary event frame interval, and generating at least one playback sequence according to a ranking result.

[0018] In some embodiments, the playback type includes a first playback type, and the step of obtaining at least one path of the playback video stream corresponding to the playback type includes obtaining, from at least one first auxiliary camera position, s i -m and e i +n as at least one path of the playback video stream, where m and n are both integers greater than or equal to 0.

[0019] In some embodiments, the playback type includes a second playback type, and the step of obtaining at least one path of the playback video stream corresponding to the playback type includes obtaining s' according to the reference video stream under the condition that the playback type is the second playback type. i and e' i and determining an area in which an event corresponding to the i-th reference event frame interval occurs according to each camera position angle; and determining an area in which an event corresponding to the i-th reference event frame interval occurs according to each camera position angle, and determining an area in which an event corresponding to the i-th reference event frame interval occurs according to each camera position angle. i and e i and obtaining a video stream between the two as at least one path of the playback video stream.

[0020] In some embodiments, the at least one reference event frame section includes an i-th reference event frame section and an (i+1)-th reference event frame section, where i is an integer equal to or greater than 1, and the step of generating a broadcast production sequence includes: Local SequenceEnd frame identifier E i is the (i+1)th reference event frame interval Local Sequence The start frame identifier S of i+1 generating a supplemental sequence under the condition that the supplemental sequence is not adjacent to the camera position, E i and S i+1 and a frame identifier of each frame image located between E i and S i+1 and a step of merging the local sequence and the supplemental sequence to obtain a broadcast production sequence.

[0021] In some embodiments, the reference position is used to provide a close-up video stream of the dribbling player, a first auxiliary camera position is used to provide a close-up video stream at a different angle on the pitch, a second auxiliary camera position is used to provide a standard video stream at a different angle on the pitch, and a third auxiliary camera position is used to provide a standard video stream at the spectator's viewing angle.

[0022] In some embodiments, generating the broadcast-staged video includes acquiring frame images corresponding to the broadcast-staged sequence according to the broadcast-staged sequence and a video stream of a camera position corresponding to a camera position identifier of the broadcast-staged sequence, and coding the frame images to acquire the broadcast-staged video.

[0023] According to a second aspect of the present disclosure, there is provided a broadcast production device comprising: an acquisition module configured to acquire a reference video stream from a reference camera position; an event recognition module configured to perform event recognition on the reference video stream to acquire at least one reference event frame interval, wherein each reference event frame interval corresponds to a unique event and each reference event frame interval includes frame identifiers of multiple consecutive images in which the same event occurs; a determination module configured to determine a local sequence for each reference event frame interval according to a correspondence between the event and a camera position identifier, wherein the local sequence includes a camera position identifier of each frame image of a video to be played corresponding to the reference event frame interval and a frame identifier corresponding to the camera position identifier; a first generation module configured to generate a broadcast production sequence according to the local sequence; and a second generation module configured to generate a broadcast production video according to the broadcast production sequence and a video stream of a camera position corresponding to the camera position identifier of the broadcast production sequence.

[0024] According to a third aspect of the present application, there is provided a broadcast production device comprising: a memory; and a processor coupled to the memory, the processor configured to perform the broadcast production method according to any of the above embodiments based on instructions stored in the memory.

[0025] According to a fourth aspect of the present disclosure, there is provided a broadcast production system comprising: a broadcast production device according to any of the above embodiments; and at least one camera configured to generate a video stream and transmit the video stream to the broadcast production device.

[0026] According to a fifth aspect of the present disclosure, there is provided a computer-storable medium having stored thereon computer program instructions which, when executed by a processor, implements a broadcast directing method according to any of the above embodiments.

[0027] The accompanying drawings, which constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the disclosure.

[0028] The present disclosure can be more clearly understood from the following detailed description taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0029] [Figure 1] FIG. 1 is a flow diagram illustrating a broadcast production method according to some embodiments of the present disclosure. [Figure 2] FIG. 1 illustrates a distribution of camera positions in a live streaming venue, according to some embodiments of the present disclosure. [Figure 3] FIG. 10 is a schematic diagram illustrating merging reference event frame intervals according to some embodiments of the present disclosure. [Figure 4] FIG. 10 is a flow diagram illustrating determining a local sequence for each reference event frame interval according to some embodiments of the present disclosure. [Figure 5a] FIG. 10 is a flow diagram illustrating extending an initial local sequence of the i-th reference event frame interval according to some embodiments of the present disclosure. [Figure 5b] FIG. 10 is a flow diagram illustrating extending an initial local sequence of the i-th reference event frame interval according to another embodiment of the present disclosure. [Figure 6a] FIG. 10 is a flow diagram illustrating extending an initial local sequence of the i-th reference event frame interval according to some embodiments of the present disclosure. [Figure 6b] 1 is a schematic diagram illustrating generating at least one extended frame interval according to some embodiments of the present disclosure. [Figure 7] FIG. 10 is a flow diagram illustrating determining a local sequence for each reference event frame interval according to another embodiment of the present disclosure. [Figure 8] 1 is a block diagram illustrating a broadcast production device according to some embodiments of the present disclosure. [Figure 9]FIG. 10 is a block diagram showing a broadcast production device according to another embodiment of the present disclosure. [Figure 10] 1 is a block diagram illustrating a broadcast production system according to some embodiments of the present disclosure. [Figure 11] FIG. 1 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0030] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangements, numerical expressions, and numerical values ​​of components and steps defined in these embodiments do not limit the scope of the present disclosure unless otherwise specified.

[0031] However, it should be understood that the sizes of parts shown in the drawings have not been drawn to scale for ease of illustration.

[0032] The following description of at least one exemplary embodiment is merely exemplary in nature and is in no way intended to limit the disclosure and its application or uses.

[0033] Techniques, methods, and devices known to those skilled in the art may not be discussed in detail, but are intended to be part of this specification, as appropriate.

[0034] In all examples shown and discussed herein, any particular values ​​should be construed as exemplary only and not limiting, and thus other examples of exemplary embodiments may have different values.

[0035] It should be noted that like reference numbers and letters refer to like items in the following drawings, and therefore, once a particular item is defined in one drawing, it need not be further discussed in subsequent drawings.

[0036] In related art, labor costs are high and real-time performance and accuracy are poor. Based on this, the present disclosure provides a broadcast production method that can reduce labor costs and improve real-time performance and accuracy of broadcast production.

[0037] Broadcast directing methods according to some embodiments of the present disclosure are described in detail below with reference to FIGS.

[0038] FIG. 1 is a flow diagram illustrating a broadcast production method according to some embodiments of the present disclosure.

[0039] FIG. 2 is a diagram illustrating the distribution of camera positions in a live streaming venue according to some embodiments of the present disclosure.

[0040] 1, the broadcast directing method includes step S10 of acquiring a reference video stream from a reference camera position, step S20 of performing event recognition on the reference video stream to acquire at least one reference event frame section, step S30 of determining a local sequence for each reference event frame section, step S40 of generating a broadcast directing sequence according to the local sequence, and step S50 of generating a broadcast directing video according to the broadcast directing sequence and a video stream at a camera position corresponding to a camera position identifier of the broadcast directing sequence. For example, the broadcast directing method is executed by a broadcast directing device.

[0041] According to the present disclosure, by event recognition, a local sequence of each reference event frame interval is obtained, and broadcast direction is performed according to the local sequence so as to realize automatic broadcast direction, which reduces labor costs and improves the real-time performance and accuracy of broadcast direction.

[0042] In addition, by realizing automatic broadcast direction, the work difficulty of the on-site broadcast direction team is greatly reduced, the on-site director only needs to instruct the cameraman to shoot the appropriate video material, and the generation and output of the broadcast direction video is completed efficiently and automatically by the computer. Furthermore, since the computer code is customizable and it is convenient to change and customize the broadcast direction logic, it is possible to realize the output of thousands of broadcast direction videos for thousands of people, greatly enriching the audience's choices.

[0043] In step S10, a reference video stream from a reference camera position is acquired. In some embodiments, the reference video stream from the reference camera position is acquired via an input interface.

[0044] For example, the reference camera position is camera CAM-2 as shown in FIG. 2. Camera CAM-2 is a 4K camera that provides a close-up video stream of a dribbling player. The lens of camera CAM-2 is a lens with over 100x magnification and is a close-up camera position in the stands. In some embodiments, the camera position identifier for camera CAM-2 is 2. For example, each camera in FIG. 2 is located on the pitch.

[0045] In step S20, event recognition is performed on the reference video stream to obtain at least one reference event frame interval, each reference event frame interval corresponding to a unique event, each reference event frame interval including frame identifiers of multiple consecutive images in which the same event occurs.

[0046] In some embodiments, event recognition is performed using a video event recognition algorithm, including, but not limited to, a P3D ResNet (Pseudo 3D Residual Network) algorithm.

[0047] For example, event recognition for a reference video stream is achieved as follows.

[0048] First, an event recognition result for each frame image in the reference video stream is obtained by using a video event recognition algorithm. In some embodiments, the event recognition result identifier for each frame image is expressed as P=[p1,...,p cls ]. p cls represents the probability that an event with the event identifier in cls occurs or the probability that the event does not occur. For example, in a soccer live streaming scene, the value of cls is an integer between 1 and 7, which respectively represent six different events and no event.

[0049] In some embodiments, events in a live soccer streaming scene include, but are not limited to, shots, free kicks, corner kicks, goal kicks, throw-ins, and player collisions.

[0050] Second, a smoothing operation is performed on the event recognition results of each frame image in the reference video stream to obtain a smoothed event recognition result for each frame image. For example, by using a time window having a length of t seconds, a smoothing operation with a stride of 1 frame is performed on the event recognition results of each frame image. In some embodiments, t is equal to 0.5. The smoothing operation can reduce the error of event recognition, so that the event recognition becomes more accurate and thus the accuracy of broadcast rendition is improved.

[0051] For example, the smoothed event recognition result is

[0052]

number

[0053] where m=f×t, where f is the frame rate of the reference video stream.

[0054]

number

[0055] represents the smoothed event recognition result of the intermediate frame between the first frame image and the m-th frame image.

[0056] Then, for each frame image, the event corresponding to the maximum probability in the smoothed event recognition results, or no event, is determined as the final event recognition result for each frame image.

[0057] Finally, frame identifiers of multiple consecutive images in which the same event occurs are merged to obtain at least one reference event frame interval. In some embodiments, multiple reference event frame intervals corresponding to the same event, separated by multiple frames of event-free images, can also be merged into one reference event frame interval.

[0058] FIG. 3 is a schematic diagram illustrating merging reference event frame intervals according to some embodiments of the present disclosure.

[0059] As shown in FIG. 3, a, b, and c respectively represent reference event frame intervals of different events. For the reference video stream, there are two reference event frame intervals c. The two reference event frame intervals c are spaced apart by multiple frames of event-free images. For example, under the condition that the number of separated frames of event-free images is equal to or less than a preset threshold, the two reference event frame intervals c are merged into one event frame interval c'. In some embodiments, the preset threshold is f×t1. For example, t1 is 0.5 seconds.

[0060] Returning to FIG. 1, after at least one reference event frame interval is obtained, step S30 is executed.

[0061] In step S30, a local sequence of each reference event frame interval is determined according to the relative relationship between the event and the camera position identifier, and the local sequence includes the camera position identifier of each frame image of the video to be played corresponding to the reference event frame interval and the frame identifier corresponding to the camera position identifier.

[0062] For example, the at least one reference event frame interval includes the i-th reference event frame interval, where i is a positive integer. In some embodiments, the local sequence of the i-th reference event frame interval can be expressed as:

[0063]

number

[0064] It can be expressed as S i and E i are the start and end frame identifiers of the local sequence of the i-th reference event frame interval, respectively. j is the frame identifier of the video to be played, and j is the S i E i where k is the camera position identifier and z is the frame identifier of the video stream corresponding to the camera position identifier. j (k,z) indicates that the jth frame image of the video to be played back corresponding to the ith reference event frame interval is the zth frame image from the camera position having the camera position identifier k.

[0065] For example, Table 1 shows the correspondence between events and camera position identifiers in a soccer live streaming scene.

[0066] [Table 1]

[0067] As shown in Table 1, in a soccer live streaming scene, shots, corner kicks, free kicks, player collisions, goal kicks, and throw-ins correspond to camera position identifiers 1, 2, 2, 1, 2, and 2, respectively. For example, the camera position with camera position identifier 1 is camera CAM-1 as shown in Figure 2. Camera CAM-1 is a 4K camera to provide a standard video stream at the spectator's viewing angle. Camera CAM-1 provides a standard lens, which is a stand panoramic camera position.

[0068] For example, step S30 is performed by the steps shown in FIG.

[0069] FIG. 4 is a flow diagram illustrating determining a local sequence for each reference event frame interval according to some embodiments of the present disclosure.

[0070] As shown in FIG. 4, determining the local sequence of each reference event frame interval includes steps S31 to S33.

[0071] In step S31, an initial local sequence of the i-th reference event frame interval is determined according to the correspondence between the event and the camera position identifier, and the start frame identifier and the end frame identifier of the initial local sequence are the start frame identifier and the end frame identifier of the i-th reference event frame interval, respectively.

[0072] For example, the event in the i-th reference event frame interval is a corner kick. According to Table 1, the camera position identifier corresponding to the corner kick is 2.

[0073] In some embodiments, the start and end frame identifiers of the i-th reference event frame interval are s i and e iThe initial local sequence of the i-th reference event frame interval is

[0074]

number

[0075] It is shown as c j (2,j) indicates that the jth frame image of the video to be played corresponding to the initial local sequence is the jth frame image from the video stream with a camera position identifier of 2.

[0076] In step S32, video streams from at least one first auxiliary camera position are acquired, for example, the first auxiliary camera position is used to provide close-up video streams at different angles on the pitch.

[0077] For example, in a soccer live streaming scene, the first auxiliary camera positions are cameras CAM-3, CAM-7, CAM-8, and CAM-10 shown in FIG. 2, and their camera position identifiers are 3, 7, 8, and 10, respectively. Cameras CAM-3, CAM-7, CAM-8, and CAM-10 are all 4K cameras, providing lenses greater than 80x, greater than 40x, greater than 80x, and greater than 80x, respectively. Cameras CAM-3, CAM-7, and CAM-10 are all ground camera positions, and camera CAM-8 is a stand camera position.

[0078] In step S33, the initial local sequence of the i-th reference event frame interval is extended by using a video stream from at least one first auxiliary camera position to obtain a local sequence of the i-th reference event frame interval.

[0079] FIG. 5a is a flow diagram illustrating extending an initial local sequence of an i-th reference event frame interval according to some embodiments of the present disclosure.

[0080] In some embodiments, the at least one reference event frame interval further includes an (i+1)th reference event frame interval, and the start frame identifier of the (i+1)th reference event frame interval is s i+1 is.

[0081] As shown in FIG. 5a, extending the initial local sequence of the i-th reference event frame interval includes steps S331 to S332.

[0082] In step S331, if i is equal to 1, i and 1 are non-adjacent, or e i and s i+1 S from at least one first auxiliary camera position, under the condition that there is at least one of i Video stream between 1 and e i and s i+1 At least one of the video streams between S and S is acquired as an extended video stream. i the difference between and 1 or s i+1 and e i and f are greater than a predetermined difference. In some embodiments, the predetermined difference is 0 or f × t2. For example, t2 is 2 seconds.

[0083] In step S332, the initial local sequence of the i-th reference event frame interval is extended by using the extended video stream to obtain the local sequence of the i-th reference event frame interval.

[0084] FIG. 5b is a flow diagram illustrating extending the initial local sequence of the i-th reference event frame interval according to another embodiment of the present disclosure.

[0085] In some embodiments, the at least one reference frame interval further includes an (i-1)th reference event frame interval, and the end frame identifier of the local sequence of the (i-1)th reference event frame interval is E i-1 is.

[0086] As shown in FIG. 5b, extending the initial local sequence of the i-th reference event frame interval includes steps S331' to S332'.

[0087] In step S331', if i is greater than 1, S i and E i-1 are non-adjacent, or e i and s i+1 S from at least one first auxiliary camera position, under the condition that there is at least one of i and E i-1 Video stream between or e i and s i+1 At least one of the video streams between S and S is acquired as an extended video stream. i and E i-1 the difference between or s i+1 and e i and the difference between the time t and the time t is greater than a predetermined difference. In some embodiments, the predetermined difference is 0 or ×t. For example, t is 2 seconds.

[0088] In some embodiments, e i and s i+1 are non-adjacent and s i+1 and e i Under the condition that the difference between i and, e i and the sum of the preset values ​​from at least one first auxiliary camera position are obtained as an augmented video stream.

[0089] In step S332', the initial local sequence of the i-th reference event frame interval is extended by using the extended video stream to obtain the local sequence of the i-th reference event frame interval.

[0090] The process of extending the initial local sequence of the i-th reference event frame interval is described in detail below in conjunction with FIGS. 6a and 6b.

[0091] FIG. 6a is a flow diagram illustrating extending an initial local sequence of an i-th reference event frame interval according to some embodiments of the present disclosure.

[0092] FIG. 6b is a schematic diagram illustrating generating at least one extended frame interval according to some embodiments of the present disclosure.

[0093] For example, the augmented video stream is a multi-pass augmented video stream, the multi-pass augmented video stream being from the same frame interval of one of a plurality of first auxiliary camera positions.

[0094] As shown in FIG. 6a, extending the initial local sequence of the i-th reference event frame interval includes steps S3321 to S3324.

[0095] In step S3321, facial recognition is performed on each pass of the multi-pass extended video stream to obtain at least one face frame interval corresponding to a pass of the multi-pass extended video stream. Each face frame interval corresponds to a unique face recognition result. Each face frame interval includes frame identifiers of multiple consecutive images having the same face recognition result. In some embodiments, the total number of frames in each face frame interval is greater than a preset total number of frames. For example, the preset total number of frames is f × t2. For example, t2 is 2 seconds. Controlling the total number of frames in each face frame interval can improve the audience's viewing experience.

[0096] In some embodiments, facial recognition is performed by using a facial detection SDK (software development kit) provided by the JingDong AI open platform Neuhub to obtain facial recognition results for each frame image of each pass of the multi-pass extended video stream. Furthermore, at least one facial frame interval is obtained according to multiple frame identifiers of multiple consecutive frame images having the same facial recognition result. For example, the facial recognition result for each frame image is a facial attribute contained in the frame image. The facial attributes include, but are not limited to, coaches, substitute players, and linesmen.

[0097] For example, s i and E i-1 are non-adjacent, or s i If camera positions 1, 2, and 3 are non-adjacent, then as shown in Figure 6b, there will be augmented video streams 1, 2, and 3, which are from the first auxiliary camera position shown in Figure 2 with camera position identifiers 3, 7, and 8, respectively.

[0098] The different paths of the multi-path extended video stream are from different first auxiliary camera positions. Extended video stream 1 corresponds to face frame interval 11 and face frame interval 12. Face frame interval 11 is [x1, x2], and face frame interval 12 is [x3, s iThe extended video stream 2 corresponds to a face frame section 21 and a face frame section 22. The face frame section 21 is [x4, x5], and the face frame section 22 is [x6, s i The extended video stream 3 corresponds to the face frame section 31, which is [x7,s i x1 -1]. <x4<x7<x2<x5<x6<x3<s i It is -1.

[0099] In S3322, at least one extended frame interval is generated according to a face frame interval of a path of the multi-path extended video stream, where each extended frame interval can be concatenated and includes at least a portion of a plurality of face frame intervals corresponding to different first auxiliary camera positions, where being concatenated refers to two face frame intervals being adjacent or overlapping.

[0100] For example, generating at least one extended frame interval according to each face frame interval of each pass of the extended video stream is performed as follows.

[0101] First, for the extended video stream of each first auxiliary camera position, a face frame interval adjacent to the i-th reference event frame interval is determined as an initial extended frame interval.

[0102] For example, for the extended video stream 1 shown in FIG. 6b, face frame section 12 is determined as the initial extended frame section.

[0103] Second, to update the initial extended frame interval, starting from the face frame interval adjacent to the i-th reference event frame interval, at least a portion of one face frame interval of a first auxiliary camera position other than the first auxiliary camera position that can be connected to the initial extended frame interval is connected to the initial extended frame interval along the direction of decreasing or increasing frame identifiers.

[0104] For example, s iand E i-1 are non-adjacent, or s i and 1 are non-adjacent, for the extended video stream 1 shown in FIG. 6b, to update the initial extended frame interval, starting from face frame interval 12, a portion [x7, x3-1] of face frame interval 31 of another first auxiliary camera position that can be concatenated to face frame interval 12 along the direction of decreasing frame identifiers is concatenated to the initial extended frame interval.

[0105] The initial extended frame interval is then cyclically updated until there are no more face frame intervals at the first auxiliary camera positions other than the first auxiliary camera position corresponding to the initial extended frame interval that can be connected to the initial extended frame interval.

[0106] For example, s i and E i-1 are non-adjacent, or s i and 1 are non-adjacent, for the enhanced video stream 1 shown in Fig. 6b, the initial enhanced frame interval is cyclically updated to the initial enhanced frame interval [x3,s i To perform the update of [x7, x3-1], a portion of the face frame section 21 of another first auxiliary camera position [x4, x7-1], which can be connected to a portion of the face frame section 31 [x7, x3-1], can continue to be connected to the initial extended frame section.

[0107] Finally, the updated initial extension frame interval is determined as the extension frame interval.

[0108] For example, s i and E i-1 are non-adjacent, or s i 6b, a part of face frame section 21 [x4, x7-1], a part of face frame section 31 [x7, x3-1], and the whole of face frame section 12 [x3, s i -1], one extended frame obtained by concatenating

[0109] In step S3323, an extended sequence is obtained according to the extended frame interval having the largest number of corresponding first auxiliary camera positions and the largest total number of frames in at least one extended frame interval, and the extended sequence includes a camera position identifier of each frame image of the video to be played corresponding to the extended frame interval and a frame identifier corresponding to the camera position identifier.

[0110] For example, s i and E i-1 are non-adjacent, or s i 6b, a part of face frame section 21 [x4, x7-1], a part of face frame section 31 [x7, x3-1], and the whole of face frame section 12 [x3, s i The extended frame interval obtained by concatenating [(1)-1] and [(2)-1] has the largest number of corresponding first auxiliary camera positions and the largest total number of frames. An extended sequence is obtained from the extended frame interval.

[0111] For example, the obtained extended sequence is

[0112]

number

[0113] is.

[0114] In step S3324, the initial local sequence of the i-th reference event frame interval is extended according to the extension sequence to obtain the local sequence of the i-th reference event frame interval.

[0115] For example, the local sequence of the i-th reference event frame interval obtained by the expansion is

[0116]

number

[0117] is.

[0118] For example, e i and s i+1 If the frames are non-adjacent, the extension can also be performed along the direction of increasing frame identifiers to obtain an extension sequence.

[0119]

number

[0120] is used to extend the initial local sequence after

[0121] In some embodiments, e i and s i+1 are non-adjacent, the start frame identifier of the extended sequence in this case is typically a frame identifier separated by a specific number of frames from the end frame identifier of the i-th event frame interval, considering that a specific reaction time is ensured for the cameraman. Then, the sequence between the start frame identifier of the extended sequence and the end frame identifier of the i-th event frame interval is supplemented by using the sequence of corresponding frame images of the third auxiliary camera position.

[0122] For example, a third auxiliary camera position is used to provide a standard video stream at the spectator's viewing angle. In some embodiments, the third auxiliary camera position is camera CAM-1 in FIG. 2 in the soccer live streaming scene. Camera CAM-1 is a 4K camera that provides a standard lens and is a standard panoramic camera position.

[0123] In some embodiments, s i and E i-1 and e i and S i+1 Both are non-adjacent, or s i and 1 and ei and s i+1 are non-adjacent, two extended sequences are obtained simultaneously to correspondingly extend the initial local sequence simultaneously.

[0124] For example, step S30 of determining the local sequence of each reference event frame interval can also be performed by the steps shown in FIG.

[0125] FIG. 7 is a flow diagram illustrating determining a local sequence for each reference event frame interval according to another embodiment of the present disclosure.

[0126] As shown in FIG. 7, determining the local sequence of each reference event frame interval includes steps S31' to S34'.

[0127] In step S31', an initial local sequence of the i-th reference event frame interval is determined according to the correspondence between the event and the camera position identifier. The start frame identifier and the end frame identifier of the initial local sequence are respectively set as s i and e i is.

[0128] In step S32′, e i and s i+1 If the i-th reference event frame interval is non-adjacent, the playback type is determined according to the event corresponding to the i-th reference event frame interval. For example, in a soccer live streaming scene, the playback type includes a first playback type and a second playback type. In some embodiments, the first playback type is a close-up camera position slow playback, and the second playback type is a standard camera position normal playback.

[0129] For example, under the condition that the event is a player collision, the replay type is close-up camera position slow replay, and under the condition that the event is a shot, corner kick, or free kick, the replay type is standard camera position normal replay.

[0130] In step S33', at least one path of the playback video stream corresponding to the playback type is obtained.

[0131] For example, under the condition that the playback type is the first playback type, s from at least one first auxiliary camera position i -m and e i +n are obtained as playback video streams, where m and n are both integers equal to or greater than 0. Generally, close-up camera positions may be in shadow, so that it cannot be guaranteed that all close-up camera positions can capture pictures of the same event when an event occurs. Therefore, to obtain playback video streams, a specific range is added before and after the start frame identifier and the end frame identifier of the ith event frame interval, respectively.

[0132] For example, under the condition that the playback type is the second playback type, s' corresponding to each frame image i and e' i The camera position angles between are obtained according to the reference video stream. Furthermore, the area where the event corresponding to the i-th reference event frame interval occurs is determined according to each camera position angle. Therefore, the s at least one second auxiliary camera position in the area i and e i The video stream located between is obtained as the playback video stream. For example, the value range of the camera position angle is [-90, 90] in degrees.

[0133] For example, a second auxiliary camera position is used to provide a standard video stream at a different angle on the pitch. In some embodiments, the second auxiliary camera positions are CAM-4, CAM-5, CAM-6, and CAM-9 in FIG. 2. Cameras CAM-4, CAM-5, CAM-6, and CAM-9, with camera position identifiers 4, 5, 6, and 9, respectively, are all 4K cameras and provide standard lenses. Cameras CAM-4 and CAM-6 are the left standoffside camera position and the left ground camera position, respectively. Cameras CAM-5 and CAM-9 are the right standoffside camera position and the right ground camera position, respectively.

[0134] For example, determining the area in which an event occurs may be performed as follows.

[0135] First, according to the camera position angle, the camera position angle sequence

[0136]

number

[0137] is generated.

[0138] Then, a monomial linear regression equation a=k×x+b of the camera position angle sequence A is calculated, where a is the angle, x is the index value x of the angle sequence A, and x∈[0,e i -s i ),x∈N. The monomial linear regression equation simply describes the change process of the camera position angle of the reference camera position within the i-th event frame interval.

[0139] For example, under the condition that k×b is greater than 0 (k and b are both positive, or k and b are both negative), the area in which the event occurs is within a specific half-pitch area. Under the condition that k is positive and b is positive, the camera position angle at the start of the event is biased toward the right half-pitch area, and the camera position angle gradually shifts to the right as the event occurs. Under the condition that k is negative and b is negative, the camera position angle at the start of the event is shifted toward the left half-pitch area, and the camera position angle gradually shifts to the left as the event occurs.

[0140] The half-pitch is crossed when an event occurs, provided that k×b is less than 0 (one of k and b is positive and the other is negative). No replay is performed for events that cross the half-pitch, as they are not considered to pose a goal threat.

[0141] In step S34', the initial local sequence is extended according to at least one pass of the playback video stream to obtain the local sequence of the i-th reference event frame interval.

[0142] For example, extending the initial local sequence according to at least one path of the playback video stream is performed as follows.

[0143] First, at least one playback sequence is generated according to at least one path of the playback video stream. Each playback sequence includes: i and s i+1 and a frame identifier corresponding to the camera position identifier.

[0144] For example, under the condition that the playback type is the first playback type, event recognition is performed on at least one path of the playback video stream to obtain at least one auxiliary event frame interval. Further, at least one playback sequence is generated according to the at least one auxiliary event frame interval. The auxiliary event frame interval includes frame identifiers of multiple consecutive images in which an event corresponding to the i-th reference event frame interval occurs.

[0145] For example, at least one auxiliary event frame interval is ranked according to the total number of frames and the weight of each auxiliary event frame interval. Further, at least one playback sequence is generated according to the ranking result. In some embodiments, under the condition that the first playback type is a close-up camera position slow playback, a frame interpolation process at a slow motion rate is performed on the playback sequence to generate a slow playback sequence. Under the condition that the camera position corresponding to the playback sequence is a high-speed camera, the frame interpolation process is not required.

[0146] In some embodiments, under the condition that the playback type is the second playback type, the s of at least one second auxiliary camera position in the area i and e i After the video streams located between are acquired as playback video streams, at least one playback sequence is generated according to the at least one playback video stream, for example, according to the start frame identifier and the end frame identifier of each playback video stream and its corresponding camera position identifier, a corresponding playback sequence is generated.

[0147] Then, after the at least one reproduction sequence is generated, the initial local sequence is extended by using the at least one reproduction sequence.

[0148] For example, after the end frame identifier of the initial local sequence, as many playback sequences as possible are concatenated to obtain the local sequence. In some embodiments, after the initial local sequence is extended by using the extension sequence, it is also possible to continue concatenating as many playback sequences as possible to obtain the local sequence.

[0149] Returning to FIG. 1, after the local sequence of each reference event frame interval is determined, step S40 is executed.

[0150] In step S40, a broadcast production sequence is generated in accordance with the local sequence.

[0151] For example, local sequences are merged to obtain a broadcast rendition sequence.

[0152] In some embodiments, the i-th reference event frame interval Local Sequence End frame identifier E i is the (i+1)th reference event frame interval Local Sequence The start frame identifier S of i+1 A supplementary sequence is generated under the condition that the camera positions and E i and S i+1 and a frame identifier of each frame image located between E i and S i+1 The camera position of each frame image located between and is the third auxiliary camera position. Furthermore, each local sequence and the supplementary sequence are merged to obtain the broadcast production sequence.

[0153] In step S50, a broadcast staged video is generated according to the broadcast staged sequence and the video stream at a camera position corresponding to the camera position identifier of the broadcast staged sequence. In some embodiments, frame images corresponding to the broadcast staged sequence are acquired according to the broadcast staged sequence and the video stream at a position corresponding to the camera position identifier of the broadcast staged sequence. Further, the frame images are encoded to obtain the broadcast staged video.

[0154] For example, after the video stream from each camera is acquired through the video input interface, the video stream is stored in a buffer, and after the broadcast production sequence is acquired, according to the camera position identifier of each frame image provided by the broadcast production sequence and the frame identifier corresponding to the camera position identifier, an image having the corresponding frame identifier is acquired from the video stream of the corresponding camera position in the buffer, and each frame image is sequentially encoded to acquire the broadcast production video.

[0155] In some embodiments, the broadcast rendition video is output via a video output interface for live streaming.

[0156] FIG. 8 is a block diagram illustrating a broadcast production device according to some embodiments of the present disclosure.

[0157] As shown in FIG. 8, the broadcast production device 8 includes an acquisition module 81, an event recognition module 82, a determination module 83, a first generation module 84, and a second generation module 85. It is equipped with:

[0158] The acquisition module 81 is configured to, for example, perform step S10 shown in FIG. 1 to acquire a reference video stream from a reference camera position.

[0159] In some embodiments, the broadcast production device 8 further includes an input interface 80. The acquisition module 81 acquires a reference video stream from a reference camera position via the input interface 80.

[0160] The event recognition module 82 is configured to perform event recognition on the reference video stream to obtain at least one reference event frame interval, for example, by performing step S20 shown in FIG. 1. Each reference event frame interval corresponds to a unique event. Each reference event frame interval includes frame identifiers of multiple consecutive images in which the same event occurs.

[0161] 1 to determine a local sequence for each reference event frame interval according to the correspondence between the event and the camera position identifier. The local sequence includes a camera position identifier for each frame image of the video to be played corresponding to the reference event frame interval and a frame identifier corresponding to the camera position identifier.

[0162] The first generating module 84 is configured to, for example, execute step S40 shown in FIG. 1 to generate a broadcast production sequence according to the local sequence.

[0163] The second generating module 85 is configured to, for example, execute step S50 shown in FIG. 1 to generate a broadcast production video according to the broadcast production sequence and the video stream of the camera position corresponding to the camera position identifier of the broadcast production sequence.

[0164] In some embodiments, the broadcast production device 8 further includes a buffer 86. The buffer 86 is configured to store a video stream of a corresponding camera position. For example, the acquisition module 81 can acquire the video stream of the camera position via the input interface 80 corresponding to the input interface and buffer the video stream in the buffer 86.

[0165] In some embodiments, the broadcast production device 8 further comprises an output interface 87. The second generation module 85 outputs the broadcast production video via the output interface 87 for live streaming.

[0166] FIG. 9 is a block diagram showing a broadcast production device according to another embodiment of the present disclosure.

[0167] 9, the broadcast directing device 9 includes a memory 91 and a processor 92 coupled to the memory 91. The memory 91 is configured to store instructions for executing a corresponding embodiment of the broadcast directing method. The processor 92 is configured to execute the broadcast directing method according to any of the embodiments of the present disclosure based on the instructions stored in the memory 91.

[0168] FIG. 10 is a block diagram illustrating a broadcast production system according to some embodiments of the present disclosure.

[0169] 10, the broadcast production system 10 includes a broadcast production device 101 and at least one camera 102. The broadcast production device 101 is a broadcast production device according to any of the embodiments of the present disclosure. The broadcast production device 101 is configured to execute a broadcast production method according to any of the embodiments of the present disclosure.

[0170] At least one camera 102 is configured to generate a video stream and transmit the video stream to the broadcast production equipment. One camera corresponds to one camera position and has a unique camera position identifier. The video streams include, but are not limited to, a reference video stream and a video stream for a corresponding camera position.

[0171] FIG. 11 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure.

[0172] 11, computer system 110 may be embodied in the form of a general-purpose computing device. Computer system 110 comprises a memory 1110, a processor 1120, and a bus 1100 that connects the different system components.

[0173] The memory 1110 may include, for example, a system memory, a non-volatile storage medium, etc. The system memory stores, for example, an operating system, applications, a boot loader, other programs, etc. The system memory may include a volatile storage medium, for example, a random access memory (RAM) and / or a cache memory. The non-volatile storage medium stores, for example, instructions for executing at least one corresponding embodiment of the broadcast production method. The non-volatile storage medium includes, but is not limited to, a magnetic disk memory, an optical memory, a flash memory, etc.

[0174] The processor 1120 may be implemented by discrete hardware components such as a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gates or transistors, etc. Thus, each module, such as the determination module and decision module, may be implemented by a central processing unit (CPU) that executes instructions in a memory that perform the corresponding steps, or may be implemented by dedicated circuitry that performs the corresponding steps.

[0175] Bus 1100 may use any of a variety of bus architectures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.

[0176] The computer system 110 may further include an input / output interface 1130, a network interface 1140, a storage interface 1150, etc. These interfaces 1130, 1140, 1150, as well as the memory 1110 and the processor 1120, may be connected via a bus 1100. The input / output interface 1130 may provide a connection interface for input / output devices such as a display, a mouse, and a keyboard. The network interface 1140 provides a connection interface for various networking devices. The storage interface 1150 provides a connection interface for external storage devices such as a floppy disk, a USB disk, and an SD card.

[0177] Various aspects of the present disclosure are described herein with reference to flow diagrams and / or block diagrams of methods, devices, and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks, can be implemented by computer-readable program instructions.

[0178] These computer-readable program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable device to create a machine, such that the instructions, when executed by the processor, create means for implementing the functions specified in one or more blocks in the flow diagrams and / or block diagrams.

[0179] These computer-readable program instructions may also be stored in a computer-readable memory, and these instructions cause a computer to operate in a particular manner to produce an article of manufacture that includes instructions to implement the functions specified in one or more blocks in the flow diagrams and / or block diagrams.

[0180] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects.

[0181] The broadcast directing method, apparatus, system, and computer-storable medium in the above embodiments reduce labor costs and improve real-time performance and accuracy of broadcast directing.

[0182] The broadcast production method, device, and system, and computer-readable storage medium according to the present disclosure have been described in detail above. In order to avoid obscuring the concept of the present disclosure, some details well known in the art have not been described. Those skilled in the art can fully understand how to implement the technical solutions disclosed herein in light of the foregoing description. [Explanation of symbols]

[0183] 8 Broadcast Production Equipment 9 Broadcasting Equipment 10 Broadcast Production System 80 input interfaces 81 Acquisition Module 82 Event Recognition Module 83 Decision Module 84 First Generation Module 85 Second generation module 86 buffers 87 Output Interface 91 memory 92 processors 101 Broadcast Production Equipment 102 Camera 110 Computer Systems 1100 Bus 1110 memory 1120 processor 1130 Input / Output Interface 1140 Network Interface 1150 storage interface

Claims

1. acquiring a reference video stream from a reference camera position; performing event recognition on the reference video stream to obtain at least one reference event frame interval, each reference event frame interval corresponding to an event, and each reference event frame interval including frame identifiers of multiple consecutive images in which the same event occurs; determining a local sequence for each reference event frame interval according to a correspondence between the event and a camera position identifier, the local sequence including a camera position identifier of each frame image of a video to be played corresponding to the reference event frame interval, and a frame identifier corresponding to the camera position identifier; generating a broadcast production sequence according to the local sequence; generating a broadcast production video according to the broadcast production sequence and a video stream of a camera position corresponding to the camera position identifier of the broadcast production sequence; A broadcast production method including:

2. the at least one reference event frame interval includes an i-th reference event frame interval, where i is a positive integer; determining a local sequence of each reference event frame interval according to a correspondence between the event and a camera position identifier; determining an initial local sequence of the i-th reference event frame interval according to the correspondence between the event and the camera position identifier, wherein a start frame identifier and an end frame identifier of the initial local sequence are the start frame identifier and the end frame identifier of the i-th reference event frame interval, respectively; acquiring a video stream from at least one first auxiliary camera position; augmenting the initial local sequence of the i-th event frame interval by using the video stream from the at least one first auxiliary camera position to obtain the local sequence of the i-th reference event frame interval; Including, The broadcast production method according to claim 1.

3. The at least one reference event frame interval further includes an (i+1)th reference event frame interval, and the start frame identifier and the end frame identifier of the i-th reference event frame interval are respectively set to s i and e i and the start frame identifier of the (i+1)th reference event frame interval is s i+1 and The step of expanding the initial local sequence of the i-th reference event frame interval includes: For i equal to 1, non-adjacent s i and 1 or non-adjacent e i and s i+1 from the at least one first auxiliary camera position, s i Video stream between 1 and e i and s i+1 obtaining at least one of the video streams between the augmenting the initial local sequence of the i-th reference event frame interval by using the augmented video stream to obtain the local sequence of the i-th reference event frame interval; Including, The broadcast production method according to claim 2.

4. The at least one reference event frame interval further includes an (i-1)th reference event frame interval, and an end frame identifier of the local sequence of the (i-1)th reference event frame interval is E i-1 and The step of expanding the initial local sequence of the i-th reference event frame interval includes: For i greater than 1, non-adjacent s i and E i-1 or non-adjacent e i and s i+1 from the at least one first auxiliary camera position, s i and E i-1 Video stream between or e i and s i+1 obtaining at least one of the video streams between the augmenting the initial local sequence of the i-th reference event frame interval by using the augmented video stream to obtain the local sequence of the i-th reference event frame interval; Including, The broadcast production method according to claim 2.

5. The augmented video stream is a multi-pass augmented video stream, and the multi-pass augmented video stream is from a plurality of first auxiliary camera positions, and the augmenting step of the initial local sequence of the i-th reference event frame interval includes: performing face recognition on each pass of the multi-pass extended video stream to obtain at least one face frame interval corresponding to a pass of the multi-pass extended video stream, each face frame interval corresponding to a unique face recognition result, each face frame interval including frame identifiers of multiple consecutive images having the same face recognition result; generating at least one extended frame interval according to the face frame intervals of the passes of the multi-pass extended video stream, each extended frame interval including at least a portion of a plurality of face frame intervals corresponding to different first auxiliary camera positions; acquiring an extended sequence according to an extended frame section that has the largest number of corresponding first auxiliary camera positions and the largest total number of frames in the at least one extended frame section, the extended sequence including a camera position identifier for each frame of the video to be played corresponding to the extended frame interval, and a frame identifier corresponding to the camera position identifier; extending the initial local sequence of the i-th reference event frame interval according to the extension sequence to obtain the local sequence of the i-th reference event frame interval; Including, 5. The broadcast production method according to claim 3 or 4.

6. generating at least one enhanced frame interval according to the face frame interval of a pass of the multi-pass enhanced video stream, determining a face frame interval adjacent to the i-th reference event frame interval as an initial extended frame interval for the multi-pass extended video stream for each first auxiliary camera position; To update the initial extended frame interval, starting from the face frame interval adjacent to the i-th reference event frame interval, linking at least a portion of one face frame interval of a first auxiliary camera position other than the first auxiliary camera position to the initial extended frame interval along a direction of decreasing or increasing frame identifiers; cyclically updating the initial extended frame interval until there are no more face frame intervals at the first auxiliary camera positions other than the first auxiliary camera position corresponding to the initial extended frame interval; determining the updated initial extension frame interval as the extension frame interval; Including, The broadcast production method according to claim 5.

7. The at least one reference event frame interval includes an i-th reference event frame interval and an (i+1)-th reference event frame interval, where i is an integer equal to or greater than 1, and a start frame identifier and an end frame identifier of the i-th reference event frame interval are respectively set to s i and e i and the (i+1)th start frame identifier is s i+1 and determining a local sequence of each reference event frame interval according to a correspondence between the event and a camera position identifier; determining an initial local sequence of the i-th reference event frame interval according to the correspondence between the event and the camera position identifier, wherein a start frame identifier and an end frame identifier of the initial local sequence are respectively set to s i and e i Steps e i and s i+1 if the i-th reference event frame interval is not adjacent, determining a playback type according to the event corresponding to the i-th reference event frame interval; obtaining at least one path of a playback video stream corresponding to the playback type; extending the initial local sequence according to at least one path of the playback video stream to obtain the local sequence for the i-th reference event frame interval; Including, The broadcast production method according to claim 1.

8. the step of extending the initial local sequence comprises: generating at least one playback sequence according to at least one path of the playback video stream, each playback sequence comprising: i and s i+1 and a frame identifier corresponding to the camera position identifier; extending the initial local sequence by using the at least one playback sequence; Including, The broadcast production method according to claim 7.

9. the playback type includes a first playback type; generating at least one playback sequence according to at least one path of the playback video stream; Under a condition that the playback type is the first playback type, performing event recognition on at least one path of the playback video stream to obtain at least one auxiliary event frame interval, wherein the auxiliary event frame interval includes frame identifiers of a plurality of consecutive images in which the event corresponding to the i-th reference event frame interval occurs; generating the at least one playback sequence according to the at least one auxiliary event frame interval; Including, The broadcast production method according to claim 8.

10. generating at least one playback sequence according to the at least one auxiliary event frame interval; ranking the at least one auxiliary event frame interval according to a total number of frames and a weight of each auxiliary event frame interval; generating the at least one playback sequence according to the ranking result; Including, The broadcast production method according to claim 9.

11. the playback type includes a first playback type; The step of obtaining at least one path of a playback video stream corresponding to the playback type includes: Under the condition that the playback type is the first playback type, from at least one first auxiliary camera position, i -m and e i +n as at least one path of the playback video stream, wherein both m and n are integers equal to or greater than 0. The broadcast production method according to claim 7.

12. the playback type includes a second playback type; The step of obtaining at least one path of a playback video stream corresponding to the playback type includes: s' according to the reference video stream under the condition that the playback type is the second playback type. i and e' i acquiring a camera position angle corresponding to each frame image between determining an area where the event occurs corresponding to the i-th reference event frame interval according to each camera position angle; From at least one second auxiliary camera position within the area, s i and e i and obtaining a video stream between the video stream and the playback video stream as at least one path of the playback video stream; Including, The broadcast production method according to claim 7.

13. the at least one reference event frame section includes an i-th reference event frame section and an (i+1)-th reference event frame section, where i is an integer equal to or greater than 1; The step of generating a broadcast production sequence includes: The end frame identifier E of the local sequence of the i-th reference event frame interval i is the start frame identifier S of the local sequence in the (i+1)th reference event frame interval. i+1 generating a supplemental sequence under the condition that the supplemental sequence is not adjacent to the camera position and E i and S i+1 and a frame identifier of each frame image located between E i and S i+1 and the camera position of each frame image located between is a third auxiliary camera position; merging the local sequence and the supplemental sequence to obtain the broadcast production sequence; Including, A broadcast production method according to any one of claims 1 to 4 and 6 to 12.

14. 14. The broadcast production method of claim 13, wherein the reference camera position is used to provide a close-up video stream of a dribbling player, a first auxiliary camera position is used to provide a close-up video stream at a different angle on the pitch, a second auxiliary camera position is used to provide a standard video stream at a different angle on the pitch, and the third auxiliary camera position is used to provide a standard video stream at a spectator's viewing angle.

15. generating a broadcast production video; Acquiring the frame image corresponding to the broadcast production sequence according to the broadcast production sequence and the video stream of the camera position corresponding to the camera position identifier of the broadcast production sequence; coding the frame images to obtain the broadcast-produced video; The broadcast production method according to claim 1, comprising:

16. an acquisition module configured to acquire a reference video stream from a reference camera position; an event recognition module configured to perform event recognition on the reference video stream to obtain at least one reference event frame interval, each reference event frame interval corresponding to an event, and each reference event frame interval including frame identifiers of multiple consecutive images in which the same event occurs; a determination module configured to determine a local sequence for each reference event frame interval according to a correspondence between the event and a camera position identifier, the local sequence including a camera position identifier for each frame image of a video to be played corresponding to the reference event frame interval, and a frame identifier corresponding to the camera position identifier; a first generation module configured to generate a broadcast production sequence according to the local sequence; a second generating module configured to generate a broadcast rendition video according to the broadcast rendition sequence and a video stream of a camera position corresponding to the camera position identifier of the broadcast rendition sequence; A broadcasting production device comprising:

17. Memory and a processor coupled to the memory, the processor configured to perform the broadcast directing method of any one of claims 1 to 15 based on instructions stored in the memory; and A broadcasting production device comprising:

18. The broadcast production device according to any one of claims 16 to 17, at least one camera configured to generate a video stream and transmit the video stream to the broadcast production device; A broadcast production system comprising:

19. 16. A computer-storable medium having stored thereon computer program instructions which, when executed by a processor, implements the broadcast directing method of any one of claims 1 to 15.

20. 16. A computer program comprising instructions which, when executed by a processor, cause the processor to perform a broadcast directing method according to any one of claims 1 to 15.

Images