Multi-camera automatic photographic method and system

Abstract

A multi-lens automatic camera movement shooting method includes the following steps: Shooting people and objects with multiple cameras, obtaining person information and object information, wherein the multiple cameras include a main camera and secondary cameras; detecting whether the person and object are in physical contact using the main camera; when the main camera detects physical contact between the person and object, selecting one of the multiple cameras to shoot based on the person information, and shooting with first shooting parameters based on the person information and object information; and when the person information changes, selecting another of the multiple cameras to shoot based on the person information, and shooting with second shooting parameters based on the person information and object information, wherein the second shooting parameters are different from the first shooting parameters.

Description

Technical Field

[0001] This invention relates to a shooting method and system, and more particularly to a multi-lens automatic camera movement shooting method and system. Prior Technology

[0002] The broadcast is a collaborative effort by a team of people with different roles; the photography team consists of multiple photographers, each with their own tasks to capture various exciting moments on the field; the director, as the soul of the broadcast, is responsible for switching appropriate shots based on the angles of the photographers. For the currently popular personal live streaming, this requires a relatively large amount of human and material resources.

[0003] In addition, single-camera shooting results are rather monotonous and have many blind spots. Automated video production systems, on the other hand, require manual operation, making it impossible for a single person to handle all shooting tasks.

[0004] To address the challenges of today's popular personal live streaming, where the resources of traditional film crews are limited, the industry has been racking its brains for solutions, but a suitable approach has yet to be developed. Therefore, automating multi-camera shooting and camera movement is a crucial research topic and a pressing need for improvement in the field. Summary of the Invention

[0005] This invention proposes a multi-lens automatic camera movement shooting method and system, which improves upon the problems of previous technologies.

[0006] In some embodiments of the present invention, the multi-lens automatic camera movement shooting method proposed in the present invention includes the following steps: shooting people and objects with multiple cameras and obtaining person information and object information, wherein the multiple cameras include a main camera and at least one secondary camera; detecting whether the person and the object are in physical contact through the main camera; when the main camera detects that the person and the object are in physical contact, selecting one of the multiple cameras to shoot based on the person information, and shooting with a first shooting parameter based on the person information and the object information; and when the person information changes, selecting another of the multiple cameras to shoot based on the person information, and shooting with a second shooting parameter based on the person information and the object information, wherein the second shooting parameter is different from the first shooting parameter.

[0007] In some embodiments of the present invention, the multi-lens automatic camera movement shooting system proposed in the present invention includes multiple cameras, including a main camera and at least one secondary camera, which are communicatively connected to each other. The multiple cameras shoot people and objects, and obtain information about people and objects. The main camera detects whether people and objects are in physical contact. When the main camera detects that people and objects are in physical contact, the main camera selects one of the multiple cameras to shoot based on the person information, and shoots with a first shooting parameter based on the person information and object information. When the person information changes, the main camera selects another of the multiple cameras to shoot based on the person information, and shoots with a second shooting parameter based on the person information and object information, wherein the second shooting parameter is different from the first shooting parameter.

[0008] In summary, the technical solution of this invention has significant advantages and beneficial effects compared with the prior art. By employing the multi-lens automatic camera movement shooting method and system of this invention, multiple cameras can be used to automatically shoot and move the camera, resulting in the production of an interesting video. By using multiple cameras to film people and objects, obtaining information about people and objects, and automatically capturing the best shot for the current situation, tasks that previously required many people are now automatically completed, significantly reducing the enormous human and material costs.

[0009] The above description will be given in detail below with reference to the embodiments, and the technical solution of the present invention will be further explained. Simple Explanation of the Diagram

[0010] To make the above and other objects, features, advantages and embodiments of the present invention more apparent and understandable, the accompanying drawings are described below: Figure 1 is a block diagram of a multi-lens automatic camera movement shooting system according to some embodiments of the present invention; Figure 2 is a flowchart of a multi-lens automatic camera movement shooting method according to some embodiments of the present invention; Figure 3 is a flowchart of a multi-lens automatic camera movement shooting method according to some embodiments of the present invention; Figure 4 is a schematic diagram of a multi-lens automatic camera movement shooting method according to one of the embodiments of the present invention; Figure 5 is a schematic diagram of a multi-lens automatic camera movement shooting method according to one of the embodiments of the present invention; and Figure 6 is a schematic diagram of a multi-lens automatic camera movement shooting method according to some embodiments of the present invention. Implementation

[0011] To make the description of this invention more detailed and complete, reference can be made to the accompanying drawings and the various embodiments described below, in which the same numbers represent the same or similar elements. Furthermore, well-known elements and steps are not described in the embodiments to avoid unnecessarily limiting the invention.

[0012] Figure 1 is a block diagram of a multi-lens automatic camera movement shooting system 100 according to some embodiments of the present invention. As shown in Figure 1, the multi-lens automatic camera movement shooting system 100 includes multiple cameras 110 distributed in different locations to capture images from different perspectives. In some embodiments, any of the multiple cameras 110 may be a PTZ camera (Pan-Tile-Zoom camera), a wide-angle camera, or a combination thereof.

[0013] In terms of architecture, the multiple cameras 110 include a main camera 111 and secondary cameras 112 and 113, which are communicatively connected to each other. For example, the main camera 111 is communicatively connected to the secondary camera 112, and the secondary camera 113 is communicatively connected to the main camera 111. It should be understood that although Figure 1 illustrates two secondary cameras 112 and 113, this does not limit the invention; in practice, the number of secondary cameras can be one or more. In use, the main camera 111 and the secondary cameras 112 and 113 can simultaneously capture images of a person 190 and an object 120, which are then processed and analyzed by the main camera 111. For example, the object 120 can be a display device, such as an electronic display screen, whiteboard, blackboard, poster, etc.

[0014] In practice, for example, the main camera 111 may include a camera, a communication device, a processor, and a storage device, all electrically connected to each other. The communication device of the main camera 111 establishes wired or wireless communication with the secondary cameras 112 and 113. The storage device of the main camera 111 stores program instructions and / or artificial intelligence models. The processor of the main camera 111 executes the program instructions and / or artificial intelligence models to process and analyze the images from the multiple cameras 110. The communication device of the main camera 111 outputs the processed images to other devices (such as a receiver for live streaming). Alternatively, the main camera 111 may include a camera, a communication device, and an external device (such as a computer), all electrically connected to each other. The external device has the processor and storage device described above, and its functions will not be elaborated further. The secondary cameras 112 and 113 may have the same hardware architecture as the main camera 111 or a simplified hardware architecture.

[0015] In use, multiple cameras 110 film a person 190 and an object 120, and obtain information about the person 190 (such as face orientation, body orientation, etc.) and the object 120 (such as the amount of movement of the object 120, etc.). The main camera 111 detects whether the person 190 and the object 120 are in physical contact. When the main camera 111 detects that the person 190 and the object 120 are in physical contact, the main camera 111 selects one of the multiple cameras 110 (such as the secondary camera 112) to film based on the person information, and films with the first shooting parameters based on the person information and the object information. When the person information changes, the main camera 111 selects another of the multiple cameras 110 to film based on the person information, and films with the second shooting parameters based on the person information and the object information, wherein the second shooting parameters are different from the first shooting parameters. Therefore, the multi-lens automatic camera movement shooting system 100 can automatically complete the shooting and camera movement of multiple cameras 110 to create an interesting video. By using multiple cameras 110 to shoot people and objects, obtaining information about people and objects, it automatically captures the best shot for the current situation, automatically completing tasks that would otherwise require more people, thus reducing huge human and material costs.

[0016] To provide a more detailed explanation of the operation of the multi-lens automatic camera movement shooting system 100, please refer to Figures 1 and 2. Figure 2 is a flowchart of a multi-lens automatic camera movement shooting method 200 according to some embodiments of the present invention. It should be understood that, unless otherwise specified, the order of the steps mentioned in the embodiments of Figure 2 can be adjusted as needed, and they can even be executed simultaneously or partially simultaneously. In practice, for example, the multi-lens automatic camera movement shooting method 200 can be executed by the multi-lens automatic camera movement shooting system 100; for example, the main camera 111, in conjunction with the secondary cameras 112 and 113, executes the multi-lens automatic camera movement shooting method 200.

[0017] In the multi-lens automatic camera movement shooting method 200, multiple cameras 110 are used to shoot people and objects, and to obtain person information of person 190 and object information of object 120. The multiple cameras 110 include a main camera 111 and at least one secondary camera (such as secondary camera 112 and / or secondary camera 113). The main camera 111 detects whether the person 190 and the object 120 are in physical contact. When the main camera detects that the person and the object are in physical contact, one of the multiple cameras 110 is selected to shoot based on the person information, and the shooting is performed with a first shooting parameter based on the person information and the object information. When the person information changes, another of the multiple cameras 110 is selected to shoot based on the person information, and the shooting is performed with a second shooting parameter based on the person information and the object information, wherein the second shooting parameter is different from the first shooting parameter. In this way, the multi-lens automatic camera movement shooting method 200 can use multiple cameras 110 to shoot people 190 and objects 120, obtain the relationship between people 190 and objects 120, and then make different camera movements and switch between multiple camera shots according to the relationship.

[0018] In some embodiments, the person information of person 190 may be action type, face orientation, body orientation, position, or a combination thereof. In some embodiments, the object information of object 120 may be size, position, amount of movement, type, displayed content, or a combination thereof. In some embodiments, physical contact may be holding, handwriting, pointing to displayed content, or a combination thereof.

[0019] In some embodiments, either the first shooting parameter or the second shooting parameter includes the PTZ value for composition, where PTZ is an abbreviation for Pan (horizontal movement), Tilt (vertical movement), or Zoom (zoom). For example, the first shooting parameter is the PTZ value of one of the plurality of cameras 110 selected above, and the second shooting parameter is the PTZ value of the other of the plurality of cameras 110 selected above. The second shooting parameter differs from the first shooting parameter, allowing for shooting and camera movement by multiple cameras to complete the shooting of an interesting video.

[0020] In some embodiments, if the main camera 111 detects that the person and the object are not in actual contact, or if the person or object information has not changed after a predetermined time, the camera switches to another of the multiple cameras to shoot, or one of the multiple cameras originally shot with a third shooting parameter, wherein the third shooting parameter is different from the first and second shooting parameters, in order to avoid monotonous footage.

[0021] As shown in Figures 1 and 2, in step S201, the main camera 111 and secondary cameras 112 and 113 simultaneously capture images, using artificial intelligence (AI) (e.g., a trained AI model) to detect the person 190 and their movements, as well as the object 120 (AI human detection, TV / blackboard detection, TV display motion detection). In step S202, it is determined whether the related movements of the person 190 and the object 120 are detected. In step S203, a close-up (e.g., a close-up of the person 190 and / or the object 120) is taken, or the best framing composition is presented. In step S204, different perspective shots are switched based on the action or time.

[0022] To provide a more detailed explanation of the operation of the multi-lens automatic camera movement shooting system 100, please refer to Figures 1-6. Figure 3 is a flowchart of a multi-lens automatic camera movement shooting method 300 according to some embodiments of the present invention, and Figures 4-6 are schematic diagrams of a multi-lens automatic camera movement shooting method 300 according to some embodiments of the present invention. It should be understood that, unless otherwise specified, the order of the steps mentioned in the embodiments of Figure 3 can be adjusted according to actual needs, and they can even be executed simultaneously or partially simultaneously. In practice, for example, the multi-lens automatic camera movement shooting method 300 can be executed by the multi-lens automatic camera movement shooting system 100; for example, the main camera 111, in conjunction with the secondary cameras 112 and 113, executes the multi-lens automatic camera movement shooting method 300.

[0023] As shown in Figures 3 and 4, in step S301, the main camera 111 and the secondary cameras 112 and 113 simultaneously capture images, using artificial intelligence (AI) (e.g., a trained AI model) to detect the person 490 and objects 420 and 450. In step S302, the associated actions of the person 490 and objects 420 and 450 are detected.

[0024] Specifically, the detection of actions related to people and objects is defined as follows: the person 190 must be in physical contact with the object (e.g., holding the object). Depending on the context, the user can specify or exclude specific objects or actions (e.g., holding a microphone, pointing, touching the display device, or writing, and the area of ​​the action must overlap with the content display device).

[0025] Additionally, the amount of movement of an object can be detected selectively through continuous images captured by any of the multiple cameras 110 to determine whether the object is a real object (e.g., a handheld item) or background footage. In some embodiments, as shown in Figure 4, the image 400A captured by the main camera 111 includes a person 490, an object 420 (e.g., a whiteboard), and an object 450 (e.g., a handheld item). Since the amount of movement of object 420 (e.g., the whiteboard) is zero in the continuous image, object 420 is determined to be background footage. Since object 450 moves with the hand of person 490 in the continuous image, object 450 is determined to be a real object.

[0026] As shown in Figures 3 and 4, in step S303, the posture of the person 490 picking up the object 450 (e.g., an item) is detected. In step S304, it is detected whether the held object 450 (e.g., an item) overlaps with the person 490. If so, in step S305, a close-up of the object 450 (e.g., an item) is taken, as shown in image 400B of Figure 4. In step S306, different perspective shots are switched according to the action or time, for example, switching between images 400B and 400C of Figure 4, to avoid monotony of the image.

[0027] Regarding the close-up shot in step S305, for example, the main camera 111 is used as the reference, and the secondary cameras 112 and 113 simultaneously shoot as the viewpoint selection. After a predetermined time, the person 490 holds the object 450 (e.g., the item) and displays it in close-up, as shown in image 400B of Figure 4.

[0028] Regarding the switching perspective in step S306: Based on action or time, different perspective shots are switched. If the person 490 does not perform any action (e.g., determined by an artificial intelligence model), then the person 490 is filmed. If the person 490 continues to perform an action, the perspective is randomly switched between the person 490 and a close-up of the object 450 (e.g., an item). Based on the positional relationship between the person 490 and the object 420 (e.g., a content display device), the best and second-best camera angles are determined (e.g., determined by an artificial intelligence model). If there is no change in action for a predetermined period of time, the best and second-best camera perspectives are automatically switched. For example, if the person 490 is facing the main camera 111, then the main camera 111 is the best camera. If the area of ​​the person 490 filmed by the secondary camera 113 is larger than the area of ​​the person 490 filmed by the secondary camera 112, then the secondary camera 113 is the second-best camera. However, this invention is not limited to this example.

[0029] Specifically, steps S303~S306, close-up of the handheld object: First, the main camera 111 and the secondary cameras 112 and 113 simultaneously capture and detect the person's information, with the main camera 111 detecting the action and the object in the hand; when it is detected that the person 490 is currently holding an object 450 (e.g., an item), the size and position of the object 450 are also detected. After a predetermined period of time (e.g., 2~3 seconds), the handheld object mode is entered, and the screen is switched to a close-up of image 400B (close-up of the object in the hand); after about another predetermined period of time (e.g., 5~8 seconds), the screen is switched, and the person 490 holding the object and the object 450 are simultaneously in the frame (the best viewing angle is selected based on the direction the person 490 is facing), for example, switching to image 400C to avoid monotony.

[0030] On the other hand, if it is detected in step S304 that the held object 450 (e.g., an item) does not overlap with the person 490, the best framing composition is presented in step S308. Specifically, when it is detected that the person 490 puts down the object 450, the mode of holding the item is exited and the search for the best framing composition is resumed (e.g., the best framing composition is determined through an artificial intelligence model).

[0031] As shown in Figures 3 and 5, in step S307, a pointing action is detected, for example, in image 500A captured by secondary camera 113, the hand 591 of person 590 points to the display content 522 of object 520. In step S308, it is detected whether object 520 (e.g., display device) overlaps with person 590. If not, in step S309, the optimal framing composition is presented (e.g., determined by an artificial intelligence model), as shown in image 500B. For example, when person 590 (e.g., speaker) points to object 520 (e.g., display device) or is facing away from main camera 111 and secondary cameras 112 and 113 (e.g., writing), the close-up of display content 522 is the main focus, with the secondary camera 113, where display content 522 is not obstructed, as the optimal angle. The optimal framing composition presented in image 500B is entirely based on display content 522.

[0032] On the other hand, as shown in Figures 3 and 6, in step S307, a pointing action is detected, for example: in image 600A captured by secondary camera 113, the hand 691 of person 690 touches object 620 (e.g., display device). In step S308, it is detected whether object 620 (e.g., display device) overlaps with person 690. If so, in step S310, the optimal composition of the display device is performed (e.g., determined by an artificial intelligence model), as shown in image 600B. For example, person 690 (e.g., speaker) is looking at secondary camera 112. Based on the relative position of person 690 (e.g., speaker) and display content 622, secondary camera 112 is selected to shoot, with the optimal secondary camera 112 not obstructing display content 622. The composition is based on display content 622 and person 690 (e.g., speaker), taking into account the relative position of person 690 and object 620 (e.g., left and right positions).

[0033] Combining Figures 3, 5, and 6, the interaction between the character and the display device is as follows: First, the main camera 111 and the secondary cameras 112 and 113 simultaneously capture and detect character information, with the main camera detecting movement and the display device detecting movement. When the character is detected to be pointing or touching, and the character and the display device are simultaneously detected to overlap, after a preset time (e.g., 2-3 seconds), the display device interaction mode is entered, and different shooting screens are switched according to the following situations. For example, when the character 590 in Figure 5 is facing the object 520 (e.g., the display device), the focus is on a close-up of the content 522. When the character 690 in Figure 6 is facing the secondary camera 112, both the character 690 and the object 620 (e.g., the display device) are captured simultaneously, and the viewing angle is determined by referring to their relative positions. After approximately a preset time (e.g., 5-8 seconds), the screen is switched, alternating between the best and second-best shots (determined by the artificial intelligence model). When the system detects that a person has left the display area, it exits the display interaction mode and returns to finding the best framing and composition.

[0034] In summary, the technical solution of this invention has significant advantages and beneficial effects compared with the prior art. By employing the multi-lens automatic camera movement shooting method 200, 300 and the multi-lens automatic camera movement shooting system 100 of this invention, multiple cameras 110 can be used to automatically shoot and move the camera, resulting in the production of an interesting video. By using multiple cameras 110 to film people 190 and objects 120, obtaining information about the people and objects, the best shot for the current situation can be automatically captured, automatically completing tasks that would otherwise require many people, thus reducing substantial human and material costs.

[0035] Although the present invention has been disclosed above by way of embodiments, it is not intended to limit the present invention. Anyone skilled in the art can make various modifications and refinements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

[0036] 100: Multi-lens automatic camera movement shooting system 110: Camera 111: Main Camera 112, 113: Secondary cameras 120, 420, 450, 520, 620: Object 190, 490, 590, 690: Characters 200, 300: Multi-lens automatic camera movement shooting method 400A, 400B, 400C: Imaging 500A, 500B, 600A, 600B: Imaging 522, 622: Display content 591, 691: Hand S201~S204, S301~S310: Steps

Claims

1. A multi-lens automatic camera movement shooting method, comprising the following steps: shooting a person and an object with multiple cameras, and obtaining person information of the person and object information of the object, wherein the multiple cameras include a main camera and at least one secondary camera; detecting whether the person and the object are in physical contact through the main camera; when the main camera detects that the person and the object are in physical contact, selecting one of the multiple cameras to shoot based on the person information, and shooting with a first shooting parameter based on the person information and the object information; and when the person information changes, selecting another of the multiple cameras to shoot based on the person information, and shooting with a second shooting parameter based on the person information and the object information, wherein the second shooting parameter is different from the first shooting parameter.

2. The multi-lens automatic camera movement shooting method as described in claim 1, wherein any one of the plurality of cameras is a PTZ camera, a wide-angle camera, or a combination thereof.

3. The multi-lens automatic camera movement shooting method as described in claim 1, wherein the person information is action type, face orientation, body orientation, position, or a combination thereof.

4. The multi-lens automatic camera movement shooting method as described in claim 1, wherein the object information is size, position, amount of movement, type, display content, or a combination thereof.

5. The multi-lens automatic camera movement shooting method as described in claim 1, wherein the physical contact is handheld, handwriting, pointing a finger at the displayed content, or a combination thereof.

6. The multi-lens automatic camera movement shooting method as described in claim 1 further includes: detecting the amount of movement of the object and determining whether the object is a real object or a background scene.

7. The multi-lens automatic camera movement shooting method as described in claim 1, wherein either the first shooting parameter or the second shooting parameter includes the PTZ value of the composition.

8. The multi-lens automatic camera movement shooting method as described in claim 1 further includes: when the main camera detects that the person and the object are not in physical contact, or when the information of the person or the object has not changed after a predetermined time, switching to shooting by the other of the multiple cameras or shooting by the other of the multiple cameras with a third shooting parameter, wherein the third shooting parameter is different from the first and second shooting parameters.

9. The multi-lens automatic camera movement shooting method as described in claim 1 further includes: using artificial intelligence to pre-specify or exclude the object.

10. A multi-lens automatic camera movement shooting system, comprising: a plurality of cameras including a main camera and at least one secondary camera, communicatively connected to each other; the plurality of cameras shooting a person and an object, and acquiring person information of the person and object information of the object; the main camera detecting whether the person and the object are in physical contact; when the main camera detects that the person and the object are in physical contact, the main camera selects one of the plurality of cameras to shoot based on the person information, and shoots with a first shooting parameter based on the person information and the object information; when the person information changes, the main camera selects another of the plurality of cameras to shoot based on the person information, and shoots with a second shooting parameter based on the person information and the object information, wherein the second shooting parameter is different from the first shooting parameter.

Images