A method and device for outputting video stream images and a camera device

Abstract

The present invention provides a method, device and camera equipment for outputting video stream images, the method comprising obtaining an original video stream image, performing object recognition on the original video stream image to determine the recognition object contained in the original video stream image; the recognition object includes an object object and / or a human object; selecting at least one target object from the recognition object, obtaining a video stream close-up image of the target object; and outputting the video stream close-up image to a display terminal to perform a close-up display of the target image. The method of the present invention can display the features of a specific recognition object by obtaining the original video stream image and identifying the object or human recognition object contained therein, and can realize free switching between the original video stream image and the close-up video stream image by automatically tracking and identifying the recognition object, thereby assisting users in realizing different scene requirements such as live broadcast and conference.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for outputting video stream images and a camera device. Background Art

[0002] The acquisition and output of video streams is an effective way to display certain scene conditions. Generally, when outputting video streams, one or more cameras are used to directly acquire video data in the scene and directly output it to a screen or terminal for display. If a person or an object in the scene needs to be enlarged, or if certain people / objects need to be enlarged, it is usually necessary to manually determine the object to be enlarged, and switching is inconvenient. The above method not only consumes a certain amount of human resources, but also is inconvenient to use due to manual operation, which affects the user experience. Summary of the invention

[0003] In view of the above problems, the present invention proposes a method, device and camera equipment for outputting video stream images, which overcome the above problems or at least partially solve the above problems.

[0004] According to one aspect of the present invention, a method for outputting a video stream image is provided, comprising:

[0005] Acquire an original video stream image, and perform object recognition on the original video stream image to determine a recognition object contained in the original video stream image; the recognition object includes an object and / or a human object;

[0006] Selecting at least one target object from the identified objects, and acquiring a video stream close-up image of the target object;

[0007] The video stream close-up image is output to a display terminal to display a close-up of the target image.

[0008] Optionally, selecting at least one target object from the identified objects includes:

[0009] selecting at least one target object from the identified objects in response to a user's voice command; and / or,

[0010] Collecting user voice information, predicting user intent based on the voice information, and selecting at least one target object from the identified objects based on the predicted user intent; and / or,

[0011] Analyzing the gestures of each of the identified objects based on the original image of the video stream, and selecting at least one target object from the identified objects according to the gestures of each of the identified objects; and / or,

[0012] At least one target object is selected according to the motion trajectory of each of the identified objects in the original image of the video stream.

[0013] Optionally, parsing the gestures of each of the identified objects based on the original image of the video stream, and selecting at least one target object from the identified objects according to the gestures of each of the identified objects includes:

[0014] The human skeleton of each identified object is obtained by detecting key points of the human body, and an association relationship is established with each identified object;

[0015] The hand features in the human skeleton corresponding to each of the identified objects are tracked, and when it is determined based on the hand features that any of the identified objects performs a preset gesture action, the identified object performing the preset gesture action is taken as a target object.

[0016] Optionally, selecting at least one target object according to the motion trajectory of each of the identified objects in the original image of the video stream includes:

[0017] Track the motion trajectory of any identified object in the original image of a continuous video stream;

[0018] When it is determined that the motion trajectory satisfies a preset motion law, the identified object is taken as a target object.

[0019] Optionally, after acquiring the original image of the video stream, the method further comprises: outputting the original image of the video stream to an operation terminal;

[0020] The selecting at least one target object from the identified objects includes: selecting at least one target object from the identified objects in response to a selection instruction triggered by the user based on the operating end.

[0021] Optionally, acquiring a video stream close-up image of the target object includes:

[0022] Performing cropping and scaling processing on the original video stream image to generate a video stream close-up image of the target object; or,

[0023] A close-up image of the video stream of the target object is obtained through a close-up camera.

[0024] Optionally, acquiring the video stream close-up image of the target object by using a close-up camera includes:

[0025] Obtaining the distance between the original camera that captures the original image of the video stream and the target object;

[0026] If it is determined that the distance is within a set threshold range, using the original camera to obtain a close-up image of the video stream of the target object;

[0027] If it is determined that the distance exceeds the set threshold range, a close-up camera is used to obtain a close-up image of the video stream of the target object.

[0028] Optionally, after outputting the video stream close-up image to a display terminal to display the target image in close-up, the method further includes:

[0029] receiving a close-up release instruction for stopping close-up display of the target object;

[0030] Identify the close-up release object corresponding to the close-up release instruction in the target object; determine a matching close-up release method according to the number and / or object type of the close-up release object, and stop close-up display of the close-up release object according to the close-up release method; or,

[0031] The original image of the video stream is output to the display terminal for display.

[0032] According to another aspect of the present invention, a device for outputting a video stream image is provided, the device comprising:

[0033] The original image acquisition module is used to acquire the original image of the video stream, and perform object recognition on the original image of the video stream to determine the recognition object contained in the original image of the video stream; the recognition object includes an object and / or a human object;

[0034] A close-up recognition module, used to select at least one target object from the recognition objects and obtain a video stream close-up image of the target object;

[0035] The display module is used to output the close-up image of the video stream to a display terminal to display a close-up of the target image.

[0036] According to another aspect of the present invention, a system for outputting video stream images is provided. The system includes at least one camera, at least one display terminal, and a video stream image output device connected to the camera and the display terminal.

[0037] According to yet another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store program codes, and the program codes are used to execute any of the above methods.

[0038] According to another aspect of the present invention, there is further provided a camera device, the camera device comprising a processor and a memory:

[0039] The memory is used to store program code and transmit the program code to the processor;

[0040] The processor is configured to execute any of the above methods according to instructions in the program code.

[0041] The present invention provides a method, device and camera equipment for outputting video stream images. The method of the present invention obtains an original video stream image and identifies an object or human body recognition object contained therein. After selecting a target object, the method generates a video stream close-up image of the target object to achieve a magnified close-up of the target object, so as to meet the video playback requirements of application scenarios such as conferences and live broadcasts, realize the prominent display of key objects or human bodies in different application scenarios, meet the user's personalized needs for video playback, and can quickly locate the object that needs a close-up without manual operation, and output a clear close-up image.

[0042] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below.

[0043] Based on the following detailed description of specific embodiments of the present invention in conjunction with the accompanying drawings, those skilled in the art will become more aware of the above and other objects, advantages and features of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0045] Figure 1 A schematic diagram showing a flow chart of a method for outputting video stream images according to an embodiment of the present invention is shown;

[0046] Figure 2 A schematic diagram showing a flow chart of a method for outputting video stream images according to another embodiment of the present invention is shown;

[0047] Figure 3 A schematic diagram showing the structure of a device for outputting video stream images according to an embodiment of the present invention is shown;

[0048] Figure 4 A schematic diagram of the structure of a video stream image output system according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0049] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0050] The embodiment of the present invention provides a method for outputting a video stream image. Figure 1 As shown, the method for outputting video stream images provided by the embodiment of the present invention may at least include the following steps S101 to S103.

[0051] S101, obtaining an original image of a video stream, and performing object recognition on the original image of the video stream to determine a recognition object contained in the original image of the video stream; the recognition object includes an object and / or a human object.

[0052] The original video stream image to be output in this embodiment can be shot in real time based on a depth camera or an ordinary camera, or can be a pre-stored uncropped video stream image. After the original video stream image to be output is obtained, it can be detected and identified to determine the identification object contained in the original video stream image, wherein the determined identification object can be an object object or a human object. Specifically, the face detection method can be used to intelligently identify each human object contained in the original video stream image. In addition, feature extraction, feature selection and matching methods can also be used to identify the object object contained in the original video stream image, such as a complete commodity, article, etc. Taking a meeting in a conference room as an example, the identification object in the original video stream image may be multiple participants in the meeting; in terms of shopping live broadcast, the identification object in the original video stream image may be each anchor, recommended commodity, assistant anchor, etc. The number and type of identification objects in the original video stream image shot in different scenes may be different, and the embodiment of the present invention does not limit this. Among them, the original video stream image is a full-frame image obtained by the camera, and the resolution and screen size of the original video stream image can be freely set according to different display terminals.

[0053] The target detection of this embodiment is to obtain the categories and target frames of different objects; based on the detected identified objects, when the target is subsequently tracked, the detected targets can be tracked, each target is assigned a unique identifier, and information such as trajectory can be obtained. The target frame corresponding to each identified object obtained based on the target detection includes the size and position information of the corresponding object, so that the designated object can be quickly and accurately zoomed in.

[0054] S102, selecting at least one target object from the identified objects, and obtaining a video stream close-up image of the target object.

[0055] There may be multiple target objects in the original video stream image. For different application scenarios, different numbers and types of objects may be taken in close-up. Therefore, it is necessary to select at least one target object based on the original video stream image and obtain a video stream close-up image of the target object, so that one target object or multiple (two or more) target objects can be displayed as the region of interest in the main area (for example, displayed in the center), and the target object can be zoomed in and close-up to zoom in and display the local details of the target object. The close-up in this embodiment can be understood as the corresponding zoom display of the target object.

[0056] In this embodiment, when obtaining a close-up image of a video stream of a target object, the following two methods may be included:

[0057] The first method is to crop and scale the original video stream image to generate a close-up video stream image of the target object. If the selected target object is a single object, the display area corresponding to the target object in the original video stream image is identified; based on the display area, the original video stream image is cropped and scaled to generate a close-up video stream image of the target object. When taking a close-up of the target object, it can be a zoomed-in close-up of multiple objects or a single object, which can be set according to different scene requirements.

[0058] If multiple target objects are selected, identify the display area corresponding to each target object in the original image of the video stream, and determine the boundary area in the display area corresponding to each target object in the original image of the video stream; crop and scale the original image of the video stream based on the boundary area to generate a video stream close-up image of the target object.

[0059] The second method is to obtain a close-up image of the target object through a close-up camera.

[0060] In this embodiment, the original video stream image and the close-up video stream image are acquired by independent cameras. After the target object is determined based on the original video stream image, the target position corresponding to the target object can be located first, and the close-up camera can be used to acquire the video stream close-up image of the target object according to the target position. The close-up camera does not need to be full-frame, first locates the target position, then selects a certain range of the target area according to the 16:9 frame and crops it out, and then scales the cropped resolution to 2K for output.

[0061] Optionally, multiple close-up cameras are provided in the application scenario. When obtaining a close-up image of the video stream of the target object through the close-up camera, the distance between the close-up camera currently in use and the target object can be determined first. Assuming that the distance is within the set threshold range, the current close-up camera can be used to locate and close-up the target object. Assuming that the distance exceeds the set threshold range, the close-up camera is used to obtain a close-up image of the video stream of the target object. This embodiment takes the use of segmented switching lenses as an example: target distance <5.0m: camera A is used to locate and close-up the target alone; target distance >5.0m: after locating the target with camera A, the corresponding camera B is switched according to the left and right positions of the target to take a close-up of the target. At the same time, the close-up area can be output as a 1080P or 720P image in 16:9. This embodiment adopts a segmented lens switching method, which can perform close-up processing on target objects in different positions, so that the obtained close-up image of the video stream of the target object can be clear and smooth.

[0062] S103, outputting the close-up image of the video stream to a display terminal to display a close-up of the target image.

[0063] After obtaining the close-up image of the video stream of the target object, it can be output to the display terminal, and the display terminal displays the close-up of the target image. In practical applications, before displaying the close-up image of the video, the display terminal may be in a state of displaying the original image of the video stream. At this time, the original image of the video stream can be updated to the close-up image of the video stream for display. The display terminal of this embodiment can be a terminal such as a mobile phone, a tablet computer, and a display screen.

[0064] In this embodiment, when the above step S103 outputs the video stream close-up image to the display terminal for display, the interface parameters of the display interface corresponding to the display terminal can be obtained first; the size of the output image is adjusted according to the interface parameters, and the adjusted video stream close-up image is output, so that the output video stream close-up image can be adjusted to the screen or interface size of the display terminal. For example, if the display terminal is a mobile phone, the video stream close-up image can be adjusted to the screen size when the mobile phone is displayed in portrait mode, so that the user can watch it with the mobile phone in hand.

[0065] An embodiment of the present invention provides a method for outputting a video stream image. By acquiring an original video stream image and identifying an object or human body recognition object contained therein, after selecting a target object, a zoomed-in close-up of the target object is achieved by generating a video stream close-up image of the target object, so as to meet the video playback requirements of application scenarios such as conferences and live broadcasts, and to achieve highlight display of key objects or human bodies in different application scenarios, thereby meeting the user's personalized needs for video playback.

[0066] The above step S102 mentions that at least one target object needs to be selected from the identified objects. In an optional embodiment of the present invention, the target object can be selected in different ways. The different ways are described in detail below.

[0067] 1. Voice recognition method

[0068] At least one target object is selected from the recognized objects in response to a user's voice instruction.

[0069] The user can specify the target object by voice. When selecting the target object according to the user's voice command, the user's voice information can be first obtained and analyzed. When it is recognized that the user's voice information contains preset instructions such as "zoom in" and "close-up", the target object contained in the user's voice information can be further identified. For example, if the user sends the instruction "zoom in to item A", at this time, it can be determined that the target object is item A, and then the position of item A in the original image of the video stream is determined through feature comparison and other processes, and a close-up of it is taken.

[0070] In actual applications, only some people may have the authority to issue voice commands. Therefore, when a voice command is detected, voiceprint recognition can be performed first. If it is determined that the user who issued the voice command has the authority, the target object will be displayed in close-up. Otherwise, the voice command will be ignored.

[0071] 2. Intention prediction method

[0072] The user's voice information is collected, and the user's intention is predicted based on the voice information, and at least one target object is selected from the recognition objects according to the predicted user intention.

[0073] For example, in a live broadcast scenario, assuming that the host is currently explaining a product, by collecting the host's voice information and predicting that the host is about to demonstrate the details of the product, the product can be automatically selected as the target object and displayed in close-up. In this embodiment, the intelligent model established and trained by the neural network can realize the prediction and analysis of user intentions based on voice information.

[0074] 3. Gesture recognition method

[0075] The gestures of each recognized object are analyzed based on the original image of the video stream, and at least one target object is selected from the recognized objects according to the gestures of each recognized object.

[0076] In this embodiment, a preset gesture for triggering a close-up with one key can be pre-set. After acquiring the original video stream image, by performing image recognition on each object in the original video stream image, when any object is detected to be making a preset gesture, the object can be taken as the target object for close-up. Of course, if multiple objects make preset gestures within a certain time range, the multiple objects making the preset gestures can all be taken as feature objects for close-up.

[0077] In an optional embodiment of the present invention, it is also possible to respond only to gestures of fixed objects. That is, when determining whether any recognized object triggers a gesture, first determine whether the recognized object is a pre-set fixed object that needs gesture recognition. If so, further determine whether the gesture corresponding to the recognized object is a preset gesture. When the judgment result is yes, respond to the gesture of the recognized object, treat the recognized object as the target object, and obtain a close-up image of the video stream of the object. Assuming that it is determined that the recognized object is not a pre-set fixed object that needs gesture recognition, ignore the gesture of the object, and continue to monitor the gestures of other recognized objects in the original image of the video stream. In this embodiment, by identifying gestures of fixed objects, the system computing power can be reduced to avoid affecting the user experience due to misidentification of the target object.

[0078] Optionally, when selecting a target object based on analyzing the gestures of each recognized object based on the original image of the video stream, the following steps A1 to A2 may be specifically included.

[0079] A1, the human skeleton of each recognition object is obtained by detecting the key points of the human body, and an association relationship is established with each recognition object. As mentioned above, each recognition object in the original image can have a unique corresponding identifier, and the recognized face image and human skeleton can both be associated with the corresponding recognition object.

[0080] A2, track the hand features in the human skeleton corresponding to each recognition object, and when judging any recognition object to make a preset gesture based on the hand features, the recognition object making the preset gesture is taken as the target object. In this embodiment, a hand tracker can be used to track the hands of each recognition object, so as to obtain the hand features of each recognition object. At the same time, a portrait tracker and a hand tracker can be used to track the portraits of each recognition object. When judging that the recognition object makes a preset gesture, in addition to verifying the category of the preset gesture, it is also necessary to judge the duration of the gesture (such as satisfying 1 second) and the stable state of the recognition object. When it is judged that the head of the recognition object is stable and the gesture is stable, the recognition object can be determined as the target object, thereby avoiding the user's short-term misoperation and misidentification of the user's intention. Optionally, when judging the stability of the gesture, the hand features can be input into a pre-created classification model, and the classification model can judge that the user has made a preset gesture for a certain number of times, so that it can be determined that the user has made a stable preset gesture, and then the object is determined to be the target object and a close-up is taken. If it is unstable, the count is re-counted. Optionally, when performing gesture recognition, the gesture recognition candidate area of ​​the recognition object can be determined first, so as to further track and recognize the user's gesture within the gesture recognition candidate area. The gesture candidate area can be within the user's shoulders, above the mouth but not above the eyebrows or forehead, and not below the upper body. By setting the gesture candidate area, the reliability of gesture control can be improved, misjudgment can be reduced, and the computing power consumption can be greatly reduced compared to conventional gesture control solutions, making it easier to implement on mid- and low-end platforms.

[0081] Taking a meeting scenario as an example, suppose there are multiple people attending a meeting in a conference room. When one of the users is speaking and needs a close-up, the user only needs to make a preset gesture to achieve a close-up display of the user, so as to highlight the person who is speaking at this time and attract the attention of the participants. When performing portrait and hand tracking, both can be based on the SORT (Simple Online and Realtime Tracking) multi-target tracking algorithm. In actual applications, only specific people may be allowed to issue gesture commands. Therefore, gesture classification and gesture command recognition can be performed only for some recognition objects, thereby reducing computing power consumption.

[0082] 4. Trajectory prediction method

[0083] At least one target object is selected based on the motion trajectory of each identified object in the original image of the video stream. Optionally, selecting at least one target object based on the motion trajectory of each identified object in the original image of the video stream includes: tracking the motion trajectory of any identified object in the continuous original image of the video stream; when it is determined that the motion trajectory meets the preset motion law, the identified object is used as the target object. Taking the live broadcast scene as an example, assuming that the anchor is holding an object, when it is determined that the anchor is holding the object and moving it toward the camera, it can be understood that the anchor needs to focus on displaying the object. At this time, the object can be used as the target object and a close-up can be taken.

[0084] In addition to the above, the target object can also be selected through the operator's instructions. Specifically, after the original image of the video stream is obtained in the above step S101, the original image of the video stream can also be output to the operation end; when selecting at least one target object from the identified objects, at least one target object can be selected from the identified objects in response to the selection instruction triggered by the user based on the operation end.

[0085] In actual applications, the original image of the video stream can also be output to the operation end, such as a video surveillance device. The operator at the operation end can not only view the camera display interface, but also select the content he wants to select in the display interface. The identified object in the frame can be used as the target object for close-up display.

[0086] The above introduces the process of taking a close-up of the target object. In the present embodiment, the close-up image of the video stream can also be switched to the original image of the video stream. Optionally, after the close-up image of the video stream is output to the display terminal in the above step S103 to display the target image in close-up, a close-up release instruction to stop displaying the close-up of the target object can also be received; the close-up release object corresponding to the close-up release instruction in the target object is identified; a matching close-up release method is determined according to the object number and / or object type of the close-up release object, and the close-up display of the close-up release object is stopped according to the close-up release method; or the original image of the video stream is directly output to the display terminal for display.

[0087] The close-up contact instruction in this embodiment can be a voice instruction or a gesture instruction, similar to the method when selecting the target object. It can support one-key gesture to switch back to the panoramic view, or it can return to the default mode with one key; when releasing the close-up, the close-up can be released for a single target object, or for multiple target objects, and it can be set according to different needs. When releasing the close-up for multiple target objects, the position of the target object to be kept in close-up can be re-determined by adjusting the camera and adjusting the close-up image of the video stream. In addition, the above embodiment also introduces judging the target object by the motion trajectory. In this case, it can be further judged according to the running trajectory that when the host holds the object and moves in the direction of the original camera, the close-up of the object can be touched and the playback mode of the original image of the original video stream can be switched.

[0088] The following describes in detail a method for outputting a video stream image provided by an embodiment of the present invention through an embodiment.

[0089] S201, using camera A to capture the original image of the video stream, and output the original image of the video stream to a display terminal, which may be a computer or a mobile phone;

[0090] S202, obtaining an original image of a video stream, and performing object recognition on the original image of the video stream to determine an identifiable object contained in the original image of the video stream; in this embodiment, the identifiable object in the original image of the video stream may include object A, object B, and object C, and an identification identifier is assigned to each object; wherein object A and object B may be the host and assistant in a live broadcast scene, and object C is an object for explaining to the host and assistant;

[0091] S203, obtaining the human skeletons of object A and object B through human key point detection, and establishing association relationships with the identification identifiers of object A and object B respectively; at this time, the position changes of object A, object B and object C, as well as the hand features of object A and object B can be tracked by the tracker;

[0092] S204, when it is determined that the object A is holding the object C, and the object C is moving steadily toward the camera, determining that the object C is the target object;

[0093] S205, determine the distance between camera A and object C. If the distance between camera A and object C is less than 5.0m, execute step S206: If the distance between camera A and object C is greater than or equal to 5.0m, execute step S207. Optionally, if the distance information can be obtained directly (for example, the device has a distance sensor, or uses a depth camera-TOF / structured light), the distance comparison is performed directly. If the distance information cannot be obtained directly, the target frame size can be compared: Based on the principle of "near big and far small", the face size of object C is compared with the preset target size (at the preset distance, such as 5m). If it is smaller than the preset size, it means that the preset distance is exceeded.

[0094] S206, using camera A to locate the target and collect a close-up image of the video stream;

[0095] S207, after positioning the target with camera A, switch the corresponding camera B according to the left and right positions of the target, and use camera B to capture a close-up image of the video stream.

[0096] S208, outputting the close-up image of the video stream to a display terminal to display a close-up of the target image.

[0097] S210, when it is determined that the object A is holding the object C and moving stably in a direction away from the camera, the close-up of the object C is released, and the original image of the video stream is output to the display terminal.

[0098] The solution provided in this embodiment can realize free switching between the original image of the video stream and the close-up image of the video stream through automatic tracking and recognition of human body movements, assisting users to realize various switching requirements of different scenes such as live broadcast and conference, and can realize automatic recognition of close-up objects without unnecessary manual operations, thereby improving the intelligent management of scenes such as live broadcast and conference.

[0099] The solution recorded in the above embodiment is an implementation method with two cameras. The embodiment of the present invention further provides a method for outputting a video stream image, which can also be applied to an application scenario with a single camera. The method for outputting a video image in this embodiment may include:

[0100] S301, using camera X to capture the original image of the video stream, and output the original image of the video stream to a display terminal, which may be a computer, a mobile phone, a tablet, a large conference screen, or a multimedia display screen;

[0101] S302, obtaining an original image of a video stream, and performing object recognition on the original image of the video stream to determine an identifiable object contained in the original image of the video stream; in this embodiment, the identifiable objects in the original image of the video stream may include object A, object B, and object C, and an identification identifier is assigned to each object; wherein object A and object B may be the host and assistant in the live broadcast scene;

[0102] S303, the human skeletons of object A and object B are obtained by detecting the key points of the human body, and the human skeletons are associated with the identification marks of object A and object B respectively; at this time, the object A, object B and their hand features can be tracked by the tracker. In this embodiment, a neural network model with multi-class target detection can be used to simultaneously detect the portrait target and the hand target in the original image of the video stream, and the portrait target and the hand target are associated, so as to determine the features of each object and hand.

[0103] S304, when it is detected that object A triggers gesture instruction 1 (the gesture instruction 1 is a preset gesture for starting a close-up, such as a hand-raising gesture), object A is determined to be the target object. At this time, the camera X can be used to locate object A and collect a video stream close-up image of object A; specifically, the display area corresponding to object A in the original video stream image can be identified; the original video stream image is cropped and scaled based on the display area to generate a video stream close-up image of object A.

[0104] S305, outputting the close-up image of the video stream to the display terminal to realize close-up display of object A. In this embodiment, when close-up display of object A is performed, the close-up image of object A can be displayed in the display terminal.

[0105] S306, when it is determined that the gesture instruction 2 of the object A is (the gesture instruction 2 is a preset gesture for exiting the close-up, such as when making a fist), the close-up of the object A is released, and the original image of the video stream is output to the display terminal.

[0106] The method of this embodiment can also realize close-ups of multiple people. For example, the original image of the video stream can be cropped to focus on displaying close-up images of multiple target objects. Users who need close-ups only need to make simple gestures to enable and exit close-ups, which is simpler and easier than traditional manual close-ups.

[0107] Based on the same inventive concept, the embodiment of the present invention further provides a device 300 for outputting video stream images, such as Figure 3 As shown, the video stream image output device 300 of this embodiment may include:

[0108] The original image acquisition module 310 is used to acquire the original image of the video stream, and perform object recognition on the original image of the video stream to determine the recognition object contained in the original image of the video stream; the recognition object includes an object and / or a human object;

[0109] A close-up recognition module 320 is used to select at least one target object from the recognition objects and obtain a close-up image of the video stream of the target object;

[0110] The display module 330 is used to output the close-up image of the video stream to the display terminal to display the close-up of the target image.

[0111] In an optional embodiment of the present invention, the close-up recognition module 320 may also be used for:

[0112] selecting at least one target object from the identified objects in response to a user's voice command; and / or,

[0113] Collecting user voice information, predicting user intent based on the voice information, and selecting at least one target object from the identified objects based on the predicted user intent; and / or,

[0114] Analyze the gestures of each identified object based on the original image of the video stream, and select at least one target object from the identified objects according to the gestures of each identified object; and / or,

[0115] At least one target object is selected according to the motion trajectory of each identified object in the original image of the video stream.

[0116] In an optional embodiment of the present invention, the close-up recognition module 320 may also be used to: obtain the human skeleton of each recognition object through human key point detection, and establish an association relationship with each recognition object;

[0117] The hand features in the human skeleton corresponding to each recognition object are tracked, and when it is determined based on the hand features whether any recognition object performs a preset gesture action, the recognition object performing the preset gesture action is taken as the target object.

[0118] In an optional embodiment of the present invention, the close-up recognition module 320 may also be used to: track the motion trajectory of any recognized object in the original image of the continuous video stream;

[0119] When it is determined that the motion trajectory meets the preset motion law, the identified object is taken as the target object.

[0120] In an optional embodiment of the present invention, the display module 330 is also used to output the original image of the video stream to the operating terminal;

[0121] The close-up recognition module 320 may also be used to select at least one target object from the recognized objects in response to a selection instruction triggered by the user based on the operating end.

[0122] In an optional embodiment of the present invention, the close-up recognition module 320 may also be used to: perform cropping and scaling processing on the original video stream image to generate a close-up image of the video stream of the target object; or,

[0123] A close-up image of a target object is obtained through a close-up camera.

[0124] In an optional embodiment of the present invention, the close-up recognition module 320 may also be used to: obtain the distance between the original camera that captures the original image of the video stream and the target object;

[0125] If the distance is determined to be within the set threshold range, the original camera is used to obtain a close-up image of the video stream of the target object;

[0126] If the distance is determined to exceed the set threshold range, a close-up camera is used to obtain a close-up image of the video stream of the target object.

[0127] In an optional embodiment of the present invention, the display module 330 is further configured to receive a close-up release instruction for stopping close-up display of the target object;

[0128] Identify the close-up release object corresponding to the close-up release instruction in the target object; determine the matching close-up release method according to the object quantity and / or object type of the close-up release object, and stop close-up display of the close-up release object according to the close-up release method; or output the original image of the video stream to the display terminal for display.

[0129] An embodiment of the present invention further provides a computer-readable storage medium, which is used to store program codes, and the program codes are used to execute the methods of the above embodiments.

[0130] The embodiment of the present invention also provides a system for outputting video stream images, such as Figure 4 As described above, the video stream image output system of this embodiment includes at least one camera 100, at least one display terminal 200, and the video stream image output device 300 of the above embodiment connected to the camera and the display terminal.

[0131] The embodiment of the present invention further provides a camera device, the camera device includes a processor and a memory: the memory is used to store program code and transmit the program code to the processor; the processor is used to execute the method described in the above embodiment according to the instructions in the program code. Of course, in addition to the processor, the camera device also includes an optical component for realizing image acquisition, and the optical component may include optical components such as lenses and filters. In addition, the camera device may also include other components that can realize the camera function of the camera device and the functions recorded in this embodiment, such as a memory, a housing, a fixing component, etc., which will not be described in detail here.

[0132] Those skilled in the art can clearly understand that the specific working processes of the systems, devices, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and for the sake of brevity, they are not further described here.

[0133] In addition, the functional units in various embodiments of the present invention may be physically independent of each other, or two or more functional units may be integrated together, or all functional units may be integrated into one processing unit. The above integrated functional units may be implemented in the form of hardware, or in the form of software or firmware.

[0134] Those skilled in the art can understand that if the integrated functional unit is implemented in the form of software and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can essentially or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, which includes a number of instructions to enable a computing device (such as a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention when running the instructions. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk and other media that can store program codes.

[0135] Alternatively, all or part of the steps of implementing the aforementioned method embodiments may be accomplished by hardware associated with program instructions (such as a computing device such as a personal computer, a server, or a network device), and the program instructions may be stored in a computer-readable storage medium. When the program instructions are executed by a processor of a computing device, the computing device executes all or part of the steps of the methods described in the embodiments of the present invention.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that within the spirit and principles of the present invention, the technical solutions described in the aforementioned embodiments can still be modified, or some or all of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not deviate from the protection scope of the present invention.

Claims

1. A method for outputting a video stream image, characterized in that: include: Acquire an original image of a video stream, and perform object recognition on the original image of the video stream to determine a recognized object contained in the original image of the video stream; The identification object includes an object and / or a human object; Selecting at least one target object from the identified objects, and acquiring a video stream close-up image of the target object; The selecting at least one target object from the identified objects comprises: tracking the motion trajectory of any identified object in the continuous video stream original image; when it is determined that the motion trajectory satisfies a preset motion law, taking the identified object as the target object; Outputting the close-up image of the video stream to a display terminal to display a close-up of the target object; Continue to track the motion trajectory of the target object and switch to the original video stream original image playback mode when the target object triggers a close-up release instruction; The acquiring of the video stream close-up image of the target object comprises: performing cropping and scaling processing on the original video stream image to generate the video stream close-up image of the target object; The selecting of at least one target object from the identified objects also includes: obtaining the human skeleton of each identified object by detecting human key points, and establishing an association relationship with each identified object; tracking the hand features of the gesture candidate area in the human skeleton corresponding to each identified object, and judging based on the hand features that any of the identified objects makes a preset gesture action and the head and gesture of the identified object are stable, the identified object that makes the preset gesture action is used as the target object.

2. The method according to claim 1, characterized in that The selecting at least one target object from the identified objects comprises: selecting at least one target object from the identified objects in response to a user's voice command; and / or, The user's voice information is collected, and the user's intention is predicted based on the voice information, and at least one target object is selected from the recognition objects according to the predicted user's intention.

3. The method according to claim 1, characterized in that: After acquiring the original video stream image, the method further includes: outputting the original video stream image to an operation terminal; The selecting at least one target object from the identified objects includes: selecting at least one target object from the identified objects in response to a selection instruction triggered by the user based on the operating end.

4. The method according to claim 1, characterized in that The acquiring of the video stream close-up image of the target object comprises: A close-up image of the video stream of the target object is obtained through a close-up camera.

5. The method according to claim 4, characterized in that The step of obtaining a close-up image of a video stream of the target object by using a close-up camera comprises: Obtaining the distance between the original camera that captures the original image of the video stream and the target object; If it is determined that the distance is within a set threshold range, using the original camera to obtain a close-up image of the video stream of the target object; If it is determined that the distance exceeds the set threshold range, a close-up camera is used to obtain a close-up image of the video stream of the target object.

6. The method according to any one of claims 1 to 5, characterized in that After outputting the close-up image of the video stream to a display terminal to display a close-up of the target object, the method further includes: receiving a close-up release instruction for stopping close-up display of the target object; Identify the close-up release object corresponding to the close-up release instruction in the target object; determine a matching close-up release method according to the number and / or object type of the close-up release object, and stop close-up display of the close-up release object according to the close-up release method; or, The original image of the video stream is output to the display terminal for display.

7. A device for outputting video stream images, characterized in that: The device comprises: The original image acquisition module is used to acquire the original image of the video stream, and perform object recognition on the original image of the video stream to determine the recognition object contained in the original image of the video stream; the recognition object includes an object and / or a human object; A close-up recognition module is used to select at least one target object from the recognition objects and obtain a close-up image of the target object in the video stream; the selecting at least one target object from the recognition objects includes: tracking the motion trajectory of any recognition object in the continuous video stream original image; when it is determined that the motion trajectory meets a preset motion law, the recognition object is used as the target object; A display module is used to output the close-up image of the video stream to a display terminal to display the close-up of the target object; continue to track the movement trajectory of the target object and switch to the original video stream original image playback mode when the target object triggers a close-up release instruction; The close-up recognition module obtains the close-up image of the video stream of the target object, including: performing cropping and scaling processing on the original image of the video stream to generate the close-up image of the video stream of the target object; The close-up recognition module is also used to: obtain the human skeleton of each recognition object through human body key point detection, and establish an association relationship with each recognition object; track the hand features of the gesture candidate area in the human skeleton corresponding to each recognition object, and based on the hand features, judge that any of the recognition objects makes a preset gesture action and the head and gesture of the recognition object are stable, and the recognition object that makes the preset gesture action is used as the target object.

8. A camera device, characterized in that: The camera device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is configured to execute the method according to any one of claims 1 to 6 according to the instructions in the program code.

9. A video stream image output system, characterized in that: The system comprises at least one camera, at least one display terminal and the video stream image output device according to claim 7 connected to the camera and the display terminal.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program codes, and the program codes are used to execute the method according to any one of claims 1 to 6.

Images