Multi-camera cascading method and device, storage medium and electronic equipment

Abstract

The invention discloses a multi-camera cascading method and device, a storage medium and electronic equipment, and the method comprises the steps: collecting the scene image information of a target scene through employing at least two camera devices, carrying out the portrait detection processing of the scene image information, obtaining the portrait information collected by each camera device, and storing the portrait information in a server; obtaining first portrait information, collected by the first camera device, of a target person, obtaining all portrait information collected by the second camera device, and carrying out portrait feature matching processing on the first portrait information and all portrait information collected by the second camera device; and acquiring first portrait information of the target person, acquiring second portrait information of the target person acquired by each camera device in the second camera device, determining target portrait information corresponding to the minimum deflection angle in the first portrait information and the second portrait information, and outputting a close-up image of the target person by adopting the target camera device corresponding to the target portrait information. According to the invention, the close-up image of the most front face of the person is output without limiting the position of the camera device.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, and particularly relates to a multi-camera cascading method and device, a storage medium and an electronic device. BACKGROUND

[0002] In the case of a large scene area or the presence of multiple people, in order to be able to collect more details in the scene, more portrait information, the prior art can use multiple cameras to collect image information of the scene. This technology can also be applied to remote video application scenarios, such as online remote meetings. The prior art can use multiple cameras to collect portrait information in the scene, and then determine the coordinates of each camera to convert to find the most frontal portrait information for display to improve the portrait display effect. However, the prior art is highly dependent on the accurate positions and coordinate conversion of each camera, so the camera cannot be moved at will, thereby increasing the user's difficulty of use, and instead cannot obtain clear and accurate portraits, affecting the display effect of the portrait. Therefore, there is a need to provide a method that is convenient to use and can obtain accurate and clear portraits. SUMMARY

[0003] Embodiments of the present application provide a multi-camera cascading method, device, storage medium and electronic device. The method can obtain portrait information of the same person by matching the portrait information collected by each camera device, and determine the camera device with the smallest deflection angle to output a close-up of the person according to the portrait information, thereby outputting a close-up image of the most frontal face in the scene without the need to limit the position of the camera device, improving the convenience of use of the camera device and the accuracy of the face image. The technical solution is as follows:

[0004] In a first aspect, the embodiments of the present application provide a multi-camera cascading method, which includes the following steps:

[0005] Collecting scene image information of a target scene by using at least two camera devices, performing portrait detection processing on the scene image information to obtain portrait information collected by each camera device, and the shooting angles of each camera device in the at least two camera devices are different for the target scene;

[0006] Obtaining first portrait information of a target person collected by a first camera device, and obtaining all portrait information collected by a second camera device, wherein the first camera device is any camera device in the at least two camera devices, and the second camera device is a camera device other than the first camera device in the at least two camera devices;

[0007] Performing portrait feature matching processing on the first portrait information and all portrait information collected by the second camera device to obtain second portrait information of the target person collected by each camera device in the second camera device;

[0008] obtaining a deflection angle of the target person relative to the camera device in each of the first portrait information and the second portrait information;

[0009] determining target portrait information corresponding to a minimum deflection angle in the first portrait information and the second portrait information;

[0010] outputting a close-up image of the target person by using a target camera device corresponding to the target portrait information, the target camera device being the first camera device or the second camera device.

[0011] In a second aspect, an embodiment of the present application provides a multi-camera cascade device, the device comprising:

[0012] a portrait information acquisition module, configured to acquire scene image information of a target scene by using at least two camera devices, and perform portrait detection processing on the scene image information to obtain portrait information collected by each camera device, the photographing angles of the camera devices being different for the target scene;

[0013] a feature matching module, configured to obtain first portrait information of a target person collected by a first camera device, and obtain all portrait information collected by a second camera device, the first camera device being any one of the at least two camera devices, and the second camera device being a camera device other than the first camera device among the at least two camera devices;

[0014] a portrait matching module, configured to perform portrait feature matching processing on the first portrait information and the all portrait information collected by the second camera device, to obtain second portrait information of the target person collected by each camera device in the second camera device;

[0015] a deflection angle acquisition module, configured to obtain a deflection angle of the target person relative to the camera device in each of the first portrait information and the second portrait information;

[0016] a target portrait acquisition module, configured to determine target portrait information corresponding to a minimum deflection angle in the first portrait information and the second portrait information;

[0017] a close-up image output module, configured to output a close-up image of the target person by using a target camera device corresponding to the target portrait information, the target camera device being the first camera device or the second camera device.

[0018] In a third aspect, an embodiment of the present application provides a computer storage medium, the computer storage medium storing a plurality of instructions, the instructions being adapted to be loaded by a processor and executed to perform the method steps described above.

[0019] In a fourth aspect, an electronic device is provided, which can include a processor and a memory. The memory stores a computer program, which is adapted to be loaded by the processor and execute the method steps described above.

[0020] In one or more embodiments of the present application, scene image information of a target scene is collected by using at least two camera devices, and portrait detection processing is performed on the scene image information to obtain portrait information collected by each camera device. The shooting angles of each camera device in the at least two camera devices are different, first portrait information of a target person collected by a first camera device is obtained, all portrait information collected by a second camera device is obtained, and portrait feature matching processing is performed on the first portrait information and the all portrait information collected by the second camera device to obtain second portrait information of the target person collected by each camera device in the second camera device. In the first portrait information and the second portrait information, a deflection angle of the target person in each portrait information relative to the camera device is obtained, target portrait information corresponding to the minimum deflection angle is determined in the first portrait information and the second portrait information, and a close-up image of the target person is output by using a target camera device corresponding to the target portrait information. By matching the portrait information collected by each camera device, portrait information of the same person is obtained, and the camera device with the minimum deflection angle is determined according to the portrait information to output a close-up image of the person, so that the close-up image of the most frontal face in the scene can be output without limiting the position of the camera device, and the use convenience of the camera device and the accuracy of the face image are improved. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0022] Figure 1 is a flowchart of an image source tracking method provided by an embodiment of the present application;

[0023] Figure 2 is a flowchart of an image source tracking method provided by an embodiment of the present application;

[0024] Figure 3 is a structural diagram of an image source tracking device provided by an embodiment of the present application;

[0025] Figure 4 is a structural diagram of a private image storage module provided by an embodiment of the present application;

[0026] Figure 5is a structural schematic diagram of an image source acquisition module provided by an embodiment of the present application.

[0027] Figure 6 is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0029] The multi-camera cascade device can use at least two cameras to collect scene image information of a target scene. The multi-camera cascade method provided by the embodiments of the present application can be implemented by relying on a computer program and can run on a multi-camera cascade device based on the von Neumann architecture. The computer program can be integrated in an application or can run as an independent tool application. Please refer to Figure 1 An example schematic diagram of close-up image acquisition in a target scene is provided by an embodiment of the present application. For example, in an online remote meeting scene, the target scene can be a conference room, and one or more meeting participants can be in the target scene. In order to collect more scene details or more portrait information of the participants in the target scene, the multi-camera cascade device can be placed with at least two cameras in the target scene. The placement positions and number of the cameras in the at least two cameras can be changed according to actual needs of a user. The shooting angles of the cameras for the target scene can be different. For example, the user can change the positions of the cameras according to the positions of the participants in the target scene, or can increase or decrease the number of the cameras according to the size of the target scene and the number of the participants. It can be understood that the positions of the cameras can also be placed by the user as needed. Since the multi-camera cascade device can be applied to a meeting scene, a display device for video display and presentation document display can be placed in the meeting scene. The cameras can be installed on the display device, or the cameras can be built-in cameras of the display device.

[0030] For example Figure 1In the target scene in the figure, there are five persons participating in a meeting, and the user can place a camera device in front of the meeting table, on the left side and on the right side to collect scene image information of the target scene. It can be understood that the scene image information obtained by the camera device can be a relatively comprehensive scene image for the target scene, and also contains portrait information of all the collected persons. The portrait information is image information containing a person, which can be used to display the appearance of the person to the user or the audience. However, the scene image information is not convenient for the user or the audience to carefully view individual persons. For example, when a meeting participant speaks during the meeting, directly viewing the scene image information can not be able to observe the facial expression of the meeting participant. Therefore, the user can use the multi-camera cascading device to obtain a close-up image of the meeting participant. For example, the user can use the multi-camera cascading device to determine the person who wants to output the close-up image as a target person. The close-up image refers to a kind of lens composition mode for close-range shooting of the object or person being shot, which can be used to highlight the details of the object or person being shot. The multi-camera cascading device can also crop the image containing the target person from the scene image information as a close-up image.

[0031] It can be understood that the portrait information collected by different camera devices for the same person is also different due to the shooting angle. If the person has a large deflection angle relative to the camera device, the portrait information cannot accurately display the face of the person, and the user and the audience cannot watch the appearance, expression or action of the person. In the traditional technical solution, when the close-up image of the person in the target scene is needed, the coordinates of each camera device are determined to find the most frontal portrait information as the close-up image display. However, this method is highly dependent on the accurate coordinates of each camera device, so the camera device cannot be moved at will, thereby increasing the difficulty of use for the user, and a clear and accurate portrait cannot be obtained.

[0032] The multi-camera cascading device in the embodiment of the present application can interact with all camera devices. The multi-camera cascading device can be any camera device or a module or application program in the camera device for implementing the multi-camera cascading method, or other electronic devices other than all camera devices. The multi-camera cascading device can obtain portrait information collected by all camera devices for the target person, determine target portrait information with the smallest deflection angle among all portrait information of the target person, and then use the target camera device collecting the target portrait information to output the close-up image of the target person. Thus, the most frontal close-up image in the scene can be output without limiting the position of the camera device. It can be understood that since the deflection angle of the target person and the target camera device is the smallest, the close-up image output by the target camera device for the target person is the most accurate image closest to the frontal face of the target person among all camera devices.

[0033] The scene image information acquired by the camera device can include multiple persons collected by the camera device, and the target person can be a person specified by the user to display a close-up image. For example, the multi-camera cascade device can display the scene image information collected by all the camera devices, and then the user can send a close-up instruction to the multi-camera cascade device for a target person in the first scene image information collected by the first camera device. After the multi-camera cascade device acquires the close-up instruction for the target person, the multi-camera cascade device can acquire first portrait information of the target person in the first scene image information, wherein the first camera device is any one of the at least two camera devices.

[0034] Since the multi-camera cascade device can be applied to a conference scene, the user can specify to display a close-up image of a person who is speaking, so the multi-camera cascade device can confirm the person who is speaking in the target scene as the target person. After acquiring the first scene image information, the multi-camera cascade device can acquire the mouth opening area of each person in the first scene image information. If the mouth opening area of a person in the first scene image information reaches a preset area, the multi-camera cascade device determines that the person is the target person. There can be one or more target persons, and the preset area can be an initial setting of the multi-camera cascade device or can be set by the user or relevant staff. For example, the preset area can be 5 square centimeters. It can be understood that since the distance between the camera device and the person, the focal length of the camera device, and other factors are different, the multi-camera cascade device can have difficulty in acquiring the specific size of the mouth opening area of the person. Therefore, the multi-camera cascade device can acquire the percentage of the mouth opening area of the person to the face area of the person. If the percentage is greater than a preset percentage, the multi-camera cascade device can confirm the person as the target person. The preset percentage can be an initial setting of the multi-camera cascade device or can be set by the user or relevant staff. The preset percentage can be 5%.

[0035] In addition to specifying one person to output a close-up image, the user can also specify multiple persons to output a close-up image at the same time. For example, the user can specify all the persons collected in the first scene image information as target persons. The multi-camera cascade device can generate a person set specified by the user based on all the persons collected in the first scene image information, and then determine the camera device outputting a close-up image for the persons in the person set in sequence. Therefore, at this time, the target person can be any person in the person set for which the camera device outputting a close-up image has not been determined. Until all the persons in the person set have determined the camera device outputting a close-up image, the multi-camera cascade device can output a close-up image of all the persons in the person set by using the camera device.

[0036] The user can also send a close-up instruction to the multi-camera cascade device for outputting close-up images of all the persons in the target scene. It can be understood that the first scene image information captured by the first camera device can not completely include all the persons in the target scene. Therefore, the multi-camera cascade device can perform portrait feature matching processing on the portrait information of the persons captured by all the camera devices in sequence, and determine the camera device outputting the close-up image until the camera device outputting the close-up image is determined for all the persons captured by all the camera devices. The multi-camera cascade device can confirm the persons captured by the camera device that has not yet determined the camera device outputting the close-up image as persons to be processed. The multi-camera cascade device can determine the camera device outputting the close-up image for the persons to be processed in the persons captured by at least two camera devices as the first camera device. The target person can be any person to be processed corresponding to the first camera device.

[0037] Optionally, in addition to determining the target person in all the scene image information, the user can also input the face image information of the target person in advance. After the multi-camera cascade device obtains the scene image information, the multi-camera cascade device can determine the portrait information in the first scene image information that matches the pre-input face image information as the first portrait information of the target person. The multi-camera cascade device can calculate the face similarity between all the portrait information in the first scene image information and the face image information, and determine the portrait information with a face similarity greater than a preset face similarity threshold as the portrait information matching the face image information. The preset face similarity threshold can be an initial setting of the multi-camera cascade device, or can be set by the user or relevant staff. For example, the preset face similarity threshold can be 80%.

[0038] The multi-camera cascade method provided by the present application will be described in detail below with reference to specific embodiments.

[0039] Please refer to Figure 2 A flowchart of a multi-camera cascade method is provided for the embodiments of the present application. As shown in Figure 1 The method of the embodiments of the present application can include the following steps S101-S106.

[0040] S101, scene image information of a target scene is captured by using at least two camera devices. Portrait detection processing is performed on the scene image information to obtain portrait information captured by each camera device. The shooting angles of each camera device in the at least two camera devices for the target scene are different.

[0041] Specifically, the user can place at least two cameras in the target scene, and then the multi-camera cascade device can collect scene image information of the target scene by using the at least two cameras. It can be understood that the shooting angles of each camera in the at least two cameras are different, so the scene image information obtained by different cameras is also different. Then the multi-camera cascade device can perform portrait detection processing on all the scene image information, so as to obtain the portrait information collected by each camera. The portrait detection processing is used to determine the portrait information from the scene image information, for example, to determine the head contour or body contour of a person. After determining the position of the person in the scene image information, it is convenient to further determine the portrait information. It can be understood that due to the different shooting angles of the cameras, there can be one or more portrait information in a scene image information, or there can be no portrait information.

[0042] In S102, first portrait information of a target person collected by a first camera and all portrait information collected by a second camera are obtained. The first camera is any camera in the at least two cameras, and the second camera is a camera other than the first camera in the at least two cameras.

[0043] Specifically, after the multi-camera cascade device obtains the portrait information in each scene image information, it also needs to perform portrait feature matching processing on the portrait information of the person in different scene image information, generate a matching result for the portrait information of the same person, and then obtain the portrait information of the target person collected by each camera from the matching result. The target person is any person in the target scene, or a person specified by the user to need to display a close-up image. For example, the user can send a close-up instruction to the multi-camera cascade device for the target person in the collected scene image information. The close-up instruction is used to instruct the multi-camera cascade device to output a close-up image of the target person.

[0044] The multi-camera cascade device can obtain first portrait information of a target person collected by a first camera. The first camera can be any camera in the at least two cameras, and the target person can be any person in the scene image information collected by the first camera for the target scene. The second camera is a camera other than the first camera in the at least two cameras. The second camera can be one or more. It can be understood that if the user uses two cameras to collect scene image information of the target scene, the second camera is only one. If the user uses three cameras to collect scene image information of the target scene, the second camera can be two.

[0045] S103, perform portrait feature matching processing on the first portrait information and all portrait information collected by the second camera device to obtain second portrait information of the target person collected by each camera device in the second camera device.

[0046] Specifically, the first portrait information is portrait information of the target person collected by the first camera device, and the multi-camera cascade device also needs to determine portrait information of the target person from the portrait information collected by the second camera device. The multi-camera cascade device can perform portrait feature matching processing on the first portrait information and all portrait information collected by the second camera device to obtain second portrait information of the target person collected by each camera device in the second camera device. It can be understood that the second camera device can be multiple, so there can be multiple second camera devices collecting portrait information of the target person, so the second portrait information can also be one or more.

[0047] S104, obtain the deflection angle of the target person in each portrait information relative to the camera device from the first portrait information and the second portrait information.

[0048] Specifically, the multi-camera cascade device can calculate the deflection angle of the target person in each portrait information relative to the camera device from the first portrait information and the second portrait information. The deflection angle can be used to determine whether the portrait information of the target person collected by the camera device is clear, or whether it is more biased towards the front face. If the deflection angle is smaller, the target person in the portrait information is closer to the front face facing the camera device, and vice versa. If the deflection angle is larger, the target person in the portrait information is closer to the side face facing the camera device.

[0049] S105, determine the target portrait information corresponding to the minimum deflection angle from the first portrait information and the second portrait information.

[0050] Specifically, the multi-camera cascade device determines the target portrait information corresponding to the minimum deflection angle from the first portrait information and the second portrait information. Since the deflection angle of the target person in the target portrait information relative to the camera device is the smallest, the target portrait information is the portrait information of the target person that is most front-facing and most clearly displayed among all portrait information of the target person.

[0051] S106, output a close-up image of the target person using the target camera device corresponding to the target portrait information. The target camera device is the first camera device or the second camera device.

[0052] Specifically, the multi-camera cascade device can determine target portrait information corresponding to a target camera, and then use the target camera to output a close-up image of the target person. For example, the multi-camera cascade device can use the target camera to capture a close-up image of the target person, or can crop the scene image information captured by the target camera to obtain a close-up image of the target person. The multi-camera cascade device can also continuously output close-up images of the target person according to the capture frequency of the target camera, so as to achieve the purpose of outputting a video stream corresponding to the close-up image of the target person.

[0053] In some embodiments, the first camera can be a camera provided by an interactive smart panel in a conference room, and the second camera can be a camera placed on a table in the conference room.

[0054] In the embodiments of the present application, at least two cameras are used to capture scene image information of a target scene, and portrait detection processing is performed on the scene image information to obtain portrait information captured by each camera. The shooting angles of each camera in the at least two cameras are different with respect to the target scene. First portrait information of a target person captured by a first camera is obtained, and all portrait information captured by a second camera is obtained. Portrait feature matching processing is performed on the first portrait information and all the portrait information captured by the second camera to obtain second portrait information of the target person captured by each camera in the second camera. In the first portrait information and the second portrait information, the deflection angle of the target person in each portrait information with respect to the camera is obtained. In the first portrait information and the second portrait information, the target portrait information corresponding to the minimum deflection angle is determined, and a close-up image of the target person is output by using a target camera corresponding to the target portrait information. By matching the portrait information captured by each camera to obtain portrait information of the same person, and by outputting a close-up image of the person according to the portrait information with the smallest deflection angle, a close-up image of the most frontal face in the scene can be output without limiting the position of the camera, thereby improving the convenience of using the camera and the accuracy of the face image.

[0055] See Figure 3 A flowchart of a multi-camera cascade method is provided for the embodiments of the present application. As Figure 3 shown, the method of the embodiments of the present application can include the following steps S201-S207.

[0056] S201, at least two cameras are used to capture scene image information of a target scene, and portrait detection processing is performed on the scene image information to obtain portrait information captured by each camera. The shooting angles of each camera in the at least two cameras are different with respect to the target scene.

[0057] Specifically, the user can place at least two cameras in the target scene, and then the multi-camera cascade device can collect scene image information of the target scene by using the at least two cameras. It can be understood that the shooting angles of each camera in the at least two cameras are different, so the scene image information obtained by different cameras is also different. Then the multi-camera cascade device can perform portrait detection processing on all the scene image information, so as to obtain the portrait information collected by the cameras. The portrait detection processing is used to determine the portrait information from the scene image information, for example, to determine the head contour or body contour of a person. After determining the position of the person in the scene image information, it is convenient to further determine the portrait information. It can be understood that due to the different shooting angles of the cameras, there can be one or more portrait information in a scene image information, or there can be no portrait information.

[0058] Optionally, the multi-camera cascade device can perform portrait detection processing on the scene image information by using a head detection algorithm or a human body detection algorithm. The head recognition algorithm can recognize the position and boundary box of the human head in the image or video, and further obtain the portrait information of the person by accurately detecting and positioning the head. The human body detection algorithm can detect the contour and position of the human body in the image or video, and further obtain the portrait information by finding the boundary box or contour of all human bodies in the image. The portrait information is image information containing a person, which can be used to display the appearance of the person to the user or the audience.

[0059] S202, obtaining first portrait information of a target person collected by a first camera, and obtaining all portrait information collected by a second camera, wherein the first camera is any camera in the at least two cameras, and the second camera is a camera other than the first camera in the at least two cameras.

[0060] Specifically, after the multi-camera cascade device obtains the portrait information in each scene image information, it also needs to perform portrait feature matching processing on the portrait information of the person in different scene image information, and generate a matching result for the portrait information of the same person. The multi-camera cascade device can obtain first portrait information of a target person collected by a first camera. The first camera can be any camera in the at least two cameras, and the target person can be any person in the scene image information collected by the first camera for the target scene. The second camera is a camera other than the first camera in the at least two cameras, and the second camera can be one or more. It can be understood that if the user uses two cameras to collect scene image information of the target scene, there is only one second camera. If the user uses three cameras to collect scene image information of the target scene, there can be two second cameras.

[0061] S203, performing feature extraction processing on the first portrait information to obtain first feature data, and performing feature extraction processing on all portrait information collected by the second camera to obtain second feature data.

[0062] Specifically, the multi-camera cascade device needs to perform portrait feature matching processing on the first portrait information and all portrait information collected by the second camera, so as to obtain the second portrait information of the target person collected by the second camera. In order to further match the portrait information, the multi-camera cascade device needs to perform feature extraction processing on the portrait information to obtain feature data. The feature data is a feature vector reflecting the appearance characteristics of the person in the portrait information. The multi-camera cascade device performs feature extraction processing on the first portrait information to obtain first feature data. The first feature data is used to reflect the appearance characteristics of the target person in the first portrait information. In addition, feature extraction processing also needs to be performed on all portrait information collected by the second camera to obtain second feature data. The second feature data contains the appearance characteristics of each person in all persons collected by the second camera. In the first feature data and the second feature data, a plurality of sets of feature data are collected. Each set of feature data corresponds to the features of a collected person.

[0063] S204, obtaining target feature data matched with the first feature data in the second feature data, and determining the portrait information corresponding to the target feature data as the second portrait information of the target person.

[0064] Matching each set of feature data in the first feature data with each set of feature data in the second feature data can obtain matched target feature data. Specifically, the multi-camera cascade device can obtain target feature data matched with the first feature data in the second feature data. Since the feature data is used to reflect the appearance characteristics of the person, if two feature data are matched, it means that the two feature data are from the same person. Therefore, if the target feature data is matched with the first feature data, the target feature data is also from the target person. The multi-camera cascade device can determine the portrait information corresponding to the target feature data as the second portrait information of the target person. It can be understood that the second camera can be multiple, so there can be multiple second cameras collecting portrait information of the target person. Therefore, the target feature data can be one or more. Similarly, the second portrait information can also be one or more.

[0065] Optionally, the multi-camera cascade device can calculate the similarity between each feature data in the second feature data and the first feature data, and then determine the feature data with a similarity greater than a similarity threshold in the second feature data as the target feature data matched with the first feature data, where the similarity threshold is used to determine whether two feature data are matched and come from the same person, and can be an initial setting of the multi-camera cascade device or set by a user or a relevant staff.

[0066] In S205, the deflection angle of the target person in each portrait information relative to the camera device is obtained from the first portrait information and the second portrait information.

[0067] Specifically, the multi-camera cascade device can calculate the deflection angle of the target person in each portrait information relative to the camera device from the first portrait information and the second portrait information. The deflection angle can be used to determine whether the portrait information of the target person obtained by the camera device is clear and whether it is more inclined to the front face. If the deflection angle is smaller, the target person in the portrait information is closer to the front face facing the camera device, and vice versa. If the deflection angle is larger, the target person in the portrait information is closer to the side face facing the camera device.

[0068] Optionally, the first portrait information and the second portrait information are both portrait information of the target person, so the multi-camera cascade device can perform angle detection processing on the first portrait information and the second portrait information, thereby obtaining the deflection angle of the target person in each portrait information relative to the camera device from the first portrait information and the second portrait information.

[0069] Optionally, the angle detection processing is used to calculate the deflection angle of the person in the portrait information relative to the camera device. The multi-camera cascade device can use an angle detection algorithm to perform angle detection processing. The angle detection algorithm is used to calculate the direction or angle of an object or person in an image, and can obtain the deflection angle of the person in the portrait information relative to the camera device. The angle detection algorithm can include an edge detection algorithm, a Hough transform, a histogram of oriented gradients, etc.

[0070] Optionally, the multi-camera cascade device can also use a convolutional neural network-based angle detection model to perform angle detection processing. The multi-camera cascade device can input all portrait information of the target person into the angle detection model, and the angle detection model can output the deflection angle corresponding to each portrait information. For example, the angle detection model can be an image pose estimation (Im2pose) model, a lightweight pose recognition (lwposr) model, and a 3D dense face alignment (3DDFA) model, etc.

[0071] In S206, the target portrait information corresponding to the minimum deflection angle is determined from the first portrait information and the second portrait information.

[0072] Specifically, the multi-camera cascade device determines the target portrait information corresponding to the minimum deflection angle from the first portrait information and the second portrait information. Since the target portrait information has the minimum deflection angle of the target person relative to the camera device, the target portrait information is the portrait information that most clearly displays the target person among all the portrait information of the target person.

[0073] Optionally, if there are two or more minimum deflection angles with the same value in the portrait information of the target person, the multi-camera cascade device can select one of the two or more minimum deflection angles to determine the corresponding portrait information as the target portrait information.

[0074] Please refer to Figure 4 An example schematic diagram for determining the target portrait information is provided for the embodiments of the present application. As shown in the figure, a user can use three camera devices to capture a target scene, which are camera device A, camera device B and camera device C. The user can view the scene image information A captured by the camera device A, and click the target person in the scene image information A through a mouse or a touch screen to send a close-up instruction of the target person captured by the camera device A to the multi-camera cascade device. The multi-camera cascade device can determine the camera device A as the first camera device, determine the scene image information captured by the camera device A as the first scene image information, and obtain the first portrait information of the target person in the first scene image information. Then, the camera device B and the camera device C are determined as the second camera device, and the scene image information captured by the camera device B and the camera device C is determined as the second scene image information. The multi-camera cascade device can obtain all the portrait information captured by the second camera device in the second scene image information, and obtain the second portrait information of the target person through portrait matching processing. Then, the multi-camera cascade device determines the target portrait information with the minimum deflection angle from the first portrait information and the second portrait information, as shown in the figure. Figure 4 The target portrait information is the portrait information captured by the camera device C, that is, the multi-camera cascade device can output the close-up image of the target person by using the camera device C.

[0075] S207, outputting the close-up image of the target person by using the target camera device corresponding to the target portrait information.

[0076] Specifically, the multi-camera cascade device can determine a target camera corresponding to the target portrait information, and then use the target camera to output a close-up image of the target person. For example, the multi-camera cascade device can use the target camera to capture a close-up image of the target person, or can crop the scene image information captured by the target camera to obtain a close-up image of the target person. The multi-camera cascade device can also continuously output close-up images of the target person according to the capture frequency of the target camera, so as to achieve the purpose of outputting a video stream corresponding to the close-up image of the target person.

[0077] Optionally, due to different camera positions and camera angles, only the first camera may have captured the target person, and the second camera may not have captured the target person. It can be understood that if the second camera does not capture the target person, the target person will not exist in the second scene image, and the target portrait information matching the first portrait information will not exist in the second portrait information. Therefore, if the target person does not exist in the second scene image information captured by the second camera, the first camera can be used to output a close-up image of the target person.

[0078] In the embodiment of the present application, at least two camera devices are used to collect scene image information of a target scene, and portrait detection processing is performed on the scene image information to obtain portrait information collected by each camera device. The camera devices in the at least two camera devices have different shooting angles for the target scene, first portrait information of a target person collected by a first camera device is obtained, and all portrait information collected by a second camera device is obtained. The target person can be a person specified by a user for outputting a close-up image, and a close-up image of all persons in the target scene can be output according to the user's demand, and the camera device outputting the close-up image is determined for all persons in turn. Feature extraction processing is performed on the first portrait information to obtain first feature data, feature extraction processing is performed on all portrait information collected by the second camera device to obtain second feature data, target feature data matched with the first feature data is obtained from the second feature data, and the portrait information corresponding to the target feature data is determined as second portrait information of the target person. The portrait information belonging to the target person is determined through feature data matching, the accuracy of the target person portrait information is ensured, the accuracy of the close-up image output is further improved, the deflection angle of the target person relative to the camera device in each portrait information is obtained from the first portrait information and the second portrait information, the target portrait information corresponding to the minimum deflection angle is determined from the first portrait information and the second portrait information, and the target camera device corresponding to the target portrait information is used to output the close-up image of the target person. The portrait information belonging to one person is obtained by matching the portrait information collected by each camera device, and the camera device with the smallest deflection angle is determined according to the portrait information to output the close-up of the person, so that the close-up image of the most frontal face in the scene can be output without limiting the position of the camera device, and the use convenience of the camera device and the accuracy of the face image are improved.

[0079] The following will be described in detail with reference to the accompanying drawings. Figure 5 The multi-camera cascade device provided in the embodiment of the present application will be described in detail. It should be noted that the multi-camera cascade device in the embodiment of the present application is used to execute the method of the embodiment of the present application shown in Figure 5 and Figure 2 and Figure 3 for the sake of brevity, only parts related to the embodiment of the present application are shown, and specific technical details not disclosed are described with reference to the embodiment shown in Figure 2 and Figure 3 .

[0080] Please refer to Figure 5 , which shows a structural schematic diagram of a multi-camera cascade device provided in an exemplary embodiment of the present application. The multi-camera cascade device can be realized by software, hardware or a combination of both to become all or part of the device. The device 1 includes a portrait information collection module 11, a feature matching module 12, a portrait matching module 13, a deflection angle obtaining module 14, a target portrait obtaining module 15 and a close-up image output module 16.

[0081] The portrait information collection module 11 is configured to collect scene image information of a target scene by using at least two camera devices, and to obtain portrait information collected by each camera device by performing portrait detection on the scene image information, wherein the camera devices are different in shooting angle for the target scene.

[0082] The portrait information acquisition module 12 is configured to acquire first portrait information of a target person collected by a first camera device, and to acquire all portrait information collected by a second camera device, wherein the first camera device is any one of the at least two camera devices, and the second camera device is a camera device other than the first camera device.

[0083] Optionally, the portrait information acquisition module 12 is specifically configured to acquire a close-up instruction of a target person in first scene image information collected by the first camera device by the user.

[0084] The first portrait information of the target person in the first scene image information is acquired.

[0085] The portrait matching module 13 is configured to perform portrait feature matching on the first portrait information and all portrait information collected by the second camera device, to obtain second portrait information of the target person collected by each camera device of the second camera device.

[0086] Optionally, the portrait matching module 13 is specifically configured to perform feature extraction on the first portrait information to obtain first feature data.

[0087] The portrait matching module 13 is specifically configured to perform feature extraction on all portrait information collected by the second camera device to obtain second feature data.

[0088] The target feature data matched with the first feature data is acquired from the second feature data, and portrait information corresponding to the target feature data is determined as the second portrait information of the target person.

[0089] Optionally, the portrait matching module 13 is specifically configured to calculate a similarity between each feature data in the second feature data and the first feature data.

[0090] The feature data in the second feature data with a similarity greater than a similarity threshold is confirmed as the target feature data matched with the first feature data.

[0091] The deflection angle acquisition module 14 is configured to acquire a deflection angle of the target person relative to the camera device in each of the first portrait information and the second portrait information.

[0092] The target portrait acquisition module 15 is configured to determine target portrait information corresponding to a minimum deflection angle from the first portrait information and the second portrait information.

[0093] Optionally, the target portrait acquisition module 15 is specifically configured to determine a minimum deflection angle with a minimum value from among all deflection angles.

[0094] Among the first portrait information and the second portrait information, target portrait information corresponding to the minimum deflection angle is determined.

[0095] Optionally, the target portrait acquisition module 15 is specifically configured to, if there are at least two minimum deflection angles, confirm the portrait information corresponding to any minimum deflection angle among the at least two minimum deflection angles as the target portrait information.

[0096] The close-up image output module 16 is configured to output a close-up image of the target person by using a target camera corresponding to the target portrait information, the target camera being the first camera or the second camera.

[0097] Optionally, the close-up image output module 16 is specifically configured to, if the target person does not exist in the second scene image information collected by the second camera, output the close-up image of the target person by using the first camera.

[0098] In this embodiment, at least two camera devices are used to collect scene image information of the target scene. The scene image information is processed by human image detection to obtain the human image information collected by each camera device. Each of the at least two camera devices has a different shooting angle for the target scene. The first human image information of the target person collected by the first camera device is obtained, and all human image information collected by the second camera device is obtained. The target person can be selected to output close-up images according to the person specified by the user, and close-up images of all people in the target scene can be output according to the user's needs. The camera device for outputting close-up images is determined for all people in turn. Feature extraction processing is performed on the first portrait information to obtain first feature data. Feature extraction processing is also performed on all portrait information collected by the second camera device to obtain second feature data. Target feature data matching the first feature data is obtained from the second feature data. The portrait information corresponding to the target feature data is determined as the second portrait information of the target person. By matching feature data, portrait information belonging to the same target person is determined, ensuring the accuracy of target person portrait information acquisition and further improving the accuracy of close-up image output. The deflection angle of the target person relative to the camera device is obtained from each of the first and second portrait information. The target portrait information corresponding to the smallest deflection angle is determined from the first and second portrait information. The target camera device corresponding to the target portrait information is used to output the close-up image of the target person. By matching the portrait information collected by each camera device to obtain portrait information belonging to the same person, and determining the camera device with the smallest deflection angle to output the close-up of the person based on this portrait information, the most frontal close-up image of the face in the scene can be output without restricting the position of the camera device, improving the ease of use of the camera device and the accuracy of the facial image.

[0099] It should be noted that the multi-camera cascaded device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the multi-camera cascaded method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the multi-camera cascaded device and the multi-camera cascaded method embodiments provided in the above embodiments belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0100] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0101] This application also provides a computer storage medium that can store multiple instructions, which are adapted to be loaded and executed by a processor as described above. Figures 1-4 The multi-camera cascading method of the illustrated embodiment can be found in the following documentation for its specific execution process: Figures 1-4The specific description of the illustrated embodiments will not be repeated here.

[0102] The present application also provides a computer program product, which stores at least one instruction loaded and executed by a processor to perform the above Figures 1-4 The specific execution process of the multi-camera cascading method of the illustrated embodiments can be referred to Figures 1-4 The specific description of the illustrated embodiments will not be repeated here.

[0103] Please refer to Figure 6 Fig. 1 shows a structural block diagram of an electronic device according to an example embodiment of the present application. The electronic device in the present application can include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 can be connected through the bus 150.

[0104] The processor 110 can include one or more processing cores. The processor 110 connects various parts within the entire electronic device through various interfaces and lines, executes various functions of the terminal 100 and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Alternatively, the processor 110 can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor 110 can integrate a combination of one or more of a central processing unit (CPU), a graphics processor (GPU), and a modem. Among them, the CPU is mainly used to process the operating system, user pages, and application programs; the GPU is used to render and draw display content; and the modem is used to process wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor 110, but can be realized by a separate communication chip.

[0105] The memory 120 can include a random access memory (RAM) and can also include a read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable storage medium. The memory 120 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 120 can include a program storage area and a data storage area, where the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described above, and the like. The operating system can be an Android system, an IOS system developed by Apple Inc., a system developed based on the Android system, a system developed based on the IOS system, or other systems.

[0106] The memory 120 can be divided into an operating system space and a user space, where the operating system runs in the operating system space, and native and third-party applications run in the user space. In order to ensure that different third-party applications can achieve good running effects, the operating system allocates corresponding system resources to different third-party applications. However, there are also differences in the demand for system resources in different application scenarios in the same third-party application. For example, in the local resource loading scenario, the third-party application has a higher requirement for the disk reading speed; in the animation rendering scenario, the third-party application has a higher requirement for the GPU performance. However, the operating system and the third-party application are independent of each other, and the operating system often cannot timely perceive the current application scenario of the third-party application, resulting in that the operating system cannot perform targeted system resource adaptation according to the specific application scenario of the third-party application.

[0107] In order to enable the operating system to distinguish the specific application scenario of the third-party application, it is necessary to open up the data communication between the third-party application and the operating system, so that the operating system can obtain the current scenario information of the third-party application at any time, and then perform targeted system resource adaptation based on the current scenario.

[0108] The input device 130 is configured to receive input instructions or data, and the input device 130 includes but is not limited to a keyboard, a mouse, a camera device, a microphone, or a touch device. The output device 140 is configured to output instructions or data, and the output device 140 includes but is not limited to a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 are a touch display screen.

[0109] The touch display screen can be designed as a full screen, a curved screen, or a special-shaped screen. The touch display screen can also be designed as a combination of a full screen and a curved screen, a combination of a special-shaped screen and a curved screen, and the present application does not limit this.

[0110] In addition, those skilled in the art can understand that the structure of the electronic device shown in the above figure does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than the figure, or combine certain components, or different component arrangements. For example, the electronic device also includes radio frequency circuit, input unit, sensor, audio circuit, wireless fidelity (Wireless Fidelity, WiFi) module, power supply, Bluetooth module and other components, which will not be repeated here.

[0111] In Figure 6 In the electronic device shown, the processor 110 can be configured to invoke the multi-camera cascading application stored in the memory 120, and specifically perform the following operations:

[0112] At least two cameras are used to collect scene image information of a target scene, and portrait detection processing is performed on the scene image information to obtain portrait information collected by each camera. The shooting angles of each camera in the at least two cameras are different for the target scene.

[0113] The first camera collects first portrait information of a target person, and the second camera collects all portrait information. The first camera is any camera in the at least two cameras, and the second camera is a camera other than the first camera in the at least two cameras.

[0114] The first portrait information and the all portrait information collected by the second camera are subjected to portrait feature matching processing to obtain second portrait information of the target person collected by each camera in the second camera.

[0115] The deflection angle of the target person in each portrait information in the first portrait information and the second portrait information is obtained.

[0116] In the first portrait information and the second portrait information, the target portrait information corresponding to the minimum deflection angle is determined.

[0117] The close-up image of the target person is output by the target camera corresponding to the target portrait information.

[0118] In one embodiment, when the processor 110 performs the operation of obtaining the first portrait information of the target person collected by the first camera, the processor 110 specifically performs the following operations:

[0119] The close-up instruction of the target person in the first scene image information collected by the first camera is obtained.

[0120] Obtain first portrait information of the target person in the first scene image information.

[0121] In an embodiment, the processor 110, when performing the portrait feature matching processing on the first portrait information and all portrait information collected by the second camera, specifically performs the following operations to obtain second portrait information of the target person collected by each camera of the second camera:

[0122] Perform feature extraction processing on the first portrait information to obtain first feature data;

[0123] Perform feature extraction processing on all portrait information collected by the second camera to obtain second feature data;

[0124] Obtain target feature data matching the first feature data in the second feature data, and determine the portrait information corresponding to the target feature data as the second portrait information of the target person.

[0125] In an embodiment, the processor 110, when performing the portrait feature matching processing on the first portrait information and all portrait information collected by the second camera, specifically performs the following operations to obtain second portrait information of the target person collected by each camera of the second camera:

[0126] Calculate the similarity of each feature data in the second feature data and the first feature data;

[0127] Confirm the feature data in the second feature data with a similarity greater than a similarity threshold as the target feature data matching the first feature data.

[0128] In an embodiment, the processor 110, when performing the portrait feature matching processing on the first portrait information and all portrait information collected by the second camera, specifically performs the following operations to obtain second portrait information of the target person collected by each camera of the second camera:

[0129] Determine the minimum deflection angle with the smallest value among all deflection angles;

[0130] Determine the target portrait information corresponding to the minimum deflection angle among the deflection angles in the first portrait information and the second portrait information.

[0131] In an embodiment, the processor 110, when performing the portrait feature matching processing on the first portrait information and all portrait information collected by the second camera, specifically performs the following operations to obtain second portrait information of the target person collected by each camera of the second camera:

[0132] If there are at least two minimum deflection angles, confirm the portrait information corresponding to any minimum deflection angle among the at least two minimum deflection angles as the target portrait information.

[0133] In one embodiment, the processor 110, when outputting the close-up image of the target person using the target camera corresponding to the target portrait information, specifically performs the following operations:

[0134] If the target person does not exist in the second scene image information collected by the second camera, the close-up image of the target person is output using the first camera.

[0135] In the embodiment, the scene image information of the target scene is collected using at least two cameras, portrait detection processing is performed on the scene image information to obtain portrait information collected by each camera, the shooting angles of each camera in the at least two cameras are different for the target scene, the first portrait information of the target person collected by the first camera is obtained, and all the portrait information collected by the second camera is obtained. The target person can be the person specified by the user to output the close-up image, and can output the close-up image of all the persons in the target scene according to the user's demand, and the camera outputting the close-up image is determined for all the persons in turn. The first portrait information is subjected to feature extraction processing to obtain first feature data, all the portrait information collected by the second camera is subjected to feature extraction processing to obtain second feature data, the target feature data matching the first feature data is obtained in the second feature data, the portrait information corresponding to the target feature data is determined as the second portrait information of the target person, the portrait information belonging to the target person is determined through feature data matching, the accuracy of the target portrait information is ensured, the accuracy of the close-up image output is further improved, the deflection angle of the target person relative to the camera in each portrait information is obtained in the first portrait information and the second portrait information, the target portrait information corresponding to the minimum deflection angle is determined in the first portrait information and the second portrait information, and the close-up image of the target person is output using the target camera corresponding to the target portrait information. The portrait information belonging to one person is obtained by matching the portrait information collected by each camera, and the camera outputting the close-up of the person is determined according to the portrait information, so that the close-up image of the most frontal face in the scene can be output without limiting the position of the camera, and the use convenience of the camera and the accuracy of the face image are improved.

[0136] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware, and the program can be stored in a computer readable storage medium. When the program is executed, the processes of the above-mentioned embodiments can be included. The storage medium can be a magnetic disc, an optical disc, a read-only memory or a random access memory, etc.

[0137] The above merely provides the preferred embodiments of the present application, and cannot be used to limit the scope of the present application. Any equivalent changes made according to the claims of the present application shall still fall within the scope of the present application.

[0138] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, the scene image information, portrait information and the like involved in the present specification are acquired under full authorization.

Claims

1. A method for multi-camera cascading, characterized in that, The method includes: At least two camera devices are used to acquire scene image information of the target scene, and human image detection processing is performed on the scene image information to obtain human image information acquired by each camera device. Each of the at least two camera devices has a different shooting angle for the target scene. Acquire the first portrait information of the target person captured by the first camera device, and acquire all portrait information captured by the second camera device, wherein the first camera device is any one of the at least two camera devices, and the second camera device is any one of the at least two camera devices other than the first camera device; The first portrait information and all portrait information collected by the second camera device are subjected to portrait feature matching processing to obtain the second portrait information of the target person collected by each camera device in the second camera device; Obtain the deflection angle of the target person relative to the camera device in each of the first and second portrait information; From the first portrait information and the second portrait information, determine the target portrait information corresponding to the minimum deflection angle; The target camera device corresponding to the target portrait information outputs a close-up image of the target person, wherein the target camera device is either the first camera device or the second camera device.

2. The method according to claim 1, characterized in that, The acquisition of the first portrait information of the target person captured by the first camera device includes: Obtain a close-up command from the user for the target person in the first scene image information captured by the first camera device; Obtain the first portrait information of the target person from the first scene image information.

3. The method according to claim 1, characterized in that, The step of performing facial feature matching processing on the first facial information and all facial information collected by the second camera device to obtain the second facial information of the target person collected by each camera device in the second camera device includes: The first portrait information is subjected to feature extraction processing to obtain the first feature data; Feature extraction processing is performed on all portrait information collected by the second camera device to obtain second feature data; Target feature data matching the first feature data is obtained from the second feature data, and the portrait information corresponding to the target feature data is determined as the second portrait information of the target person.

4. The method according to claim 3, characterized in that, The step of obtaining target feature data that matches the first feature data from the second feature data includes: Calculate the similarity between each feature data in the second feature data and the first feature data; In the second feature data, the feature data with a similarity greater than the similarity threshold is identified as the target feature data that matches the first feature data.

5. The method according to claim 1, characterized in that, The step of determining the target portrait information corresponding to the minimum deflection angle from the first portrait information and the second portrait information includes: Among all the deflection angles mentioned, determine the minimum deflection angle with the smallest value; From the first portrait information and the second portrait information, determine the target portrait information corresponding to the minimum deflection angle.

6. The method according to claim 5, characterized in that, Determining the target portrait information corresponding to the minimum deflection angle from the first portrait information and the second portrait information includes: If there are at least two minimum deflection angles, then the portrait information corresponding to any one of the at least two minimum deflection angles is identified as the target portrait information.

7. The method according to claim 1, characterized in that, The step of using the target camera device corresponding to the target portrait information to output a close-up image of the target person includes: If the target person is not present in the second scene image information captured by the second camera device, then the first camera device outputs a close-up image of the target person.

8. A multi-camera cascade device, characterized in that, The device includes: A human image information acquisition module is used to acquire scene image information of a target scene using at least two camera devices, and to perform human image detection processing on the scene image information to obtain human image information acquired by each camera device. The at least two camera devices have different shooting angles for the target scene. The feature matching module is used to obtain the first portrait information of the target person collected by the first camera device and to obtain all portrait information collected by the second camera device, wherein the first camera device is any one of the at least two camera devices, and the second camera device is any one of the at least two camera devices other than the first camera device. The portrait matching module is used to perform portrait feature matching processing on the first portrait information and all portrait information collected by the second camera device to obtain the second portrait information of the target person collected by each camera device in the second camera device; The deflection angle acquisition module is used to acquire the deflection angle of the target person relative to the camera device in each of the first portrait information and the second portrait information; The target portrait acquisition module is used to determine the target portrait information corresponding to the minimum deflection angle from the first portrait information and the second portrait information; The close-up image output module is used to output a close-up image of the target person using the target camera device corresponding to the target portrait information, wherein the target camera device is the first camera device or the second camera device.

9. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions, which are adapted to be loaded by a processor and executed as method steps as claimed in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and executed the method steps as claimed in any one of claims 1 to 7.

Images