An image processing method, device and equipment suitable for a multi-person group photo scene, and a storage medium

Abstract

This application discloses an image processing method, apparatus, device, and storage medium applicable to group photo scenarios, relating to the field of image processing technology. The method includes: detecting faces in each image frame of a video stream using a face detection model to obtain an initial result containing bounding boxes and key points; filtering each frame using preset conditions to obtain a target face set; determining a composition strategy based on the number of faces in the set: if there is one face, a first cropping box is determined based on a preset single-face bounding box, an ideal composition ratio, and composition points; if there are at least two faces, an outer envelope rectangle is constructed based on each face bounding box, and a second cropping box is determined based on a preset expansion coefficient and a target aspect ratio; determining the change in the number of faces between consecutive frames in each image frame; smoothly adjusting the first and second cropping boxes to obtain a target cropping box; and cropping image frames based on the target cropping box to generate a centered image of the person, thereby improving the efficiency of centering the image of the person in a group photo scenario.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, and storage medium suitable for group photos. Background Technology

[0002] Currently, in photographic art, there are specialized techniques to address the complex needs of group photography. By carefully placing the core group of people in the center of the frame, a stable and prominent visual focus is created, ensuring that each individual in the group photo is clearly presented and the overall composition is harmonious and unified. This effectively guides the viewer's attention to the interaction and emotional expression of the people. With technological advancements, automated centering composition technology based on face detection, pose recognition, and intelligent algorithms can analyze the positional relationships of multiple people and elements in the frame in real time, automatically cropping or adjusting the image to center the group of people and optimize spatial allocation, thus revolutionizing the traditional group photo process. This technology significantly improves shooting efficiency and image quality, reduces human coordination and post-processing time, and greatly lowers the technical threshold for professional composition. It empowers ordinary users to easily create professional-quality group images, promoting the intelligent and convenient development of group photography, and has profound significance for promoting social sharing, commemorative creation, and the digital transformation of the imaging industry.

[0003] Currently, group photos and live video streaming scenarios suffer from challenges in composition efficiency and final image quality due to dynamic changes in the number of people and distances. Traditional shooting methods for group photos often require repeated manual adjustments to focus and framing, which is not only cumbersome and slow to respond, but also prone to cropping people or creating an unbalanced composition, severely impacting the shooting experience and content expression. Especially in scenarios with high real-time requirements such as mobile live streaming, users struggle to balance ensuring people are in the frame with harmonious composition, resulting in unprofessional images and compromised communication effectiveness.

[0004] As can be seen from the above, how to improve the efficiency of centering the portraits in a group photo is an urgent problem to be solved in the image processing process applicable to group photo scenarios. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide an image processing method, apparatus, device, and storage medium suitable for group photos, which can improve the efficiency of centering the portraits in a group photo during image processing. The specific solution is as follows: Firstly, this application provides an image processing method suitable for group photos, including: A face detection model is used to detect faces in each image frame of a video stream in a group photo scene, resulting in initial detection results including bounding boxes and key points of several face regions. These initial detection results are then filtered based on preset filtering conditions to obtain a set of target faces corresponding to each image frame. The preset filtering conditions include a preset frontal face condition and a preset subject size condition. The preset frontal face condition is a condition for filtering face pose based on a preset frontal face pose; the preset subject size condition is a condition for filtering image size based on the size corresponding to the bounding boxes of the face regions. The composition strategy corresponding to the image frame is determined based on the number of faces in the target face set; When the number of faces is one, the first cropping box is determined by using the composition strategy corresponding to a single person and based on the preset single face region bounding box, the preset ideal composition ratio and the preset composition point. When there are at least two faces, an outer envelope rectangle is constructed based on the bounding boxes of each face region. Then, a second cropping frame is determined based on the outer envelope rectangle, the preset expansion coefficient, and the target aspect ratio using a preset group photo composition mode and the corresponding composition strategy. The state machine determines the change data of the number of faces in the preceding and following frames of each image frame, and the first cropping box and the second cropping box are smoothly adjusted based on the change data to obtain a target cropping box. Each image frame is then cropped based on the target cropping box to obtain a centered image of the human figure.

[0006] Optionally, the face detection model is used to perform face detection on each image frame in the video stream data of the group photo scene to obtain an initial detection result including several face region bounding boxes and key points. The initial detection result is then filtered based on preset filtering conditions to obtain a set of target faces corresponding to each image frame, including: The video stream data, which includes several consecutive image frames, is acquired, and a face detection model is used to perform face detection on the current image frame to obtain an initial detection result including several face region bounding boxes and corresponding face key points, wherein the face key points include binocular key points. Based on the initial detection results, the straight-line distance between the key points of each person's eyes is determined, and the straight-line distance is compared with a preset eye distance threshold to obtain the comparison result; If the comparison result indicates that the straight-line distance is not less than the preset eye distance threshold, then the pose corresponding to the face is set as a frontal pose and retained; if the comparison result indicates that the straight-line distance is less than the preset eye distance threshold, then the pose corresponding to the face is set as a side pose and filtered out. Determine the width and height values ​​corresponding to the bounding boxes of each face region in the initial detection results, and set the maximum value of the width and height values ​​as the current size corresponding to the face. Then, construct a size set based on the current size of all faces that are determined to be frontal faces in the current image frame. The current size with the largest value in the size set is set as the target size threshold, and the data to be compared is determined based on the target size threshold and the preset ratio threshold. Then, the current size of each face is compared with the data to be compared. If the current size is not smaller than the data to be compared, the corresponding face is set as the main face and retained. If the current size is smaller than the data to be compared, the corresponding face is set as the secondary face and filtered out, thus obtaining the target face set corresponding to each image frame.

[0007] Optionally, determining the composition strategy corresponding to the image frame based on the number of faces in the target face set includes: The number of faces in the target face set is counted, and the corresponding target composition strategy is determined from each preset composition strategy based on the number of faces; The preset composition strategies include a first composition strategy for handling cases without valid faces, a second composition strategy for handling single-person portraits, and a third composition strategy for handling group photos.

[0008] Optionally, when the number of faces is one, determining the corresponding first cropping box using a composition strategy corresponding to a single person and based on a preset single-face region bounding box, a preset ideal composition ratio, and preset composition points includes: When the number of faces is one, determine the upper left and lower right corner coordinates of a single face region in the current image frame, and construct a face bounding box based on the upper left and lower right corner coordinates; The face width and face height are determined based on the face bounding box, and the face center coordinates are determined based on the face width and face height. Then, the target composition point coordinates, including the horizontal and vertical composition point coordinates, are obtained. Based on user needs, determine the preset composition ratio, preset target aspect ratio, preset scaling factor, preset proportion range, and preset offset threshold; A first ratio between the face width and the image width of the current image frame is determined, and a second ratio between the face height and the image height of the current image frame is determined. Then, the larger of the first ratio and the second ratio is set as the current proportion of the face. Determine the offset of the face center coordinates relative to the preset composition point coordinates, determine whether the current proportion of the face is within the preset proportion range, and then determine whether the offset is greater than the preset offset threshold. If the current proportion of the face is not within the preset proportion range, or if the offset is greater than the preset offset threshold, then the cropping frame width and cropping frame height corresponding to the cropping frame are determined based on the ratio between the face width and the face height and the preset target aspect ratio. If the ratio is not less than the preset target aspect ratio, then the cropping frame width is determined based on the face width, the preset ideal composition ratio, and the preset scaling factor, and then the cropping frame height is determined based on the cropping frame width and the preset target aspect ratio. If the ratio is less than the preset target aspect ratio, the cropping frame height is determined based on the face height, the preset ideal composition ratio, and the preset scaling factor, and the cropping frame width is determined based on the cropping frame height and the preset target aspect ratio. The cropping frame center coordinates are determined based on the face center coordinates, the cropping frame width, the cropping frame height, and the preset composition point coordinates, so as to determine the corresponding first cropping frame based on the cropping frame center coordinates, the cropping frame width, and the cropping frame height.

[0009] Optionally, when the number of faces is at least two, constructing an outer envelope rectangle based on the bounding boxes of each face region, and then determining a second cropping box based on the outer envelope rectangle, a preset expansion coefficient, and a target aspect ratio using a preset group photo composition mode and corresponding composition strategy, includes: When the number of faces is at least two, the number of faces is compared with a preset multi-person determination threshold. When the number of faces is not less than the preset multi-person determination threshold, the face bounding boxes of each face region are used to extract the corresponding upper left horizontal coordinate, upper left vertical coordinate, lower right horizontal coordinate and lower right vertical coordinate from each face bounding box based on a preset coordinate region. The minimum values ​​of the left-left corner x-coordinates and the left-left corner y-coordinates are respectively set as the left boundary and the upper boundary of the outer envelope rectangle, and the maximum values ​​of the right-right corner x-coordinates and the lower boundary of the outer envelope rectangle are respectively set as the right boundary and the lower boundary of the outer envelope rectangle, so as to construct an outer envelope rectangle based on the left boundary, the upper boundary, the right boundary, and the lower boundary of the outer envelope rectangle; A preset expansion coefficient and a preset target aspect ratio are determined. Then, based on the width and height of the outer envelope rectangle and the preset expansion coefficient, the expanded width and expanded height of the expanded rectangle are determined. The expanded rectangle is a rectangle that expands outward from the center of the outer envelope rectangle by a preset range. The expanded rectangle is adjusted based on the expanded width, expanded height, and the preset target aspect ratio to obtain the second cropping frame.

[0010] Optionally, the step of determining the change data of the number of faces in the preceding and following frames of each image frame based on a state machine, and smoothly adjusting the first cropping box and the second cropping box based on the change data to obtain the target cropping box, includes: Obtain the third cropping box corresponding to the previous image frame, and determine the cropping box to be processed based on the first cropping box and the second cropping box; the third cropping box is the actual cropping area corresponding to the cropping operation performed on the previous image frame; Based on the mapping relationship between the third cropping box and the cropping box to be processed, the cropping box to be processed is mapped from the cropping map coordinate system of the previous image frame back to the original image coordinate system to obtain the mapped target cropping box; The corresponding smoothing adjustment method is determined by using a preset face count change confirmation mechanism and based on the comparison between the face count of the current image frame and the face count of the previous image frame. If the comparison result indicates an increase in the number of faces, then the adjustment step size of each image frame is determined based on the preset adjustment step size and the difference between the target cropping box and the current cropping box, so as to update the target cropping box based on the smooth adjustment method and the adjustment step size to obtain a new target cropping box; When the comparison result indicates that the number of faces has not increased, the adjustment step size of each image frame is determined by using a preset smoothing adjustment method and based on a preset smoothing factor and the difference, so as to update the target cropping box based on the adjustment step size and obtain a new target cropping box. If no valid face is detected in a consecutive preset number of image frames, the current cropping box is reset to the target cropping box.

[0011] Optionally, cropping each of the image frames based on the target cropping box to obtain a centered image of the person includes: Based on the shooting mode of the current image frame and the position of the target cropping box in the current image frame, determine whether the current image frame meets the conditions for switching between landscape and portrait modes; If the shooting mode is portrait mode and the lower boundary of the target cropping frame exceeds the bottom of the current image frame, then the current image frame is determined to meet the landscape / portrait switching condition, and the shooting mode is switched to landscape mode. Based on the switched shooting mode, the corresponding target output resolution, including landscape output resolution and portrait output resolution, is determined. Then, the target cropping box is used to crop the current image frame to obtain the cropped image. If the shooting mode is landscape mode, the cropped image is scaled to the landscape output resolution based on the target output resolution; if the shooting mode is portrait mode, the cropped image is scaled to a preset intermediate resolution, and then filled on the left and right sides of the cropped image based on the portrait output resolution to obtain a standardized image. An initial visualization mask image is generated based on the target cropping box and the face region bounding box, and an identifier graphic corresponding to the face region bounding box and the target cropping box is drawn on the initial visualization mask image to obtain a target visualization mask image; the visualization mask image includes a visualization effect of blurring the non-cropped areas. A centered human figure composition image is constructed based on the standardized image and the target visualization mask image.

[0012] Secondly, this application provides an image processing apparatus suitable for group photo scenarios, comprising: The detection result generation module is used to perform face detection on each image frame in the video stream data of the group photo scene using a face detection model, to obtain an initial detection result including several face region bounding boxes and key points, and to filter the initial detection result based on preset filtering conditions to obtain a set of target faces corresponding to each image frame; the preset filtering conditions include preset frontal face conditions and preset subject size conditions; the preset frontal face conditions are conditions for filtering face pose based on preset frontal face pose; the preset subject size conditions are conditions for filtering image size based on the size corresponding to the face region bounding boxes; The composition strategy determination module is used to determine the composition strategy corresponding to the image frame based on the number of faces in the target face set; The first cropping frame determination module is used to determine the corresponding first cropping frame when the number of faces is one, by using a composition strategy corresponding to a single person and based on a preset single face region bounding box, a preset ideal composition ratio and a preset composition point. The second cropping frame determination module is used to construct an outer envelope rectangle based on the bounding boxes of each face region when the number of faces is at least two, and then determine the second cropping frame based on the outer envelope rectangle, the preset expansion coefficient and the target aspect ratio using a preset group photo composition mode and the corresponding composition strategy. The composition image generation module is used to determine the change data of the number of faces in the preceding and following frames of each image frame based on a state machine, and to smoothly adjust the first cropping box and the second cropping box based on the change data to obtain a target cropping box, and to crop each image frame based on the target cropping box to obtain a composition image with the human face centered.

[0013] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned image processing method suitable for group photos.

[0014] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned image processing method applicable to multi-person group photo scenarios.

[0015] As can be seen from the above, before performing image processing applicable to group photo scenarios, this application needs to use a face detection model to perform face detection on each image frame in the video stream data of the group photo scenario, obtaining initial detection results including bounding boxes and key points of several face regions. Then, the initial detection results are filtered based on preset filtering conditions to obtain target face sets corresponding to each image frame. The preset filtering conditions include preset frontal face conditions and preset subject size conditions. The composition strategy corresponding to the image frame is determined based on the number of faces in the target face set. When the number of faces is one, the composition strategy corresponding to a single person is used. The first cropping frame is determined based on the preset single-face region bounding box, the preset ideal composition ratio, and the preset composition points. When there are at least two faces, an outer envelope rectangle is constructed based on the bounding boxes of each face region. Then, a second cropping frame is determined based on the preset group photo composition mode and the corresponding composition strategy, the outer envelope rectangle, the preset expansion coefficient, and the target aspect ratio. The change data of the number of faces in the previous and next frames of each image frame is determined based on the state machine. The first and second cropping frames are then smoothly adjusted based on the change data to obtain the target cropping frame. Each image frame is then cropped based on the target cropping frame to obtain a composition image with the portrait centered.

[0016] Therefore, this application first needs to use a face detection model to perform face detection on each image frame in the video stream data of the group photo scene, obtaining an initial detection result including several face region bounding boxes and key points. Then, the initial detection result is filtered based on preset filtering conditions to obtain the target face set corresponding to each image frame. Secondly, the composition strategy corresponding to the image frame is determined based on the number of faces in the target face set. When the number of faces is one, a composition strategy corresponding to a single person is used, based on a preset single face region bounding box, a preset ideal composition ratio, and a preset composition... The first cropping frame is determined by clicking on the corresponding frame. Then, when the number of faces is no greater than a preset number of faces, an outer envelope rectangle is constructed based on the bounding boxes of each face region. A second cropping frame is then determined using a preset group photo composition mode and corresponding composition strategy, based on the outer envelope rectangle, a preset expansion coefficient, and the target aspect ratio. Finally, the change in the number of faces between consecutive frames is determined using a state machine, and the first and second cropping frames are smoothly adjusted based on this change data to obtain the target cropping frame. Each image frame is then cropped based on this target cropping frame to obtain a centered image. This improves the efficiency of centering portraits in group photo scenarios, thus enhancing the user experience. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 This application discloses a flowchart of an image processing method applicable to group photos. Figure 2 This is a schematic diagram of a specific image processing method for group photos disclosed in this application; Figure 3 This is a schematic diagram of an image processing device suitable for group photo scenarios disclosed in this application; Figure 4 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Currently, in photographic art, there are methods specifically designed to address the complex needs of group photography. By carefully placing the core group of people in the center of the frame, a stable and prominent visual focus is created, ensuring that each individual in the group photo is clearly presented and the overall composition is harmonious and unified. This effectively guides the viewer's attention to the interaction and emotional expression of the people. Currently, group photos and live video streaming scenarios suffer from challenges in composition efficiency and image quality due to dynamic changes in the number of people and distance. Traditional shooting methods for group photos often require repeated manual adjustments to the focus and framing, which is not only cumbersome and slow to respond, but also prone to cropping people or creating an unbalanced composition, severely impacting the shooting experience and content expression. Therefore, this application provides an image processing method suitable for group photo scenarios, which can improve the efficiency of centering portraits in group photo scenarios during image processing.

[0021] See Figure 1 As shown, this embodiment of the invention discloses an image processing method suitable for group photos, including: Step S11: Use a face detection model to perform face detection on each image frame in the video stream data of the group photo scene, and obtain initial detection results including several face region bounding boxes and key points. Then, filter the initial detection results based on preset filtering conditions to obtain the target face set corresponding to each image frame. The preset filtering conditions include preset frontal face conditions and preset subject size conditions. The preset frontal face conditions are conditions for filtering face pose based on preset frontal face pose. The preset subject size conditions are conditions for filtering image size based on the size corresponding to the face region bounding box.

[0022] In this embodiment, the overall process for image processing applicable to group photo scenarios is as follows: Figure 2 As shown: First, real-time face detection and filtering are performed: the system captures the input image in real time. First, a lightweight face detection model is loaded and initialized, the core of which is to quickly and accurately output the bounding boxes and key points of the face regions. Then, the detection results are filtered to remove interference, ensuring that the image focuses on the valid subject.

[0023] The filtering criteria are defined by the following formula: Frontal face determination: Calculate the Euclidean distance between the key points of both eyes: ; Subsequently, retain human face ( (Use a threshold, such as 20 pixels), to filter out side profiles.

[0024] Next, the main body size is determined, that is, the face size is calculated: ; in, The set of all face sizes in the current frame. This is a proportional threshold, such as 0.45.

[0025] Subsequently, retain The faces are analyzed, and irrelevant faces that are too small or too far away are filtered out to obtain the effective set of faces. their quantity As a key input for subsequent decision-making.

[0026] Specifically, the step of using a face detection model to perform face detection on each image frame in the video stream data of a group photo scene, obtaining an initial detection result including several face region bounding boxes and key points, and filtering the initial detection result based on preset filtering conditions to obtain a set of target faces corresponding to each image frame, may include: acquiring video stream data including several consecutive image frames, and using a face detection model to perform face detection on the current image frame to obtain an initial detection result including several face region bounding boxes and corresponding face key points, wherein the face key points include binocular key points; determining the straight-line distance between the binocular key points of each face based on the initial detection result, and comparing the straight-line distance with a preset eye distance threshold to obtain a comparison result; if the comparison result indicates that the straight-line distance is not less than the preset eye distance threshold, then the pose corresponding to the face is set as a frontal pose and retained ... If the straight-line distance is less than the preset eye distance threshold, the pose corresponding to the face is set as a side-face pose and filtered out. The width and height values ​​of the bounding boxes of each face region in the initial detection results are determined, and the maximum value of the width and height values ​​is set as the current size corresponding to the face. Then, a size set is constructed based on the current sizes of all faces determined to be frontal poses in the current image frame. The current size with the largest value in the size set is set as the target size threshold, and comparison data is determined based on the target size threshold and a preset ratio threshold. Then, the current size of each face is compared with the comparison data. If the current size is not less than the comparison data, the corresponding face is set as the main face and retained; if the current size is less than the comparison data, the corresponding face is set as a secondary face and filtered out, resulting in a set of target faces corresponding to each image frame.

[0027] Step S12: Determine the composition strategy corresponding to the image frame based on the number of faces in the target face set.

[0028] In this embodiment, the present application requires the generation of an adaptive dynamic composition strategy based on the number of faces, that is, based on the number of valid faces. Automatically selects preset composition strategies to achieve a seamless transition from single-person close-ups to group photos: (1) Mode M0 (N=0): No valid face, use the default panoramic view or the preset center frame.

[0029] (2) Mode M1 (N=1): Single portrait mode. Initiates a detailed composition calculation process centered on the face and conforming to portrait aesthetics.

[0030] (3) Mode M2 ​​(N≥2): Group photo mode. Initiates a relaxed composition calculation process centered on the group, ensuring that everyone is in the frame and the layout is coordinated.

[0031] It is worth mentioning that the above steps, as a routing layer, direct the image stream to different dedicated graphing calculation modules.

[0032] Specifically, determining the composition strategy corresponding to the image frame based on the number of faces in the target face set may include: counting the number of faces in the target face set, and determining the corresponding target composition strategy from each preset composition strategy based on the number of faces; wherein, the preset composition strategies include a first composition strategy for handling cases without valid faces, a second composition strategy for handling single portrait cases, and a third composition strategy for handling group photos.

[0033] Step S13: When the number of faces is one, the first cropping frame is determined by using the composition strategy corresponding to a single person and based on the preset single face region bounding box, the preset ideal composition ratio and the preset composition point.

[0034] In this embodiment, the algorithm performs refined centering composition for a single-person scene: when in mode M1, the algorithm performs refined composition for a single face. Let the coordinates of the upper left and lower right corners of the face frame be... Its center is Width and height are Then define the target aspect ratio. (Depending on the shooting mode, portrait mode is 3:4, landscape mode is 4:3).

[0035] It is worth mentioning that the above algorithm is designed to calculate the clipping frame. This allows the human face to be in ideal proportion. (e.g., 0.275) is a preset composition point located within the frame. For example, around (0.5, 0.35), where, , The coordinates of the cropping frame center are: , This is for the width and height of the cropping frame. The calculation logic is as follows: (1) Determine if adjustment is needed: Calculate the current proportion of the face relative to the image frame. and its center offset: ; Among them, if or This triggers the composition clipping calculation.

[0036] (2) Calculate the target clipping frame: like : ; otherwise: , , ; in, The scaling factor is used to obtain the final integer value. .

[0037] Specifically, when the number of faces is one, determining the corresponding first cropping box using a composition strategy corresponding to a single person and based on a preset single-face region bounding box, a preset ideal composition ratio, and preset composition points may include: when the number of faces is one, determining the upper-left and lower-right coordinates of a single face region in the current image frame, and constructing a face bounding box based on the upper-left and lower-right coordinates; determining the face width and face height based on the face bounding box, and determining the face center coordinates based on the face width and face height, and then obtaining the target including horizontal and vertical composition point coordinates. The system determines the following based on user needs: composition point coordinates; a preset composition ratio, a preset target aspect ratio, a preset scaling factor, a preset proportion range, and a preset offset threshold; a first ratio between the face width and the image width of the current image frame, and a second ratio between the face height and the image height of the current image frame, then sets the larger of the first and second ratios as the current proportion of the face; the system determines the offset of the face center coordinates relative to the preset composition point coordinates, and determines whether the current proportion of the face is within the preset proportion range, then determines whether the offset is greater than the preset offset threshold. Furthermore, if the current proportion of the face is not within the preset proportion range, or if the offset is greater than the preset offset threshold, then the cropping frame width and height corresponding to the cropping frame are determined based on the ratio between the face width and the face height and the preset target aspect ratio; if the ratio is not less than the preset target aspect ratio, then the cropping frame width is determined based on the face width, the preset ideal composition ratio, and the preset scaling factor, and then the cropping frame height is determined based on the cropping frame width and the preset target aspect ratio; if the ratio is less than the preset target aspect ratio, then the cropping frame height is determined based on the face height, the preset ideal composition ratio, and the preset scaling factor, and the cropping frame width is determined based on the cropping frame height and the preset target aspect ratio; the cropping frame center coordinates are determined based on the face center coordinates, the cropping frame width, the cropping frame height, and the preset composition point coordinates, so as to determine the corresponding first cropping frame based on the cropping frame center coordinates, the cropping frame width, and the cropping frame height.

[0038] Step S14: When the number of faces is at least two, construct an outer envelope rectangle based on the bounding boxes of each face region, and then use a preset group photo composition mode and corresponding composition strategy to determine a second cropping frame based on the outer envelope rectangle, a preset expansion coefficient and a target aspect ratio.

[0039] In this embodiment, the present application requires calculation of the group inclusion and centering composition of a multi-person scene: In mode M2, the algorithm aims to include all faces while maintaining a harmonious layout. Let the set of all face bounding boxes be... Its corresponding outer envelope rectangle is , where are the coordinates of the leftmost, rightmost, topmost, and bottommost points of the rectangle, respectively. Then, the expansion coefficient is defined. (e.g., 2.0, 0.3) are used to add appropriate margins. The target clipping box calculation process is shown below: (1) Calculate the core area of ​​the composition: based on the maximum face size Based on Determine the core areas that should be included: ; ; ; ; (2) Generate a centered cropping box: centering on the core area Based on the target aspect ratio and upward factor (e.g., 2.5, to position the face at the top of the image), calculate the cropping frame: When in portrait mode: ; ; ; ; When in landscape mode: ; ; ; ; in, As the expansion factor, the final result is obtained by rounding. .

[0040] Specifically, when the number of faces is at least two, constructing an outer envelope rectangle based on the bounding boxes of each face region, and then determining a second cropping frame based on the outer envelope rectangle, a preset expansion coefficient, and a target aspect ratio using a preset group photo composition mode and corresponding composition strategy, may include: when the number of faces is at least two, comparing the number of faces with a preset multi-person determination threshold; when the number of faces is not less than the preset multi-person determination threshold, extracting the corresponding upper-left horizontal coordinate, upper-left vertical coordinate, lower-right horizontal coordinate, and lower-right vertical coordinate from each face bounding box based on the bounding boxes of each face region and a preset coordinate region; setting the minimum value among the upper-left horizontal coordinate and the upper-left vertical coordinate as the left and right sides of the outer envelope rectangle. The boundary and the upper boundary of the outer envelope rectangle are defined, and the maximum value of each of the lower right corner horizontal coordinates and each of the lower right corner vertical coordinates is set as the right boundary and the lower boundary of the outer envelope rectangle, respectively. An outer envelope rectangle is constructed based on the left boundary, the upper boundary, the right boundary, and the lower boundary of the outer envelope rectangle. A preset expansion coefficient and a preset target aspect ratio are determined. Then, based on the width and height of the outer envelope rectangle and the preset expansion coefficient, the expanded width and expanded height of the expanded rectangle are determined. The expanded rectangle is a rectangle that expands outward from the center of the outer envelope rectangle by a preset range. The expanded rectangle is adjusted based on the expanded width, expanded height, and the preset target aspect ratio to obtain the second clipping frame.

[0041] Step S15: Determine the change data of the number of faces in the preceding and following frames of each image frame based on the state machine, and smoothly adjust the first cropping box and the second cropping box based on the change data to obtain the target cropping box. Crop each image frame based on the target cropping box to obtain a centered image of the human face.

[0042] In this embodiment, the clipping frame needs to be smoothly adjusted, its coordinates mapped, and its robustness controlled based on a state machine. Furthermore, to ensure visual continuity and system stability, the algorithm introduces a state machine for intelligent adjustment.

[0043] First, the embodiments of this application require coordinate mapping: for frame t>0, the composition calculation is performed on the cropped image of the previous frame. Proceed to the next step. New coordinates. Map back to the original image by translation: ; Secondly, perform adaptive smoothing adjustments: Quick Adjustment: Triggered when the number of faces N increases. (Total number of steps) Then the first The adjustment amount for the step is: ; Subsequently, updates .

[0044] Gradual adjustment: Triggered when the number of people decreases or becomes 1.

[0045] Let smoothing factor The adjustment amount per frame is: ; Subsequently, updates .

[0046] Furthermore, robustness enhancement operations are implemented: a counting mechanism is employed. Increase the number of confirmed cases to avoid false positives due to flickering; set a timeout counter for cases without a face. After the timeout, it will automatically reset to the default panoramic view.

[0047] Specifically, the step of determining the change data of the number of faces in the preceding and following frames of each image frame based on a state machine, and smoothly adjusting the first and second cropping boxes based on the change data to obtain the target cropping box, may include: obtaining a third cropping box corresponding to the previous image frame, and determining a cropping box to be processed based on the first and second cropping boxes; the third cropping box is the actual cropping area corresponding to the cropping operation performed on the previous image frame; mapping the cropping box to be processed from the cropping map coordinate system of the previous image frame back to the original image coordinate system based on the mapping relationship between the third cropping box and the cropping box to be processed, to obtain the mapped target cropping box; and using a preset face number change confirmation mechanism and based on the number of faces in the current image frame and the change data... The comparison result between the number of faces in the previous image frame determines the corresponding smoothing adjustment method; if the comparison result indicates that the number of faces has increased, the adjustment step size of each image frame is determined based on a preset adjustment step size and the difference between the target cropping box and the current cropping box, so as to update the target cropping box based on the smoothing adjustment method and the adjustment step size to obtain a new target cropping box; when the comparison result indicates that the number of faces has not increased, the preset smoothing adjustment method is used and the adjustment step size of each image frame is determined based on a preset smoothing factor and the difference, so as to update the target cropping box based on the adjustment step size to obtain a new target cropping box; when no valid face is detected in a consecutive preset number of image frames, the current cropping box is reset to the target cropping box.

[0048] In this embodiment, the system needs to perform landscape / portrait screen adaptation, standardized image cropping, and real-time visualization output. That is, the system dynamically selects the landscape / portrait screen mode based on the composition content. And generate the final output. Specifically, the embodiments of this application require mode adaptation: for example, when Furthermore, in portrait mode, the cropping frame touches the bottom of the screen. When [the screen is in landscape mode], it automatically switches to landscape mode to accommodate more content. Image cropping and standardization are performed: based on the final determined [screen format]. Crop from the original image The image is then scaled to a standard resolution (1280x960 for landscape mode, and first scaled to 720x960 for portrait mode, then filled with black borders to reach 1280x960), and a standardized image is output. Finally, real-time visualization and interaction are achieved: a visualization mask is generated synchronously. Blur the non-frame area and overlay a grayscale layer, while simultaneously drawing a red face frame and a cropping frame. and Used for real-time display, recording, or streaming, completing the entire intelligent mapping process from perception to output.

[0049] Specifically, the step of cropping each image frame based on the target cropping frame to obtain a centered portrait image may include: determining whether the current image frame meets the landscape / portrait switching condition based on the shooting mode of the current image frame and the position of the target cropping frame in the current image frame; if the shooting mode is portrait mode and the lower boundary of the target cropping frame extends beyond the bottom of the current image frame, then the current image frame is determined to meet the landscape / portrait switching condition, and the shooting mode is switched to landscape mode; based on the switched shooting mode, a corresponding target output resolution including landscape output resolution and portrait output resolution is determined, and then the target cropping frame is used to crop the current image frame to obtain the cropped image; if the shooting mode is landscape... If the shooting mode is portrait mode, the cropped image is scaled to the horizontal output resolution based on the target output resolution; if the shooting mode is portrait mode, the cropped image is scaled to a preset intermediate resolution, and then filled on the left and right sides of the cropped image based on the portrait output resolution to obtain a standardized image; an initial visualization mask image is generated based on the target cropping box and the face region bounding box, and a marker graphic corresponding to the face region bounding box and the target cropping box is drawn on the initial visualization mask image to obtain a target visualization mask image; the visualization mask image includes a visualization effect that blurs the non-cropped areas; a centered composition image of the portrait is constructed based on the standardized image and the target visualization mask image.

[0050] As can be seen from the above, the embodiments of this application first need to use a face detection model to perform face detection on each image frame in the video stream data of the group photo scene, to obtain an initial detection result including several face region bounding boxes and key points, and then filter the initial detection result based on preset filtering conditions to obtain each target face set corresponding to each image frame; secondly, the composition strategy corresponding to the image frame is determined based on the number of faces in the target face set; when the number of faces is one, the composition strategy corresponding to a single person is used and based on the preset single face region bounding box, preset ideal composition ratio, and preset... The process involves several steps: First, determining the first cropping frame based on the composition points. Then, when the number of faces is no greater than a preset number, an outer bounding rectangle is constructed based on the bounding boxes of each face region. Next, a second cropping frame is determined using a preset group photo composition mode and corresponding composition strategy, based on the outer bounding rectangle, a preset expansion coefficient, and the target aspect ratio. Finally, a state machine is used to determine the change in the number of faces between consecutive image frames. Based on this change data, the first and second cropping frames are smoothly adjusted to obtain the target cropping frame. This target cropping frame is then used to crop each image frame, resulting in a centered image. This method improves the efficiency of centering portraits in group photos, enhancing the user experience.

[0051] Accordingly, see Figure 3 As shown, this application also provides an image processing apparatus suitable for group photos, comprising: The detection result generation module 11 is used to perform face detection on each image frame in the video stream data of the group photo scene using a face detection model, to obtain an initial detection result including several face region bounding boxes and key points, and to filter the initial detection result based on preset filtering conditions to obtain a set of target faces corresponding to each image frame; the preset filtering conditions include preset frontal face conditions and preset subject size conditions; the preset frontal face conditions are conditions for filtering face pose based on preset frontal face pose; the preset subject size conditions are conditions for filtering image size based on the size corresponding to the face region bounding box; The composition strategy determination module 12 is used to determine the composition strategy corresponding to the image frame based on the number of faces in the target face set; The first cropping frame determination module 13 is used to determine the corresponding first cropping frame when the number of faces is one, by using a composition strategy corresponding to a single person and based on a preset single face region bounding box, a preset ideal composition ratio and a preset composition point. The second cropping frame determination module 14 is used to construct an outer envelope rectangle based on the boundary boxes of each face region when the number of faces is at least two, and then determine the second cropping frame based on the outer envelope rectangle, the preset expansion coefficient and the target aspect ratio using a preset group photo composition mode and the corresponding composition strategy. The composition image generation module 15 is used to determine the change data of the number of faces in the previous and next frames of each image frame based on a state machine, and to smoothly adjust the first cropping box and the second cropping box based on the change data to obtain a target cropping box, and to crop each image frame based on the target cropping box to obtain a composition image with the portrait centered.

[0052] In some specific embodiments, the detection result generation module 11 may specifically include: The face detection unit is used to acquire video stream data including several consecutive image frames, and to perform face detection on the current image frame using a face detection model to obtain an initial detection result including several face region bounding boxes and corresponding face key points, wherein the face key points include binocular key points. The comparison result determination unit is used to determine the straight-line distance between the key points of each eye on each face based on the initial detection result, and compare the straight-line distance with a preset eye distance threshold to obtain the comparison result; A straight-line distance judgment unit is used to set the pose corresponding to the face as a frontal pose and retain it if the comparison result indicates that the straight-line distance is not less than the preset eye distance threshold, and to set the pose corresponding to the face as a side-face pose and filter it out if the comparison result indicates that the straight-line distance is less than the preset eye distance threshold. The size set construction unit is used to determine the width and height values ​​corresponding to the bounding boxes of each face region in the initial detection result, and set the maximum value of the width and height values ​​as the current size corresponding to the face. Then, it constructs a size set based on the current size of all faces that are determined to be frontal faces in the current image frame. The current size comparison unit is used to set the current size with the largest value in the size set as the target size threshold, and to determine the data to be compared based on the target size threshold and the preset ratio threshold, and then compare the current size of each face with the data to be compared. The target face set generation unit is used to set the corresponding face as the main face and retain it if the current size is not smaller than the data to be compared, and to set the corresponding face as the secondary face and perform a filtering operation if the current size is smaller than the data to be compared, so as to obtain the target face sets corresponding to each image frame.

[0053] In some specific embodiments, the mapping strategy determination module 12 may specifically include: A face count unit is used to count the number of faces in the target face set and determine the corresponding target composition strategy from each preset composition strategy based on the number of faces; wherein, the preset composition strategies include a first composition strategy for handling cases with no valid faces, a second composition strategy for handling cases with single portraits, and a third composition strategy for handling cases with group photos.

[0054] In some specific embodiments, the first cropping frame determination module 13 may specifically include: A face bounding box construction unit is used to determine the upper left corner coordinates and lower right corner coordinates of a single face region in the current image frame when the number of faces is one, so as to construct a face bounding box based on the upper left corner coordinates and lower right corner coordinates; The face center coordinate determination unit is used to determine the face width and face height based on the face bounding box, and to determine the face center coordinates based on the face width and face height, and then obtain the target composition point coordinates including the horizontal composition point coordinates and the vertical composition point coordinates; The composition ratio determination unit is used to determine the preset composition ratio, preset target aspect ratio, preset scaling factor, preset proportion range, and preset offset threshold based on user needs. The face current proportion determination unit is used to determine a first ratio between the face width and the image width of the current image frame, and to determine a second ratio between the face height and the image height of the current image frame, and then set the larger value between the first ratio and the second ratio as the face current proportion; The offset determination unit is used to determine the offset of the face center coordinates relative to the preset composition point coordinates, and to determine whether the current proportion of the face is within the preset proportion range, and then to determine whether the offset is greater than the preset offset threshold. The offset determination unit is used to determine the cropping frame width and cropping frame height based on the ratio between the face width and the face height and the preset target aspect ratio if the current proportion of the face is not within the preset proportion range, or if the offset is greater than the preset offset threshold. The cropping frame height determination unit is used to determine the cropping frame width based on the face width, the preset ideal composition ratio, and the preset scaling factor if the ratio is not less than the preset target aspect ratio, and then determine the cropping frame height based on the cropping frame width and the preset target aspect ratio; The cropping frame width determination unit is used to determine the cropping frame height based on the face height, the preset ideal composition ratio, and the preset scaling factor if the ratio is less than the preset target aspect ratio, and to determine the cropping frame width based on the cropping frame height and the preset target aspect ratio. The first cropping frame determination subunit is used to determine the center coordinates of the cropping frame based on the face center coordinates, the cropping frame width, the cropping frame height, and the preset composition point coordinates, so as to determine the corresponding first cropping frame based on the center coordinates of the cropping frame, the cropping frame width, and the cropping frame height.

[0055] In some specific embodiments, the second cropping frame determining module 14 may specifically include: The coordinate extraction unit is used to compare the number of faces with a preset multi-person determination threshold when the number of faces is at least two, and to extract the corresponding upper left horizontal coordinate, upper left vertical coordinate, lower right horizontal coordinate and lower right vertical coordinate from each face boundary box based on the face bounding box of each face region and based on the preset coordinate region. An outer envelope rectangle construction unit is used to set the minimum value of each of the upper left corner horizontal coordinates and each of the upper left corner vertical coordinates as the left boundary and the upper boundary of the outer envelope rectangle, respectively, and to set the maximum value of each of the lower right corner horizontal coordinates and each of the lower right corner vertical coordinates as the right boundary and the lower boundary of the outer envelope rectangle, respectively, so as to construct an outer envelope rectangle based on the left boundary, the upper boundary, the right boundary, and the lower boundary of the outer envelope rectangle; An expansion width and height determination unit is used to determine a preset expansion coefficient and a preset target aspect ratio, and then determine the expansion width and expansion height of the expanded rectangle based on the width and height of the outer envelope rectangle and the preset expansion coefficient; the expanded rectangle is a rectangle that expands a preset range outward from the center of the outer envelope rectangle; The second cropping frame determination sub-unit is used to adjust the expanded rectangle based on the expanded width, expanded height, and the preset target aspect ratio to obtain the second cropping frame.

[0056] In some specific embodiments, the composition image generation module 15 may specifically include: The cropping box determination unit is used to obtain the third cropping box corresponding to the previous image frame, and determine the cropping box to be processed based on the first cropping box and the second cropping box; the third cropping box is the actual cropping area corresponding to the cropping operation performed on the previous image frame; The target cropping box determination unit is used to map the cropping box to be processed from the cropping map coordinate system of the previous image frame back to the original image coordinate system based on the mapping relationship between the third cropping box and the cropping box to be processed, so as to obtain the mapped target cropping box. The smoothing adjustment method determination unit is used to determine the corresponding smoothing adjustment method by using a preset face number change confirmation mechanism and based on the comparison result between the face number of the current image frame and the face number of the previous image frame. The step size determination unit is used to determine the adjustment step size of each image frame based on a preset adjustment step size and the difference between the target cropping box and the current cropping box if the comparison result indicates an increase in the number of faces, so as to update the target cropping box based on the smooth adjustment method and the adjustment step size to obtain a new target cropping box. The step size generation unit is used to determine the adjustment step size of each image frame by using a preset smooth adjustment method and based on a preset smoothing factor and the difference when the comparison result indicates that the number of faces has not increased, so as to update the target cropping box based on the adjustment step size and obtain a new target cropping box. The target cropping box generation unit is used to reset the current cropping box to the target cropping box when no valid face is detected in a consecutive preset number of image frames.

[0057] In some specific embodiments, the composition image generation module 15 may specifically include: The first image frame determination unit is used to determine whether the current image frame meets the landscape / portrait switching conditions based on the shooting mode of the current image frame and the position of the target cropping box in the current image frame. The second image frame determination unit is used to determine that the current image frame meets the landscape / portrait switching conditions if the shooting mode is portrait mode and the lower boundary of the target cropping frame exceeds the bottom of the current image frame, and then switch the shooting mode to landscape mode. The cropped image generation unit is used to determine the target output resolution, including horizontal and vertical output resolutions, based on the switched shooting mode, and then use the target cropping box to perform a cropping operation on the current image frame to obtain the cropped image. A standardized image generation unit is used to scale the cropped image to the target output resolution based on the target output resolution if the shooting mode is landscape mode; and to scale the cropped image to a preset intermediate resolution if the shooting mode is portrait mode, and then fill the left and right sides of the cropped image based on the portrait output resolution to obtain a standardized image. A visualization mask image generation unit is used to generate an initial visualization mask image based on the target cropping box and the face region bounding box, and to draw an identifier graphic corresponding to the face region bounding box and the target cropping box on the initial visualization mask image to obtain a target visualization mask image; the visualization mask image includes a visualization effect of blurring the non-cropped area. The composition image generation unit is used to construct a composition image with the human figure centered based on the standardized image and the target visualization mask image.

[0058] Furthermore, embodiments of this application also disclose an electronic device, Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. Specifically, the electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the image processing method for multi-person group photo scenarios disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be a computer.

[0059] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0060] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0061] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the image processing method for multi-person group photo scenarios disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0062] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned image processing method suitable for multi-person group photo scenarios. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.

[0063] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.

[0064] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0065] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0066] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0067] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image processing method suitable for group photos, characterized in that, include: Face detection is performed on each image frame in the video stream data of the group photo scene using a face detection model to obtain an initial detection result including several face region bounding boxes and key points. The initial detection result is then filtered based on preset filtering conditions to obtain a set of target faces corresponding to each image frame. The preset filtering conditions include preset front view conditions and preset body size conditions; The preset frontal face condition is a condition for filtering facial poses based on a preset frontal face pose; The preset subject size condition is a condition for filtering image size based on the size corresponding to the bounding box of the face region; The composition strategy corresponding to the image frame is determined based on the number of faces in the target face set; When the number of faces is one, the first cropping box is determined by using the composition strategy corresponding to a single person and based on the preset single face region bounding box, the preset ideal composition ratio and the preset composition point. When there are at least two faces, an outer envelope rectangle is constructed based on the bounding boxes of each face region. Then, a second cropping frame is determined based on the outer envelope rectangle, the preset expansion coefficient, and the target aspect ratio using a preset group photo composition mode and the corresponding composition strategy. The state machine determines the change data of the number of faces in the preceding and following frames of each image frame, and the first cropping box and the second cropping box are smoothly adjusted based on the change data to obtain a target cropping box. Each image frame is then cropped based on the target cropping box to obtain a centered image of the human figure.

2. The image processing method for group photos according to claim 1, characterized in that, The method involves using a face detection model to detect faces in each image frame of the video stream data in a group photo scene, obtaining initial detection results including bounding boxes and key points of several face regions. These initial detection results are then filtered based on preset filtering conditions to obtain a set of target faces corresponding to each image frame, including: The video stream data, which includes several consecutive image frames, is acquired, and a face detection model is used to perform face detection on the current image frame to obtain an initial detection result including several face region bounding boxes and corresponding face key points, wherein the face key points include binocular key points. Based on the initial detection results, the straight-line distance between the key points of each person's eyes is determined, and the straight-line distance is compared with a preset eye distance threshold to obtain the comparison result; If the comparison result indicates that the straight-line distance is not less than the preset eye distance threshold, then the pose corresponding to the face is set as a frontal pose and retained; if the comparison result indicates that the straight-line distance is less than the preset eye distance threshold, then the pose corresponding to the face is set as a side pose and filtered out. Determine the width and height values ​​corresponding to the bounding boxes of each face region in the initial detection results, and set the maximum value of the width and height values ​​as the current size corresponding to the face. Then, construct a size set based on the current size of all faces that are determined to be frontal faces in the current image frame. The current size with the largest value in the size set is set as the target size threshold, and the data to be compared is determined based on the target size threshold and the preset ratio threshold. Then, the current size of each face is compared with the data to be compared. If the current size is not smaller than the data to be compared, the corresponding face is set as the main face and retained. If the current size is smaller than the data to be compared, the corresponding face is set as the secondary face and filtered out, thus obtaining the target face set corresponding to each image frame.

3. The image processing method for group photos according to claim 2, characterized in that, The step of determining the composition strategy corresponding to the image frame based on the number of faces in the target face set includes: The number of faces in the target face set is counted, and the corresponding target composition strategy is determined from each preset composition strategy based on the number of faces; The preset composition strategies include a first composition strategy for handling cases without valid faces, a second composition strategy for handling single-person portraits, and a third composition strategy for handling group photos.

4. The image processing method for group photos according to claim 3, characterized in that, When the number of faces is one, the first cropping box is determined using a composition strategy corresponding to a single face and based on a preset single-face region bounding box, a preset ideal composition ratio, and preset composition points, including: When the number of faces is one, determine the upper left and lower right corner coordinates of a single face region in the current image frame, and construct a face bounding box based on the upper left and lower right corner coordinates; The face width and face height are determined based on the face bounding box, and the face center coordinates are determined based on the face width and face height. Then, the target composition point coordinates, including the horizontal and vertical composition point coordinates, are obtained. Based on user needs, determine the preset composition ratio, preset target aspect ratio, preset scaling factor, preset proportion range, and preset offset threshold; A first ratio between the face width and the image width of the current image frame is determined, and a second ratio between the face height and the image height of the current image frame is determined. Then, the larger of the first ratio and the second ratio is set as the current proportion of the face. Determine the offset of the face center coordinates relative to the preset composition point coordinates, determine whether the current proportion of the face is within the preset proportion range, and then determine whether the offset is greater than the preset offset threshold. If the current proportion of the face is not within the preset proportion range, or if the offset is greater than the preset offset threshold, then the cropping frame width and cropping frame height corresponding to the cropping frame are determined based on the ratio between the face width and the face height and the preset target aspect ratio. If the ratio is not less than the preset target aspect ratio, then the cropping frame width is determined based on the face width, the preset ideal composition ratio, and the preset scaling factor, and then the cropping frame height is determined based on the cropping frame width and the preset target aspect ratio. If the ratio is less than the preset target aspect ratio, the cropping frame height is determined based on the face height, the preset ideal composition ratio, and the preset scaling factor, and the cropping frame width is determined based on the cropping frame height and the preset target aspect ratio. The cropping frame center coordinates are determined based on the face center coordinates, the cropping frame width, the cropping frame height, and the preset composition point coordinates, so as to determine the corresponding first cropping frame based on the cropping frame center coordinates, the cropping frame width, and the cropping frame height.

5. The image processing method for group photos according to claim 1, characterized in that, When the number of faces is at least two, an outer envelope rectangle is constructed based on the bounding boxes of each face region. Then, a second cropping box is determined using a preset group photo composition mode and corresponding composition strategy, based on the outer envelope rectangle, a preset expansion coefficient, and a target aspect ratio. This includes: When the number of faces is at least two, the number of faces is compared with a preset multi-person determination threshold. When the number of faces is not less than the preset multi-person determination threshold, the face bounding boxes of each face region are used to extract the corresponding upper left horizontal coordinate, upper left vertical coordinate, lower right horizontal coordinate and lower right vertical coordinate from each face bounding box based on a preset coordinate region. The minimum values ​​of the left-left corner x-coordinates and the left-left corner y-coordinates are respectively set as the left boundary and the upper boundary of the outer envelope rectangle, and the maximum values ​​of the right-right corner x-coordinates and the lower boundary of the outer envelope rectangle are respectively set as the right boundary and the lower boundary of the outer envelope rectangle, so as to construct an outer envelope rectangle based on the left boundary, the upper boundary, the right boundary, and the lower boundary of the outer envelope rectangle; A preset expansion coefficient and a preset target aspect ratio are determined. Then, based on the width and height of the outer envelope rectangle and the preset expansion coefficient, the expanded width and expanded height of the expanded rectangle are determined. The expanded rectangle is a rectangle that expands outward from the center of the outer envelope rectangle by a preset range. The expanded rectangle is adjusted based on the expanded width, expanded height, and the preset target aspect ratio to obtain the second cropping frame.

6. The image processing method for group photos according to claim 1, characterized in that, The step of determining the change data of the number of faces in consecutive frames of each image frame based on a state machine, and smoothly adjusting the first cropping box and the second cropping box based on the change data to obtain the target cropping box includes: Obtain the third cropping box corresponding to the previous image frame, and determine the cropping box to be processed based on the first cropping box and the second cropping box; the third cropping box is the actual cropping area corresponding to the cropping operation performed on the previous image frame; Based on the mapping relationship between the third cropping box and the cropping box to be processed, the cropping box to be processed is mapped from the cropping map coordinate system of the previous image frame back to the original image coordinate system to obtain the mapped target cropping box; The corresponding smoothing adjustment method is determined by using a preset face count change confirmation mechanism and based on the comparison between the face count of the current image frame and the face count of the previous image frame. If the comparison result indicates an increase in the number of faces, then the adjustment step size of each image frame is determined based on the preset adjustment step size and the difference between the target cropping box and the current cropping box, so as to update the target cropping box based on the smooth adjustment method and the adjustment step size to obtain a new target cropping box; When the comparison result indicates that the number of faces has not increased, the adjustment step size of each image frame is determined by using a preset smoothing adjustment method and based on a preset smoothing factor and the difference, so as to update the target cropping box based on the adjustment step size and obtain a new target cropping box. If no valid face is detected in a consecutive preset number of image frames, the current cropping box is reset to the target cropping box.

7. The image processing method for group photos according to any one of claims 1 to 6, characterized in that, The step of cropping each image frame based on the target cropping box to obtain a centered human portrait image includes: Based on the shooting mode of the current image frame and the position of the target cropping box in the current image frame, determine whether the current image frame meets the conditions for switching between landscape and portrait modes; If the shooting mode is portrait mode and the lower boundary of the target cropping frame exceeds the bottom of the current image frame, then the current image frame is determined to meet the landscape / portrait switching condition, and the shooting mode is switched to landscape mode. Based on the switched shooting mode, the corresponding target output resolution, including landscape output resolution and portrait output resolution, is determined. Then, the target cropping box is used to crop the current image frame to obtain the cropped image. If the shooting mode is landscape mode, the cropped image is scaled to the landscape output resolution based on the target output resolution; if the shooting mode is portrait mode, the cropped image is scaled to a preset intermediate resolution, and then filled on the left and right sides of the cropped image based on the portrait output resolution to obtain a standardized image. An initial visualization mask image is generated based on the target cropping box and the face region bounding box, and an identifier graphic corresponding to the face region bounding box and the target cropping box is drawn on the initial visualization mask image to obtain a target visualization mask image; the visualization mask image includes a visualization effect of blurring the non-cropped areas. A centered human figure composition image is constructed based on the standardized image and the target visualization mask image.

8. An image processing device suitable for group photos, characterized in that, include: The detection result generation module is used to perform face detection on each image frame in the video stream data of the group photo scene using a face detection model, to obtain an initial detection result including several face region bounding boxes and key points, and to filter the initial detection result based on preset filtering conditions to obtain a set of target faces corresponding to each image frame. The preset filtering conditions include preset front view conditions and preset body size conditions; The preset frontal face condition is a condition for filtering facial poses based on a preset frontal face pose; The preset subject size condition is a condition for filtering image size based on the size corresponding to the bounding box of the face region; The composition strategy determination module is used to determine the composition strategy corresponding to the image frame based on the number of faces in the target face set; The first cropping frame determination module is used to determine the corresponding first cropping frame when the number of faces is one, by using a composition strategy corresponding to a single person and based on a preset single face region bounding box, a preset ideal composition ratio and a preset composition point. The second cropping frame determination module is used to construct an outer envelope rectangle based on the bounding boxes of each face region when the number of faces is at least two, and then determine the second cropping frame based on the outer envelope rectangle, the preset expansion coefficient and the target aspect ratio using a preset group photo composition mode and the corresponding composition strategy. The composition image generation module is used to determine the change data of the number of faces in the preceding and following frames of each image frame based on a state machine, and to smoothly adjust the first cropping box and the second cropping box based on the change data to obtain a target cropping box, and to crop each image frame based on the target cropping box to obtain a composition image with the human face centered.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the image processing method for group photo scenarios as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the image processing method for group photo scenarios as described in any one of claims 1 to 7.

Images