Picture correction method, device and equipment based on face detection and storage medium

Abstract

The invention relates to the technical field of image processing, solves the problem that a camera cannot be adjusted to a correct visual angle in the prior art, and provides an image correction method and device based on face detection, equipment and a storage medium. The method comprises the following steps: acquiring real-time video data shot by a camera at a preset visual angle, and decomposing the real-time video data into multiple frames of real-time images; judging whether each real-time image is a positive direction image or the rotating image corresponding to each real-time image is a positive direction image, and obtaining a judgment result; obtaining a positive direction image according to the judgment result; according to the positive direction image, calculating a rotation angle needing to be adjusted of a camera; and according to the rotation angle, completing shooting angle correction processing of the camera. According to the invention, the camera can be adjusted to a correct visual angle, and the correctness and stability of a monitoring picture are ensured.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image correction method, apparatus, device, and storage medium based on face detection. Background Technology

[0002] In modern society, CCTV surveillance systems play a crucial role in security and management. Whether in public places, corporate offices, or homes, the proper installation and calibration of CCTV cameras are essential to ensuring the accuracy and stability of the footage. However, in practical applications, due to various reasons, camera viewing angles may deviate, resulting in inaccurate surveillance footage and ineffective coverage of the target area. Therefore, researching and applying effective methods to adjust cameras to the correct viewing angle and ensure the accuracy and stability of the surveillance footage is an important and urgent problem that needs to be solved.

[0003] Chinese patent CN110460772A discloses an automatic camera adjustment method, apparatus, device, and storage medium. The method includes: acquiring a captured image; when a face is detected in the image, determining a first position of the face in the image; determining whether the first position meets adjustment conditions; if so, determining an offset angle of the camera based on a second position and the first position, where the second position includes the center position of the image, and the offset angle has four directions: up, down, left, and right; generating a control command based on the offset angle, the control command being used to control a motor connected to the camera to move the camera by the offset angle. The above solution has a certain delay from image acquisition, detection, and offset angle calculation to generating control commands and camera adjustment, and may not be able to track fast-moving targets in real time.

[0004] Therefore, it is necessary to adjust the camera to the correct angle and ensure the accuracy and stability of the monitoring footage. Summary of the Invention

[0005] In view of this, the present invention provides a face detection-based image correction method, apparatus, device and storage medium to solve the problem in the prior art that the camera cannot be adjusted to the correct viewing angle.

[0006] The technical solution adopted in this invention is: In a first aspect, the present invention provides an image correction method based on face detection, the method comprising: Acquire real-time video data captured by a camera from a preset viewpoint, and decompose the real-time video data into multiple frames of real-time images; Determine whether each of the real-time images is a positive orientation image or the corresponding rotated image of each real-time image is a positive orientation image, and obtain the determination result; Based on the judgment result, obtain the positive direction image; Based on the positive direction image, calculate the required rotation angle of the camera; Based on the rotation angle, the shooting angle correction process of the camera is completed.

[0007] Preferably, the determination of whether each of the real-time images is a positive-direction image or the corresponding rotated image of each real-time image is a positive-direction image, and the determination result includes: The real-time images are rotated according to a preset rotation angle to obtain rotated images. Each of the real-time images and each of the rotated images are input into a pre-trained face detection model to obtain a first face image corresponding to the real-time image and a second face image corresponding to the rotated image; Facial landmark detection is performed on the first face image and the second face image respectively to obtain the first facial landmark set and the corresponding first landmark confidence sequence in the first face image, and the second facial landmark set and the corresponding second landmark confidence sequence in the second face image. Based on the first keypoint confidence sequence and the second keypoint confidence sequence, and combined with a preset confidence threshold, it is determined whether each real-time image is a positive image or the corresponding rotated image of each real-time image is a positive image, and the determination result is obtained.

[0008] Preferably, the step of determining whether each real-time image is a positive-direction image or the corresponding rotated image of each real-time image is a positive-direction image based on the first keypoint confidence sequence and the second keypoint confidence sequence, combined with a preset confidence threshold, and obtaining the determination result includes: The first confidence sequence and the second confidence sequence are respectively mean calculated to obtain the first confidence mean and the second confidence mean; The first confidence mean and the second confidence mean are compared. If the first confidence mean is greater than the second confidence mean, then the first confidence mean is taken as the target confidence mean. If the first confidence mean is less than or equal to the second confidence mean, then the second confidence mean is taken as the target confidence mean. Based on the keypoint confidence sequence corresponding to the target confidence mean and the confidence threshold, it is determined whether each real-time image is a positive orientation image or the rotated image corresponding to each real-time image is a positive orientation image, and the determination result is obtained.

[0009] Preferably, the step of determining whether each of the real-time images is a forward-oriented image or the rotated image corresponding to each real-time image is a forward-oriented image based on the keypoint confidence sequence corresponding to the target confidence mean and the confidence threshold, and obtaining the determination result includes: Extract the confidence of each key point in the key point confidence sequence corresponding to the target confidence mean to obtain the target key point confidence; The confidence level of the target key point is compared with the confidence threshold to obtain the comparison result; Based on the comparison results, the number of target key points with a confidence level greater than the confidence threshold is obtained; When the number of target key points is greater than a preset key point number threshold, determine whether the key point confidence sequence corresponding to the target confidence average is a first key point confidence sequence or a second key point confidence sequence. If the keypoint confidence sequence corresponding to the target confidence mean is the first keypoint confidence sequence, then the judgment result is that the real-time image is a positive direction image; If the keypoint confidence sequence corresponding to the target confidence mean is the second keypoint confidence sequence, then the judgment result is that the rotated image is a positive direction image.

[0010] Preferably, calculating the required rotation angle of the camera based on the positive direction image includes: The positive direction image is input into a pre-trained bed detection model to obtain a bed region image; Edge detection is performed on the image of the bed area to obtain a sequence of edge contour points; The edge contour point sequence is input into a pre-trained Hough transform line detection model, and the detected lines are filtered to obtain each horizontal line. The rotation angle is calculated based on the Hough polar coordinate line parameters corresponding to each horizontal line.

[0011] Preferably, calculating the rotation angle based on the Hough polar coordinate line parameters corresponding to each horizontal line includes: Based on the image of the bed area, obtain the coordinates of the bed area location; Based on the Hough polar coordinate line parameters corresponding to each transverse line and the position coordinates of the bed area, each transverse line is filtered to obtain the filtered target transverse lines. Calculate the angle between the horizontal line of each target and the horizontal direction based on the target Hough polar coordinate line parameters corresponding to the horizontal line of the target. The rotation angle is calculated based on the angle between the horizontal line of each target and the horizontal direction.

[0012] Preferably, calculating the rotation angle based on the angle between the horizontal line of each target and the horizontal direction includes: Determine the median value of the angle between the horizontal line of each target and the horizontal direction; The rotation angle is obtained by calculating the difference between the median of the included angle and the angle corresponding to the horizontal direction.

[0013] Secondly, the present invention provides an image correction device based on face detection, the device comprising: The image acquisition module is used to acquire real-time video data captured by a camera from a preset viewpoint and decompose the real-time video data into multiple frames of real-time images. The judgment module is used to determine whether each of the real-time images is a positive orientation image or the corresponding rotated image of each real-time image is a positive orientation image, and to obtain the judgment result; A forward-facing image acquisition module is used to acquire a forward-facing image based on the judgment result; The rotation angle calculation module is used to calculate the required rotation angle of the camera based on the positive direction image. The correction module is used to correct the shooting angle of the camera based on the rotation angle.

[0014] Thirdly, embodiments of the present invention also provide an electronic device, including: at least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method of the first aspect described above.

[0015] Fourthly, embodiments of the present invention also provide a storage medium storing computer program instructions thereon, which, when executed by a processor, implement the method of the first aspect described above.

[0016] In summary, the beneficial effects of the present invention are as follows: The present invention provides a face detection-based image correction method, apparatus, device, and storage medium. The method includes: acquiring real-time video data captured by a camera at a preset viewpoint; decomposing the real-time video data into multiple real-time images; determining whether each real-time image is a positive-direction image or a rotated image corresponding to each real-time image is a positive-direction image, and obtaining a determination result; acquiring a positive-direction image based on the determination result; calculating the rotation angle that the camera needs to adjust based on the positive-direction image; and completing the shooting angle correction processing of the camera based on the rotation angle. This invention first performs face detection and keypoint detection on a real-time image and its 180-degree rotated image, respectively. The average confidence score of the keypoints in both images, along with the "number of keypoints reaching a threshold," is used as the criterion to select the side with the higher confidence score as the positive orientation image, avoiding misjudgments based solely on texture or sensor data. Within the obtained positive orientation image, a bed detection model is used to delineate the bed region. Edge detection is performed only within this region, and Hough line detection is input. Irrelevant line segments are filtered out using the bed region coordinates, retaining horizontal lines, and their polar coordinate parameters are used to calculate the angle between each line and the horizontal direction. The median of these angles is taken to suppress interference from local occlusion and weak edges. The difference between the median and the horizontal angle is used as the rotation angle to be adjusted, driving the camera to perform angle correction. Through a chain of face confidence-based orientation determination + bed geometry as a reference + robust estimation of the median angle, the camera can be automatically adjusted to the correct viewing angle under different installation postures and complex backgrounds, overcoming the problems of stable orientation determination and accurate correction in existing technologies. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, and these are all within the protection scope of the present invention.

[0018] Figure 1 This is a schematic diagram illustrating the overall workflow of the face detection-based image correction method in Embodiment 1 of the present invention. Figure 2 This is a flowchart illustrating the process of determining whether each real-time image is a positive-direction image or the corresponding rotated image of each real-time image is a positive-direction image in Embodiment 1 of the present invention. Figure 3 This is a flowchart illustrating the process of calculating the required rotation angle of the camera in Embodiment 1 of the present invention. Figure 4 This is a structural block diagram of the face detection-based image correction device in Embodiment 2 of the present invention; Figure 5 This is a schematic diagram of the electronic device in Embodiment 3 of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. In the description of the present invention, it should be understood that the terms "center," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, the element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Where there is no conflict, embodiments of the present invention and the various features thereof can be combined with each other, all of which are within the scope of protection of the present invention.

[0020] Example 1 Please see Figure 1 Embodiment 1 of the present invention discloses an image correction method based on face detection, the method comprising: Acquire real-time video data captured by a camera from a preset viewpoint, and decompose the real-time video data into multiple frames of real-time images; Specifically, the system continuously captures real-time video data from a camera, which is mounted on a crib with its field of view facing downwards using a bracket. The system then segments the real-time video data into individual frames in chronological order, serving as a unified input for subsequent orientation and angle estimation. This approach discretizes the continuous real-time video data into frame-level samples that can be processed in parallel. This facilitates pairing with subsequent rotating reference frames and provides a stable, equally spaced data source for the subsequent detection model, thereby improving the real-time performance and robustness of the overall processing.

[0021] Determine whether each of the real-time images is a positive orientation image or the corresponding rotated image of each real-time image is a positive orientation image, and obtain the determination result; Specifically, a 180-degree rotated reference image is generated for each original frame. The two images are then fed into the face detection and face key point detection modules, respectively, to obtain two sets of key point confidence sequences. First, the mean confidence values ​​of the two sides are compared, and the side with the larger mean is selected as the candidate. Then, it is required that at least a number of key points on that side have confidence values ​​exceeding a threshold before it is confirmed as the positive direction. This avoids misjudgments caused by inverted installation, occlusion, or weak textures, and significantly improves the stability and interpretability of positive and negative determination.

[0022] Based on the judgment result, obtain the positive direction image; Specifically, when the mean confidence score of the key points in the original real-time image is higher and meets the key point number threshold, the real-time image is directly used as the positive direction image; otherwise, a rotated reference image is selected as the positive direction image. This threshold selection mechanism combines mean evaluation with key point number compliance, which takes into account both statistical stability and maintains output consistency under conditions such as illumination changes and occlusion, providing reliable input for subsequent angle estimation.

[0023] Based on the positive direction image, calculate the required rotation angle of the camera; Specifically, based on the positive direction image, the bed region is first delineated in the positive direction image using a target detection model. Edge detection and Hough line detection are performed only in this region, and short line segments are deleted according to the rule of length threshold as a percentage of bed width, retaining the main horizontal edge. Then, the angle between these horizontal lines and the horizontal direction is calculated, and the difference between the median angle and the ideal horizontal angle is taken as the rotation angle to be adjusted. This combination of region constraint → horizontal detection only → length filtering → median robust estimation can effectively suppress background texture and short-side noise, ensuring the accuracy and anti-interference of angle estimation.

[0024] Based on the rotation angle, the shooting angle correction process of the camera is completed.

[0025] Specifically, the aforementioned rotation angle is used as an execution quantity to drive the camera to complete automatic correction, restoring the field of view to a horizontal and stable correct orientation. This closed-loop action connects visual estimation with mechanical execution, enabling continuous self-correction even when the user's installation angle is inaccurate or the environment is constantly changing, ensuring the accuracy and long-term stability of the monitoring image.

[0026] Preferably, please refer to Figure 2 The determination of whether each real-time image is a positive-direction image or the corresponding rotated image of each real-time image is a positive-direction image, and the determination result includes: The real-time images are rotated according to a preset rotation angle to obtain rotated images. Specifically, each real-time image Pic1 is rotated 180° around the image center (using bilinear interpolation while maintaining the same resolution) to form a rotated image Pic2 that corresponds one-to-one with the original real-time image, which serves as a reference for subsequent comparison and determination of the orientation. This transforms the uncertainty of whether the image is inverted into a discriminable problem between the original image and the rotated image, avoiding misjudgments caused by making orientation assumptions based on a single frame.

[0027] Each of the real-time images and each of the rotated images are input into a pre-trained face detection model to obtain a first face image corresponding to the real-time image and a second face image corresponding to the rotated image; Specifically, the real-time image Pic1 and the rotated image Pic2 are input into a pre-trained face detection model. The face detection model is trained based on the YOLOv8s architecture and can detect face regions in the image. The face detection model outputs face bounding boxes in the real-time image and the rotated image, and cropped to obtain a first face image F1 corresponding to the real-time image Pic1 and a second face image F2 corresponding to the rotated image Pic2. Simultaneous detection of faces in two orientations can improve recall under complex lighting and occlusion conditions, and at the same time provide symmetrical and interpretable input for subsequent key point and confidence comparison.

[0028] Facial landmark detection is performed on the first face image and the second face image respectively to obtain the first facial landmark set and the corresponding first landmark confidence sequence in the first face image, and the second facial landmark set and the corresponding second landmark confidence sequence in the second face image. Specifically, two face images, F1 and F2, are input into a keypoint detection network (e.g., MTCNN face keypoint detection network), which outputs the coordinates of five keypoints and the confidence sequence of each point. Specifically, the first face keypoint set KeyPoint1(p1,p2,p3,p4,p5) and the corresponding first keypoint confidence sequence Z1(r1,r2,r3,r4,r5) in the first face image, and the second face keypoint set KeyPoint2(p1,p2,p3,p4,p5) and the corresponding second keypoint confidence sequence Z2(r1,r2,r3,r4,r5) in the second face image. The keypoints and their confidence measures the consistency of facial structure. When inverted, the stability and confidence of the keypoints usually decrease, providing a direct and comparable measurement basis for the next step of positive and negative determination.

[0029] Based on the first keypoint confidence sequence and the second keypoint confidence sequence, and combined with a preset confidence threshold, it is determined whether each real-time image is a positive image or the corresponding rotated image of each real-time image is a positive image, and the determination result is obtained.

[0030] Specifically, the mean values ​​of the confidence sequences on both sides are calculated and compared, and the side with the larger mean value is selected as the candidate. Furthermore, it is required that the confidence values ​​of at least a number of key points on this side are higher than the threshold before the corresponding image is identified as the positive direction. This dual criterion of mean-based evaluation combined with the number threshold is both statistically robust and engineering feasible, and can significantly reduce the probability of misjudgment under conditions of noise, occlusion, etc.

[0031] Preferably, the step of determining whether each real-time image is a positive-direction image or the corresponding rotated image of each real-time image is a positive-direction image based on the first keypoint confidence sequence and the second keypoint confidence sequence, combined with a preset confidence threshold, and obtaining the determination result includes: The first confidence sequence and the second confidence sequence are respectively mean calculated to obtain the first confidence mean and the second confidence mean; Specifically, the confidence scores of the two sets of key points are first screened for validity (e.g., undetected points are removed, and abnormal small values ​​are interpolated according to adjacent frames). Then, normalization is performed according to the face bounding box scale to eliminate systematic bias caused by distance and blur. Subsequently, the arithmetic mean of the two sequences is calculated as the face stability index of the frame in the original image direction and rotation direction, to obtain the first confidence mean Z1_avg and the second confidence mean Z2_avg. Robust statistics (such as Windsorization or truncated mean) can be added to reduce the impact of individual distortion points, and the mean is smoothed with a short window to suppress inter-frame jitter. The mean values ​​obtained in this way can comprehensively reflect the consistency of the facial key points and have good noise resistance and comparability, providing an interpretable scalar basis for subsequent direction determination.

[0032] The first confidence mean and the second confidence mean are compared. If the first confidence mean is greater than the second confidence mean, then the first confidence mean is taken as the target confidence mean. Specifically, a simple and stable discrimination rule is constructed, with direct comparison of the means on both sides as the core. When the first mean is significantly higher, it is selected as the target mean and associated with the original image direction as a candidate positive direction. At the same time, a difference hysteresis interval can be set, and the switch is only made when the first mean exceeds the second mean by a certain margin, so as to avoid frequent directional jitter when the two sides are close. This approach has low implementation cost, good real-time performance, and can quickly provide a reliable preferred direction in most normal orientation scenarios.

[0033] If the first confidence mean is less than or equal to the second confidence mean, then the second confidence mean is taken as the target confidence mean. Specifically, if the first confidence mean is less than or equal to the second confidence mean, the mirroring adopts a symmetrical branch, directly selecting the second mean as the target and associating the rotation direction as the candidate positive direction; when the two means are equal or very close, a conservative strategy can be triggered (e.g., using the direction already confirmed in the previous frame, or entering the enhanced detection path) to avoid the direction switching back and forth under boundary conditions; this mirroring rule ensures the completeness and simplicity of the algorithm, enabling the system to still provide robust direction candidates when the image is installed upside down or the original image quality is poor.

[0034] Based on the keypoint confidence sequence corresponding to the target confidence mean and the confidence threshold, it is determined whether each real-time image is a positive orientation image or the rotated image corresponding to each real-time image is a positive orientation image, and the determination result is obtained.

[0035] Specifically, based on the keypoint confidence sequence corresponding to the target confidence mean, and taking the keypoint confidence sequence on the side to which the target confidence mean belongs as a basis, each point is compared with a threshold and the number of keypoints above the threshold is counted. Only when the number reaches the set keypoint number threshold is the side finally confirmed as the positive direction. The threshold can be combined with environmental adaptation (such as moderately lowering it in low light or strong backlight) and superimposed with temporal consistency constraints (such as confirming it only if it meets the requirements for multiple consecutive frames) to further suppress occasional noise and occlusion. Through the three-level checks of mean evaluation, number of points meeting the standard, and temporal consistency, the output direction determination has both statistical significance and engineering robustness, thus providing a reliable input for subsequent angle estimation based on the bedside straight line.

[0036] Preferably, the step of determining whether each of the real-time images is a forward-oriented image or the rotated image corresponding to each real-time image is a forward-oriented image based on the keypoint confidence sequence corresponding to the target confidence mean and the confidence threshold, and obtaining the determination result includes: Extract the confidence of each key point in the key point confidence sequence corresponding to the target confidence mean to obtain the target key point confidence; Specifically, after determining the side to which the target confidence mean belongs, the confidence of key points such as the corner of the eye, the tip of the nose, and the corner of the mouth are read point by point from the key point confidence sequence of that side. To improve reliability, validity screening can be performed first (undetected points are recorded as empty or interpolated according to adjacent frames), normalization can be performed according to the face bounding box scale to reduce the system bias caused by distance and blur, and Windsor smoothing or short window smoothing can be performed on abnormally low values ​​of single points. The target key point confidence obtained in this way maintains point-by-point resolution and has comparability across frames and scenes.

[0037] The confidence level of the target key point is compared with the confidence threshold to obtain the comparison result; Specifically, the confidence level of each target key point is compared with a preset confidence threshold one by one to generate a pass mark sequence arranged in order of key points. The confidence threshold can be a basic threshold plus an adaptive correction method, which is dynamically fine-tuned according to quality indicators such as light intensity, motion blur degree, and face bounding box area, so as not to over-reject in low light, side face or light occlusion, and to maintain sufficient discrimination in high-quality images. The continuous probability quantity is transformed into discrete pass results, which facilitates subsequent stable decision-making.

[0038] Based on the comparison results, the number of target key points with a confidence level greater than the confidence threshold is obtained; Specifically, the number of target key points that reach the threshold is obtained by counting the labeled sequence; optionally, differentiated weights are assigned to key points (e.g., higher weights are given to the eyes and the tip of the nose), or points with obvious abnormal spatial relationships (such as geometric structures that do not meet the relative position constraints of the eyes and nose) are removed before counting to reduce the impact of random noise on the overall judgment; the counting result will serve as a hard condition for whether to accept the candidate of that direction, ensuring that the decision is supported by minimum geometric evidence.

[0039] When the number of target key points is greater than a preset key point number threshold, determine whether the key point confidence sequence corresponding to the target confidence average is a first key point confidence sequence or a second key point confidence sequence. Specifically, if the number of qualified candidates is not less than the threshold, the direction is accepted as a valid candidate, and the assignment is determined based on the identity of the sequence to which the target mean belongs (the first sequence corresponds to the original image direction, and the second sequence corresponds to the rotation direction). When the number of qualified candidates is insufficient, a conservative strategy is triggered (using the direction of the previous frame or entering the enhanced detection branch), and hysteresis and multi-frame consistency constraints can be used to avoid frequent switching back and forth under edge conditions. This design significantly improves the stability of direction determination while ensuring sensitivity.

[0040] If the keypoint confidence sequence corresponding to the target confidence mean is the first keypoint confidence sequence, then the judgment result is that the real-time image is a positive direction image; Specifically, when the target mean comes from the confidence sequence of the first key point, it means that the real-time image provides more stable geometric evidence of the face in the current frame and the number of key points meets the standard. Therefore, each real-time image is directly confirmed as a positive orientation image and enters the subsequent angle estimation and correction calculation. At the same time, the number of consecutive frames of orientation confirmation is recorded for subsequent time consistency constraints. This reduces unnecessary image rotation and recalculation overhead and ensures that the downstream processing receives higher quality input.

[0041] If the keypoint confidence sequence corresponding to the target confidence mean is the second keypoint confidence sequence, then the judgment result is that the rotated image is a positive direction image.

[0042] Specifically, when the keypoint confidence sequence corresponding to the target confidence mean is the second keypoint confidence sequence, it indicates that the face structure of the rotated image is clearer and more stable, and thus each rotated image is identified as a positive orientation image. Subsequently, this positive orientation image is used as the input for subsequent bed region detection and straight line angle estimation, and the current installation posture can be marked as having an inverted risk, which facilitates the execution side to directly give the corresponding angle correction command. This branch ensures that the correct orientation image can still be obtained quickly and robustly even when the image is inverted or the original image quality is poor.

[0043] Preferably, please refer to Figure 3 The step of calculating the required rotation angle of the camera based on the positive direction image includes: The positive direction image is input into a pre-trained bed detection model to obtain a bed region image; Specifically, the forward-facing image is fed into a pre-trained bed detection model based on the YOLOv8 architecture. The model outputs the bounding box and class score of the bed target on the entire image. Based on this, a local image corresponding to the bounding box is cropped from the original image as the bed region image. To improve stability, the bounding box can be smoothed by combining multi-frame temporal consistency, and a small number of edges are adaptively expanded according to the ratio of the bounding box to the entire image to avoid the edges truncating the bed edge line. Subsequent processing is limited to the region most relevant to the horizontal reference, which significantly reduces the interference of background texture, bedding patterns and other objects on subsequent line and angle estimation, which is beneficial to improving computational efficiency and robustness.

[0044] Edge detection is performed on the image of the bed area to obtain a sequence of edge contour points; Specifically, the image of the bed region is first normalized in brightness and lightly denoised. Then, an edge detection operator is executed to obtain the edge response. Based on connectivity, contour tracking is performed on the response pixels, and the point sequence corresponding to each contour is output according to the spatial adjacency order to form the edge contour point sequence. If necessary, horizontal structuring elements can be superimposed for morphological processing to enhance the approximately horizontal long side and suppress fine textures. This step purifies the continuous bed edge structure from complex textures into an ordered set of contour points, providing higher quality and more ordered input for line fitting and parameter estimation, thereby reducing the impact of breaks and noise on subsequent detection.

[0045] The edge contour point sequence is input into a pre-trained Hough transform line detection model, and the detected lines are filtered to obtain each horizontal line. Specifically, the edge contour point sequence is voted into the Hough parameter space, and candidate lines corresponding to the peaks are searched under the preset angle and distance resolution. In order to retain only the horizontal targets, the angle range in the parameter space is limited to accept only candidates that are close to the horizontal direction. A second screening is carried out in combination with rules such as length threshold, vote number threshold, and maximum gap of fracture, and finally each horizontal line is obtained. This process can remove irrelevant terms from the possible vertical, diagonal and short miscellaneous lines, and retain the set of horizontal lines that are most consistent with the bedside geometry and most representative, providing reliable samples for angle estimation.

[0046] The rotation angle is calculated based on the Hough polar coordinate line parameters corresponding to each horizontal line.

[0047] Specifically, the Hough polar coordinate line parameters of each horizontal line are read, and the angle between each horizontal line and the ideal horizontal direction is calculated to obtain a set of angle observations. To enhance robustness, the angle observations can be weighted according to the line segment length or the number of votes, and abnormal deviation values ​​can be eliminated or the median value can be used as the overall representative. Finally, the difference between the representative angle and the ideal horizontal angle is calculated, and the sign and limit range are determined in combination with the current image orientation, thus obtaining the rotation angle used for correction. This approach uses the consensus of multiple horizontal lines to resist local occlusion, stripe interference, and false detection of short sides, making the calculated rotation angle both accurate and insensitive to noise.

[0048] Preferably, calculating the rotation angle based on the Hough polar coordinate line parameters corresponding to each horizontal line includes: Based on the image of the bed area, obtain the coordinates of the bed area location; Specifically, the positive-direction image is input into the bed detection model, which outputs the bed region location bounding box and records its position and size parameters Bed(x, y, w, h) as the bed region location coordinates; where w and h represent the width and height of the bed region, respectively, and subsequent steps use w as the length reference. This limits the calculation to the region most relevant to the horizontal reference, reducing the interference of background lines and textures on detection, and providing a unified scale benchmark for subsequent normalization filtering based on bed width.

[0049] Based on the Hough polar coordinate line parameters corresponding to each transverse line and the position coordinates of the bed area, each transverse line is filtered to obtain the filtered target transverse lines. Specifically, using the width w of the bed area location coordinates as the scale, the visible length of each line in the set of transverse lines Lines1(ρ, θ) obtained by Hough transform is counted in line segment form, and lines with a length less than 0.85*w are deleted to form a new set of line parameters Lines2(ρ, θ). This selection rule, which is normalized by bed width, takes advantage of the prior knowledge that the bed edge is usually the longest line in the area, effectively eliminating short and messy lines such as bedding folds and bed rail stripes, and retaining the candidate lines that best represent the geometry of the bed edge, thereby improving the stability of subsequent angle estimation.

[0050] Calculate the angle between the horizontal line of each target and the horizontal direction based on the target Hough polar coordinate line parameters corresponding to the horizontal line of the target. Specifically, for each target horizontal line in the filtered set of line parameters Lines2(ρ, θ), the angle between the line and the horizontal direction is directly calculated from its Hough polar coordinate line parameter θ (in the polar coordinate normal representation, the reference angle corresponding to the horizontal direction is fixed, so the angle can be obtained by the difference with the reference angle); this step maps the geometry into a comparable angle quantity, which is convenient for subsequent robust statistics.

[0051] The rotation angle is calculated based on the angle between the horizontal line of each target and the horizontal direction.

[0052] Preferably, calculating the rotation angle based on the angle between the horizontal line of each target and the horizontal direction includes: Determine the median value of the angle between the horizontal line of each target and the horizontal direction; The rotation angle is obtained by calculating the difference between the median of the included angle and the angle corresponding to the horizontal direction.

[0053] Specifically, the set of angles between the horizontal lines of each target is compiled, the midpoint of which is taken as the overall observation angle. This midpoint is then subtracted from the ideal horizontal angle θ_ideal=0° to obtain the rotation angle that the camera needs to adjust. Δθ = θ_ideal-median(θ); Wherein, median(θ) is the median of the detected horizontal straight line angle. The median statistics are not sensitive to abnormal line segments and can effectively suppress the deviation caused by local occlusion or weak edges, so that the obtained rotation angle is both robust and can be directly used for subsequent mechanism correction.

[0054] Example 2 Please see Figure 4 Embodiment 2 of the present invention also provides an image correction device based on face detection, the device comprising: The image acquisition module is used to acquire real-time video data captured by a camera from a preset viewpoint and decompose the real-time video data into multiple frames of real-time images. The judgment module is used to determine whether each of the real-time images is a positive orientation image or the corresponding rotated image of each real-time image is a positive orientation image, and to obtain the judgment result; A forward-facing image acquisition module is used to acquire a forward-facing image based on the judgment result; The rotation angle calculation module is used to calculate the required rotation angle of the camera based on the positive direction image. The correction module is used to correct the shooting angle of the camera based on the rotation angle.

[0055] Specifically, the face detection-based image correction device provided in this embodiment of the invention includes: an image acquisition module for acquiring real-time video data captured by a camera at a preset viewing angle, and decomposing the real-time video data into multiple frames of real-time images; a judgment module for determining whether each of the real-time images is a positive-direction image or a rotated image corresponding to each real-time image is a positive-direction image, and obtaining a judgment result; a positive-direction image acquisition module for acquiring a positive-direction image based on the judgment result; a rotation angle calculation module for calculating the rotation angle that the camera needs to adjust based on the positive-direction image; and a correction module for completing the shooting angle correction processing of the camera based on the rotation angle. This device first performs face detection and keypoint detection on the real-time image and its 180-degree rotated image, respectively. The mean confidence score of the keypoints in both images and the number of keypoints reaching a threshold are used as criteria to select the side with higher confidence as the positive orientation image, avoiding misjudgments based solely on texture or sensor data. Within the obtained positive orientation image, a bed detection model is used to delineate the bed region. Edge detection is performed only within this region, and Hough line detection is input. Irrelevant line segments are filtered out using the bed region coordinates, retaining horizontal lines, and their polar coordinate parameters are used to calculate the angle between each line and the horizontal direction. The median of these angles is taken to suppress interference from local occlusion and weak edges. The difference between the median and the horizontal angle is used as the rotation angle to be adjusted, driving the camera to perform angle correction. Through a chain of face confidence-based orientation determination + bed geometry as a reference + robust estimation of the median angle, the camera can be automatically adjusted to the correct viewing angle under different installation postures and complex backgrounds, overcoming the problems of stable orientation determination and accurate correction in existing technologies.

[0056] Example 3 In addition, combined Figure 1 The face detection-based image correction method described in Embodiment 1 of the present invention can be implemented by an electronic device. Figure 5 A schematic diagram of the hardware structure of the electronic device provided in Embodiment 3 of the present invention is shown.

[0057] Electronic devices may include processors and memory storing computer program instructions.

[0058] Specifically, the processor may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement embodiments of the present invention.

[0059] The memory may include a large-capacity storage device for data or instructions. For example, and not limitingly, the memory may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disk drive, a magneto-optical disk drive, magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory may include removable or non-removable (or fixed) media. Where appropriate, the memory may be internal or external to a data processing device. In a particular embodiment, the memory is a non-volatile solid-state memory. In a particular embodiment, the memory includes a read-only memory (ROM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0060] The processor reads and executes computer program instructions stored in memory to implement any of the face detection-based image correction methods in the above embodiments.

[0061] In one example, the electronic device may also include a communication interface and a bus. For example, Figure 5 As shown, the processor, memory, and communication interface are connected via a bus and communicate with each other.

[0062] The communication interface is mainly used to enable communication between various modules, devices, units and / or equipment in the embodiments of the present invention.

[0063] A bus, including hardware, software, or both, couples components of the device together. For example, and not limitingly, a bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, a bus may include one or more buses. While specific buses are described and illustrated in embodiments of the invention, the invention contemplates any suitable bus or interconnect.

[0064] Example 4 In addition, in conjunction with the face detection-based image correction method in Embodiment 1 above, Embodiment 4 of the present invention can also provide a computer-readable storage medium for implementation. This computer-readable storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the face detection-based image correction methods in the above embodiments.

[0065] In summary, the embodiments of the present invention provide a method, apparatus, device, and storage medium for image correction based on face detection.

[0066] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0067] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the required tasks. The programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0068] It should also be noted that the exemplary embodiments mentioned in this invention describe methods or systems based on a series of steps or apparatus. However, this invention is not limited to the order of the steps described above; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0069] The above description is merely a specific embodiment of the present invention. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the protection scope of the present invention.

Claims

1. A face detection-based image correction method, characterized in that, The method includes: Acquire real-time video data captured by a camera from a preset viewpoint, and decompose the real-time video data into multiple frames of real-time images; Determine whether each of the real-time images is a positive orientation image or the corresponding rotated image of each real-time image is a positive orientation image, and obtain the determination result; Based on the judgment result, obtain the positive direction image; Based on the positive direction image, calculate the required rotation angle of the camera; Based on the rotation angle, the shooting angle correction process of the camera is completed.

2. The image correction method based on face detection according to claim 1, characterized in that, The determination of whether each real-time image is a positive-oriented image or the corresponding rotated image of each real-time image is a positive-oriented image includes the following results: The real-time images are rotated according to a preset rotation angle to obtain rotated images. Each of the real-time images and each of the rotated images are input into a pre-trained face detection model to obtain a first face image corresponding to the real-time image and a second face image corresponding to the rotated image; Facial landmark detection is performed on the first face image and the second face image respectively to obtain the first facial landmark set and the corresponding first landmark confidence sequence in the first face image, and the second facial landmark set and the corresponding second landmark confidence sequence in the second face image. Based on the first keypoint confidence sequence and the second keypoint confidence sequence, and combined with a preset confidence threshold, it is determined whether each real-time image is a positive image or the corresponding rotated image of each real-time image is a positive image, and the determination result is obtained.

3. The image correction method based on face detection according to claim 2, characterized in that, The step of determining whether each real-time image is a positive-direction image or the corresponding rotated image of each real-time image is a positive-direction image based on the first keypoint confidence sequence and the second keypoint confidence sequence, combined with a preset confidence threshold, and obtaining the determination result includes: The first confidence sequence and the second confidence sequence are respectively mean calculated to obtain the first confidence mean and the second confidence mean; The first confidence mean and the second confidence mean are compared. If the first confidence mean is greater than the second confidence mean, then the first confidence mean is taken as the target confidence mean. If the first confidence mean is less than or equal to the second confidence mean, then the second confidence mean is taken as the target confidence mean. Based on the keypoint confidence sequence corresponding to the target confidence mean and the confidence threshold, it is determined whether each real-time image is a positive orientation image or the rotated image corresponding to each real-time image is a positive orientation image, and the determination result is obtained.

4. The image correction method based on face detection according to claim 3, characterized in that, The step of determining whether each real-time image is a positive-oriented image or the rotated image corresponding to each real-time image is a positive-oriented image based on the keypoint confidence sequence corresponding to the target confidence mean and the confidence threshold, and obtaining the determination result includes: Extract the confidence of each key point in the key point confidence sequence corresponding to the target confidence mean to obtain the target key point confidence; The confidence level of the target key point is compared with the confidence threshold to obtain the comparison result; Based on the comparison results, the number of target key points with a confidence level greater than the confidence threshold is obtained; When the number of target key points is greater than a preset key point number threshold, determine whether the key point confidence sequence corresponding to the target confidence average is a first key point confidence sequence or a second key point confidence sequence. If the keypoint confidence sequence corresponding to the target confidence mean is the first keypoint confidence sequence, then the judgment result is that the real-time image is a positive direction image; If the keypoint confidence sequence corresponding to the target confidence mean is the second keypoint confidence sequence, then the judgment result is that the rotated image is a positive direction image.

5. The image correction method based on face detection according to any one of claims 1-4, characterized in that, The step of calculating the required rotation angle of the camera based on the positive direction image includes: The positive direction image is input into a pre-trained bed detection model to obtain a bed region image; Edge detection is performed on the image of the bed area to obtain a sequence of edge contour points; The edge contour point sequence is input into a pre-trained Hough transform line detection model, and the detected lines are filtered to obtain each horizontal line. The rotation angle is calculated based on the Hough polar coordinate line parameters corresponding to each horizontal line.

6. The image correction method based on face detection according to claim 5, characterized in that, The calculation of the rotation angle based on the Hough polar coordinate line parameters corresponding to each horizontal line includes: Based on the image of the bed area, obtain the coordinates of the bed area location; Based on the Hough polar coordinate line parameters corresponding to each transverse line and the position coordinates of the bed area, each transverse line is filtered to obtain the filtered target transverse lines. Calculate the angle between the horizontal line of each target and the horizontal direction based on the target Hough polar coordinate line parameters corresponding to the horizontal line of the target. The rotation angle is calculated based on the angle between the horizontal line of each target and the horizontal direction.

7. The image correction method based on face detection according to claim 6, characterized in that, The calculation of the rotation angle based on the angle between the horizontal line of each target and the horizontal direction includes: Determine the median value of the angle between the horizontal line of each target and the horizontal direction; The rotation angle is obtained by calculating the difference between the median of the included angle and the angle corresponding to the horizontal direction.

8. A face detection-based image correction device, characterized in that, The device includes: The image acquisition module is used to acquire real-time video data captured by a camera from a preset viewpoint and decompose the real-time video data into multiple frames of real-time images. The judgment module is used to determine whether each of the real-time images is a positive orientation image or the corresponding rotated image of each real-time image is a positive orientation image, and to obtain the judgment result; A forward-facing image acquisition module is used to acquire a forward-facing image based on the judgment result; The rotation angle calculation module is used to calculate the required rotation angle of the camera based on the positive direction image. The correction module is used to correct the shooting angle of the camera based on the rotation angle.

9. An electronic device, characterized in that, include: At least one processor, at least one memory, and computer program instructions stored in the memory, which, when executed by the processor, implement the method as described in any one of claims 1-7.

10. A storage medium storing computer program instructions thereon, characterized in that, The method as described in any one of claims 1-7 is implemented when the computer program instructions are executed by the processor.

Images