An attitude detection and autofocus method

Abstract

This invention proposes a posture detection and automatic zoom method, comprising: S11, using a target detection algorithm to identify and locate human information in the image; the human information includes the number of human bodies in the image and the position data of several human bodies; the position data includes the center coordinates and dimensions of the detection box; S12, filtering the target people to select those for posture detection; S13, classifying the target people using a clustering algorithm to obtain several target group classes; S14, segmenting the image according to the target group classes to obtain segmented images with the same number of target group classes; S15, enlarging the segmented images to a preset resolution; S16, performing posture detection on the enlarged segmented images; the above method can be applied to various sports competition scenarios, achieving adaptive image segmentation and posture detection, and finding more human postures with fewer detections.

Description

Technical Field

[0001] This invention relates to the field of intelligent camera technology, specifically to a posture detection and automatic zoom method. Background Technology

[0002] Currently, human target detection technology is quite mature. However, due to the large number of key detection points on the human body, the recognition effect of human posture and behavior is greatly affected by factors such as the pixel size of the human body in the image and the image clarity. Human posture detection is essentially a small target detection problem. Small targets are small in size in the original image. After several downsampling processes by the neural network, their size on the feature map may only be a few pixels, resulting in a low final recognition rate.

[0003] To address the aforementioned issues, common solutions include data augmentation (randomly cropping, enlarging, and stitching images), increasing image acquisition resolution, reducing downsampling rates, and adding small target detection layers. However, these methods all have limitations: they reduce the rate of single-shot recognition and increase the false recognition rate. Enlarging small targets for recognition is another solution; however, how to segment the image and how many times to segment it is also a challenge. Finding the optimal solution between the number of athlete poses and the number of detections—that is, finding as many athlete poses as possible with as few detections as possible—has become a key problem that human pose detection technology needs to solve. Summary of the Invention

[0004] To address the aforementioned constraints and limitations, this invention proposes a posture detection and automatic zoom method that achieves more accurate posture detection by adaptively cropping and scaling the image; simultaneously, automatic zoom functionality can be achieved based on posture detection.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] A method for attitude detection and automatic zoom, characterized in that,

[0007] The method comprises the following steps:

[0008] S11. Use target detection algorithms to identify human information in the location image;

[0009] The human body information refers to the number of human bodies in the image and the position data of several human bodies; the position data refers to the center position coordinates and size value of the detection box.

[0010] S12. Perform target person filtering to select target persons who need to undergo posture detection.

[0011] S13. Use clustering algorithms to classify the target individuals and obtain several target group categories;

[0012] S14. Segment the screen according to the target audience group to obtain the same number of segmented screens as the target audience group group;

[0013] S15. Enlarge the cut image to the preset resolution;

[0014] S16. Perform posture detection on the enlarged cut image.

[0015] The method also includes an automatic zoom step, specifically including:

[0016] S21. Detect the pose of the target person in the shooting scene using S11-16, and simultaneously detect the first and second landmarks using the target detection algorithm.

[0017] S22. When a first landmark is present on the screen, and when it is detected that more than a first preset number of people are moving toward the landmark, the zoom module performs a zoom operation.

[0018] When the first marker is not present in the frame, and the target person exceeds the second preset number of people, moves away from the marker or leaves the frame, the zoom module performs a zoom reduction operation to restore the lens to its original focal length.

[0019] S23. When the target people are all far away in the frame and are gathered together, the zoom module performs a zoom operation.

[0020] When the target person on the left moves to the left and the target person on the right moves to the right, or when a target person exceeding the third preset number of people moves closer, the zoom module performs a zoom reduction operation to restore the lens to its original focal length.

[0021] S24. When there are a first landmark and a second landmark in the picture, if the target person moves in the same way as the second landmark and the target person moves toward the first landmark, the zoom module performs a zoom operation.

[0022] If there is no second marker in the frame within the preset capture time, or if there is no first marker in the frame, the zoom module performs a zoom reduction operation to restore the lens to its original focal length.

[0023] Compared with existing technologies, this invention has the following advantages: By adaptively segmenting and scaling the image, more accurate posture detection is achieved; more human postures can be detected with fewer detections, achieving a balance between recognition speed and accuracy. Furthermore, automatic zooming based on posture detection can be implemented automatically without human intervention. Since the method of this invention only performs slight zoom magnification and shrinks back to the original size when the image changes, it also prevents information loss caused by magnifying the image during zooming, thus avoiding impact on image tracking.

[0024] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0025] Figure 1 This is a flowchart illustrating the steps of an attitude detection method provided in an embodiment of the present invention.

[0026] Figure 2 A flowchart illustrating the steps of target person filtering provided in an embodiment of the present invention.

[0027] Figure 3 This is a schematic diagram illustrating zooming when target individuals gather, as provided in an embodiment of the present invention.

[0028] Figure 4 This is a schematic diagram illustrating the zooming effect when a target person is gathered at a distance, as provided in an embodiment of the present invention.

[0029] Figure 5 This is a schematic diagram illustrating the zooming effect when a target person moves, as provided in an embodiment of the present invention.

[0030] Figure 6 This is a structural diagram of an attitude detection and automatic zoom device provided in an embodiment of the present invention. Detailed Implementation

[0031] The following specific embodiments illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. To further understand the present invention, the present invention will be further described in detail below with reference to the preferred embodiments.

[0032] The inventive point of this invention is to provide an attitude detection and automatic zoom method for achieving adaptive attitude detection, and automatic zoom can be completed based on the attitude detection results.

[0033] One aspect of the present invention is a posture detection method, wherein the posture detection first uses a human target detection algorithm to obtain the position of the target person and filters it, and then cuts and enlarges the image area according to the position information, thereby achieving adaptive posture recognition and detection.

[0034] refer to Figure 1 As shown, the attitude detection includes the following steps:

[0035] S11. Use a target detection algorithm to identify and locate human information in the image; the human information is the number of human bodies in the image and the position data of several human bodies; the position data is the center position coordinates and size value of the detection box.

[0036] S12. Perform target person filtering, select the target persons who need to be pose detected, and obtain target person data;

[0037] The purpose of target person filtering is to eliminate interference from non-target persons (such as off-field players and spectators in sports games);

[0038] S13. Use clustering algorithms to classify the target individuals and obtain several target group categories;

[0039] S14. Cut the screen according to the target group category, so that several target characters in each target group category are in the same cut screen, and obtain the same number of cut screens as the target group category.

[0040] S15. Enlarge the cut image to the preset resolution;

[0041] S16. Perform posture detection on the enlarged cut image.

[0042] As one embodiment, the object detection algorithm in S11 includes any one or more combinations of the R-CNN series algorithms, YOLO series algorithms, and SSD algorithms. All of the above object detection algorithms are publicly available and mature technical solutions, and will not be described in detail here.

[0043] As one embodiment, reference Figure 2 The target person filtering process in S12 specifically includes the following steps:

[0044] S121. Filter out the location data where the foot points are within the boundary line of the field from the location data obtained in S11 to obtain the data of the people in the field.

[0045] The foot point refers to the bottom center point of the detection frame; the site boundary line is obtained through visual algorithm recognition.

[0046] S122. Filter out the position data of the detection box that meets the size requirements from the data of people in the field to obtain the target person data.

[0047] The size requirement is that the width and height of the detection frame must meet the size threshold.

[0048] It should be noted that the purpose of S122 in setting the detection frame size is to filter out people who are far from the camera. Specifically, S122 filters players based on the principle that objects appear larger when closer and smaller when farther away in a camera's image. The detection frame height and width are smaller for people who are far from the camera.

[0049] The size threshold can be preset or set during the shooting process by recording the size value of the human detection frame at the distant boundary line.

[0050] As one embodiment, the clustering algorithm described in S13 is implemented using the three-dimensional K-Means clustering algorithm. When using the three-dimensional K-Means clustering algorithm for classification, the three-dimensional coordinates of each target person are clustered.

[0051] The first two dimensions of the three-dimensional coordinates are the horizontal and vertical coordinates of the center point of the detection box of the target person, and the third dimension of the three-dimensional coordinates is the height of the detection box.

[0052] As one example, the number of target population groups should meet the following conditions:

[0053]

[0054] Where N is the number of target population groups, N player The number of target individuals.

[0055] Generally, considering the limitations of computing power in smart hardware, N should not be greater than 3.

[0056] As one embodiment, in S15, the segmented image can be enlarged to a preset resolution using a super-resolution algorithm.

[0057] The super-resolution algorithm includes any one or more of the following algorithms: Bicubic Interpolation, Edge-Directed Super-Resolution (EDSR), Deep Learning-based Super-Resolution, Sparse Representation-based Super-Resolution (SRCNN), and Interpolation-based Super-Resolution (FSRCNN).

[0058] The super-resolution algorithms described above are all publicly available and mature technical solutions. Those skilled in the art can implement them smoothly based on the description of the embodiments, and will not be elaborated further here.

[0059] As one embodiment, the pose detection of the magnified segmented image in S16 can be achieved by any one of the OpenPose algorithm, Mask R-CNN algorithm, AlphaPose algorithm, HRNet algorithm, and DensePose algorithm. The above pose detection algorithms are all publicly available and mature technical solutions, and will not be described in detail here.

[0060] Another aspect of the present invention is a method for automatic zoom based on attitude detection, which enables precise automatic zooming in automated shooting tasks that are independent of human control. The automatic zooming includes the following steps:

[0061] S21. Detect the pose of the target person in the shooting scene using S11-16, and simultaneously detect the first and second landmarks using the target detection algorithm.

[0062] S22, Reference Figure 3 -a, When a first landmark is present in the frame and more than a first preset number of people are detected moving and gathering toward the landmark, the zoom module performs a zoom operation.

[0063] refer to Figure 3 -b When the first marker is not present in the frame, and the target person exceeds the second preset number of people, moves away from the marker or leaves the frame, the zoom module performs a zoom reduction operation to restore the lens to its original focal length.

[0064] S23, Reference Figure 4 -a, when the target people are all far away in the frame and are gathered together, the zoom module performs a zoom operation;

[0065] refer to Figure 4 -b, when the target person on the left moves to the left and the target person on the right moves to the right, or when a target person exceeding the third preset number of people moves closer, the zoom module performs a zoom reduction operation to restore the lens to its original focal length.

[0066] S24. Reference Figure 5 When there are a first landmark and a second landmark in the frame, if the target person moves in the same way as the second landmark and moves toward the first landmark, the zoom module performs a zoom operation.

[0067] If there is no second marker in the frame within the preset capture time, or if there is no first marker in the frame, the zoom module performs a zoom reduction operation to restore the lens to its original focal length.

[0068] As one embodiment, in S22, when the target person moves toward the marker, if the average distance between the target person and the marker decreases, the magnification of the zoom operation is higher.

[0069] Generally, the magnification ranges from 110% to 130%.

[0070] As one embodiment, the method in S21-24 can be applied to the filming of a football match; in this case, the target is the player, the first marker is the goal, and the second marker is the football.

[0071] In S22, when more than 2 / 3 of the players move toward the goal, the zoom module performs a zoom operation, with the magnification ranging from 110% to 130%. The closer the players are to the goal, the higher the magnification.

[0072] When the goal is out of the frame, or when more than half of the players in the frame move to the opposite side of the goal or leave the frame, the zoom module performs a zoom-out operation.

[0073] In S23, when players are far away from the screen and clustered together, the zoom module performs a zoom-in operation; when players spread out (i.e., players on the left side of the screen move to the left and players on the right side move to the right), or when more than half of the players move closer to the screen, the zoom module performs a zoom-out operation.

[0074] In S24, when the goal and the football are detected in the picture, and a player is dribbling the ball and moving towards the goal, the zoom module performs a zoom operation. The zoom factor ranges from 110% to 130%. The closer the player is to the goal, the higher the zoom factor.

[0075] If no football is detected within 2 seconds or no goal is visible in the frame, the zoom module performs a zoom-out operation.

[0076] As one embodiment, the present invention also provides an attitude detection and automatic zoom device, with reference to... Figure 6 The device consists of an image acquisition module, a zoom module, and a detection and control module.

[0077] The image capture module is used to capture video and obtain video footage;

[0078] The zoom module is used to zoom the lens of the image acquisition module according to the zoom control command of the detection control module.

[0079] The detection and control module is used to perform attitude detection on the image captured by the image acquisition module according to S11-16 and generate zoom control commands according to S21-24 based on the detection results.

[0080] As one embodiment, the method described in this invention can be implemented in software and / or a combination of software and hardware, for example, using an application-specific integrated circuit (ASIC), a general-purpose computer, or any other similar hardware device.

[0081] The method described in this invention can be implemented as a software program, which can be executed by a processor to achieve the steps or functions described above. Similarly, the software program (including associated data structures) can be stored in a computer-readable recording medium, such as RAM memory, magnetic or optical drives, floppy disks, and similar devices.

[0082] In addition, some steps or functions of the method described in this invention can be implemented in hardware, for example, as a circuit that works with a processor to perform the various steps or functions.

[0083] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0084] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0085] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for attitude detection and automatic zoom, characterized in that, The method includes the following steps: S11. Use target detection algorithms to identify human information in the location image; The human body information refers to the number of human bodies in the image and the position data of several human bodies; the position data refers to the center position coordinates and size value of the detection frame, and the size value includes the width and height of the detection frame; S12. Perform target person filtering to identify the target persons who need to undergo pose detection, specifically including: S121. From the position data obtained in S11, filter out the position data where the foot points are within the boundary line of the venue to obtain the data of people in the venue; the foot point refers to the bottom center point of the detection box; the boundary line of the venue is identified by a visual algorithm; S122. Select the location data from the data of people in the field whose detection frame size values ​​meet the size requirements to obtain the target person data; the size requirements are that the width and height of the detection frame meet the size threshold. S13. Use a clustering algorithm to classify the target people. When the clustering algorithm is used for classification, the three-dimensional coordinates of each target person are clustered. The first two dimensions of the three-dimensional coordinates are the horizontal and vertical coordinates of the center point of the detection box of the target person, and the third dimension is the height of the detection box, so as to obtain several target people groups. The number of the target population groups should meet the following conditions: Where N is the number of target population groups, N player The number of target individuals, where N is no greater than 3; S14. Cut the screen according to the target group category, so that several target characters in each target group category are in the same cut screen, and obtain the same number of cut screens as the target group category. S15. Enlarge the cut image to a preset resolution using a super-resolution algorithm; S16. Perform posture detection on the enlarged cut image.

2. The method according to claim 1, characterized in that, The object detection algorithms in S11 include any one or more combinations of the R-CNN series algorithms, YOLO series algorithms, and SSD algorithms.

3. The method according to claim 1, characterized in that, The clustering algorithm described in S13 is implemented using the three-dimensional K-Means clustering algorithm.

4. The method according to claim 1, characterized in that, The super-resolution algorithm includes any one or more of the following algorithms: bicubic interpolation, edge-based super-resolution, deep learning-based super-resolution, sparse representation-based super-resolution, and interpolation-based super-resolution.

5. The method according to claim 1, wherein In S16, pose detection of the magnified cropped image can be achieved using any one of the following algorithms: OpenPose, Mask R-CNN, AlphaPose, HRNet, or DensePose.

6. The method according to claim 1, characterized in that... , The method also includes an automatic zoom step, specifically including: S21. Detect the pose of the target person in the shooting scene using S11-16, and simultaneously detect the first and second landmarks using the target detection algorithm. S22. When a first landmark is present on the screen, and when it is detected that more than a first preset number of people are moving toward the landmark, the zoom module performs a zoom operation. When the first marker is not present in the frame, and the target person exceeds the second preset number of people, moves away from the marker or leaves the frame, the zoom module performs a zoom reduction operation to restore the lens to its original focal length. S23. When the target people are all far away in the frame and are gathered together, the zoom module performs a zoom operation. When the target person on the left moves to the left and the target person on the right moves to the right, or when a target person exceeding the third preset number of people moves closer, the zoom module performs a zoom reduction operation to restore the lens to its original focal length. S24. When there are a first landmark and a second landmark in the picture, if the target person moves in the same way as the second landmark and the target person moves toward the first landmark, the zoom module performs a zoom operation. If there is no second marker in the frame within the preset capture time, or if there is no first marker in the frame, the zoom module performs a zoom reduction operation to restore the lens to its original focal length.

Images