Panoramic photography control method, device, computer equipment, and storage medium

Abstract

The present disclosure relates to a method, device, computer equipment, and storage medium for controlling panoramic photography. The method includes: determining a target area in a viewfinder of a panoramic photography preview interface; during the panoramic photography process, determining limb features appearing in the target area, wherein the limb features at least contain body parts of a human body or object features associated with body parts; in response to the limb features not including gesture features, based on the limb features, performing target recognition in a panoramic image previewed in the panoramic photography preview interface, determining a body area associated with the limb features in the panoramic image, wherein the body area contains limb features and a hand area; identifying gesture information from a hand area in the body area, and controlling the panoramic photography process based on the gesture information. The method can conveniently control the photography effect during the photography process, thereby reducing the photography time.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a method, device, computer equipment, and storage medium for controlling panoramic photography. Background Art

[0002] With the development of mobile devices, when controlling mobile devices to take photos, the photo button of the mobile device is usually used to take photos. However, when using a selfie stick to take photos, it is necessary to repeatedly adjust various photo settings in the mobile device (such as adjusting the photo mode, focal length, etc.). This method is inconvenient. For this reason, there is a method of controlling mobile devices to take photos based on gesture recognition.

[0003] However, when controlling a mobile device to take photos based on gesture recognition, if there are multiple people within the recognition range of the mobile device at the same time, the gestures of multiple people need to be recognized at the same time, which will increase the shooting time. Summary of the invention

[0004] Based on this, it is necessary to provide a panoramic photography control method, device, computer equipment, and storage medium to address the above technical issues.

[0005] In a first aspect, the present disclosure provides a method for controlling panoramic photography, which is applied to a panoramic camera, and the method includes:

[0006] Determine the target area in the viewfinder of the panoramic shooting preview interface;

[0007] During the panoramic shooting process, determining limb features appearing in the target area, wherein the limb features at least include body parts of a human body or object features associated with the body parts;

[0008] In response to the limb features not including gesture features, target recognition is performed in the panoramic image previewed in the panoramic shooting preview interface based on the limb features, and a body region associated with the limb features in the panoramic image is determined, where the body region contains the limb features and a hand region;

[0009] Gesture information is recognized from a hand region in the body region, and a panoramic shooting process is controlled based on the gesture information.

[0010] In one embodiment, the panoramic camera includes: a first fisheye lens and a second fisheye lens, and determining the target area in the viewfinder of the panoramic shooting preview interface includes:

[0011] Determine an overlapping area of ​​a first image obtained by the panoramic camera through the first fisheye lens and a second image obtained by the second fisheye lens during shooting;

[0012] The overlapping area includes a first overlapping area and a second overlapping area, and the first overlapping area and the second overlapping area are symmetrically distributed in a panoramic image stitched by the first image and the second image; the first overlapping area is located at a first angle in the panoramic image, and the second overlapping area is located at a second angle in the panoramic image that is symmetrical to the first angle, and the angle of the panoramic image is 360°;

[0013] Determine a field of view center of the panoramic image, and determine the first overlapping area or the second overlapping area as a target overlapping area based on differences between the field of view center and the first angle and the second angle, respectively;

[0014] According to the target overlapping area, a target area is determined in a viewfinder of the panoramic shooting preview interface.

[0015] In one of the embodiments, determining the target area in the viewfinder of the panoramic shooting preview interface includes:

[0016] In response to receiving the area selection instruction, a target area is determined in the viewfinder frame of the panoramic shooting preview interface based on a selection box indicated by the area selection instruction.

[0017] In one embodiment, the performing target recognition in the panoramic image previewed in the panoramic shooting preview interface based on the limb feature and determining the body area associated with the limb feature in the panoramic image includes:

[0018] Performing edge detection on the limb feature to determine the shape of the limb feature, and determining a first body part corresponding to the limb feature according to the shape of the limb feature;

[0019] Performing target recognition in the panoramic image previewed in the panoramic shooting interface to determine a target feature in the panoramic image that is directly or indirectly connected to the limb feature;

[0020] Performing edge detection on the target feature to determine the shape of the target feature;

[0021] A body region associated with the limb feature in the panoramic image is determined based on a predetermined human body model, a shape of the target feature, and the first body part.

[0022] In one embodiment, determining the body region associated with the limb feature in the panoramic image based on a predetermined human body model, the shape of the target feature and the first body part comprises:

[0023] Determining an initial body part of the target feature based on the shape of the target feature;

[0024] Adjusting the initial body part according to the association relationship between the human body parts in the predetermined human body model and the first body part;

[0025] According to the adjusted initial body part, a body region associated with the limb feature in the panoramic image is determined.

[0026] In one embodiment, the identifying gesture information from the hand region in the body region includes:

[0027] Extracting hand key points in the hand region in the body region, and fitting the hand key points to obtain a hand skeleton model;

[0028] Based on the similarity between the hand skeleton model and a plurality of preset standard skeleton models, gesture information of the hand skeleton model is identified.

[0029] In one embodiment, the identifying gesture information of the hand skeleton model based on the similarity between the hand skeleton model and a plurality of preset standard skeleton models includes:

[0030] Determine a first spatial coordinate set of the key points of the hand in the hand skeleton model, and a second spatial coordinate set of the key points of the hand in a plurality of the standard skeleton models;

[0031] Calculating the Hausdorff distance between the first spatial coordinate set and a plurality of the second spatial coordinate sets;

[0032] Determining the similarity between the hand skeleton model and a plurality of preset standard skeleton models according to the Hausdorff distance;

[0033] Determining a target skeleton model among the plurality of standard skeleton models according to the similarity;

[0034] Based on the gesture information matched by the target skeleton model, the gesture information of the hand skeleton model is determined.

[0035] In a second aspect, the present disclosure further provides a control device for panoramic photography. The device comprises:

[0036] A target area determination module is used to determine the target area in the viewfinder of the panoramic shooting preview interface;

[0037] A limb feature determination module, used to determine limb features appearing in the target area during panoramic shooting, wherein the limb features at least include body parts of a human body or object features associated with the body parts;

[0038] a target recognition module, configured to, in response to the limb feature not including a gesture feature, perform target recognition in the panoramic image previewed in the panoramic shooting preview interface based on the limb feature, and determine a body region associated with the limb feature in the panoramic image, wherein the body region contains the limb feature and a hand region;

[0039] The gesture control module is used to identify gesture information from the hand area in the body area, and control the panoramic shooting process based on the gesture information.

[0040] In a third aspect, the present disclosure further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps in any of the above method embodiments are implemented.

[0041] In a fourth aspect, the present disclosure further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0042] In a fifth aspect, the present disclosure further provides a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor, the steps in any of the above method embodiments are implemented.

[0043] In the above embodiments, by determining the target area in the viewfinder of the panoramic shooting preview interface, the limb features appearing in the target area are determined during the panoramic shooting process. Since the process of panoramic shooting needs to be controlled, the controller cannot be accurately identified when there are a large number of people, and if the recognition is based on facial information, it will be difficult to identify the controller when the controller does not show his face (such as wearing a mask or a hat), and the recognition speed is slow when there are a large number of people. Therefore, the target area can be used for recognition to improve the recognition speed. In some cases, due to certain restrictions, the controller's gesture is not placed in the target area, and in response to the limb features not including the gesture features, in order to enable the controller to control, the target can be recognized in the panoramic image previewed in the panoramic shooting preview interface based on the limb features, and the body area associated with the limb features in the panoramic image is determined. In this way, other body areas of the controller related to the limb features can be found. Then, the gesture information is recognized from the hand area in the body area, and the process of panoramic shooting is controlled based on the gesture information. There is no need to additionally recognize the gestures of multiple people, nor is there any need to recognize faces. As long as the body area can be recognized according to the limb features of the target area, the shooting speed can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 is a flow chart of a method for controlling panoramic photography in one embodiment;

[0046] Figure 2 Schematic diagram of the process of step S102 in one embodiment;

[0047] Figure 3 is a schematic diagram of a first overlapping region and a second overlapping region in one embodiment;

[0048] Figure 4 is a schematic diagram of the center of the field of view in one embodiment;

[0049] Figure 5 This is a schematic diagram of the process of step S106 in one embodiment;

[0050] Figure 6 This is a schematic diagram of the process of step S308 in one embodiment;

[0051] Figure 7This is a schematic diagram of the process of step S108 in one embodiment;

[0052] Figure 8 is a schematic diagram of the process of step S504 in one embodiment;

[0053] Fig. 9 is a schematic block diagram of the structure of a panoramic photography control device in one embodiment;

[0054] Fig.10 Schematic diagram of the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solution and advantages of the present disclosure more clear, the present disclosure is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure and are not used to limit the present disclosure.

[0056] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0057] In this article, the term "and / or" is only a description of the association relationship between related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the related objects before and after are in an "or" relationship.

[0058] In one embodiment, Figure 1 As shown, a method for controlling panoramic shooting is provided, which is described by taking the method applied to a panoramic camera as an example, and includes the following steps:

[0059] S102: determining a target area in a viewfinder of a panoramic shooting preview interface.

[0060] The panoramic shooting preview interface may generally be an interface of a panoramic image previewed by a panoramic camera during the process of shooting a panoramic image or video. The viewfinder may generally be a frame for displaying a panoramic image. The panoramic preview interface may be different depending on different situations, so the panoramic preview interface may be the same as the viewfinder, and the viewfinder may also be included in the panoramic preview interface.

[0061] Specifically, when shooting with a panoramic camera, the user can select a certain area in the viewfinder, and determine the target area according to the selected area, or determine the target area according to certain rules in the panoramic image stitching process. The target area can be flexibly set according to actual conditions, and the determination process of the target area is not limited in this embodiment.

[0062] S104, during the panoramic shooting process, determining limb features appearing in the target area, wherein the limb features at least include body parts of a human body or object features associated with the body parts.

[0063] Among them, the body parts of the human body may include, in some embodiments of the present disclosure, the head, arms, palms, etc. Object features associated with body parts, such as object features of a hat, gloves, glasses, shoes, watches, etc.

[0064] Specifically, during the panoramic shooting process, since the target area has been determined, it is necessary to control the panoramic shooting process, such as controlling the focal length, controlling whether to shoot continuously, controlling the shooting mode, etc. Therefore, the user can place body features, such as the head, arms, palms, etc., in the target area to control the panoramic shooting process. At this time, the body features appearing in the target area can be determined.

[0065] S106, in response to the limb features not including gesture features, target recognition is performed in the panoramic image previewed in the panoramic shooting preview interface based on the limb features, and a body region associated with the limb features in the panoramic image is determined, wherein the body region contains the limb features and a hand region.

[0066] The gesture feature may be a feature of a palm. The object recognition may be a method of detecting and identifying a body region associated with a specific limb feature by analyzing a panoramic image. The hand region may be a region containing a palm in a panoramic image.

[0067] Specifically, when the limb features do not include gesture features, it is necessary to identify gestures from the panoramic image obtained by panoramic shooting. Therefore, it is necessary to determine which areas in the panoramic image are associated with the limb features by analyzing the key points of the human body according to the limb features and combining image segmentation technology, and the areas may generally be body areas.

[0068] S108, identifying gesture information from the hand region in the body region, and controlling the panoramic shooting process based on the gesture information.

[0069] The gesture information may be a hand gesture, and different gestures may correspond to different control methods. Those skilled in the art may set the control method of the gesture information during the panoramic shooting process according to actual conditions. For example, when an "OK" gesture is made, the panoramic camera may take a photo or record a video. When a "V" gesture is made, the panoramic camera may switch the photo to the video or the video to the photo.

[0070] Specifically, since the body region contains the hand region, the hand region in the body region may be determined first. The hand region in the body region may be determined by image segmentation and edge detection. Then, gesture information is determined to be recognized from the hand region. The panoramic shooting process is controlled according to the recognized gesture information and a pre-set control method matching each gesture information.

[0071] In the above-mentioned panoramic shooting control method, by determining the target area in the viewfinder of the panoramic shooting preview interface, the limb features appearing in the target area are determined during the panoramic shooting process. Since the process of panoramic shooting needs to be controlled, the controller cannot be accurately identified when there are a large number of people, and if it is identified based on facial information, it will be difficult to identify the controller when the controller does not show his face (such as wearing a mask or a hat), and the recognition speed is also slow when there are a large number of people. Therefore, the target area can be used for identification to improve the recognition speed. In some cases, due to certain restrictions, the controller's gesture is not placed in the target area, and in response to the limb features not including the gesture features, in order to enable the controller to control, the target can be identified in the panoramic image previewed in the panoramic shooting preview interface based on the limb features, and the body area associated with the limb features in the panoramic image is determined. In this way, other body areas of the controller related to the limb features can be found. Then, the gesture information is identified from the hand area in the body area, and the process of panoramic shooting is controlled based on the gesture information. There is no need to additionally identify the gestures of multiple people, nor is there any need to recognize faces. As long as the body area can be identified according to the limb features of the target area, the shooting speed can be improved.

[0072] In one embodiment, Figure 2As shown, the panoramic camera includes: a first fisheye lens and a second fisheye lens, and determining the target area in the viewfinder of the panoramic shooting preview interface includes:

[0073] S202: Determine an overlapping area of ​​a first image obtained by the panoramic camera through a first fisheye lens and a second image obtained by the panoramic camera through a second fisheye lens during a shooting process.

[0074] Among them, Figure 3 As shown, the overlapping area includes a first overlapping area and a second overlapping area, and the first overlapping area and the second overlapping area are symmetrically distributed in a panoramic image stitched by the first image and the second image; the first overlapping area is located at a first angle in the panoramic image, and the second overlapping area is located at a second angle in the panoramic image that is symmetrical to the first angle, and the angle of the panoramic image is 360°. Figure 3 As shown, for example, if the first overlapping area is located at 0° of the panoramic image, the second overlapping area may be located at 180° of the panoramic image.

[0075] Specifically, during the shooting process of the panoramic camera, since the panoramic camera uses two fisheye lenses, the first fisheye lens and the second fisheye lens are used to shoot and then stitch the images to generate a panoramic image. Since the optical centers of the first fisheye lens and the second fisheye lens do not overlap, the images generated by the first fisheye lens and the second fisheye lens will have an overlapping area. Therefore, the overlapping area can be determined.

[0076] S204: Determine a field of view center of the panoramic image, and determine the first overlapping area or the second overlapping area as a target overlapping area based on differences between the field of view center and the first angle and the second angle, respectively.

[0077] Among them, in some embodiments of the present disclosure, the center of the field of view may be the center point of the panoramic image during the shooting process, and the position of the center of the field of view may be different depending on the angle at which the controller moves.

[0078] Specifically, the difference between the center of the field of view to the first angle and the second angle can be determined, the corresponding first angle or second angle with the smallest difference is selected, and then the corresponding overlapping area is determined as the target overlapping area according to the first angle or second angle with the smallest difference.

[0079] In some exemplary embodiments, Figure 4As shown, for example, the first angle corresponding to the first overlapping area is 0°, and the second angle corresponding to the second overlapping area is 180°. According to the three different shooting angles, there are three different fields of view centers, A1, A2 and A3, wherein the angle difference between the field of view center of A1 and the first angle of the first overlapping area can be 60°, and the angle difference between the field of view center of A1 and the second angle of the second overlapping area can be 120°. Therefore, when A1 is the field of view center, the target overlapping area can be the first overlapping area. The angle difference between A2 and the first angle of the first overlapping area can be 135°, and the angle difference between A2 and the second angle of the second overlapping area can be 45°. Therefore, when A2 is the field of view center, the target overlapping area can be the second overlapping area. The angle difference between A3 and the first angle of the first overlapping area can be 90°, and the angle difference between A3 and the second angle of the second overlapping area can be 90°. Therefore, one of the first overlapping area and the second overlapping area can be selected as the target overlapping area.

[0080] S206: Determine a target area in a viewfinder of a panoramic shooting preview interface according to the target overlapping area.

[0081] Specifically, after the target overlapping region is determined, all or a part of the target overlapping region may be used as the target region.

[0082] In this embodiment, the target area is determined by the angle difference between the center of the field of view and different overlapping areas, so that the target area can be set closer to the center of the field of view, thereby more conveniently controlling the panoramic shooting.

[0083] In one embodiment, determining the target area in the viewfinder of the panoramic shooting preview interface includes:

[0084] In response to receiving the area selection instruction, a target area is determined in the viewfinder frame of the panoramic shooting preview interface based on a selection box indicated by the area selection instruction.

[0085] The area selection instruction may generally be an instruction input into the panoramic camera by a user or a photographer.

[0086] Specifically, the user or photographer can input the area selection instruction in the panoramic camera. The area selection instruction can be input by touching a function menu on the screen of the panoramic camera, or by a key in the panoramic camera. Some embodiments of the present disclosure do not limit how the area selection instruction is input into the panoramic camera. After receiving the area selection instruction, the user can use a selection box in the panoramic camera to select a certain area, thereby determining the target area according to the selected area.

[0087] It should be noted that when the user inputs an area selection instruction, the target area will first be determined according to the selection indicated by the area selection instruction.

[0088] In this embodiment, the target area is determined in the viewfinder of the panoramic shooting preview interface through the selection box indicated by the area selection instruction. The target area can be flexibly set according to actual needs, thereby facilitating control during the panoramic shooting process.

[0089] In one embodiment, Figure 5 As shown, the target recognition is performed in the panoramic image previewed in the panoramic shooting preview interface based on the limb feature, and the body area associated with the limb feature in the panoramic image is determined, including:

[0090] S302, performing edge detection on the limb feature to determine the shape of the limb feature, and determining a first body part corresponding to the limb feature according to the shape of the limb feature.

[0091] Edge detection is a technique used in image processing and computer vision to identify the edges or boundaries of objects in an image. Edges usually represent areas where the brightness of an image changes significantly, usually corresponding to the outline of an object.

[0092] Specifically, edge detection operators can be used to perform edge detection on limb features, thereby determining the shape of the limb features. Edge detection operators may include: Sobel operator, Laplacian operator, Prewitt operator, etc. After edge detection, the shape of the limb feature can be determined, and the first body part corresponding to the limb feature can be roughly determined based on the shape of the limb feature. For example, if the shape of the limb feature is identified as a circle, it can be roughly determined that the first body part corresponding to the limb feature is the head. If the shape of the limb feature is identified as a cylinder or a rectangle, it can be roughly determined that the first body part corresponding to the limb feature is an arm or a leg. It can be understood that the above is only used for illustration, and the shape identified in the actual application process may not be a very regular shape, and it is necessary to use a matching algorithm or a similarity algorithm based on the identified shape to determine the corresponding first body part.

[0093] S304, performing target recognition in the panoramic image previewed in the panoramic shooting interface, and determining a target feature in the panoramic image that is directly or indirectly connected to the limb feature.

[0094] Specifically, after the limb features are identified, since the parts of the human body may be directly or indirectly connected, target recognition may be performed according to the limb features to determine the target features that are directly or indirectly connected to the limb features.

[0095] S306: Perform edge detection on the target feature to determine the shape of the target feature.

[0096] S308: Determine a body region associated with the limb feature in the panoramic image based on a predetermined human body model, the shape of the target feature, and the first body part.

[0097] The human body model includes the connection relationship between various human body parts.

[0098] Specifically, after determining the target feature, edge detection can be performed on the target feature to determine the shape of the target feature. (The method of performing edge detection can refer to the above steps and will not be repeated here.) In addition, since not all target features are human features, misidentification may occur. Therefore, the target features can be screened and matched according to the connection relationship between the various human body parts in the predetermined human body model, the shape of the target feature, and the first body part, so as to determine the target feature containing the human body part, and determine the body area associated with the limb feature in the panoramic image according to the target feature containing the human body part.

[0099] In this embodiment, by using edge detection and a human body model, the body region associated with the limb features can be accurately identified, thereby improving the accuracy of subsequent judgments.

[0100] In one embodiment, Figure 6 As shown, the determining of the body region associated with the limb feature in the panoramic image based on a predetermined human body model, the shape of the target feature and the first body part comprises:

[0101] S402: Determine an initial body part of the target feature according to the shape of the target feature.

[0102] S404: Adjust the initial body part according to the predetermined association relationship between the human body parts in the human body model and the first body part.

[0103] S406: Determine a body region associated with the limb feature in the panoramic image according to the adjusted initial body part.

[0104] Specifically, after the shape of the target feature is determined, the shape of the target feature can be used to initially determine the initial body part. It should be noted that the initial body part determined can be one or more, for example, a target feature is determined to be a left arm or a right arm. Then, the initial body part is adjusted according to the association relationship between the various human body parts in the human body model and the first body part. For example, some unreasonable parts can be adjusted, or some initial body parts that are incorrectly determined can be modified. Then, based on the adjusted initial body part, the body area associated with the limb feature in the panoramic image is determined.

[0105] In some exemplary embodiments, for example, the initial shape of the target feature is determined to be a forearm or an upper arm. The first body part is the head. Since the head is close to an end point of the upper arm among human body parts, if an end point of the target feature is close to the first body part, the target feature can be determined to be an upper arm. It is understood that the above is only for illustration.

[0106] In this embodiment, through the association relationship between the human body parts in the human body model and the first body part, it is possible to accurately determine which body part the target feature belongs to, thereby improving the accuracy of control in panoramic shooting.

[0107] In one embodiment, Figure 7 As shown, the identifying gesture information from the hand region in the body region includes:

[0108] S502, extracting hand key points in the hand region in the body region, and fitting the hand key points to obtain a hand skeleton model.

[0109] Among them, the key points of the hand can usually be the various joints in the palm. (such as the thumb, index finger, etc.) and the proximal joints and distal joints in the palm and other locations. The hand skeleton model is a simplified model used to represent the structure of the hand, which is usually used in gesture recognition, motion capture or virtual reality applications in the fields of computer vision, graphics and robotics. This model helps to better understand and analyze the dynamics and postures of the hand by defining the relationship between the various parts of the hand and the connection method of the joints. The hand skeleton model includes: 1. Skeleton composition: Joints: The hand skeleton model usually includes multiple joints, such as the wrist, palm joints, and joints of each finger (including the proximal segment, middle segment and distal segment). Bones: The part connecting the joints represents the "bone" structure of the fingers and palm. 2. Joint connection: Range of motion: Each joint has its specific range of motion, which can simulate the movements of the hand accordingly. Hierarchical structure: The hand skeleton model is presented as a tree structure, starting from the wrist and connecting to the palm and each finger in turn. 3. Coordinate system 3D coordinates: The hand skeleton model can define the position of each joint in 3D space, usually expressed as (X, Y, Z) coordinates. Relative position: The relative position and angle relationship between joints help the model understand how the fingers bend and extend.

[0110] Specifically, the key points of the hand in the hand area can be extracted by using a key point detection algorithm (such as Posenet, OpenPose, etc.), and then the identified hand key points are fitted according to the shape of the hand to obtain a hand skeleton model.

[0111] S504: Identify gesture information of the hand skeleton model based on the similarity between the hand skeleton model and a plurality of preset standard skeleton models.

[0112] The standard skeleton model may be a pre-set standard gesture.

[0113] Specifically, since the gestures recognized during panoramic shooting may be different from standard gestures and are not exactly the same, the similarity between the hand skeleton model and multiple preset standard skeleton models can be calculated, and the gesture information can be recognized based on the similarity.

[0114] In this embodiment, gesture information is identified by the similarity between the gesture skeleton model and the preset standard skeleton model. Since the gesture skeleton model simplifies the gesture into the structure of joints and bones, the amount of data to be processed is reduced, making the similarity calculation more efficient. Due to the reduction in the amount of data, gesture recognition and processing can be performed faster, which is suitable for real-time applications such as virtual reality and augmented reality. In addition, the gesture skeleton model is usually more robust to noise and motion blur, and can better adapt to gesture input in different environments or devices. It can also handle gesture deformation caused by changes in the position and posture of the hand, increasing the reliability of gesture recognition. A variety of different gestures can be recognized through the skeleton model, and only the joint position and angle need to be compared and analyzed. Regardless of the size or shape of the hand, the structural analysis provided by the skeleton model enables the recognition system to adapt to different users.

[0115] In one embodiment, Figure 8 As shown, the identifying gesture information of the hand skeleton model based on the similarity between the hand skeleton model and a plurality of preset standard skeleton models includes:

[0116] S602, determining a first spatial coordinate set of hand key points in the hand skeleton model and a second spatial coordinate set of hand key points in a plurality of standard skeleton models.

[0117] Specifically, since the hand skeleton model contains multiple hand key points, the hand key points can be used to calculate the similarity. Therefore, the hand key points contained in the hand skeleton model can be determined to generate a first set of spatial coordinates, and a second set of spatial coordinates generated using the hand key points in each standard skeleton model. It should be noted that the coordinate system corresponding to the hand skeleton model and each standard skeleton model is the same, and in some embodiments of the present disclosure, there is no restriction on how the coordinate system corresponding to the hand skeleton model and each standard skeleton model is established.

[0118] S604: Calculate the Hausdorff distance between the first spatial coordinate set and a plurality of the second spatial coordinate sets.

[0119] Among them, Hausdorff distance is a mathematical concept that measures the similarity between two point sets. It is used to describe the "maximum distance" between two sets, that is, the maximum value of the minimum distance from each point in one set to the other set.

[0120] Specifically, take two point sets as an example, where point set A is the first spatial coordinate set and point set B is the second spatial coordinate set. The Hausdorff distance is .

[0121]

[0122]

[0123]

[0124] in, Represents the maximum distance between a point in set A and the nearest point in set B. It means the maximum distance between the points in set B and the nearest points in set A. Therefore, Hausdorff distance It measures the distance of the farthest point in one set relative to its "closest point" in the other set.

[0125] S606: Determine the similarity between the hand skeleton model and a plurality of preset standard skeleton models according to the Hausdorff distance.

[0126] S608: Determine a target skeleton model among the multiple standard skeleton models according to the similarity.

[0127] S610: Determine gesture information of the hand skeleton model based on the gesture information matched by the target skeleton model.

[0128] Specifically, after determining the Hausdorff distance, the Hausdorff distance can be used as the similarity. Then, the standard skeleton model with the smallest similarity is found, and the standard skeleton model with the smallest similarity is determined as the target skeleton model. Since each standard skeleton model has matching gesture information, the matching gesture information of the target skeleton model is determined, and the gesture information of the hand skeleton model is determined based on the gesture information.

[0129] In the present embodiment, the overall similarity between two shapes or sets can be effectively evaluated by the Hausdorff distance, rather than just local features, so the accuracy of the evaluation can be improved. Since the Hausdorff distance focuses on the farthest point between the two sets, some tiny noises and irregularities can be effectively ignored, thereby improving the robustness of the algorithm. Whether it is a point set, a curve or a more complex shape, the Hausdorff distance can be applied, providing a unified similarity measurement method. The Hausdorff distance has a clear geometric meaning, i.e., the distance from any point in the two sets to the farthest point of the other set, and this intuitiveness makes it easier to understand and apply in practical applications.

[0130] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0131] Based on the same inventive concept, the disclosed embodiment also provides a panoramic photography control device for implementing the panoramic photography control method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations of one or more panoramic photography control device embodiments provided below can refer to the limitations of the panoramic photography control method above, and will not be repeated here.

[0132] In one embodiment, Fig. 9 As shown, a panoramic shooting control device 700 is provided, which is applied to a panoramic camera, and includes: a target area determination module 702, a limb feature determination module 704, a target recognition module 706 and a gesture control module 708, wherein:

[0133] The target area determination module 702 is used to determine the target area in the viewfinder of the panoramic shooting preview interface.

[0134] The limb feature determination module 704 is used to determine the limb features appearing in the target area during the panoramic shooting process, wherein the limb features at least include body parts of the human body or object features associated with the body parts.

[0135] The target recognition module 706 is used to, in response to the limb features not including gesture features, perform target recognition in the panoramic image previewed in the panoramic shooting preview interface based on the limb features, and determine the body area associated with the limb features in the panoramic image, wherein the body area contains the limb features and the hand area.

[0136] The gesture control module 708 is used to identify gesture information from the hand area in the body area, and control the panoramic shooting process based on the gesture information.

[0137] In one embodiment of the device, the panoramic camera includes: a first fisheye lens and a second fisheye lens. The target area determination module 702 is further used to determine the overlapping area of ​​the first image obtained by the panoramic camera through the first fisheye lens and the second image obtained by the second fisheye lens during the shooting process; wherein the overlapping area includes a first overlapping area and a second overlapping area, and the first overlapping area and the second overlapping area are symmetrically distributed in the panoramic image stitched by the first image and the second image; the first overlapping area is located at a first angle in the panoramic image, and the second overlapping area is located at a second angle in the panoramic image that is symmetrical to the first angle, and the angle of the panoramic image is 360°; determine the field of view center of the panoramic image, and determine the first overlapping area or the second overlapping area as the target overlapping area based on the difference between the field of view center and the first angle and the second angle respectively; determine the target area in the viewfinder of the panoramic shooting preview interface according to the target overlapping area.

[0138] In one embodiment of the device, the target area determination module 702 is further used to determine the target area in the viewfinder of the panoramic shooting preview interface based on the selection box indicated by the area selection instruction in response to receiving the area selection instruction.

[0139] In one embodiment of the device, the target identification module 706 includes:

[0140] The first edge detection module is used to perform edge detection on the limb feature, determine the shape of the limb feature, and determine the first body part corresponding to the limb feature according to the shape of the limb feature.

[0141] The feature recognition module is used to perform target recognition in the panoramic image previewed in the panoramic shooting interface, and determine the target feature in the panoramic image that is directly or indirectly connected to the limb feature.

[0142] The second edge detection module is used to perform edge detection on the target feature to determine the shape of the target feature.

[0143] The body region determination module is used to determine the body region associated with the limb feature in the panoramic image based on a predetermined human body model, the shape of the target feature and the first body part.

[0144] In one embodiment of the device, the body area determination module is further used to determine the initial body part of the target feature based on the shape of the target feature; adjust the initial body part based on the association relationship between the human body parts in the predetermined human body model and the first body part; and determine the body area associated with the limb feature in the panoramic image based on the adjusted initial body part.

[0145] In one embodiment of the device, the body area determination module is further used to determine a first spatial coordinate set of hand key points in the hand skeleton model and a second spatial coordinate set of hand key points in multiple standard skeleton models; calculate the Hausdorff distance between the first spatial coordinate set and multiple second spatial coordinate sets; determine the similarity between the hand skeleton model and multiple preset standard skeleton models based on the Hausdorff distance; determine a target skeleton model among the multiple standard skeleton models based on the similarity; and determine the gesture information of the hand skeleton model based on the gesture information matching the target skeleton model.

[0146] Each module in the above-mentioned panoramic photography control device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0147] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for controlling panoramic shooting is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0148] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present disclosure, and does not constitute a limitation on the computer device to which the scheme of the present disclosure is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0149] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0150] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0151] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0152] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided by the present disclosure can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided by the present disclosure may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided by the present disclosure may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, etc., but are not limited to this.

[0153] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0154] The above-described embodiments only express several implementation methods of the present disclosure, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present disclosure, and these all belong to the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the attached claims.

Claims

1. A method for controlling panoramic photography, characterized in that: Applied to a panoramic camera, the method comprises: Determine the target area in the viewfinder of the panoramic shooting preview interface; During the panoramic shooting process, determining limb features appearing in the target area, wherein the limb features at least include body parts of a human body or object features associated with the body parts; In response to the limb feature not including a gesture feature, based on the limb feature, target recognition is performed in the panoramic image previewed in the panoramic shooting preview interface, and a body region associated with the limb feature is determined in the panoramic image, wherein the body region contains the limb feature and a hand region; based on the limb feature, target recognition is performed in the panoramic image previewed in the panoramic shooting preview interface, and a body region associated with the limb feature is determined in the panoramic image, including: determining the region associated with the limb feature in the panoramic image according to the limb feature and key points of the human body in combination with image segmentation technology; Gesture information is recognized from a hand region in the body region, and a panoramic shooting process is controlled based on the gesture information.

2. The method according to claim 1, characterized in that The panoramic camera includes: a first fisheye lens and a second fisheye lens, and determining the target area in the viewfinder of the panoramic shooting preview interface includes: Determine an overlapping area of ​​a first image obtained by the panoramic camera through the first fisheye lens and a second image obtained by the second fisheye lens during shooting; The overlapping area includes a first overlapping area and a second overlapping area, and the first overlapping area and the second overlapping area are symmetrically distributed in a panoramic image stitched by the first image and the second image; the first overlapping area is located at a first angle in the panoramic image, and the second overlapping area is located at a second angle in the panoramic image that is symmetrical to the first angle, and the angle of the panoramic image is 360°; Determine a field of view center of the panoramic image, and determine the first overlapping area or the second overlapping area as a target overlapping area based on differences between the field of view center and the first angle and the second angle, respectively; According to the target overlapping area, a target area is determined in a viewfinder of the panoramic shooting preview interface.

3. The method according to claim 1, characterized in that Determining the target area in the viewfinder of the panoramic shooting preview interface includes: In response to receiving the area selection instruction, a target area is determined in the viewfinder frame of the panoramic shooting preview interface based on a selection box indicated by the area selection instruction.

4. The method according to claim 1, characterized in that: The step of performing target recognition in the panoramic image previewed in the panoramic shooting preview interface based on the limb feature and determining a body region associated with the limb feature in the panoramic image includes: Performing edge detection on the limb feature to determine the shape of the limb feature, and determining a first body part corresponding to the limb feature according to the shape of the limb feature; Performing target recognition in the panoramic image previewed in the panoramic shooting interface to determine a target feature in the panoramic image that is directly or indirectly connected to the limb feature; Performing edge detection on the target feature to determine the shape of the target feature; A body region associated with the limb feature in the panoramic image is determined based on a predetermined human body model, a shape of the target feature, and the first body part.

5. The method according to claim 4, characterized in that The determining of a body region associated with the limb feature in the panoramic image based on a predetermined human body model, a shape of the target feature, and the first body part comprises: Determining an initial body part of the target feature based on the shape of the target feature; Adjusting the initial body part according to the association relationship between the human body parts in the predetermined human body model and the first body part; According to the adjusted initial body part, a body region associated with the limb feature in the panoramic image is determined.

6. The method according to claim 1, characterized in that The identifying gesture information from the hand region in the body region includes: Extracting hand key points in the hand region in the body region, and fitting the hand key points to obtain a hand skeleton model; Based on the similarity between the hand skeleton model and a plurality of preset standard skeleton models, gesture information of the hand skeleton model is identified.

7. The method according to claim 6, characterized in that The identifying gesture information of the hand skeleton model based on the similarity between the hand skeleton model and a plurality of preset standard skeleton models includes: Determine a first spatial coordinate set of the key points of the hand in the hand skeleton model, and a second spatial coordinate set of the key points of the hand in a plurality of the standard skeleton models; Calculating the Hausdorff distance between the first spatial coordinate set and a plurality of the second spatial coordinate sets; Determining the similarity between the hand skeleton model and a plurality of preset standard skeleton models according to the Hausdorff distance; Determining a target skeleton model among the plurality of standard skeleton models according to the similarity; Based on the gesture information matched by the target skeleton model, the gesture information of the hand skeleton model is determined.

8. A panoramic photography control device, characterized in that: Applied to a panoramic camera, the device comprises: A target area determination module, used to determine the target area in the viewfinder of the panoramic shooting preview interface; A limb feature determination module, used to determine limb features appearing in the target area during panoramic shooting, wherein the limb features at least include body parts of a human body or object features associated with the body parts; The target recognition module is used for, in response to the limb feature not including the gesture feature, performing target recognition in the panoramic image previewed in the panoramic shooting preview interface based on the limb feature, and determining the body region associated with the limb feature in the panoramic image, wherein the body region contains the limb feature and the hand region; performing target recognition in the panoramic image previewed in the panoramic shooting preview interface based on the limb feature and determining the body region associated with the limb feature in the panoramic image, comprising: determining the region associated with the limb feature in the panoramic image according to the limb feature and key points of the human body in combination with image segmentation technology; The gesture control module is used to identify gesture information from the hand area in the body area, and control the panoramic shooting process based on the gesture information.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images