Image acquisition method and device of target sphere, computer equipment and storage medium

Abstract

The invention relates to a picture acquisition method and device of a target sphere, computer equipment, a storage medium and a computer program product. The method comprises the following steps: detecting a motion track of each motion object in a motion object group according to a multi-frame picture; determining group movement information of a movement object group according to the movement track of each movement object; determining group transition information according to a transition condition conforming to the group movement information; and according to the group transition information, carrying out image acquisition on the target sphere. Therefore, the accuracy is not influenced by the shielding relationship, and whether transition is needed can be more accurately judged, so that the stability of the image acquisition process is good.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method, device, computer equipment, storage medium and computer program product for capturing images of a target sphere. Background Art

[0002] In stadium-related application scenarios, there is a demand for video recording, that is, users want to record the moment of scoring, attack and defense transitions and other highlight scenes during the game. Due to the small field of view (FOV) of the mobile phone, it is difficult to include the entire stadium in the picture. To record the entire game video, you can only manually move the camera to keep up with the movement of the crowd.

[0003] In the prior art, the user's demand is video recording. The user wants to record the goal moment, attack and defense transition and other highlight scenes during the game. During the attack and defense transition of the player, the target ball will be frequently blocked. If the pan / tilt is guided to rotate based on the target ball and the picture is collected, the stability of the collected picture is poor. Summary of the invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for capturing images of a target sphere in order to improve stability.

[0005] In a first aspect, the present application provides a method for capturing an image of a target sphere, the method comprising:

[0006] Detecting the motion trajectory of each moving object in the moving object group according to multiple frames;

[0007] Determining group motion information of a group of moving objects according to the motion trajectory of each of the moving objects;

[0008] Determining group transition information according to the transition situation that the group movement information conforms to;

[0009] According to the group transition information, the image of the target sphere is captured.

[0010] In one embodiment, determining group motion information of a group of moving objects according to the motion trajectory of each of the moving objects includes:

[0011] Determining a target picture area and different transition critical positions in a reference picture of the multiple frames;

[0012] Among the motion trajectories of the moving objects passing through the target picture area, the motion trajectories of the moving objects intersecting at the different transition critical positions are respectively counted to obtain the number of moving objects in different transition directions;

[0013] The determining of group transition information according to the transition situation that the group movement information conforms to includes:

[0014] The group transition information is determined according to the transition interval to which the number of moving objects in different transition directions belongs.

[0015] In one embodiment, the critical value of the transition interval includes a first critical value and a second critical value, and the first critical value is greater than the second critical value;

[0016] The determining of group transition information according to the transition interval to which the number of moving objects in different transition directions belongs includes:

[0017] If the number of moving objects in the first transition direction is greater than the first critical value, and the number of moving objects in the second transition direction is less than the second critical value, determining the group transition direction to be the first transition direction;

[0018] If the number of moving objects in the first transition direction is less than the second critical value, and the number of moving objects in the second transition direction is greater than the first critical value, determining the group transition direction to be the second transition direction;

[0019] Wherein, the first transition direction and the second transition direction are opposite directions.

[0020] In one embodiment, determining group motion information of a group of moving objects according to the motion trajectory of each of the moving objects includes:

[0021] Determine the individual position of each of the moving objects at the same time according to the moving trajectories of each of the moving objects at different times;

[0022] Generating a group position at the same moment according to the individual positions of the objects at the same moment;

[0023] Generate group displacements of at least two groups of adjacent moments according to the group positions of at least two groups of adjacent moments;

[0024] The determining of group transition information according to the transition situation that the group movement information conforms to includes:

[0025] If the group displacements of the at least two groups of moments have the same transition direction, and in the same transition direction, the group displacements of the at least two groups of moments respectively exceed the transition length threshold, it is determined that the group transition information includes the same transition direction.

[0026] In one of the embodiments, when the group motion information includes the number of moving objects in different transition directions and does not include group displacements at at least two groups of adjacent moments, the transition interval used to determine the transition situation is the first transition interval;

[0027] When the group motion information does not include the number of moving objects in different transition directions and includes at least two groups of group displacements at adjacent moments, the transition length threshold used to determine the transition situation is a first transition length threshold;

[0028] In a case where the group motion information includes the number of moving objects in different transition directions and includes at least two groups of group displacements at adjacent moments, the transition interval is a second transition interval, and the transition length threshold is a second transition length threshold;

[0029] The first transition interval covers the second transition interval, and the range of the first transition interval is smaller than the range of the second transition interval, and the first transition length threshold is greater than the second transition length threshold.

[0030] In one embodiment, determining the group transition information according to the transition situation that the group movement information matches includes:

[0031] When the group motion information does not meet the group transition information generation condition, optical flow estimation is performed on the moving object group according to the multiple frames to obtain the optical flow displacement of each frame;

[0032] Accumulating the optical flow displacement of the picture of each frame to obtain a picture motion feature map corresponding to the multiple frames; the picture motion feature map includes a plurality of feature map pixels, and the direction of each feature map pixel is determined by the accumulation result of the optical flow displacement of the picture of each frame;

[0033] According to the directions of the pixels of each feature map, the number of pixels of the feature map that are displaced along the opposite transition direction is counted to obtain the number of pixels in the opposite transition direction;

[0034] A comparison result between the numbers of pixels in opposite transition directions is determined; the comparison result pertaining to the group motion information.

[0035] In one embodiment, determining the group transition information according to the transition situation that the group movement information matches includes:

[0036] When the group motion information does not meet the group transition information generation condition, a pixel quantity difference is determined based on a difference between the numbers of pixels in opposite transition directions; the number of pixels in the opposite transition directions is obtained based on the multiple frames;

[0037] Obtaining a pixel quantity difference ratio based on a quantity ratio between the pixel quantity difference value and the quantity of pixels included in the reference picture;

[0038] When the pixel quantity difference ratio exceeds a pixel quantity difference ratio threshold, the group transition direction is determined.

[0039] In one embodiment, before performing optical flow estimation on a group of moving objects according to the multiple frames, the method includes:

[0040] In case that the motion trajectories of the plurality of moving objects are detected according to the plurality of frames, it is determined that the motion trajectories of the plurality of moving objects satisfy a small number of trajectories condition; and / or it is detected that the plurality of frames satisfy a reference object lack condition.

[0041] In one embodiment, the step of capturing images of the target sphere according to the group transition information includes:

[0042] Determining whether the group transition information satisfies the image acquisition condition of the target sphere;

[0043] If the conditions are met, image collection is performed according to the group transition information to obtain an image of the target sphere.

[0044] In one embodiment, after determining whether the group transition information satisfies the image acquisition condition of the target sphere, the method further includes:

[0045] If not, the ball transition information is determined according to the ball motion trajectory of the target ball in the plurality of frames;

[0046] According to the sphere transition information, the image of the target sphere is captured.

[0047] In one embodiment, determining the ball transition information according to the ball motion trajectory of the target ball in the multiple frames includes:

[0048] Identify the sphere motion trajectory of the target sphere according to the multiple frames;

[0049] In the spherical reference images of the multiple frames, determining a target image area and an image critical position;

[0050] If the ball motion trajectory passes through the target screen area and there is a screen critical position intersecting with the ball motion trajectory, the ball transition direction is determined according to the screen critical position intersecting with the ball motion trajectory.

[0051] In one embodiment, the step of identifying the sphere motion trajectory of the target sphere according to the multiple frames includes:

[0052] Identify the moving sphere according to the trajectory of the sphere area in multiple frames;

[0053] Determine a target ball from each of the moving balls according to the confidence that each of the moving balls belongs to a preset ball type;

[0054] The spherical motion trajectory of the target sphere is determined from the spherical region trajectories in the multiple frames.

[0055] In one embodiment, the determining whether the group transition information satisfies the image acquisition condition of the target sphere includes:

[0056] Determining whether the ball trajectory belongs to a long-trajectory ball motion scene;

[0057] If yes, the group transition information does not meet the image acquisition condition of the target sphere;

[0058] If not, the group transition information satisfies the image acquisition condition of the target sphere.

[0059] In one embodiment, the step of capturing images of the target sphere according to the group transition information includes:

[0060] If the group transition information satisfies the image acquisition condition of the target sphere, the pan / tilt rotation speed is adjusted according to the movement speed in the group movement information to obtain an adjusted pan / tilt rotation speed;

[0061] If the group transition information does not meet the image acquisition condition of the target sphere, the pan / tilt rotation speed is adjusted according to the movement speed of the target sphere to obtain an adjusted pan / tilt rotation speed;

[0062] In the process of controlling the rotation of the pan-tilt head according to the adjusted pan-tilt speed, the image of the target sphere is collected through the pan-tilt head.

[0063] In one of the embodiments, the process of capturing the image of the target sphere is performed through a pan-tilt head, and the pan-tilt head is provided with a camera movement button.

[0064] In a second aspect, the present application also provides a device for collecting images of a target sphere. The device comprises:

[0065] A trajectory detection module is used to detect the motion trajectory of each moving object in a group of moving objects based on multiple frames;

[0066] A group detection module, used to determine group motion information of a group of moving objects according to the motion trajectory of each of the moving objects;

[0067] A transition detection module, used to determine group transition information according to the transition situation that the group movement information conforms to;

[0068] The image acquisition module is used to acquire images of the target sphere according to the group transition information.

[0069] In a third aspect, the present application also provides a handheld gimbal, comprising a motor and a processor, wherein the motor is used to control the rotation of the gimbal, and the processor implements the steps of capturing the image of the target sphere in any of the above embodiments when executing the computer program.

[0070] In a fourth aspect, the present application also provides a handheld gimbal, which includes a motor, and the motor is used to control the rotation of the gimbal. The terminal processor installed on the handheld gimbal is used to implement the steps of capturing the image of the target sphere in any of the above embodiments when executing the computer program.

[0071] In a fifth aspect, the present application further provides a computer device, wherein the computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the step of capturing the image of the target sphere in any of the above embodiments is implemented.

[0072] In a sixth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of capturing the image of the target sphere in any of the above embodiments.

[0073] In a seventh aspect, the present application further provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the step of capturing the image of the target sphere in any of the above embodiments is implemented.

[0074] The above-mentioned target sphere image acquisition method, device, computer equipment, storage medium and computer program product detect the motion trajectory of each moving object in the moving object group based on multiple frames of images, so that the multi-target detection method can be directly applied to the motion trajectory detection of each moving object, and the recognition speed and accuracy are guaranteed by a relatively mature detection method; on this basis, the group motion information of the moving object group is determined according to the motion trajectory of each moving object, and the group motion information is obtained by further performing data statistics through the motion trajectory of each moving object; and the group motion information can reflect the motion trend of the target sphere, and will not affect the accuracy due to occlusion. Therefore, according to the transition situation that the group motion information conforms to, when the target sphere is frequently occluded, the group transition direction can be accurately and quickly determined to better ensure the stability of image acquisition. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 A diagram showing an application environment of a method for capturing a target sphere image in an embodiment;

[0076] Figure 2 It is a schematic flow chart of a method for capturing images of a target sphere in one embodiment;

[0077] Figure 3 A schematic diagram of a transition process in an embodiment;

[0078] Figure 4 is an application environment diagram of a transition process in an embodiment;

[0079] Figure 5 is an application environment diagram of a transition process in another embodiment;

[0080] Figure 6 A schematic diagram of a transition process in another embodiment;

[0081] Figure 7 A schematic diagram of a pan / tilt during a transition process in yet another embodiment;

[0082] Figure 8 is a structural block diagram of a target sphere image acquisition device in one embodiment;

[0083] Fig. 9 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0085] The target sphere image acquisition method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 can be, but is not limited to, various cameras, video cameras, panoramic cameras, sports cameras, personal computers, laptops, smart phones, tablet computers, gimbal bodies and portable wearable devices, and the portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The terminal 102 can be fixed to the gimbal body by welding or the like, and can also be detachably connected or rotatably connected to the gimbal body. Optionally, the gimbal body can be a handheld gimbal; the handheld gimbal can be configured or equipped with a gimbal bracket, and the gimbal bracket can be a tripod or a monopod.

[0086] In one embodiment, Figure 2 As shown, a method for collecting images of a target sphere is provided, and the method is applied to Figure 1 The terminal 102 in the example is used as an example to illustrate, and the following steps are included:

[0087] Step 202: Detect the motion trajectory of each moving object in the moving object group according to multiple frames.

[0088] The multi-frame pictures are pictures acquired by the terminal at different times; optionally, pictures acquired in sequence at consecutive moments can be taken as the multi-frame pictures; pictures acquired in sequence at a certain time interval can also be taken as the multi-frame pictures; pictures collected in real time at different moments can also be taken as the multi-frame pictures.

[0089] The motion object group is a group of objects that can control the target ball to move; the motion object group includes multiple motion objects. The motion object group can control the target ball to move by performing certain actions, or enable the target ball to trigger corresponding ball events.

[0090] Optionally, the moving objects can be screened based on the motion information of multiple moving objects, and the screened moving objects are determined as the moving objects in the moving object group. Optionally, the moving objects contained in the moving object group are variable, and the motion trajectory of the moving objects is formed by the known positions of the moving objects in different pictures, and then the unknown positions are determined by the motion trajectory of the moving objects. Among them, the moving objects that exist in some frames have known positions in these frames, while the moving objects do not exist in other frames. In this case, the positions of the moving objects in the above-mentioned other frames are unknown positions.

[0091] The motion trajectory of each moving object is a position sequence of each moving object arranged in time sequence; the position sequence can be a coordinate sequence or a picture region sequence, the picture region sequence is a picture region arranged in time sequence, and each picture region is used to represent the position of each moving object in the reference picture at a certain moment. Among them, the position sequence of each moving object can be an ID sequence of each character.

[0092] In an optional embodiment, the motion trajectory of each moving object in the moving object group is detected based on multiple frames of images, including: using a spherical detector to identify people in the picture frame by frame; using a target detection algorithm, the detector scheme includes but is not limited to YOLO (You Only Look Once) and SSD (Single Shot MultiBox); using a multi-target tracking algorithm to assign a tracking ID to each person, analyzing the motion trajectory of each ID, filtering out stationary balls on the sidelines and people with smaller movements, and obtaining the motion trajectory of each moving object in the moving object group.

[0093] Step 204: determine group motion information of the group of moving objects according to the motion trajectory of each moving object.

[0094] The group motion information is motion information generated based on the motion information of the moving objects in the moving object group; the group motion information is used to characterize the motion trend of the moving object group; the moving objects in the moving object group refer to multiple moving objects belonging to the moving object group. Optionally, the group motion information is generated for the moving objects in the moving object group; wherein the group motion information may include, but is not limited to, the statistical results of the position change information of the moving objects, and the group position change information determined based on the position change information of the moving objects. Specifically, the above statistical results may be obtained by performing position change information statistics on moving objects in certain directions; the above group position change information may be obtained by performing averaging and other processing on the position change information of moving objects in certain directions in chronological order.

[0095] In one embodiment, the group motion information of the group of moving objects is determined according to the motion trajectory of each moving object, including: according to the motion trajectory of each moving object, the position change information of each moving object in each transition direction is counted to obtain the moving object information in each transition direction; and the group motion information represented by the moving object information in each transition direction is determined. Thus, the position change information of the moving object is first determined, and then the position change information is counted to obtain the moving object information in each transition direction, so that the group motion information can be more accurately calculated at the individual position angle.

[0096] In another embodiment, the group motion information of the group of moving objects is determined according to the motion trajectory of each moving object, including: determining the position of each moving object at different times in the sequence of picture areas of each moving object arranged in time order; averaging the position of each moving object at each time to obtain the group position information at each time; determining the group position information at each continuous time according to the position change of the group position information at each time; accumulating the group position information at each continuous time to obtain the group motion information of the group of moving objects. Thus, the group position information is first determined, and then the group position information at each continuous time is determined. By accumulating the group position information at each continuous time, the group motion information of the group of moving objects is obtained, thereby realizing accurate calculation of the group motion information from the group position perspective. Optionally, the analysis methods for the two perspectives of individual position and group position have their own limitations, but both can guarantee recognition speed and accuracy.

[0097] Step 206, determining group transition information according to the transition situation that the group movement information matches.

[0098] The group transition information is a transition result determined based on the group movement information. The group transition information belongs to one of the transition category and the non-transition category. The transition category is used to indicate that the picture acquisition process requires a transition; the non-transition category is used to determine that the picture acquisition process does not require a transition; the group transition information of the non-transition category can be non-existent data or represented by some prompt information. Optionally, when the group movement information meets the transition situation, the group transition direction, group transition speed and other information can be determined according to the transition situation that the group movement information meets; when the group movement information does not meet the transition situation, the group movement information is not used to control the picture acquisition.

[0099] In an optional embodiment, group transition information is determined based on a transition situation that the group motion information conforms to, including: judging whether the group motion information of the moving object group in multiple frames triggers a transition event; if so, characterizing the transition situation that the group motion information conforms to according to the transition event, and determining the group transition information of the transition category through the transition situation; if not, obtaining the group motion information of the non-transition category.

[0100] Optionally, the group transition information can be represented by a transition signal; the transition signal can be used only to represent that the picture acquisition process requires a transition, or can be used to represent that the picture acquisition process requires a transition and the corresponding group transition direction. Optionally, the transition signal can be sent by the terminal to the pan-tilt body, or by the processor of the pan-tilt body when controlling the movement of its own motor.

[0101] In an optional implementation, the group transition information is determined based on the transition conditions that the group movement information conforms to, including: determining the group transition information based on individual position statistics that the group movement information conforms to; and / or determining the group transition information based on group displacement changes that the group movement information conforms to.

[0102] Step 208, capturing images of the target sphere according to the group transition information.

[0103] The target ball is a ball in motion. Optionally, the target ball is a ball in a certain sport, which belongs to a certain preset ball type; illustratively, the target ball belongs to basketball.

[0104] In one embodiment, the target sphere is captured based on the group transition information, including: if the group transition information is used to determine the transition, the image is captured in a transition direction opposite to the previous transition direction.

[0105] In another embodiment, images of the target sphere are captured based on the group transition information, including: controlling the rotation of the pan-tilt head according to the transition direction contained in the group transition information; capturing images of the target sphere during the rotation of the pan-tilt head; wherein the pan-tilt head rotation may be a preset value or may be dynamically changed.

[0106] In an optional implementation, the pan / tilt head is controlled to rotate according to the transition direction contained in the group transition information, including: if the transition direction contained in the group transition information is a first transition direction, the pan / tilt head is controlled to rotate in the first transition direction; if the transition direction contained in the group transition information is a second transition direction, the pan / tilt head is controlled to rotate in the second transition direction.

[0107] The process of capturing the image of the target sphere is carried out through the pan-tilt, which is provided with a camera movement button, which can be a mechanical button on the pan-tilt, a virtual button on the mobile phone screen, or a virtual button on the pan-tilt screen.

[0108] Since the process of collecting images of the target sphere is carried out through the gimbal, the gimbal is equipped with a camera movement button, which can be used for manual intervention in scenarios such as stealing that cause the transition signal to fail.

[0109] In this embodiment, the motion trajectory of each moving object in the moving object group is detected based on multiple frames of images, so that the multi-target detection method can be directly applied to the motion trajectory detection of each moving object, and the recognition speed and accuracy are guaranteed by a relatively mature detection method; on this basis, the group motion information of the moving object group is determined according to the motion trajectory of each moving object, and the group motion information is obtained by further performing data statistics through the motion trajectory of each moving object; and the group motion information can reflect the movement trend of the target sphere, and will not affect the accuracy due to occlusion. Therefore, according to the transition situation that the group motion information conforms to, when the target sphere is frequently occluded, the group transition direction can be accurately and quickly determined to better ensure the stability of picture acquisition.

[0110] In one embodiment, the analysis method for the angle of individual position is described in detail. The group motion information of the group of moving objects is determined according to the motion trajectory of each moving object, including: determining the target picture area and different transition critical positions in the reference picture of multiple frames; in the motion trajectory of each moving object passing through the target picture area, the motion trajectory of the moving object intersecting at different transition critical positions is respectively counted to obtain the number of moving objects in different transition directions.

[0111] Correspondingly, the group transition information is determined according to the transition situation that the group motion information conforms to, including: determining the group transition direction according to the transition interval to which the number of motion objects in different transition directions belongs.

[0112] The reference picture is a picture of a group of moving objects, which is the picture where the motion trajectory of each moving object is located. Optionally, the picture area sequence of each moving object can be arranged in chronological order to form the motion trajectory of each moving object; and the picture where the motion trajectory of each moving object is located is the reference picture. Optionally, the reference picture can be a frame picture of multiple frames, or a picture generated based on multiple frames; illustratively, the reference picture can be a picture within a certain time period, or a picture obtained by averaging the pixel positions of multiple frames.

[0113] The transition critical position is the critical position for transition judgment in the reference picture. Optionally, the transition critical position is a critical line in the direction of a certain side of the reference picture and perpendicular to the direction of this side; optionally, the transition critical position is a critical line in the width direction of the reference picture and perpendicular to the width direction of the reference picture; optionally, the picture width of the reference picture can be calculated according to the ratio of the transition critical position to obtain the transition critical line in the picture width direction. Exemplarily, the picture width of the reference picture is Width, the picture width is the positive direction of the pixel x-axis, the pixel x-axis coordinate interval is (0~Width), and 0.2*Width and 0.8*Width are set as the critical lines where the picture critical position is located. Optionally, the transition critical position is defined according to a certain position on the court.

[0114] The target picture area is the area where the target position of the reference picture is located; optionally, the target picture area can be a picture area within a certain range of the target position, and this range can be a fixed value or can change dynamically based on picture attributes such as picture width and number of pixels.

[0115] The transition direction is at least one expected conversion direction of the image acquisition process; optionally, the transition direction is determined based on at least one set of opposite directions, and the image acquisition process can be controlled to be reversible through a set of opposite directions. Optionally, each transition direction corresponds to at least one field critical position, and the side length direction of the reference picture that is closest to the transition critical position and does not intersect is the transition direction; for example: a certain transition critical position is set along the width direction of the reference picture, and is perpendicular to the width direction of the reference picture, and is closest to the left direction of the reference picture, then the field critical position corresponding to the field critical position is the left direction; a certain transition critical position is set along the width direction of the reference picture, and is perpendicular to the width direction of the reference picture, and is close to the right direction of the reference picture, then the field critical position corresponding to the field critical position is the right direction.

[0116] The transition interval is a number interval set for the number of moving objects in different transition directions. When the number of moving objects in different transition directions respectively belongs to the transition interval, it can be determined that the group transition information is of the transition type, and the group transition direction can be generated.

[0117] In this embodiment, in the reference picture of multiple frames, the target picture area and different transition critical positions are determined, and the group movement change trend of the group movement objects with different transition directions can be reflected through the different transition critical positions; in the movement trajectories of the movement objects that pass through the target picture area, the movement trajectories of the movement objects that intersect at different transition critical positions are respectively counted, and the number of movement objects in different transition directions is obtained to determine the group movement information from the statistical perspective of individual positions; on this basis, the group transition direction is determined according to the transition interval to which the number of movement objects in different transition directions belongs, and it can be determined that the picture acquisition process needs to be transitioned, and the group transition information is clarified. Therefore, the group movement information can be more accurately calculated from the perspective of individual positions to ensure the recognition accuracy of the group transition information.

[0118] The critical value of the transition interval includes a first critical value and a second critical value, the first critical value is greater than the second critical value; the first critical value and the second critical value are respectively used to limit the number of moving objects in the transition direction, and the first critical value and the second critical value correspond to the opposite trend of the movement of the group of objects. Optionally, the first critical value is 4, 5, or 6 moving objects, and the second critical value is 1 or 2 moving objects.

[0119] In a feasible implementation, among the motion trajectories of the moving objects passing through the target picture area, the motion trajectories of the moving objects intersecting at different transition critical positions are respectively counted to obtain the number of moving objects in different transition directions, including: among the motion object trajectories passing through the target picture area, the motion object trajectories intersecting at a first transition critical position are counted to obtain the number of moving objects in the first transition direction; among the motion object trajectories passing through the target picture area, the motion object trajectories intersecting at a second transition critical position are counted to obtain the number of moving objects in the second transition direction.

[0120] In a feasible implementation, the group transition direction is determined according to the transition situation that the group motion information conforms to, including: if the number of moving objects in the first transition direction is greater than a first critical value, and the number of moving objects in the second transition direction is less than a second critical value, then the group transition direction is determined to be the first transition direction; if the number of moving objects in the first transition direction is less than the second critical value, and the number of moving objects in the second transition direction is greater than the first critical value, then the group transition direction is determined to be the second transition direction; wherein the first transition direction and the second transition direction are opposite directions.

[0121] The first transition direction and the second transition direction, the first transition direction and the second transition direction are a set of opposite directions; the first transition direction is used to characterize that the transition direction of the picture acquisition is the first transition direction, and it is necessary to control the picture to rotate in the first transition direction; the second transition direction is used to characterize that the transition direction of the picture acquisition is in the second transition direction, and it is necessary to control the picture to rotate to the second transition direction; exemplarily, the first transition direction is the left transition direction, and the second transition direction is the right transition direction.

[0122] Specifically, if the number of moving objects in the first transition direction is greater than the first critical value, and the number of moving objects in the second transition direction is less than the second critical value, it means that the majority of moving objects in the moving object group move toward the first transition direction, and a minority of moving objects in the moving object group move toward the second transition direction, and thus the group transition direction is determined to be the first transition direction. If the number of moving objects in the first transition direction is less than the second critical value, and the number of moving objects in the second transition direction is greater than the first critical value, it means that a minority of moving objects in the moving object group move toward the first transition direction, and a majority of moving objects in the moving object group move toward the second transition direction, and thus the group transition direction is determined to be the second transition direction.

[0123] In an exemplary embodiment, the group motion information of the group of moving objects is determined according to the motion trajectory of each moving object, including: the picture width of the reference picture is Width, the picture width is the positive direction of the pixel x-axis, the interval of the pixel x-axis coordinate is (0~Width), and 0.2*Width and 0.8*Width are set as the critical lines where the critical position of the picture is located. The motion trajectory of the moving object in the historical time period (such as 2 seconds) is analyzed. When the motion trajectory of the moving object passes through the critical line 0.2*Width from the center area of ​​the picture, it is considered that the transition direction of the moving object at the current moment is the left direction; when the motion trajectory of the moving object passes through the critical line 0.8*Width from the center area of ​​the picture, it is considered that the spherical transition direction of the moving object at the current moment is the right direction.

[0124] Correspondingly, the group transition direction is determined according to the transition interval to which the number of moving objects in different transition directions belongs, including: if the number of moving object IDs transitioning in the first transition direction exceeds the first critical value, and the number of moving object IDs in the second transition direction is lower than the second critical value, it is considered that the moving object group has a transition trend, that is, the transition event is recognized, and the algorithm outputs a transition signal.

[0125] In one embodiment, Figure 3As shown in Figure (a), the number of moving objects in the left transition direction is one, and the number of moving objects in the right transition direction is three. In this case, if the first critical value is three and the second critical value is two, the group transition direction belongs to the transition type, and the group transition direction is that the gimbal transitions to the right; if the first critical value is four, or the second critical value is one, the group transition direction belongs to the non-transition type.

[0126] In another embodiment, Figure 3 As shown in Figure (b), the number of moving objects in the left transition direction is three, and the number of moving objects in the right transition direction is one; in this case, if the first critical value is three and the second critical value is two, the group transition direction belongs to the transition type, and the PTZ transitions to its left side; if the first critical value is four, or the second critical value is one, the group transition direction belongs to the non-transition type. For example, Figure 3 The application scenario of the picture (b) during shooting is as follows Figure 4 shown.

[0127] Alternatively, if Figure 5 As shown, the terminal in this application scenario is set close to the midfield line of the court. At this time, in the picture taken by the terminal, no matter how the camera's field of view changes, the number of moving objects in the left transition direction is less than or equal to two, and the number of moving objects in the right transition direction is less than or equal to two. In this case, if the first critical value is three and the second critical value is two, the group transition direction belongs to the non-transition type and no transition is required.

[0128] Therefore, among the number of moving objects in the first transition direction and the number of moving objects in the second transition direction, the maximum value of the two needs to be compared by the first critical value, and the minimum value of the two needs to be compared by the second critical value. By comparing the number of moving objects in these two transition directions at the same time, misjudgment can be avoided and a higher recognition accuracy can be ensured.

[0129] In another embodiment, the analysis method for the group position is described in detail. The group motion information of the group of moving objects is determined according to the motion trajectory of each moving object, including: determining the individual position of each moving object at the same time according to the motion trajectory of each moving object at different times; generating the group position at the same time according to the individual position of each object at the same time; generating at least two groups of group displacements at adjacent times according to at least two groups of group positions at adjacent times.

[0130] Correspondingly, the group transition direction is determined according to the transition situation that the group motion information conforms to, including: if the group displacements of at least two groups of moments have the same transition direction, and in the same transition direction, the group displacements of at least two groups of moments respectively exceed the transition length thresholds corresponding to the group displacements, then it is determined that the group transition information includes the same transition direction.

[0131] The individual object position is the position of each moving object; it can be determined based on the position area where each moving object is located, or it can be determined by the coordinate position in the reference picture. The individual object position is the detection result of the target detection. Optionally, the individual object position is the center point of the tracking frame (person detection frame) stored in the tracking sequence.

[0132] The transition length threshold is a threshold used to judge the transition situation; when the group displacements of at least two groups of moments exceed the transition length thresholds corresponding to the group displacements, a transition is required; when the group displacements of only one group of moments exceed the transition length thresholds corresponding to the group displacements, no transition is required.

[0133] The group position is a position obtained by statistics based on the individual positions of the objects, and the group displacement is a displacement generated based on the group position; optionally, the individual positions of the objects and the group position are both coordinate positions, and the group displacement can be a difference vector or difference matrix of the group position at adjacent moments. Optionally, the group displacement can be a two-dimensional displacement, in which case the group displacement is a displacement along the x-axis.

[0134] Adjacent moments may be adjacent in terms of timestamps, or may be moments separated by a preset time period, or may be continuous moments. A group of adjacent moments includes two moments, and at least two groups of adjacent moments include at least three adjacent moments.

[0135] In a feasible implementation, the individual object positions of each moving object at the same time are determined according to the movement trajectories of each moving object at different times, including: in the movement trajectories of each moving object at different times, selecting the individual object positions with the same timestamp according to each moving object.

[0136] In a feasible implementation, the group position at the same moment is generated based on the individual positions of the objects at the same moment, including: averaging the individual positions of the objects at the same moment to obtain the individual position averaging result; and generating the group position at the same moment based on the individual position averaging result.

[0137] Optionally, the individual position averaging result can be used as the group position at the same moment. Optionally, the group position at the same moment is generated according to the individual position averaging result, including: at each moment, according to the distance between the individual position averaging result and the individual position of each object, the individual position of the object whose distance is outside the screening range is filtered to obtain the filtered individual position of the object; the filtered individual position of the object is averaged to obtain the group position at the same moment; wherein the screening range can be a preset value or a value obtained in real time. In this way, the individual position of the object that is too far away from the individual position averaging result is filtered out, so that the accuracy of the group position is higher.

[0138] In one embodiment, generating at least two groups of group displacements at adjacent moments according to group positions at at least two groups of adjacent moments includes: determining at least two groups of group positions at adjacent moments in time order among the group positions at the same moment; performing difference calculation based on the group positions at at least two groups of adjacent moments to obtain at least two groups of group displacements at adjacent moments; wherein the group displacements at each group of adjacent moments correspond to the group positions at at least one group of adjacent moments. Exemplarily, the group positions at each group of adjacent moments may be two coordinate positions, and the group position is a vector calculated based on the two coordinate positions.

[0139] In a feasible implementation, if at least two groups of group displacements at adjacent moments have the same transition direction, and in the same transition direction, at least two groups of group displacements at adjacent moments respectively exceed the transition length threshold corresponding to the group displacement, then it is determined that the group transition information includes the same transition direction. The judgment process includes: if it is determined that the transition directions of at least two groups of group displacements at adjacent moments are the same, and in the above-mentioned same transition direction, the lengths of the group displacements at at least two groups of adjacent moments are both greater than the transition length threshold, then it is determined that the group transition information includes the above-mentioned same transition direction.

[0140] In an exemplary embodiment, the moving object group is a crowd, and the moving object is each athlete in the moving object group; according to the moving trajectory corresponding to each moving object ID, the center points of the tracking frames (person detection frames) stored in all tracking sequences at the moment are collected, and the average x-coordinate value ave_x of the center points is calculated, and the center points far from the average ave_x are filtered out, and the filtered center points are used to recalculate the average x-coordinate value at time t0, which is recorded as ave_t0. According to the calculation method at time t0, the average x-coordinate values ​​at time t1 and time t2 are calculated and recorded as ave_t1 and ave_t2 respectively. Finally, the x-coordinate offset between time t0 and time t1 is calculated and recorded as x 01 , calculate the x coordinate offset between time t1 and time t2 and record it as x 12 , if x 01 and x 12The offset direction is consistent and the total offset exceeds the transition length threshold D thr When the group of people is considered to have a continuous overall deviation in the same direction from time t0 to t2, it is considered that a transition event occurs at this moment, and a transition signal can be output. The group transition information represented by the transition signal indicates the group transition direction.

[0141] In this embodiment, the motion trajectory of each moving object at different moments is the result of multi-target detection, and according to the motion trajectory of each moving object at different moments, the individual position of each moving object at the same moment is determined, thereby realizing the refinement of the result of multi-target detection; and according to the individual position of each object at the same moment, the group position at the same moment is generated, so that the detection process of the individual position is changed to the detection process of the group position; according to at least two groups of group positions at adjacent moments, at least two groups of group displacements at adjacent moments are generated; and the group displacements at at least two groups of moments have the same transition direction to reflect the transition plan, and the group displacements at at least two groups of moments respectively exceed the transition length threshold, thereby realizing the process of re-verification, so that the group transition information matches the transition occurrence process with higher accuracy, thereby instructing the gimbal to transition in the transition direction.

[0142] Among them, when the group motion information includes the number of moving objects in different transition directions and does not include at least two groups of group displacements at adjacent moments, the transition interval used to judge the transition situation is the first transition interval; when the group motion information does not include the number of moving objects in different transition directions and includes at least two groups of group displacements at adjacent moments, the transition length threshold used to judge the transition situation is the first transition length threshold; when the group motion information includes the number of moving objects in different transition directions and includes at least two groups of group displacements at adjacent moments, the transition interval is the second transition interval, and the transition length threshold is the second transition length threshold; wherein the first transition interval covers the second transition interval, and the range of the first transition interval is smaller than the range of the second transition interval, and the first transition length threshold is greater than the second transition length threshold.

[0143] In the case where the group motion information includes the number of moving objects in different transition directions and does not include at least two groups of group displacements at adjacent moments, in this case, the terminal uses the detection of individual positions alone. Exemplarily, the first critical value of the first transition interval is 5, and the second critical value is 1. Then, when the terminal uses the detection of individual positions alone, when more than 5 objects move to the right and less than 1 object moves to the left, a right transition signal is output.

[0144] In the case where the group motion information does not include the number of moving objects in different transition directions, and includes at least two groups of group displacements at adjacent moments, in this case, the terminal uses the detection of the group position alone. Exemplarily, when the transition direction of at least two groups of group displacements at adjacent moments is the right direction, and the total amount of the offset of the group displacements at at least two groups of adjacent moments exceeds the first transition length threshold 0.1*img_height, a transition signal to the right direction is output; wherein img_height is the reference image height.

[0145] In the case where the group motion information includes the number of moving objects in different transition directions and includes at least two groups of group displacements at adjacent moments, in this case, the terminal uses detection of individual positions and group positions at the same time.

[0146] Specifically, when the group motion information includes the number of moving objects in different transition directions and includes at least two groups of group displacements at adjacent moments, the transition interval is the second transition interval, and the transition length threshold is the second transition length threshold; while the first transition interval covers the second transition interval, and the range of the first transition interval is smaller than the range of the second transition interval, and the first transition length threshold is greater than the second transition length threshold. In this case, the detection results of the individual position and the group position are complementary, and belong to the algorithm coupling use, which can further improve the recognition speed and accuracy of the transition algorithm; specifically, on the one hand, the first transition interval covers the second transition interval, and the range of the first transition interval is smaller than the range of the second transition interval, so that the range of the transition interval is relaxed, and when the number of moving objects in different transition directions is small, it can still be accurately judged; on the other hand, the first transition length threshold is greater than the second transition length threshold, so that when the group displacements of at least two groups of moments are small, it can still be accurately judged.

[0147] For example, the first critical value of the first transition interval is 5, and the second critical value is 1; the first critical value of the second transition interval is 3, and the second critical value of the second transition interval can remain unchanged; and in the case of the first transition length threshold of 0.1*img_height, the first transition length threshold is relaxed to 0.05*img_height. At this time, the judgment logic becomes: when the number of rightward movement is greater than or equal to 3, the number of leftward movement is less than or equal to 1, and the total amount of offset exceeds 0.05*img_height, a rightward transition signal is output.

[0148] Thus, group movement information in three cases is formed, and each of them can be used to obtain group transition information. The terminal can apply any of the three cases separately, and when used in conjunction, its recognition precision and accuracy are better.

[0149] In one embodiment, group transition information is determined according to the transition situation that the group motion information meets, including: when the group motion information does not meet the group transition information generation conditions, optical flow estimation is performed on the moving object group according to multiple frames to obtain the optical flow displacement of each frame; the optical flow displacement of each frame is accumulated to obtain a picture motion feature map corresponding to the multiple frames; the picture motion feature map includes multiple feature map pixels, and the direction of each feature map pixel is determined by the accumulation result of the optical flow displacement of each frame; according to the direction of each feature map pixel, the number of pixels of the feature map pixels displaced along the opposite transition direction are counted respectively to obtain the number of pixels in the opposite transition direction; the comparison result between the number of pixels in the opposite transition direction is determined; the comparison result belongs to the group motion information.

[0150] The group transition information generation condition is a condition set for the correspondence between the group motion information and the group transition information. The group transition information generation condition can be set based on the number of motion tracks of the motion object, or whether a reference object exists.

[0151] The optical flow displacement is the result of optical flow estimation based on multiple frames, and can be used to characterize the motion trend of a group of moving objects. Optionally, in multiple frames, the optical flow displacement can be obtained based on the brightness changes of each adjacent frame. Optionally, the optical flow estimation can be performed along a certain transition direction and the opposite transition direction of this transition direction to obtain the optical flow displacement of each frame.

[0152] The picture motion feature map is the result of accumulating the optical flow displacement of multiple frames in a certain interval. Optionally, the optical flow displacement can be accumulated based on multiple frames in a certain number interval to obtain the picture motion feature map; the optical flow displacement can be accumulated for multiple frames in a certain time interval to obtain the picture motion feature map. Optionally, each pixel in the picture motion feature map is obtained by optical flow estimation based on the brightness value of each pixel, or by optical flow estimation based on each pixel and its neighboring pixels.

[0153] The feature map pixel is a pixel in the picture motion feature map, and each feature map pixel is used to characterize the optical flow estimation result within a certain interval. The direction of each feature map pixel is the optical flow direction; optionally, if the optical flow displacement is set along the transition direction, the direction of the feature map pixel is set along a certain transition direction. The opposite transition direction is at least one pair of opposite transition directions.

[0154] The comparison result between the numbers of pixels in opposite transition directions can be determined based on the ratio of the numbers of pixels in opposite transition directions, or based on the difference of the numbers of pixels in opposite transition directions, and the difference of the number of pixels has higher accuracy. For example, in a 1080p reference picture, if there are 1000 pixels to the right and 1 pixel to the left, the ratio will reach 1000. The difference in the number of pixels can represent how many pixels have changed, the ratio of the number of pixels cannot reflect the degree of such change, and the difference in the number of pixels can reflect the corresponding degree of change.

[0155] In one embodiment, the optical flow of the moving object group is estimated based on multiple frames to obtain the optical flow displacement of each frame, including: in a certain number interval of frames, the optical flow between adjacent frames is estimated, and the optical flow displacement of each frame in this number interval is obtained. Exemplarily, for the Kth frame and the K+1th frame, the optical flow between the Kth frame and the K+1th frame is calculated to obtain the displacement of the moving object group in the x direction in the K+1th frame. By analogy, the optical flow results of N frames are formed for analyzing the optical flow displacement in the N frames, where N is a preset value, which can be 10, 30, etc.

[0156] In one embodiment, the optical flow displacement of each frame is accumulated to obtain a picture motion feature map corresponding to the multiple frames, including: for each pixel of the picture, the optical flow displacement of each picture within a certain number interval is accumulated to obtain a picture motion feature map corresponding to the multiple frames.

[0157] In one embodiment, when the comparison result is group motion information; determining the group transition information according to the transition situation that the group motion information meets includes: determining the group transition information according to the transition situation that the comparison result meets. Optionally, when the comparison result is greater than a certain threshold, it can be determined to perform a transition; when the comparison result is less than a certain threshold, no transition can be performed.

[0158] Exemplarily, the optical flow displacement of N frames is accumulated to obtain the motion feature map of the optical flow in the x direction; the optical flow threshold is set to Thr, and the motion amount (the length of the optical flow displacement) less than the threshold Thr is set to 0. The difference between the number of pixels moving to the left and the number of pixels moving to the right is calculated. When the difference exceeds a certain proportion of the total number of pixels in the picture, it is considered that the target in the picture has moved as a whole in a certain direction, that is, the occurrence of the attack and defense conversion event is recognized, and the algorithm outputs a transition signal.

[0159] Therefore, through the optical flow calculation method, without relying on the detection algorithm of the moving object group, the corresponding group transition information can still be determined relatively accurately to ensure the recognition accuracy. In this process, the moving object group is a significant target in the movement process. Through the group motion information of the moving object group, it can accurately reflect whether a transition is needed, and it will not be affected by the occlusion relationship; thus, it can be more accurately judged whether a transition is needed, so that the process of collecting pictures is more stable.

[0160] In one embodiment, the conditions for generating group transition information are limited. Before performing optical flow estimation on a group of moving objects based on multiple frames, the method includes: determining that the motion trajectories of multiple moving objects meet a small number of trajectories condition when detecting motion trajectories of multiple moving objects based on multiple frames; and / or detecting that multiple frames meet a reference object lack condition.

[0161] The small number of trajectory condition is a trajectory number standard, which is used to characterize that there are few motion trajectories for multiple moving objects in multiple frames. The reference object lack condition is a detection condition for a certain reference object; optionally, the reference object is designed for a sports scene; illustratively, in a basketball scene, the reference object can be a basket or a backboard.

[0162] In an optional implementation, determining that the motion trajectories of multiple moving objects satisfy a small number of trajectories condition includes: determining that the number of motion trajectories of the multiple moving objects is less than a preset small number of trajectories threshold of the terminal; or determining that the number of motion trajectories of the multiple moving objects is less than a value input before the picture is captured.

[0163] In an optional embodiment, detecting that multiple frames satisfy a reference object lack condition includes: if a preset number of frames among the multiple frames are detected and multiple frames are not detected, then the multiple frames satisfy the reference object lack condition; if features of the reference object are not extracted based on the multiple frames, then the multiple frames satisfy the reference object lack condition.

[0164] Therefore, through the conditions of a small number of trajectories and the lack of reference objects, scenarios such as the gimbal being blocked by a close distance and the backboard being blocked by a person are set. In these scenarios, the group motion trend can be calculated more accurately through optical flow estimation.

[0165] In one embodiment, group transition information is determined based on a transition situation that group motion information conforms to, including: when the group motion information does not meet the group transition information generation conditions, a pixel number difference is determined based on the difference between the numbers of pixels in opposite transition directions; the number of pixels in the opposite transition direction is obtained based on multiple frames; the number of pixels in the opposite transition direction belongs to the group motion information; a pixel number difference ratio is obtained based on a ratio between the pixel number difference and the number of pixels contained in a reference picture; when the pixel number difference ratio exceeds a pixel number difference ratio threshold, the group transition direction is determined.

[0166] In one embodiment, based on the difference between the number of pixels in opposite transition directions, the pixel number difference is determined, including: performing optical flow estimation on the moving object group according to multiple frames to obtain the optical flow displacement of each frame; accumulating the optical flow displacement of each frame to obtain the picture motion feature map corresponding to the multiple frames; the picture motion feature map includes multiple feature map pixels, and the direction of each feature map pixel is determined by the accumulated result of the optical flow displacement of each frame; according to the direction of each feature map pixel, the pixel number of the feature map pixels displaced along the opposite transition direction is counted respectively to obtain the number of pixels in the opposite transition direction. Thus, combined with the optical flow scheme, the number of pixels in the opposite transition direction can be accurately obtained.

[0167] In another embodiment, feature points such as corner points and edge points between each frame image are first detected, and then matching is performed in each frame image through the combined information of the feature points to obtain matching points. Finally, the moving direction between the matching points is calculated, and the matching points with different moving directions are counted to obtain the number of pixels in the opposite transition direction.

[0168] In one embodiment, determining the pixel number difference based on the difference between the pixel numbers in opposite transition directions includes: performing difference calculation on the pixel number in the previous transition direction and the pixel number in the current transition direction that is the opposite transition direction to the previous transition direction to obtain the pixel number difference;

[0169] Correspondingly, when the pixel number difference ratio exceeds the pixel number difference ratio threshold, the group transition direction is determined, including: when the pixel number difference ratio exceeds the pixel number difference ratio threshold, the group transition direction is determined as the current transition direction.

[0170] In another embodiment, determining the pixel number difference based on the difference between the pixel numbers in opposite transition directions includes: performing difference calculation on the pixel number in the first transition direction and the pixel number in the second transition direction to obtain the pixel number difference;

[0171] Correspondingly, when the pixel number difference ratio exceeds the pixel number difference ratio threshold, the group transition direction is determined, including: when the pixel number difference ratio exceeds the pixel number difference ratio threshold, the maximum number of pixels between the number of pixels in the first transition direction and the number of pixels in the second transition direction is determined, and the group transition direction is determined according to the transition direction where the maximum number of pixels is located.

[0172] Therefore, based on the ratio between the pixel quantity difference and the number of pixels contained in the reference picture, the pixel quantity difference ratio is obtained, which can be adaptively adjusted according to the different sizes of the pictures taken by the camera, so that the transition judgment is more accurate.

[0173] In one embodiment, capturing images of the target sphere according to the group transition information includes: determining whether the group transition information satisfies image capturing conditions of the target sphere; if so, capturing images according to the group transition information to obtain images of the target sphere.

[0174] The image acquisition condition is an indicator for image acquisition based on group transition information. It can be judged whether the group transition information meets the image acquisition condition of the target sphere based on the group transition information, or it can be judged whether the group transition information meets the image acquisition condition of the target sphere based on the motion scene of the target sphere in multiple frames.

[0175] In one embodiment, the target sphere is captured based on the group transition information, including: if the group transition information is used to determine the transition, the image is captured in a transition direction opposite to the previous transition direction.

[0176] In another embodiment, images of the target sphere are captured based on the group transition information, including: controlling the rotation of the pan-tilt head according to the transition direction contained in the group transition information; capturing images of the target sphere during the rotation of the pan-tilt head; wherein the pan-tilt head rotation may be a preset value or may be dynamically changed.

[0177] In one embodiment, after determining whether the group transition information satisfies the image acquisition condition of the target sphere, the method further includes: if not, determining the sphere transition information according to the sphere motion trajectory of the target sphere in multiple frames; and performing image acquisition on the target sphere based on the sphere transition information.

[0178] In this embodiment, the target sphere is a sphere that can be used to control whether the picture acquisition process switches. If the picture acquisition process is judged to switch based on the position of the target sphere alone, its stability is relatively weak, so the target sphere can be used as a control factor to judge whether a switch is needed. Optionally, the target sphere is a sphere determined from the moving sphere; the moving sphere is a sphere identified from multiple spheres, and the moving sphere is a non-stationary sphere.

[0179] The ball motion trajectory is a trajectory generated based on the position of the target ball in multiple frames. Optionally, the ball motion trajectory can be a ball region trajectory where the target ball is located; or it can be determined based on ball region trajectory mimicry data used to characterize the motion of the target ball; the ball region trajectory mimicry data is used to reflect the ball motion trajectory of the target ball in multiple frames; the ball region trajectory mimicry data can be data processed by a model used to determine the ball motion trajectory.

[0180] The ball transition information is the transition result determined according to the ball motion trajectory. The ball transition information belongs to one of the transition category and non-transition category. The transition category is used to indicate that the picture acquisition process requires a transition; the non-transition category is used to determine that the picture acquisition process does not require a transition; the ball transition information of the non-transition category can be non-existent data, or it can be represented by some prompt information. Optionally, when the ball motion information meets the transition situation, the ball transition direction, ball transition speed and other information can be determined according to the transition situation that the ball motion information meets; when the ball motion information does not meet the transition situation, the ball motion information is not used to control the picture acquisition.

[0181] In an optional embodiment, based on the ball transition information, the target ball is captured, including: determining whether the ball motion trajectory of the target ball in multiple frames triggers a transition event; if so, characterizing the transition event to match the transition situation of the ball motion trajectory, and determining the ball transition information of the transition category through the transition situation; if not, obtaining the ball motion information of the non-transition category.

[0182] Optionally, the spherical transition information can be represented by a transition signal; the transition signal can be used only to represent that the picture acquisition process needs a transition, or it can be used to represent that the picture acquisition process needs a transition and the corresponding group transition direction. Optionally, the transition signal can be sent by the terminal to the pan-tilt body, or it can be sent by the processor of the pan-tilt body when controlling the movement of its own motor. Optionally, the transition signal used to represent the spherical transition information and the transition signal used to represent the group transition information can be the same or different.

[0183] In one embodiment, the target sphere is captured based on the sphere transition information, including: if the sphere transition information is used to determine the transition, the image is captured in a transition direction opposite to the previous transition direction.

[0184] In another embodiment, images of the target sphere are captured based on the sphere transition information, including: controlling the rotation of the pan-tilt head according to the transition direction contained in the sphere transition information; capturing images of the target sphere during the rotation of the pan-tilt head; wherein the pan-tilt head rotation may be a preset value or may be dynamically changed.

[0185] In an optional implementation, the pan-tilt head is controlled to rotate according to the transition direction contained in the spherical transition information, including: if the transition direction contained in the spherical transition information is the first transition direction, the pan-tilt head is controlled to rotate in the first transition direction; if the transition direction contained in the spherical transition information is the second transition direction, the pan-tilt head is controlled to rotate in the second transition direction.

[0186] In this embodiment, when the group transition information does not meet the image acquisition conditions of the target sphere, the sphere transition information is determined according to the sphere motion trajectory of the target sphere in multiple frames, which can avoid the target sphere from blocking and affecting the transition judgment, and accurately reflect whether a transition is needed; based on the sphere motion scene to which the sphere motion trajectory belongs, a transition information is selected from the group transition information and the sphere transition information to more accurately judge whether a transition is needed for different sphere motion scenes, which can avoid data conflicts between the group transition information and the sphere transition information and ensure stable operation.

[0187] In one embodiment, according to the ball motion trajectory of the target ball in multiple frames, the ball transition information is determined, including: identifying the ball motion trajectory of the target ball according to the multiple frames; determining the target picture area and the picture critical position in the ball reference picture of the multiple frames; if the ball motion trajectory passes through the target picture area and there is a picture critical position intersecting with the ball motion trajectory, then determining the ball transition direction according to the picture critical position intersecting with the ball motion trajectory.

[0188] The sphere reference picture is a picture for the target sphere, which is the picture where the sphere motion track of the target sphere is located; the reference picture and the reference picture for the group object can be the same or different reference pictures. The target picture area for the target sphere is the critical area of ​​the change amplitude of the sphere motion track, which is used to determine whether the target sphere is within the reference area, and it can be an area or a different area from the target picture area for the group object.

[0189] The picture critical position is the critical position of the spherical reference picture, and is used to determine the boundary of the spherical reference picture. Optionally, the picture critical position is a critical line in the direction of a certain side of the spherical reference picture and perpendicular to the direction of the side; optionally, the picture critical position is a critical line in the width direction of the spherical reference picture and perpendicular to the width direction of the spherical reference picture; optionally, the picture critical position and the above-mentioned transition critical position can be the same or different positions.

[0190] In one embodiment, the ball motion trajectory of the target ball is identified according to multiple frames, including: identifying the moving ball according to the ball area trajectory in the multiple frames; determining the target ball from each moving ball according to the confidence that each moving ball belongs to a preset ball type; and determining the ball motion trajectory of the target ball from the ball area trajectory in the multiple frames. Thus, by identifying the moving ball through the ball area trajectory formed by the ball area of ​​multiple frames, the movement trend of the ball can be included in the range of ball detection, static balls are excluded, and the recognition accuracy of the ball area in the picture is improved; then, according to the confidence that each moving ball belongs to a preset ball type, the target ball is determined from each moving ball, and the target ball such as a basketball is more accurately identified; in this case, the position information of the target ball is used as a control factor in the picture acquisition process, and the group motion information is assisted by this control factor to collect the picture of the target ball, so that the automatic shooting process of the target ball is more stable.

[0191] Among them, the areas belonging to the same sphere that change sequentially in multiple frames can be correlated between frames to obtain the sphere area trajectory of the sphere; if the sphere area trajectory of the sphere represents that the sphere is moving, it can be identified that the sphere is a moving sphere, and then based on whether the characteristics of the moving sphere are consistent with the characteristics of the preset ball type, the confidence that the moving sphere belongs to the preset ball type is calculated, thereby more accurately identifying target spheres such as basketballs.

[0192] In one embodiment, in a spherical reference picture of multiple frames, determining the target picture area and the picture critical position includes: determining the picture center area in the spherical reference picture of multiple frames, and converting the width of the spherical reference picture according to the picture critical position ratio along the width direction of the spherical reference picture to obtain the picture critical position. The picture critical position ratio can be multiplied by the width of the spherical reference picture to obtain the picture critical position.

[0193] In one embodiment, the sphere transition direction is determined according to the critical position of the picture intersecting the sphere motion trajectory, including: if the critical position of the picture intersecting the sphere motion trajectory is located in the first transition direction of the target picture area, then the sphere transition direction is determined to be the first transition direction; if the critical position of the picture intersecting the sphere motion trajectory is located in the second transition direction of the target picture area, then the sphere transition direction is determined to be the second transition direction.

[0194] Among them, the first transition direction and the second transition direction are opposite directions; exemplarily, when the first transition direction is the left direction, the second transition direction is the right direction; when the first transition direction is the right direction, the second transition direction is the left direction.

[0195] In an exemplary embodiment, identifying the ball motion trajectory of a target ball according to multiple frames includes: first, using a ball detector to identify the ball in the picture frame by frame; wherein the detector scheme includes but is not limited to YOLO (You Only Look Once) and SSD (Single Shot MultiBox); then, using a multi-target tracking algorithm to assign a tracking ID to each ball, analyzing the motion trajectory of each ID, filtering out static balls on the sidelines and balls with a small movement amplitude, so as to select a target ball sequence with the most significant movement from a picture area sequence of multiple balls; generating a ball motion trajectory of the target ball based on the target ball sequence;

[0196] Correspondingly, if the ball's motion trajectory passes through the target screen area and there is a critical screen position that intersects with the ball's motion trajectory, the ball's transition direction is determined according to the critical screen position that intersects with the ball's motion trajectory, including: if the target ball's motion trajectory moves significantly from the center to both sides and reaches the critical screen position, so that the target ball disappears at the edge of the reference screen, it is considered that the basketball has passed out of the reference screen from this critical screen position of the screen. Thus, the transition event is identified, and the algorithm outputs a transition signal.

[0197] In another exemplary embodiment, the screen width of the sphere reference screen is Width, the screen width is the positive direction of the pixel x-axis, the pixel x-axis coordinate interval is (0-Width), and 0.1*Width and 0.9*Width are set as the critical lines where the critical positions of the screen are located. The tracking trajectory of the target sphere in the historical time period (such as 2 seconds) is analyzed. When the sphere motion trajectory of the target sphere passes through the critical line 0.1*Width from the center area of ​​the screen, it is considered that the sphere transition direction of the target sphere at the current moment is the left direction; when the sphere motion trajectory of the target sphere passes through the critical line 0.9*Width from the center area of ​​the screen, it is considered that the sphere transition direction of the target sphere at the current moment is the right direction.

[0198] In this embodiment, the relative position relationship between the target image area and the image critical position, as well as the sphere motion trajectory of the target sphere, is used to estimate whether the sphere transition direction exists. Therefore, when the target sphere is less obstructed, it is possible to more accurately determine whether the image acquisition process requires a transition, and based on the image critical position intersecting with the sphere motion trajectory, the sphere transition direction can be more accurately determined.

[0199] In one embodiment, determining whether the group transition information satisfies the image acquisition condition of the target sphere includes: determining whether the sphere trajectory belongs to a long-trajectory sphere motion scene; if so, the group transition information does not meet the image acquisition condition of the target sphere; if not, the group transition information meets the image acquisition condition of the target sphere.

[0200] The ball motion scene is a motion scene determined based on the conditions satisfied by the ball motion trajectory. Optionally, the motion scene determined based on the conditions satisfied by the ball motion trajectory can be determined based on the properties of the ball motion trajectory; the properties of the ball motion trajectory include but are not limited to duration, the area passed by the trajectory, etc.

[0201] The long-trajectory ball movement scene is a scene where the ball movement trajectory meets certain conditions. The long-trajectory ball movement scene can be judged based on factors such as the duration and length of the ball trajectory itself, and the accuracy of the judgment based on various factors is relatively high. The long-trajectory ball movement scene includes but is not limited to the long pass scene and fast break scene in ball sports.

[0202] In an optional implementation, determining whether the ball trajectory belongs to a long-trajectory ball motion scene includes: when the duration of the ball motion trajectory is greater than a long-trajectory time threshold, determining that the ball trajectory belongs to the long-trajectory ball motion scene; when the duration of the ball motion trajectory is less than the long-trajectory time threshold, the duration of the ball motion trajectory is not used for ball motion scene determination.

[0203] In another optional implementation, determining whether the ball trajectory belongs to a long-trajectory ball motion scene includes: when the length of the ball motion trajectory is greater than a long-trajectory length threshold, determining that the ball trajectory belongs to the long-trajectory ball motion scene; when the length of the ball motion trajectory is less than the long-trajectory length threshold, the length of the ball motion trajectory is not used for ball motion scene judgment.

[0204] In an exemplary embodiment, Figure 6 As shown, Figure 6 Figures (a), (b), and (c) are executed sequentially. Figure 6 After a moving object in Figure (a) throws a ball, the image captured based on the target ball is as follows Figure 6 As shown in Figure (b) in Figure 6 After the moment shown in Figure (c), the target sphere approaches the moving object and the movement of the long-trajectory sphere ends.

[0205] In the long-trajectory ball motion scene, the target ball is less blocked, and the movement of the character is much slower than that of the ball. The probability of target ball blocking is low, and the ball transition information generated by the ball area trajectory has higher reliability, higher corresponding credibility, and higher transition accuracy. In the case of non-long-trajectory ball motion scenes, the probability of target ball blocking is high, and the group transition information generated for group motion objects has higher reliability, higher corresponding credibility, and higher transition accuracy.

[0206] In one embodiment, images of a target sphere are captured based on group transition information, including: if the group transition information meets the image capture conditions of the target sphere, the pan-tilt rotation speed is adjusted according to the movement speed in the group movement information to obtain an adjusted pan-tilt rotation speed; if the group transition information does not meet the image capture conditions of the target sphere, the pan-tilt rotation speed is adjusted according to the movement speed of the target sphere to obtain an adjusted pan-tilt rotation speed; in the process of controlling the rotation of the pan-tilt according to the adjusted pan-tilt rotation speed, the image of the target sphere is captured through the pan-tilt.

[0207] In an exemplary embodiment, "transition" refers to identifying the offensive and defensive conversion on the court based on the movement of the crowd and basketball in the court, and then controlling the pan-tilt to switch from the current half court to the other half court, ensuring that the main body of the character is in the picture. The pan-tilt is placed at the edge of the court midfield, and the crowd movement trend and basketball movement trend in the picture are automatically identified by the visual algorithm, and a transition signal is output, thereby controlling the pan-tilt to switch from the current half court to the other half court, ensuring that the rotation of the pan-tilt can keep up with the offensive and defensive conversion on the court, and achieving the effect of automatically tracking and shooting the full-court basketball game. Among them, the real-time video stream shot by the user's mobile phone is obtained, the transition recognition algorithm is called for multiple frames continuously, the target movement trend within a period of time is identified, the transition signal is output, and then the pan-tilt is guided to rotate. The real-time video stream shot by the user's mobile phone can be obtained, the transition recognition algorithm is called for multiple frames continuously, the target movement trend within a period of time is identified, and the transition signal is output, thereby guiding the pan-tilt to rotate. The transition recognition algorithm includes but is not limited to the optical flow scheme, the basketball trajectory recognition scheme, the individual position analysis scheme, the group position analysis scheme, and the combination of the above schemes. In these solutions, the training materials for the human detector and basketball detector are mainly based on the human materials on the basketball court to ensure the detection rate of players and balls. Based on this, the entire basketball game is automatically tracked and filmed to replace the user's manual camera movement and shooting, realizing the unattended function and freeing the user's hands. The solution is different from the existing basketball tracking solution. The key to the algorithm is to identify the transition signal. The "player group" is the most prominent target on the court. Using computer vision processing algorithms to analyze its movement trends can accurately identify the offense and defense transitions on the court. Then, using basketball trajectory information as an aid, it can finally quickly and accurately identify the transition signal and complete the automatic control of the gimbal according to the transition signal.

[0208] Alternatively, on an embedded platform or a mobile phone platform with low computing power, the above four solutions can be reasonably matched according to the hardware performance to maximize the use of the platform computing power and obtain the highest transition recognition accuracy. Optionally, a manual camera movement button can be designed to take manual intervention measures in scenarios such as stealing and other scenarios that cause the transition signal to fail.

[0209] Therefore, in one case, the pan-tilt rotation speed is adjusted according to the movement speed in the group movement information so that the pan-tilt rotation speed moves synchronously with the crowd; in another case, the pan-tilt rotation speed is adjusted according to the movement speed of the target sphere so that the pan-tilt rotation speed moves synchronously with the target sphere.

[0210] In an exemplary embodiment, Figure 7 As shown, after the mobile phone is installed on the gimbal, when the corresponding solution of any of the above embodiments is executed by the mobile phone, the gimbal is controlled to rotate to change the direction of the mobile phone camera, so as to collect the image of the target ball. At this time, when collecting images towards one half of the field, the gimbal and the mobile phone are as follows: Figure 7 As shown in Figure (a), when collecting images towards the other half of the field, the gimbal and the mobile phone Figure 7 As shown in Figure (b); Figure 7 During the image acquisition process between (a) and (b), the gimbal keeps rotating.

[0211] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0212] Based on the same inventive concept, the embodiment of the present application also provides a target sphere image acquisition device for implementing the target sphere image acquisition method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations of the one or more target sphere image acquisition device embodiments provided below can refer to the limitations of the target sphere image acquisition method above, and will not be repeated here.

[0213] In an exemplary embodiment, Figure 8As shown, a target sphere image acquisition device is provided, comprising:

[0214] A trajectory detection module 802 is used to detect the motion trajectory of each moving object in the moving object group according to multiple frames;

[0215] A group detection module 804 is used to determine group motion information of a group of moving objects according to the motion trajectory of each of the moving objects;

[0216] A transition detection module 806, configured to determine group transition information according to the transition situation that the group motion information conforms to;

[0217] The image acquisition module 808 is used to acquire images of the target sphere according to the group transition information.

[0218] In one embodiment, the group detection module 804 is used to:

[0219] Determining a target picture area and different transition critical positions in a reference picture of the multiple frames;

[0220] Among the motion trajectories of the moving objects passing through the target picture area, the motion trajectories of the moving objects intersecting at the different transition critical positions are respectively counted to obtain the number of moving objects in different transition directions;

[0221] Correspondingly, the transition detection module 806 is used to:

[0222] The group transition information is determined according to the transition interval to which the number of moving objects in different transition directions belongs.

[0223] In one embodiment, the critical value of the transition interval includes a first critical value and a second critical value, and the first critical value is greater than the second critical value;

[0224] The transition detection module 806 is used to:

[0225] If the number of moving objects in the first transition direction is greater than the first critical value, and the number of moving objects in the second transition direction is less than the second critical value, determining the group transition direction to be the first transition direction;

[0226] If the number of moving objects in the first transition direction is less than the second critical value, and the number of moving objects in the second transition direction is greater than the first critical value, determining the group transition direction to be the second transition direction;

[0227] Wherein, the first transition direction and the second transition direction are opposite directions.

[0228] In one embodiment, the group detection module 804 is used to:

[0229] Determine the individual position of each of the moving objects at the same time according to the moving trajectories of each of the moving objects at different times;

[0230] Generating a group position at the same moment according to the individual positions of the objects at the same moment;

[0231] Generate group displacements of at least two groups of adjacent moments according to the group positions of at least two groups of adjacent moments;

[0232] Correspondingly, the transition detection module 806 is used to:

[0233] If the group displacements of the at least two groups of moments have the same transition direction, and in the same transition direction, the group displacements of the at least two groups of moments respectively exceed the transition length threshold, it is determined that the group transition information includes the same transition direction.

[0234] In one of the embodiments, when the group motion information includes the number of moving objects in different transition directions and does not include group displacements at at least two groups of adjacent moments, the transition interval used to determine the transition situation is the first transition interval;

[0235] When the group motion information does not include the number of moving objects in different transition directions and includes at least two groups of group displacements at adjacent moments, the transition length threshold used to determine the transition situation is a first transition length threshold;

[0236] In a case where the group motion information includes the number of moving objects in different transition directions and includes at least two groups of group displacements at adjacent moments, the transition interval is a second transition interval, and the transition length threshold is a second transition length threshold;

[0237] The first transition interval covers the second transition interval, and the range of the first transition interval is smaller than the range of the second transition interval, and the first transition length threshold is greater than the second transition length threshold.

[0238] In one embodiment, the transition detection module 806 is used to:

[0239] When the group motion information does not meet the group transition information generation condition, optical flow estimation is performed on the moving object group according to the multiple frames to obtain the optical flow displacement of each frame;

[0240] Accumulating the optical flow displacement of the picture of each frame to obtain a picture motion feature map corresponding to the multiple frames; the picture motion feature map includes a plurality of feature map pixels, and the direction of each feature map pixel is determined by the accumulation result of the optical flow displacement of the picture of each frame;

[0241] According to the directions of the pixels of each feature map, the number of pixels of the feature map that are displaced along the opposite transition direction is counted to obtain the number of pixels in the opposite transition direction;

[0242] A comparison result between the numbers of pixels in opposite transition directions is determined; the comparison result pertaining to the group motion information.

[0243] In one embodiment, the transition detection module 806 is used to:

[0244] When the group motion information does not meet the group transition information generation condition, a pixel quantity difference is determined based on a difference between the numbers of pixels in opposite transition directions; the number of pixels in the opposite transition directions is obtained based on the multiple frames;

[0245] Obtaining a pixel quantity difference ratio based on a quantity ratio between the pixel quantity difference value and the quantity of pixels included in the reference picture;

[0246] When the pixel quantity difference ratio exceeds a pixel quantity difference ratio threshold, the group transition direction is determined.

[0247] In one embodiment, the transition detection module 806 is used to:

[0248] In case that the motion trajectories of the plurality of moving objects are detected according to the plurality of frames, it is determined that the motion trajectories of the plurality of moving objects satisfy a small number of trajectories condition; and / or it is detected that the plurality of frames satisfy a reference object lack condition.

[0249] In one embodiment, the picture acquisition module 808 is used to:

[0250] Determining whether the group transition information satisfies the image acquisition condition of the target sphere;

[0251] If the conditions are met, image collection is performed according to the group transition information to obtain an image of the target sphere.

[0252] In one embodiment, the picture acquisition module 808 is used to:

[0253] If not, the ball transition information is determined according to the ball motion trajectory of the target ball in the plurality of frames;

[0254] According to the sphere transition information, the image of the target sphere is captured.

[0255] In one embodiment, the picture acquisition module 808 is used to:

[0256] Identify the sphere motion trajectory of the target sphere according to the multiple frames;

[0257] In the spherical reference images of the multiple frames, determining a target image area and an image critical position;

[0258] If the ball motion trajectory passes through the target screen area and there is a screen critical position intersecting with the ball motion trajectory, the ball transition direction is determined according to the screen critical position intersecting with the ball motion trajectory.

[0259] In one embodiment, the picture acquisition module 808 is used to:

[0260] Identify the moving sphere according to the trajectory of the sphere area in multiple frames;

[0261] Determine a target ball from each of the moving balls according to the confidence that each of the moving balls belongs to a preset ball type;

[0262] The spherical motion trajectory of the target sphere is determined from the spherical region trajectories in the multiple frames.

[0263] In one embodiment, the picture acquisition module 808 is used to:

[0264] Determining whether the ball trajectory belongs to a long-trajectory ball motion scene;

[0265] If yes, the group transition information does not meet the image acquisition condition of the target sphere;

[0266] If not, the group transition information satisfies the image acquisition condition of the target sphere.

[0267] In one embodiment, the picture acquisition module 808 is used to:

[0268] If the group transition information satisfies the image acquisition condition of the target sphere, the pan / tilt rotation speed is adjusted according to the movement speed in the group movement information to obtain an adjusted pan / tilt rotation speed;

[0269] If the group transition information does not meet the image acquisition condition of the target sphere, the pan / tilt rotation speed is adjusted according to the movement speed of the target sphere to obtain an adjusted pan / tilt rotation speed;

[0270] In the process of controlling the rotation of the pan-tilt head according to the adjusted pan-tilt speed, the image of the target sphere is collected through the pan-tilt head.

[0271] In one of the embodiments, the process of capturing the image of the target sphere is performed through a pan-tilt head, and the pan-tilt head is provided with a camera movement button.

[0272] Each module in the above-mentioned target sphere image acquisition device can be implemented in whole or in part by software, hardware or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0273] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig. 9 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for capturing a picture of a target sphere is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0274] Those skilled in the art will understand that Fig. 9 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0275] In one embodiment, a handheld gimbal is provided, including a motor and a processor, wherein the motor is used to control the rotation of the gimbal, and the processor implements the steps of capturing the image of the target sphere in any of the above embodiments when executing a computer program.

[0276] In one embodiment, a handheld gimbal is provided, including a motor for controlling the rotation of the gimbal. A terminal processor installed on the handheld gimbal is used to implement the steps of capturing the image of the target sphere in any of the above embodiments when executing a computer program.

[0277] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0278] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0279] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0280] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0281] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0282] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0283] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for capturing images of a target sphere, characterized in that: The method comprises: Detecting the motion trajectory of each moving object in the moving object group according to multiple frames; Determining group motion information of a group of moving objects according to the motion trajectory of each of the moving objects; Determining group transition information according to the transition situation that the group movement information conforms to; According to the group transition information, the image of the target sphere is captured.

2. The method according to claim 1, characterized in that The determining the group motion information of the group of moving objects according to the motion trajectory of each of the moving objects comprises: Determining a target picture area and different transition critical positions in a reference picture of the multiple frames; Among the motion trajectories of the moving objects passing through the target picture area, the motion trajectories of the moving objects intersecting at the different transition critical positions are respectively counted to obtain the number of moving objects in different transition directions; The determining of group transition information according to the transition situation that the group movement information conforms to includes: The group transition information is determined according to the transition interval to which the number of moving objects in different transition directions belongs.

3. The method according to claim 2, characterized in that The critical value of the transition interval includes a first critical value and a second critical value, and the first critical value is greater than the second critical value; The determining of group transition information according to the transition interval to which the number of moving objects in different transition directions belongs includes: If the number of moving objects in the first transition direction is greater than the first critical value, and the number of moving objects in the second transition direction is less than the second critical value, determining the group transition direction to be the first transition direction; If the number of moving objects in the first transition direction is less than the second critical value, and the number of moving objects in the second transition direction is greater than the first critical value, determining the group transition direction to be the second transition direction; Wherein, the first transition direction and the second transition direction are opposite directions.

4. The method according to claim 1, characterized in that: The determining the group motion information of the group of moving objects according to the motion trajectory of each moving object includes: Determine the individual position of each of the moving objects at the same time according to the moving trajectories of each of the moving objects at different times; Generating a group position at the same moment according to the individual positions of the objects at the same moment; Generate group displacements of at least two groups of adjacent moments according to the group positions of at least two groups of adjacent moments; The determining of group transition information according to the transition situation that the group movement information conforms to includes: If the group displacements of the at least two groups of moments have the same transition direction, and in the same transition direction, the group displacements of the at least two groups of moments respectively exceed the transition length threshold, it is determined that the group transition information includes the same transition direction.

5. The method according to claim 1, characterized in that When the group motion information includes the number of moving objects in different transition directions and does not include at least two groups of group displacements at adjacent moments, the transition interval used to determine the transition situation is the first transition interval; When the group motion information does not include the number of moving objects in different transition directions and includes at least two groups of group displacements at adjacent moments, the transition length threshold used to determine the transition situation is a first transition length threshold; In a case where the group motion information includes the number of moving objects in different transition directions and includes at least two groups of group displacements at adjacent moments, the transition interval is a second transition interval, and the transition length threshold is a second transition length threshold; The first transition interval covers the second transition interval, and the range of the first transition interval is smaller than the range of the second transition interval, and the first transition length threshold is greater than the second transition length threshold.

6. The method according to claim 1, characterized in that The determining of group transition information according to the transition situation that the group movement information conforms to includes: When the group motion information does not meet the group transition information generation condition, optical flow estimation is performed on the moving object group according to the multiple frames to obtain the optical flow displacement of each frame; Accumulating the optical flow displacement of the picture of each frame to obtain a picture motion feature map corresponding to the multiple frames; the picture motion feature map includes a plurality of feature map pixels, and the direction of each feature map pixel is determined by the accumulation result of the optical flow displacement of the picture of each frame; According to the directions of the pixels of each feature map, the number of pixels of the feature map that are displaced along the opposite transition direction is counted to obtain the number of pixels in the opposite transition direction; A comparison result between the numbers of pixels in opposite transition directions is determined; the comparison result pertaining to the group motion information.

7. The method according to claim 1 or 6, characterized in that: The determining of group transition information according to the transition situation that the group movement information conforms to includes: When the group motion information does not meet the group transition information generation condition, a pixel quantity difference is determined based on a difference between the numbers of pixels in opposite transition directions; the number of pixels in the opposite transition directions is obtained based on the multiple frames; Obtaining a pixel quantity difference ratio based on a quantity ratio between the pixel quantity difference value and the quantity of pixels included in the reference picture; When the pixel quantity difference ratio exceeds a pixel quantity difference ratio threshold, the group transition direction is determined.

8. The method according to claim 6, characterized in that Before the optical flow estimation of the moving object group is performed according to the multiple frames, the method includes: In the case where the motion trajectories of the plurality of moving objects are detected according to the plurality of frames, determining that the motion trajectories of the plurality of moving objects satisfy a small number of trajectories condition; and / or, It is detected that the multiple frames satisfy a reference object lack condition.

9. The method according to claim 1, characterized in that: The step of collecting images of the target sphere according to the group transition information includes: Determining whether the group transition information satisfies the image acquisition condition of the target sphere; If the conditions are met, image collection is performed according to the group transition information to obtain an image of the target sphere.

10. The method according to claim 9, characterized in that After determining whether the group transition information meets the image acquisition condition of the target sphere, the method further includes: If not, the ball transition information is determined according to the ball motion trajectory of the target ball in the plurality of frames; According to the sphere transition information, the image of the target sphere is captured.

11. The method according to claim 10, characterized in that Determining the ball transition information according to the ball motion trajectory of the target ball in the multiple frames includes: Identify the sphere motion trajectory of the target sphere according to the multiple frames; In the spherical reference images of the multiple frames, determining a target image area and an image critical position; If the ball motion trajectory passes through the target screen area and there is a screen critical position intersecting with the ball motion trajectory, the ball transition direction is determined according to the screen critical position intersecting with the ball motion trajectory.

12. The method according to claim 11, characterized in that The step of identifying the sphere motion trajectory of the target sphere according to the multiple frames includes: Identify the moving sphere according to the trajectory of the sphere area in multiple frames; Determine a target ball from each of the moving balls according to the confidence that each of the moving balls belongs to a preset ball type; The spherical motion trajectory of the target sphere is determined from the spherical region trajectories in the multiple frames.

13. The method according to claim 10, characterized in that The determining whether the group transition information satisfies the image acquisition condition of the target sphere includes: Determining whether the ball trajectory belongs to a long-trajectory ball motion scene; If yes, the group transition information does not meet the image acquisition condition of the target sphere; If not, the group transition information satisfies the image acquisition condition of the target sphere.

14. The method according to claim 1, characterized in that The step of collecting images of the target sphere according to the group transition information includes: If the group transition information satisfies the image acquisition condition of the target sphere, the pan / tilt rotation speed is adjusted according to the movement speed in the group movement information to obtain an adjusted pan / tilt rotation speed; If the group transition information does not meet the image acquisition condition of the target sphere, the pan / tilt rotation speed is adjusted according to the movement speed of the target sphere to obtain an adjusted pan / tilt rotation speed; In the process of controlling the rotation of the pan-tilt head according to the adjusted pan-tilt speed, the image of the target sphere is collected through the pan-tilt head.

15. The method according to claim 1, characterized in that The process of capturing the image of the target sphere is performed through a pan / tilt platform, which is provided with a camera movement button.

16. A target sphere image acquisition device, characterized in that: The device comprises: A trajectory detection module is used to detect the motion trajectory of each moving object in a group of moving objects based on multiple frames; A group detection module, used to determine group motion information of a group of moving objects according to the motion trajectory of each of the moving objects; A transition detection module, used to determine group transition information according to the transition situation that the group movement information conforms to; The image acquisition module is used to acquire images of the target sphere according to the group transition information.

17. A handheld gimbal, characterized in that: It comprises a motor and a processor, wherein the motor is used to control the rotation of the pan / tilt head, and the processor is used to implement the steps of any one of the methods of claims 1 to 15.

18. A handheld gimbal, characterized in that: It comprises a motor, which is used to control the rotation of the pan-tilt head. A terminal processor installed on the handheld pan-tilt head is used to implement the steps of any one of the methods of claims 1 to 15.

19. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 15 are implemented.

20. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 15 are implemented.

Images