Target operation execution method, terminal and computer readable medium

Abstract

The invention provides a target operation execution method, which comprises the following steps of: determining state change information of a target object in a preview picture under the condition of confirming that the target object exists in the preview picture; the state change information comprises at least one of motion state change of the target object in the preview picture and morphological change of the target object in the preview picture; and under the condition that the state change information of the target object meets a preset condition, executing the target operation. The invention further provides a target operation execution terminal and a computer readable medium.

Description

Technical Field

[0001] The present disclosure relates to the field of electronic technology, and in particular to a method for executing a target operation, a terminal for executing a target operation, and a computer-readable medium. Background Art

[0002] Target motion capture technology is a relatively broad concept. At present, the existing motion capture technologies in the industry (such as drone capture, camera continuous capture, etc.) mainly focus on judging the motion range of the target in the current video frame, such as obtaining the target's motion distance, whether the target's center position is stable within a certain range, etc.

[0003] However, there is no analysis or related application of changes in motion states. Since these changes occur in a short period of time and the current automatic capture technology is still immature, it is very difficult to obtain and analyze object information, which makes it difficult to accurately capture the moment when the relevant state changes. Summary of the invention

[0004] The embodiments of the present disclosure provide a method for executing a target operation, a terminal for executing a target operation, and a computer-readable medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for executing a target operation, the method comprising: upon confirming that a target object exists in a preview screen, determining state change information of the target object in the preview screen; the state change information comprises at least one of a change in motion state of the target object in the preview screen and a change in morphology of the target object in the preview screen; and executing the target operation when the state change information of the target object satisfies a preset condition.

[0006] In a second aspect, an embodiment of the present disclosure provides an execution terminal for a target operation, comprising: one or more processors; a memory on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the execution method of the target operation described in the first aspect of the embodiment of the present disclosure.

[0007] In a third aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method for executing the target operation described in the first aspect of the embodiment of the present disclosure.

[0008] In the method for executing target operations provided by the embodiments of the present disclosure, the motion state and morphological changes of the selected target object are analyzed in the continuous frame preview screen, and the operation is performed according to the state change information corresponding to the motion state and morphological changes. When executing the preview screen shooting, the state change moment of the target can be accurately grasped, thereby improving the success rate of motion capture of the target object in the screen. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a flowchart of a method for executing a target operation in an embodiment of the present disclosure;

[0010] Figure 2 is a flowchart of some steps in another method for executing a target operation in an embodiment of the present disclosure;

[0011] Figure 3 is a flowchart of some steps in another method for executing a target operation in an embodiment of the present disclosure;

[0012] Figure 4 is a flowchart of some steps in another method for executing a target operation in an embodiment of the present disclosure;

[0013] Figure 5 is a flowchart of some steps in another method for executing a target operation in an embodiment of the present disclosure;

[0014] Figure 6 is a flowchart of some steps in another method for executing a target operation in an embodiment of the present disclosure;

[0015] Figure 7 is a flowchart of some steps in another method for executing a target operation in an embodiment of the present disclosure;

[0016] Figure 8 is a flowchart of some steps in another method for executing a target operation in an embodiment of the present disclosure;

[0017] Fig. 9 is a flowchart of some steps in another method for executing a target operation in an embodiment of the present disclosure;

[0018] Fig.10 is a schematic diagram of the structure of an execution terminal of a target operation in an embodiment of the present disclosure;

[0019] Fig.11 is a schematic diagram of the structure of a computer-readable medium in an embodiment of the present disclosure;

[0020] Fig.12 is a schematic diagram of the steps of another method for executing a target operation in an embodiment of the present disclosure;

[0021] Fig.13 It is a schematic diagram of the motion change of the target object in the camera preview screen in the embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the technical solution of the present disclosure, the execution method, terminal and computer-readable medium of the target operation provided by the present disclosure are described in detail below with reference to the accompanying drawings.

[0023] Example embodiments will be described more fully below with reference to the accompanying drawings, but the example embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. On the contrary, the purpose of providing these embodiments is to make the present disclosure thorough and complete and to enable those skilled in the art to fully understand the scope of the present disclosure.

[0024] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0025] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0026] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a", "an" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "comprising" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded.

[0027] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0028] Technical terminology. In this disclosure, unless otherwise specified, the following technical terms should be understood as follows:

[0029] The present disclosure relates to analyzing the change of motion state of a target object in a preview image to be processed, where the change of motion state refers to the change of the shape, position, speed, acceleration, etc. of the target object in the preview images of consecutive frames. In some examples, the change of the shape of the target object is represented by the change of the center of mass, and the change of the position of the target object is represented by the change of the displacement.

[0030] The centroid is the average position of all pixels in the area occupied by the target object in the preview image. In some examples, the centroid of the target object is the average of the weighted mass (such as pixel value) of all pixels in the area occupied by the target object multiplied by their positions. x ,C y ) can be obtained through as well as It is calculated, where f(x,y) is the weighted mass of the pixel with coordinates (x,y), and “Σ” represents the summation.

[0031] It is worth noting that the pixel value represents the brightness or color information of the pixel in the image. For grayscale images, the pixel value is the grayscale value, and the value range is [0,255], where 0 represents pure black, 255 represents pure white, and other values ​​represent different degrees of gray; for color images, the pixel value is a vector containing three values, representing the brightness of the three basic colors of red, green and blue (RGB). The brightness value range of each color is [0,255], and different RGB combinations are used to represent different colors.

[0032] Displacement is the vector change of the target object from one position to another in the continuous frame. In some examples, the position of the target object in the first frame is (x1, y1), and the position in the nth frame of the continuous frame is (x n ,y n ), then the displacement of the object from the 1st frame to the nth frame is (Δx, Δy) = (x n -x1,y n -y1).

[0033] Figure 1 FIG. 1 is a flow chart of a method for executing a target operation according to an embodiment of the present disclosure. Figure 1 , the present disclosure provides a method for executing a target operation, which may include:

[0034] S1. When confirming that a target object exists in the preview picture, determining state change information of the target object in the picture; the state change information includes: a change in the motion state of the target object in the preview picture, and at least one of a change in the shape of the target object in the preview picture;

[0035] S2. When the state change information of the target object meets the preset conditions, execute the target operation.

[0036] The embodiments of the present disclosure are intended to perform operations at the moment when the motion state of the target object changes. For example, when a mobile phone is used to shoot a subject (such as a cat, dog, bird, etc.), a preset target operation is performed when the preview screen corresponding to the moment when the subject changes from stillness to motion, from motion to stillness, or the moment when the form changes, etc., and the moment refers to the frame corresponding to the preview screen.

[0037] The embodiments of the present disclosure can be applied to target motion capture, such as capture of the morphological change of the target object, capture of the acceleration state of the target object, capture of the deceleration state of the target object, etc. The state change information includes: the change of the motion state of the target object in the preview screen of the continuous frames, and at least one of the morphological changes of the target object in the preview screen, wherein the motion state change can be a change in displacement, acceleration, speed, motion trajectory or other changes that represent the motion state, and the morphological change can be a change in shape, centroid or contour or other changes that represent the shape.

[0038] By determining the state change information of the motion state and morphological change of the selected target object in the preview screen of continuous frames, and performing the target operation according to the state change information of the motion state and morphological change, when performing preview screen shooting, the state change moment of the target can be accurately grasped, thereby improving the success rate of motion capture of the target object in the preview screen. In some examples, when the target operation is to shoot the preview screen, combining photography skills with prediction algorithms can recommend and help users shoot wonderful images of the target object at the moment when the motion state changes, reducing the user's demand for relevant technical reserves and improving the user's shooting experience.

[0039] The embodiments of the present disclosure do not limit the number and type of target objects, and the target object in the same preview screen can be one or more. In some examples, when there are multiple target objects, interval image object detection can be used to update the various target objects in the preview screen in real time, and the same processing can be executed or triggered for each target object. For example, multiple people are detected and tracked in the video frame at the same time, and the people in the video frame are target objects. The status changes of multiple people are analyzed, and the target operation is performed when the status change information of each person meets the preset conditions.

[0040] Reference Figure 2 In some embodiments, before S1, the following may also be included:

[0041] S01. Determine at least one trackable object in a preview image.

[0042] S02: In response to a first input instruction, determine a trackable object corresponding to the first input instruction as the target object.

[0043] In the embodiments of the present disclosure, there is no limitation on the method of identifying the trackable object in the preview screen, and it may be to detect whether the screen contains a specific target through object detection (Object detect). The target object in the trackable object may be automatically detected, or may be selected from at least one trackable object, and the present disclosure is not limited to this. At the same time, there is no limit on the number of trackable objects in the preview screen. In some examples, a first identification processing is performed on at least one trackable object detected in the preview screen, and the first identification processing may be to add a tracking box to the trackable object in the preview screen, or to add an identification arrow to the trackable object in the preview screen, etc., and the present disclosure is not limited to this.

[0044] The embodiments of the present disclosure do not limit the process of obtaining the first input instruction. The first input instruction may be an instruction representing a user selecting a target object from a plurality of trackable objects, or may be an instruction automatically generated based on preset selection conditions and trackable objects to represent the selected target object.

[0045] In other examples, a second identification process is performed on the target object corresponding to the first input instruction so that the identification of the target object can be distinguished from the identification of the trackable object.

[0046] Accordingly, in some embodiments, reference Figure 3 , S1 may include:

[0047] S11, obtaining the state parameters of the target object in the multi-frame preview image;

[0048] S12: Determine the state change information of the target object in the multi-frame preview image according to the state parameter.

[0049] In the embodiments of the present disclosure, the state parameters include parameters of the position information of the target object in the corresponding preview screen, parameters of the center of mass information of the target object in the current screen, etc. In some embodiments, the multi-frame screen is a continuous multi-frame screen; wherein the state parameters of the multi-frame screen can be obtained by real-time detection, or by predicting the state parameters (such as position information) of the target object in each frame screen through target tracking (tracking). Wherein, "frame" refers to a video frame, and the preview screen corresponding to each frame is a static screen. The frame is the basic unit of the video, and the visual dynamic effect formed by the rapid switching of multiple continuous frames of preview screens constitutes the video playback.

[0050] In addition, it is worth noting that in some embodiments, the basic unit of a static image is a pixel, and each pixel stores color information corresponding to a specific position. Each color can be represented by one or more numerical values. In the embodiments of the present disclosure, the position coordinates, color information, etc. of the pixel in the image can be used as state parameters to analyze the state change.

[0051] Wherein, in some embodiments, reference Figure 4 , S12 may include:

[0052] S121. Determine, according to the state parameters of the target object in a plurality of preview frames, the state change information of the target object in the last preview frame of the plurality of preview frames.

[0053] In an embodiment of the present disclosure, the state parameter of the target object can reflect the change of the target object in multiple preview frames, and the change includes not only the difference of the area occupied by the target object in the multiple preview frames, but also the change trend of the target object obtained by analyzing the multiple preview frames. In an exemplary implementation, the change can be determined by calculating the state parameter corresponding to each preview frame.

[0054] In an exemplary embodiment, the state change information may correspond to the state parameter. The state change information used to characterize the motion displacement may be determined according to the parameters of the position information in the preview screen corresponding to the target object. The state change information of the motion displacement is used to reflect the change of the motion state of the target object in the preview screen. In some examples, the motion state change may also be represented by other state change information, such as the speed of change, acceleration, etc.; the state change information used to characterize the motion center of mass is determined according to the parameters of the center of mass information in the preview screen corresponding to the target object. The state change information of the motion center of mass is used to reflect the morphological change of the target object in the preview screen. In some examples, the morphological change may also be represented by other state change information, such as the change in the size of the area corresponding to the target object. The state parameter is a parameter used to quantify the state change. In some examples, the state change information used to characterize the motion displacement and the motion center of mass is not limited thereto, and may also be other state changes.

[0055] It is worth noting that the state change information of the motion center of mass can be determined based on the changes in the center of mass coordinates of the target object in the last frame preview screen and other preview screens. In some examples, the center of mass coordinates here can be simplified to the center coordinates of the above-mentioned tracking box (such as a rectangle). Combined with semantic segmentation, the morphological information of the target can be accurately obtained, thereby obtaining accurate center of mass coordinates.

[0056] In some embodiments, reference Figure 5, when the total number of frames of the multi-frame preview screen is N, and the state change information includes the case of the change in the motion state of the target object within the preview screen, S121 may include:

[0057] S1211. Determine at least two first absolute differences between the position parameter of the Nth frame and the position parameter of the ith frame, where N and i are both positive integers, 0 < i < N, and the first absolute difference represents the change in the motion displacement of the same target object from the ith frame preview screen to the Nth frame preview screen;

[0058] S1212. Determine the change in the motion state of the target object in the Nth frame preview screen according to the difference between at least two of the first absolute differences. The Nth frame preview screen is the last frame preview screen in the multi-frame preview screen with a total number of frames N, and the change in the motion state is used to represent the change in the motion displacement.

[0059] The embodiments of the present disclosure do not limit the number of parameters corresponding to the state change information, which may be one or more.

[0060] It should be noted that for the calculation of displacement, the positive and negative values of the calculation result are usually used to represent the direction and / or angle of the motion of the target object. In the embodiments of the present disclosure, the actual requirements of the application scenario are fully considered, and the absolute value of the displacement change difference of the target object in the N-frame preview screen (i.e., the first absolute difference) is calculated, and the amplitude of its state change is determined according to the first absolute difference. In some examples, the ith frame may be the previous frame of the Nth frame or several previous frames of the Nth frame, and the present disclosure is not limited thereto.

[0061] In one example, the Nth frame is specifically the 2nd frame, the ith frame is specifically the 1st frame, the position of the target object in the 1st frame preview screen is (x t0 , y t0 ), and the position of the target object in the 2nd frame preview screen is (x t1 , y t1 ). The state change information is calculated through formula (1):

[0062] ΔT dist [t1] = abs(x t0 - x t1 ) - abs(y t0 - y t1 ) (1)

[0063] where abs(·) represents taking the absolute value, and ΔT dist [t1] represents the displacement amplitude (i.e., the state change information) of the target object from the 1st frame to the 2nd frame, and the coordinate values x t0 and y t0, and the coordinate values x at the second frame t1 and y t1 are state parameters.

[0064] It should be noted that the position coordinates can be the center, centroid, etc. of the target object, and the present disclosure is not limited thereto.

[0065] In some embodiments, referring to Figure 6 , for example, when the total number of frames of the multi-frame preview screen is N and the state change information includes the morphological change of the target object within the preview screen, S121 may include:

[0066] S1211': Determine at least two second absolute differences between the centroid parameters of the Nth frame and the centroid parameters of the jth frame, where N and j are both positive integers, 0 < j < N, and the second absolute difference characterizes the change in the moving centroid of the same target object from the jth frame preview screen to the Nth frame preview screen;

[0067] S1212': Determine the morphological change of the target object in the Nth frame preview screen according to the sum value of at least two of the second absolute differences. The Nth frame preview screen is the last frame preview screen in the multi-frame preview screen with a total number of frames of N, and the morphological change characterizes the change in the moving centroid.

[0068] In the embodiments of the present disclosure, the morphological change of the target object can be represented by the state change information characterizing the moving centroid.

[0069] The state change information regarding the motion form can be determined by the change in the barycentric coordinates of the target object in the Nth frame, or can be determined by the change in the length value and width value of the target object. The barycentric coordinates include at least one centroid parameter.

[0070] In one example, the Nth frame is specifically the second frame, the jth frame is specifically the first frame, the length of the target object in the first frame preview screen is Rect w_t0 and the width is Rect h_t0 , the length of the target object in the second frame preview screen is Rect w_t1 and the width is Rect h_t1 , and the state change information is calculated through formula (2):

[0071] ΔT center [t1]=abs(Rect w_t1 -Rect w_t0 )-abs(Rect h_t1 -Rect h_t0 ) (2)

[0072] Where abs(·) means to find the absolute value, ΔT center [t1] represents the change in shape of the target object from the first frame to the second frame (i.e., the state change information). The length of the Rect w_t0 , Width Rect h_t0 , in the second frame, the long Rect w_t1 And the width of Rect h_t1 These are all status parameters.

[0073] In some embodiments, after S1, the process may further include: storing the value corresponding to the state change information in an array. In one example, the value corresponding to the state change information is the state change information used to characterize the change in motion displacement, and the state change information ΔT of each frame is stored. dist [·], when the total number of preview frames is n, the array is ΔT dist [t1, t2, t3, …, tn], which is an array used to store the rate of change of motion state ΔT dist […]; In another example, the value corresponding to the state change information is the state change information used to characterize the change of the moving center of mass, and the state change information ΔT of each frame is saved. center [·], when the total number of preview frames is n, the array is ΔT center [t1, t2, t3, …, tn], which is an array used to store the rate of change of the center of mass coordinates ΔT center […], where t1, t2, t3, …, tn represent the preview images of n consecutive sequence frames. It is worth noting that this data can also be used for target tracking.

[0074] In some embodiments, reference Figure 7 , S2, may include:

[0075] S21, comparing the state change information of the target object corresponding to at least one preview frame among the multiple preview frames with a preset state change threshold;

[0076] S22: When the state change information of the target object corresponding to the at least one frame of the preview screen is greater than or equal to a preset state change threshold, confirm that the state change information of the target object meets a preset condition, and execute the target operation.

[0077] In the embodiments of the present disclosure, there is no restriction on the number of preset state change thresholds, and the preset state change thresholds may be one or more. In some examples, the state change information includes at least one item of state change information for characterizing motion displacement and state change information for characterizing motion center of mass, and the number of preset state change thresholds may be set corresponding to the number of state change information. In other examples, the number of target objects in at least one frame may be one or more, and the preset state change thresholds may be set separately according to different target objects.

[0078] The value of the preset state change threshold, the increase or decrease of the quantity, etc. can be adjusted by inputting an input instruction of the execution terminal of the target operation. It is worth noting that when the terminal is started and no input instruction for indicating the adjustment of the preset state change threshold is input, the terminal will perform the target operation according to the screen according to the default state change threshold.

[0079] In some embodiments, before S2, the following may also be included:

[0080] The preset condition is obtained.

[0081] The embodiments of the present disclosure do not limit the method for obtaining the preset conditions, which may be set according to user needs, or automatically matched according to historically executed target operations, or automatically started according to the activation of the target application or automatic recognition of the scene, etc.

[0082] In some embodiments, acquiring the preset condition may include: adjusting the preset state change threshold in response to a third input instruction.

[0083] In an embodiment of the present disclosure, the third input instruction can be input according to user needs, and the third input instruction is used to indicate the change of the numerical value of the preset state change threshold. In some embodiments, the numerical value of the preset state change threshold can also be automatically matched according to the target operation executed historically.

[0084] In some embodiments, the method for executing the target operation may further include:

[0085] In response to the second input instruction, the preset state change threshold is determined to be a first preset state change threshold for characterizing a motion displacement change threshold, and / or a second preset state change threshold for characterizing a motion center of mass change threshold.

[0086] In an embodiment of the present disclosure, the second input instruction may be used to indicate the increase or decrease or setting of the number of preset state change thresholds. In an embodiment of the present disclosure, different operation settings and function activation instructions may be provided to the user.

[0087] In some exemplary embodiments, at least one preset state change threshold can be selected, that is, any preset state change threshold can be enabled or set, and the judgment mode of the preset state change threshold can also be selected or set. For example, if only the second preset state change threshold is selected, then the first preset state change threshold will not take effect during the operation execution; the judgment mode of multiple preset state change thresholds requires that the state change information of the target object simultaneously meets multiple preset state change thresholds to trigger the execution of the target operation; if only one of the preset state change thresholds is selected, the other preset state change thresholds will not take effect.

[0088] In one example, the terminal is a terminal for performing a target operation on a preview screen. In the absence of an input instruction, the target operation is performed using a default state change threshold; in the third input instruction, a first preset state change threshold T0 for characterizing a motion displacement change threshold is set. dist_thresh In the case of the first preset state change threshold, according to the first preset state change threshold, the target operation is performed when the state change information of the target object is greater than or equal to the first preset state change threshold; in the third input instruction, a second preset state change threshold T for characterizing the motion center of mass change threshold is set center_thresh In the case of a second preset state change threshold, according to the second preset state change threshold, the target operation is performed when the state change information of the target object is greater than or equal to the second preset state change threshold; when the third input instruction also sets the first preset state change threshold T0 dist_thresh and the second preset state change threshold T center_thresh In this case, the target operation is performed according to the dual threshold combination of the first preset state change threshold and the second preset state change threshold.

[0089] In some embodiments, reference Figure 8 , S22, may include:

[0090] S221, comparing the state change information of the target object in the last preview frame of the multiple preview frames with a preset state change threshold;

[0091] S222: confirming that the state change information of the target object meets a preset condition when the state change information of the target object in the last frame preview image is greater than or equal to a preset state change threshold;

[0092] S223: Execute a target operation on the last frame preview picture.

[0093] In an embodiment of the present disclosure, determining whether to perform a target operation in the last frame may be triggered according to a comparison result between a state change of a target object in a preview screen of the last frame and a preset state change threshold.

[0094] In one example, the first preset state change threshold is set to T0 dist_thresh , for the target object in the preview image of the Nth frame, when the state change information used to characterize the motion displacement is greater than or equal to the first preset state change threshold, that is, ΔT dist [tN]>=T0 dist_thresh In the case of , the target operation is performed, where tN represents the preview picture of the Nth frame.

[0095] In some embodiments, reference Fig. 9 , S22, may include:

[0096] S221′, comparing the state change information of the target object corresponding to the first preset number of preview frames before the last preview frame in the multiple preview frames with a preset state change threshold;

[0097] S222', in the case where a second preset number of the state change information of the target object corresponding to the first preset number of frames of preview screen exists and is greater than or equal to a preset state change threshold, confirming that the state change information of the target object meets a preset condition, wherein both the first preset number and the second preset number are less than the total number of frames of the multi-frame preview screen, and the first preset number is greater than or equal to the second preset number;

[0098] S223', performing a target operation on the last frame preview picture.

[0099] In an embodiment of the present disclosure, whether to perform a target operation in the preview screen of the last frame may also be determined by comparing state change information of preview screens of a second preset number of frames before the preview screen of the last frame with a preset state change threshold, and the number value corresponding to the second preset number of frames may be set according to actual needs.

[0100] It is worth noting that when tracking multiple target objects in the preview screen at the same time, different judgment rules can be set for different target objects. For example, there are two target objects A and B in the preview screen at the same time, then A can trigger the execution of the target operation according to the comparison result of the state change information of the last frame preview screen and the preset state change threshold, and B can execute the target operation according to the comparison result of the state change information of the preview screen corresponding to the second preset number of frames before the last frame preview screen and the preset state change threshold.

[0101] In one example, there is a target object in the preview image, and determining whether the target object performs the target operation in the Nth frame is to determine whether the (Nm) frames before the Nth frame preview image all meet the condition (ΔT center [tk]>=T center_thresh ) or condition 2 (ΔT dist[tk]>=T0 dist_thresh ), and when all conditions are met, the target operation on the Nth frame preview screen is triggered, wherein tk represents the (Nm) frames before the Nth frame preview screen, (Nm)≤k≤N, (Nm) is a first preset number of frames, and in this example, whether (Nm) frames all meet condition one or condition two, the target operation on the Nth frame preview screen is triggered, and thus (Nm) in this example is a second preset number of frames, i.e., the first preset number is equal to the second preset number.

[0102] In some embodiments, performing a target operation includes:

[0103] Execute at least one of shooting, fatigue detection, slow motion, focus, and continuous capture.

[0104] The embodiments of the present disclosure analyze the motion state and morphological changes of the selected target object in the preview screen of continuous frames, and perform target operations on the preview screen according to the state change information of the motion state and morphological changes. When executing preview screen shooting, the state change moment of the target can be accurately grasped, thereby improving the success rate of motion capture of the target object in the preview screen.

[0105] The embodiments of the present disclosure can be applied to various application scenarios. Not only can the preview screen capture effect be improved during the shooting process of the preview screen, but also the application scenarios can be increased (such as being applied to drone capture, camera continuous capture, and wonderful action capture, etc.). The technical concept of the embodiments of the present disclosure can also be used for fatigue detection, slow motion, intelligent focus operation, and continuous capture to recommend photos of highlight moments, etc. Among them, fatigue detection is combined with eye detection technology, and the execution terminal of the target operation is used to judge the state of the eye centroid coordinates, and then determine the mental state of the human body. Slow motion refers to inserting frames to slow down the playback of the image. In the process of processing static video, when the target object in the preview screen is detected to have an activated state, the execution method of the target operation disclosed in the present disclosure is used to realize the slow motion function. Intelligent focus operation refers to the video preview process, by detecting that the target object changes in the motion state and body state, so as to obtain the position information of the target and focus the focus center on the target object. Continuous capture refers to recommending photos of highlight moments, but the interval between the continuous capture image frames should not be too large, otherwise it may cause tracking failure. Continuous capture can be achieved by setting the execution time interval of the target operation of the preview screen.

[0106] Second, refer to Fig.10 , the embodiment of the present disclosure provides an execution terminal for a target operation, which includes:

[0107] One or more processors 1001;

[0108] A memory 1002 having one or more programs stored thereon, which, when executed by one or more processors, enables the one or more processors to implement the execution method of any one of the above target operations;

[0109] One or more I / O interfaces 1003 are connected between the processor and the memory and are configured to implement information exchange between the processor and the memory.

[0110] Among them, the processor 1001 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 1002 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory (FLASH); the I / O interface (read-write interface) 1003 is connected between the processor 1001 and the memory 1002, and can realize information interaction between the processor 1001 and the memory 1002, including but not limited to a data bus (Bus), etc.

[0111] In some embodiments, the processor 1001 , the memory 1002 , and the I / O interface 1003 are connected to each other via a bus 1004 , and further connected to other components of the computing device.

[0112] Fig.10 The structure shown is for illustration only and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0113] Thirdly, refer to Fig.11 The embodiment of the present disclosure provides a computer-readable medium on which a computer program is stored. When the program is executed by a processor, the execution method of any one of the above-mentioned target operations is implemented.

[0114] In order to enable those skilled in the art to more clearly understand the technical solutions provided by the embodiments of the present disclosure, the technical solutions provided by the embodiments of the present disclosure are described in detail below through specific embodiments:

[0115] Reference Fig.12 , taking the camera as the execution terminal of the target operation, taking a snapshot of the target object, the kitten, in the preview screen to be operated, is achieved through the following steps:

[0116] Step 1201, place the camera and start it, select the target object to be captured as a kitten (of course, it can also be other subjects in the preview screen, and the kitten is used as an example here for explanation).

[0117] In step 1202, the camera terminal detects whether the specified target object (ie, the kitten) exists in the current camera preview image according to the set target object.

[0118] Step 1203: When a kitten is detected, the kitten is marked by a tracking frame, the preview image of the current frame is marked as the t0 frame image, and the position coordinates of the kitten in the preview image are determined as (x t0 ,y t0 ), the width of the tracking box occupied is Rect w_t0 , height is Rect h_t0 , stop detecting the target object and enable the target tracking algorithm, where the width and length represent the morphological information of the target object. If the kitten is not detected, the camera continues to detect.

[0119] Step 1204, refer to Fig.13 , update the position coordinates, length and width of the kitten in the preview screen in real time. For example, the current frame is frame t1, and the position coordinates of frame t1 are obtained by tracking algorithm as (x t1 ,y t1 ), the width of the tracking box occupied is Rect w_t1 , height is Rect h_t1 At the same time, the position coordinates and shape information of the preview image of frame t0 and frame t1 are stored in the array.

[0120] Step 1205, perform motion state analysis, and calculate the state change information used to characterize the change of the motion center of mass by formula (3):

[0121] ΔT center [t1] = abs(Rect w_t1 -Rect w_t0 )+abs(Rect h_t1 -Rect h_t0 ) (3)

[0122] The state change information used to characterize the motion displacement change is calculated by formula (4):

[0123] ΔT dist [t1] = abs(x t0 -x t1 )-abs(y t0 -y t1 ) (4)

[0124] Store ΔT for each frame center and T dist , that is, ΔT center [t1, t2, t3, …, tn] and ΔT dist[t1, t2, t3, …, tn], where t1, t2, t3, …, tn represent preview images of n sequence frames, and the state change information of the target object in the current frame image is estimated based on the saved data.

[0125] According to the first preset state change threshold T0 used to characterize the motion displacement change threshold dist_thresh , and a second preset state change threshold T for characterizing the motion center of mass change threshold center_thresh , dual threshold combination control is used to stimulate automatic photography.

[0126] Step 1206: for the current continuous frames of images satisfying ΔT center [tk]≥T center_thresh and ΔT dist [tk]≥T0 dist_thresh Any of the following conditions can automatically trigger the photo-taking process, where tk represents the tkth frame image in the shooting process.

[0127] In the above process, relevant instructions are automatically sent to the terminal device according to the detection results in the preview screen. When it is predicted that the motion state of the target object is about to change, the terminal to be executed is prepared, the camera is turned on, and object detection is first used to determine whether there is a corresponding target object in the current frame content, and the position and morphological information of the target object is obtained. The target tracking algorithm is enabled to keep tracking the target object, and the position and width and height information of the target are predicted and updated in real time (the width and height information can be used to calculate the center of mass coordinates of the target, thereby representing the morphological information), and the motion state of the target in the current frame is determined based on the real-time updated data information.

[0128] The embodiments of the present disclosure can be used to automatically realize the function of snapshot, and can capture many inadvertent wonderful moments in life and shoot shocking images.

[0129] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0130] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A method for executing a target operation, the method comprising: When it is confirmed that there is a target object in the preview screen, determining status change information of the target object within the preview screen; The status change information includes at least one of a change in the motion state of the target object within the preview screen and a change in the form of the target object within the preview screen; When the status change information of the target object meets a preset condition, executing the target operation.

2. The method for executing a target operation according to claim 1, wherein: The determining the status change information of the target object within the preview screen includes: Obtaining status parameters of the target object in multiple frames of the preview screen; According to the status parameters, determining the status change information of the target object in the multiple frames of the preview screen.

3. The method for executing a target operation according to claim 2, wherein: The determining the status change information of the target object in the multiple frames of the preview screen according to the status parameters includes: According to the status parameters of the target object in multiple frames of the preview screen, determining the status change information of the target object in the last frame of the multiple frames of the preview screen.

4. The method for executing a target operation according to claim 3, wherein: When the total number of frames of the multiple frames of the preview screen is N and the status change information includes a change in the motion state of the target object within the preview screen, determining the status change information of the target object in the last frame of the multiple frames of the preview screen according to the status parameters of the target object in multiple frames of the preview screen includes: Determining at least two first absolute differences between the position parameter of the Nth frame and the position parameter of the ith frame, where N and i are both positive integers, 0 < i < N, and the first absolute difference represents the change in the motion displacement of the same target object from the ith frame of the preview screen to the Nth frame of the preview screen; According to the difference between at least two of the first absolute differences, determining the change in the motion state of the target object in the Nth frame of the preview screen, where the Nth frame of the preview screen is the last frame of the multiple frames of the preview screen with a total number of frames of N, and the change in the motion state is used to represent the change in the motion displacement.

5. The method for executing a target operation according to claim 3, wherein: When the total number of frames of the multiple frames of the preview screen is N and the status change information includes a change in the form of the target object within the preview screen, determining the status change information of the target object in the last frame of the multiple frames of the preview screen according to the status parameters of the target object in multiple frames of the preview screen includes: Determining at least two second absolute differences between the centroid parameter of the Nth frame and the centroid parameter of the jth frame, where N and j are both positive integers, 0 < j < N, and the second absolute difference represents the change in the motion centroid of the same target object from the jth frame of the preview screen to the Nth frame of the preview screen; According to the sum value of at least two of the second absolute differences, determining the change in the form of the target object in the Nth frame of the preview screen, where the Nth frame of the preview screen is the last frame of the multiple frames of the preview screen with a total number of frames of N, and the change in the form represents the change in the motion centroid.

6. The method for executing a target operation according to claim 2, wherein: The when the status change information of the target object meets a preset condition, executing the target operation includes: Comparing the status change information of the target object corresponding to at least one frame of the multiple frames of the preview screen with a preset status change threshold; When the state change information of the target object corresponding to the at least one frame of the preview screen is greater than or equal to a preset state change threshold, it is confirmed that the state change information of the target object meets a preset condition, and the target operation is performed.

7. The method for executing a target operation according to claim 6, wherein: When the state change information of the target object corresponding to the at least one preview frame is greater than or equal to a preset state change threshold, confirming that the state change information of the target object meets a preset condition and performing the target operation includes: Comparing the state change information of the target object in the last preview frame of the multiple preview frames with a preset state change threshold; When the state change information of the target object in the last frame preview picture is greater than or equal to a preset state change threshold, confirming that the state change information of the target object meets a preset condition; The target operation is performed on the last frame preview picture.

8. The method for executing a target operation according to claim 6, wherein: When the state change information of the target object corresponding to the at least one preview frame is greater than or equal to a preset state change threshold, confirming that the state change information of the target object meets a preset condition and performing the target operation includes: Comparing the state change information of the target object corresponding to the first preset number of preview frames before the last preview frame in the multiple preview frames with a preset state change threshold; In the case where a second preset number of the state change information of the target object corresponding to the first preset number of frames of preview screen exists and is greater than or equal to a preset state change threshold, confirming that the state change information of the target object meets a preset condition, wherein both the first preset number and the second preset number are less than the total number of frames of the multi-frame preview screen, and the first preset number is greater than or equal to the second preset number; The target operation is performed on the last frame preview picture.

9. The method for executing a target operation according to any one of claims 6 to 8, wherein: The method for executing the target operation also includes: In response to the second input instruction, the preset state change threshold is determined to be a first preset state change threshold for characterizing a motion displacement change threshold, and / or a second preset state change threshold for characterizing a motion center of mass change threshold.

10. The method for executing a target operation according to any one of claims 1 to 8, wherein: Before determining the state change information of the target object in the preview screen, the method further includes: determining at least one trackable object within the preview frame; In response to a first input instruction, a trackable object corresponding to the first input instruction is determined as the target object.

11. The method for executing a target operation according to any one of claims 1 to 8, wherein: The executing target operation includes: Execute at least one of shooting, fatigue detection, slow motion, focus, and continuous capture.

12. A target operation execution terminal, comprising: one or more processors; A memory having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the execution method of the target operation according to any one of claims 1 to 11.

13. A computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method for executing the target operation according to any one of claims 1 to 11 is implemented.

Images