Target object identification method and device, video generation method and device, equipment and storage medium

Abstract

The invention provides a target object recognition method and device, a video generation method and device, equipment and a storage medium, and the target object recognition method comprises the steps: obtaining a target video, and determining a target video frame used for representing the motion track of a target object from the target video; in the target video frame, identifying at least two feature sequences related to the target object, different feature sequences representing different types of features of the target object respectively; wherein each feature sequence corresponds to the respective quality information sequence, and the quality information sequence is used for representing the quality of each feature in the corresponding feature sequence; and according to the quality information sequence, respectively determining respective representative features from each feature sequence, respectively matching each representative feature with a pre-stored base library feature of a corresponding type, and identifying the target object based on a matching result. According to the technical scheme provided by the invention, the recognition accuracy of the target object can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to a target object recognition, video generation method, device, equipment and storage medium. Background Art

[0002] In some practical application scenarios, it is often necessary to identify target objects in images or videos. However, due to the variability of the actual application scene environment and the uneven quality of video frames, the recognition accuracy of existing recognition algorithms is not high. Summary of the invention

[0003] In view of this, one or more embodiments of the present disclosure provide a method, apparatus, device, and storage medium for identifying a target object and generating a video, which can improve the recognition accuracy of the target object.

[0004] On the one hand, the present disclosure provides a method for identifying a target object, the method comprising: acquiring a target video, and determining a target video frame for characterizing a motion trajectory of the target object from the target video; in the target video frame, identifying at least two feature sequences related to the target object, different feature sequences respectively characterizing different types of features of the target object; wherein each feature sequence corresponds to a respective quality information sequence, and the quality information sequence is used to characterize the quality of each feature in the corresponding feature sequence; according to the quality information sequence, determining respective representative features from each feature sequence, and matching each representative feature with a pre-stored underlying database feature of the corresponding type, and identifying the target object based on the matching result.

[0005] On the other hand, the present disclosure also provides a device for identifying a target object, the device comprising: a video frame determination unit, used to acquire a target video, and determine a target video frame for characterizing a motion trajectory of the target object from the target video; a feature sequence identification unit, used to identify at least two feature sequences related to the target object in the target video frame, different feature sequences respectively characterizing different types of features of the target object; wherein each feature sequence corresponds to a respective quality information sequence, and the quality information sequence is used to characterize the quality of each feature in the corresponding feature sequence; an object identification unit, used to determine respective representative features from each feature sequence according to the quality information sequence, and match each representative feature with a pre-stored underlying library feature of the corresponding type, and identify the target object based on the matching result.

[0006] On the other hand, the present disclosure further provides an electronic device, which includes a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the above-mentioned target object recognition method is implemented.

[0007] On the other hand, the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the above-mentioned target object recognition method is implemented.

[0008] On the other hand, the present disclosure also provides a video generation method, which includes: in response to a user's video generation request, obtaining multiple target videos; based on the above-mentioned target object recognition method, identifying the target object contained in each of the target videos; determining the target object matching the user from the identified target objects; extracting the to-be-processed video frames of the target object matching the user from the multiple target videos, and generating the video of the user based on the to-be-processed video frames.

[0009] The technical solution provided by one or more embodiments of the present disclosure can identify multiple different types of feature sequences related to the target object in the target video frames after determining multiple target video frames that characterize the motion trajectory of the target object from the target video. These different types of feature sequences can describe the target object from different angles. For each feature sequence, it may include features of corresponding types identified from each target video frame. In addition, each feature sequence has its own quality information sequence, and the quality information sequence can characterize the quality of each feature in the feature sequence. Subsequently, based on the quality information sequence, a representative feature can be determined for each feature sequence, and the representative feature can be used as the feature in the feature sequence that best reflects the characteristics of the target object. By providing representative features that meet the quality requirements, the recognition result of the target object is more accurate.

[0010] After determining the representative features of each feature sequence, each representative feature can be matched with the corresponding type of base database features in turn. The base database features can have their own identifiers. By matching the representative features with the base database features, the identifiers corresponding to each representative feature can be characterized by the matching results. In this way, by statistically analyzing the matching results of each representative feature, the target object can be identified.

[0011] It can be seen from the above description that the technical solution provided by one or more embodiments of the present disclosure can obtain multiple feature sequences that describe the target object from different angles by analyzing the motion trajectory of the target object. In combination with the quality feature sequence, the representative features of the feature sequence can be obtained, and feature matching can be performed based on the representative features in each feature sequence, so as to obtain a more accurate recognition result. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The features and advantages of the embodiments of the present disclosure may be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the present disclosure in any way. In the accompanying drawings:

[0013] Figure 1 A schematic diagram of the steps of a method for identifying a target object in one embodiment of the present disclosure is shown;

[0014] Figure 2 A schematic diagram showing feature matching in one embodiment of the present disclosure is shown;

[0015] Figure 3 A schematic diagram of identifying a target object in one embodiment of the present disclosure is shown;

[0016] Figure 4 A schematic diagram of functional modules of a target object recognition device in one embodiment of the present disclosure is shown;

[0017] Figure 5 A schematic diagram of the structure of an electronic device in one embodiment of the present disclosure is shown;

[0018] Figure 6 A schematic diagram showing the steps of a video generation method in one embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0020] In some scenarios where target objects need to be identified, the recognition accuracy may be low due to reasons such as the target objects moving too fast or their clothing being similar.

[0021] In view of this, the present disclosure provides a method for identifying a target object, which can generate multiple different types of feature sequences related to the target object, obtain multiple representative features of the target object from different angles, and thus improve the recognition accuracy of the target object.

[0022] See also Figure 1 , a method for identifying a target object provided by an embodiment of the present disclosure may include the following steps.

[0023] S1: Acquire a target video, and determine a target video frame for representing a motion trajectory of a target object from the target video.

[0024] In this embodiment, the target object may be the user to be identified. It is understandable that before using the technical solutions disclosed in each embodiment of the present disclosure, the type, scope of use, and usage scenarios of the personal information involved in the present disclosure should be informed to the user in an appropriate manner in accordance with relevant laws and regulations and the user's authorization should be obtained.

[0025] For example, before implementing the technical solutions of various embodiments of the present disclosure, a prompt message may be sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0026] As an optional but non-limiting implementation, the prompt information can be sent to the user in a pop-up window, in which the prompt information can be presented in text. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree".

[0027] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0028] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0029] It can be understood that in the specific implementation of this application, identity information, video data and other related data are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards.

[0030] In this embodiment, through the camera deployed in the offline scene, when the target object is active in the offline scene, the target video containing the target object can be collected. Of course, the subsequent processing of the target video to identify the target object is carried out after the authorization of the target object has been obtained. In practical applications, the target video can be analyzed frame by frame by the object detection algorithm. For any video frame in the target video, after processing with the object detection algorithm, the position information of one or more bounding boxes corresponding to the video frame can be output. Among them, each bounding box can represent a human body in the video frame, and the human body existing in the video frame can be identified and located through the position information of the bounding box. In addition, after passing the object detection algorithm, each bounding box representing the human body will have its own bounding box identifier, which can be a globally unique number, for example. After object detection processing is performed on different video frames of the target video, the position information and bounding box identifier of the above-mentioned bounding box can be output for each video frame. Among them, bounding boxes with the same bounding box identifier can represent the same human body. In this way, after detecting the target video frame by frame, the motion trajectory of the same human body can be tracked in different video frames by identifying the bounding boxes with the same bounding box identifier.

[0031] In this embodiment, the target object can be any human body to be identified in the target video. In the target video, the target object may only appear in part of the video frames and be identified by the object detection algorithm, and then this part of the video frames can be used as the target video frames representing the motion trajectory of the target object. In this way, by performing frame-by-frame object detection on the target video, the target video frames representing the motion trajectory of the target object can be determined in the target video.

[0032] It should be noted that for the same target video, it may include multiple target objects to be identified. Each target object can be processed by the above-mentioned object detection algorithm. Subsequently, multiple bounding box sets can be obtained by classification according to the bounding box identification, and different bounding box sets can correspond to different target objects. Accordingly, each target object will have its own target video frame representing the motion trajectory in the target video, and the target object can be identified by analyzing the target video frame of the target object.

[0033] It can be seen that when the target video contains multiple target objects, these target objects can be identified one by one. In the embodiments of the present disclosure, one of the target objects can be used as an example to illustrate the solution, and the identification process of other target objects can be deduced in the same way.

[0034] In one embodiment, after acquiring the target video, the above-mentioned object detection algorithm can be used to identify each candidate object contained in the target video, and determine the human body quality information of each candidate object in the target video. Among them, a human posture recognition (Pose Estimation) algorithm can be used to identify the posture information corresponding to the human body features of the candidate object. By comparing the posture information with the standard posture information, the human body quality information of the candidate object can be generated. Among them, the closer the recognized posture information is to the standard posture information, the better the quality of the human body quality information representation. In addition, the human body quality information can also be represented by parameters such as image contrast, image brightness, and image blur. According to the quality level represented by the human body quality information, each candidate object can be screened, so as to determine the candidate object whose quality level meets the requirements in the target video. These candidate objects whose quality levels meet the requirements can be used as the target objects to be identified in the target video. Subsequently, the target video frame used to represent the motion trajectory of the target object can be determined from the target video. By screening the candidate objects in the target video through the human body quality information, the candidate objects whose image quality does not meet the requirements can be filtered out, so that the subsequent recognition accuracy of the target object is higher.

[0035] S3: In the target video frame, identify at least two feature sequences related to the target object, and different feature sequences respectively characterize different types of features of the target object; wherein each feature sequence corresponds to its own quality information sequence, and the quality information sequence is used to characterize the quality of each feature in the corresponding feature sequence.

[0036] In this embodiment, after the target video frame is screened out from the target video, a more refined feature recognition can be performed on the target object in the target video frame. Specifically, different types of feature sequences related to the target object can be extracted from different angles. Taking the skiing scene as an example, the above different types of feature sequences can include at least one of a head feature sequence, a human feature sequence, and a clothing icon feature sequence. The number of feature sequences can be flexibly determined according to the requirements of identity recognition accuracy and data processing efficiency in actual application scenarios.

[0037] For example, in a specific application example, the above-mentioned target video frames can be processed frame by frame, and the head image features, body features and clothing icon features of the target object can be extracted from each target video frame. By summarizing the different types of features extracted from each target video frame, three different types of feature sequences (head image feature sequence, body feature sequence, clothing icon feature sequence) can be obtained.

[0038] In one embodiment, taking the feature sequence of any target type as an example, the type features related to the target object and conforming to the target type can be identified from each target video frame. In practical applications, the above-mentioned type features can be extracted from the target video frame using a feature extraction network. Specifically, the feature extraction network can be flexibly selected from a variety of different network models according to the needs of the actual application scenario. The above-mentioned various different network models may include, for example, a VGG model, an Inception model, a ResNet model, etc. Before the target video frame is input into the feature extraction network, the target video frame can be cropped to different degrees based on the different target types. For example, when the target type is an avatar type, the head area of ​​the target object can be cropped from the target video frame, and the cropped area image can be input into the feature extraction network, thereby ensuring that the features output by the feature extraction network are avatar features. Similarly, if the target type is a human body type, it can be cropped directly according to the bounding box of the target object in the target video frame, and the cropped area image can be input into the feature extraction network.

[0039] In this embodiment, after the corresponding type features are identified from each target video frame, the feature sequence composed of the identified type features can be used as the feature sequence of the target type. For different target types, respective feature sequences can be generated in a similar manner. It can be seen that a feature sequence can include multiple type features of the same type. For example, in an avatar feature sequence, avatar features of the target object identified from each target video frame can be included. For another example, in a clothing icon feature sequence, clothing icon features of the target object identified from each target video frame can be included.

[0040] In one embodiment, when extracting features from a target video frame and generating different types of type features, quality information of the type features can also be generated. The quality information of the type features can characterize the quality of the currently extracted type features. For example, in some examples, the quality information can characterize at least one of the contrast and brightness of each type of feature. In practical applications, the target object in the target video frame may be blocked or move too fast, resulting in the type features of the target object extracted from the target video frame being not ideal, and the quality information can quantitatively evaluate the quality of each type of feature. Different models can be used to generate quality information of type features for different type features.

[0041] For example, for avatar features, a megaface model can be used to output quality information of avatar features. For human body features, a human posture recognition (Pose Estimation) algorithm can be used to identify posture information corresponding to human body features. By comparing the posture information with the standard posture information, quality information corresponding to the human body features can be generated. Among them, the closer the recognized posture information is to the standard posture information, the better the quality of the quality information representation of the human body features. For clothing icon features, quality information can be represented by parameters such as image contrast and image brightness. In this way, for different types of type features, corresponding quality information can be generated. In practical applications, by combining the feature extraction model and the quality information generation model, when generating different types of feature sequences, the quality information of each type of feature in the feature sequence can also be generated together. In this way, the quality information of each type of feature can constitute a quality information sequence of the feature sequence. Among them, different feature sequences can correspond to different quality information sequences. For the corresponding feature sequence and quality information sequence, each quality information in the quality information sequence can correspond one-to-one to each type of feature in the feature sequence, so as to represent the quality of each type of feature.

[0042] S5: According to the quality information sequence, determine the respective representative features from each of the feature sequences, match each of the representative features with the pre-stored base library features of the corresponding type, and identify the target object based on the matching results.

[0043] In this embodiment, when a user enters an offline scene, image information of the user that meets the preset rules can be collected with the user's authorization. For example, when a visitor enters an activity venue, he or she can take a clear frontal photo under the guidance of the staff. The above preset rules can be limited to the user's clothing, posture and other information when collecting image information.

[0044] In one embodiment, after obtaining the current image information of the user, the human body quality information of the user in the current image information can be identified in the manner described in step S1. Then, it can be determined whether the quality level of the current image information meets the standard based on the human body quality information. If it meets the standard, the current image information can be used as image information that complies with the preset rules; if it does not meet the standard, the user's image information can be re-collected. In other words, only when the quality level represented by the human body quality information reaches the quality level specified by the preset rules, the current image information will be used as image information that complies with the preset rules, otherwise the image information will be re-collected. This ensures that image information that meets the preset rules can be collected, providing an accurate data basis for the subsequent construction of the base library features.

[0045] In this embodiment, after collecting the image information of the user that meets the preset rules, at least two preset types of type features related to the user can be identified from the collected image information according to the feature extraction method in the aforementioned steps. Among them, the at least two preset types identified correspond to the types represented by the at least two feature sequences extracted in step S3. For example, the user's head portrait features, body features and clothing icon features can be extracted from the collected image information. Since the image information is collected under relatively ideal conditions, the type features of each preset type identified from the image information can be used as the user's base library features, and the base library features can accurately represent the user's features. At the same time, after collecting the user's base library features, the user's standard identifier can also be bound to these base library features. The standard identifier can be, for example, the user's name, or a number representing the user's name. In this way, for the same user, multiple base library features extracted based on the user's image information can be bound to the user's annotation identifier. Among them, the standard identifier can be entered by the user after obtaining the user's authorization when collecting the image information.

[0046] Through the above method, any user can obtain the user's image information that meets the preset rules. Subsequently, at least two preset types of type features related to the user can be identified from the image information, and each of the identified type features is used as the user's base database feature. At the same time, the user's base database feature has a binding relationship with the user's standard identifier.

[0047] In this embodiment, after identifying multiple different types of feature sequences of the target object from the target video frame, a representative feature can be determined from each feature sequence. Generally speaking, the representative feature can be the feature with the best quality in the feature sequence of the corresponding type, or it can be the result of the weighted summation of the features of each type in the feature sequence. Subsequently, the target object can be identified based on the representative features in each feature sequence.

[0048] In one embodiment, when determining the representative features of each feature sequence, taking any target feature sequence as an example, the quality information of each type of feature in the target feature sequence can be counted. The quality information can usually be a numerical value that can characterize the quality. Based on the statistical quality information, the type feature with the best quality can be used as the representative feature of the target feature sequence. The quality can be determined by the numerical value. The larger the numerical value, the better the quality. Of course, if there are multiple types of features with the best quality in the same feature sequence, then one can be randomly selected from the multiple types of features with the best quality as the representative feature of the feature sequence.

[0049] In addition, in one embodiment, the weight value of each type of feature in the target feature sequence can also be determined based on the statistical quality information. The higher the quality level represented by the quality information, the higher the corresponding weight value can be. Subsequently, the weighted fusion of each type of feature in the target feature sequence can be performed based on the weight value to obtain the fused representative feature.

[0050] See also Figure 2 In this embodiment, after determining multiple representative features of the target object, each representative feature can be matched with the corresponding type of base database features in turn, so as to determine the base database features that match the representative features in the pre-stored base database features. Specifically, taking any target representative feature as an example, the target base database features that belong to the same type as the target representative feature can be first determined in the pre-stored base database features. For example, assuming that the type of the target representative feature is an avatar type, when performing feature matching, the target representative feature will only be matched with the base database features that represent the avatar features, and will not be matched with the base database features that represent the human body features.

[0051] In practical applications, the matching process between the representative feature and the base database feature can be performed by calculating the similarity. Specifically, the similarity between the target representative feature and each target base database feature of the same type can be calculated respectively, and based on the similarity and a preset threshold value corresponding to the type of the target representative feature, the matching result of the target representative feature in the pre-stored base database features is determined.

[0052] In practical applications, the highest similarity can be determined from the calculated similarities, and when the highest similarity is greater than or equal to the preset threshold, the target base database feature corresponding to the highest similarity is used as the base database feature that matches the target representative feature; wherein, different types of target representative features may have different corresponding preset thresholds. Wherein, both the target representative feature and the target base database feature can be represented in the form of a vector, then the similarity between the target representative feature and the target base database feature can be represented by the Euclidean distance between the two vectors. The closer the Euclidean distance, the more similar the two features are.

[0053] That is to say, in practical applications, if the highest value of the calculated similarity is still lower than a certain threshold, it means that the representative feature is not very matched with the pre-stored base feature. In this case, it is not suitable to use the base feature with the highest similarity as the base feature that matches the representative feature. In view of this, when determining the target base feature with the highest similarity, it can be further determined whether the highest similarity is greater than or equal to the preset threshold corresponding to the type of the target representative feature. If it is greater than or equal to the preset threshold, the target base feature with the highest similarity can be used as the base feature that matches the target representative feature. However, if the highest similarity is less than the preset threshold, it can be determined that the target representative feature does not have a matching base feature. Of course, for different types of target representative features, the corresponding preset thresholds can also be different. For example, in a skiing scene, since the target object wears corresponding skiing gear (such as a helmet and / or goggles) on its head, it will cause the head features of different target objects to have a higher similarity. In this case, in order to match the accurate base feature, it is necessary to set the preset threshold corresponding to the head feature to be relatively high, so as to obtain an accurate matching result.

[0054] In one embodiment, after matching the base database features with each representative feature, a corresponding matching result can be obtained. When identifying the target object based on the matching result, it can be determined whether each representative feature has its own matching base database feature. If each representative feature has its own matching base database feature, then the standard identifiers of the matching base database features can be counted, and then the target object can be identified based on the counted standard identifiers.

[0055] See also Figure 3 Specifically, if the statistically obtained standard identifiers are the same, then the standard identifier can be used as the recognition result of the target object. For example, after the three representative features of the target object are matched with the base database features, the matching base database features A, B, and C can be obtained. The standard identifiers bound to these three base database features A, B, and C all point to user M, so user M can be used as the recognition result of the target object.

[0056] On the other hand, if the statistically obtained standard identifiers are not exactly the same, then the users corresponding to the representative features are also inconsistent, and a prompt message representing that the target object cannot be identified can be generated at this time. For example, after the three representative features of the target object are matched with the bottom library features, matching bottom library features D, E, and F can be obtained, but the standard identifiers bound to the bottom library features D and E point to user K, while the standard identifier bound to the bottom library feature F points to user L, indicating that there is a contradiction in the matching result and the target object cannot be identified. The target object can be further identified through other video data later, but in the data processing process of the current round, the recognition result of the target object cannot be obtained. When the recognition result of the target object cannot be obtained, it indicates that the target object has not completed registration in advance. In other words, for the same target object, in response to the matching result that any representative feature related to the target object does not match the bottom library feature, the target object is determined to be an object that has not been pre-registered.

[0057] In addition, in one embodiment, considering that not every representative feature can search for a matching base database feature, when identifying a target object, it is possible to first determine whether each representative feature has a matching base database feature. If at least one representative feature among the representative features does not have a matching base database feature, a prompt message indicating that the target object cannot be identified can be directly generated. In this way, through a preliminary detection of the matching results, it is possible to directly determine the situation where the identification result cannot be obtained, thereby avoiding the subsequent statistical process for standard identification and reducing the complexity of data processing.

[0058] In this embodiment, after the target object is identified, the target object's play highlights can be edited from the target video, and the edited play highlights can be pushed to the target object's terminal device.

[0059] In a specific application example, users who enter an activity venue can complete the registration of their identity information in the application or applet provided by the operator of the activity venue. During the registration process, users can take images that meet the preset rules. After the image information is processed in the background, the user's head features, body features, and clothing icon features can be generated, and these three features are bound to the user's registered identity information as the base database features and stored in the database.

[0060] When users are playing in the activity venue, the cameras deployed in the activity venue can shoot the user's play video and upload the shot video to the background for processing. When the background processes the video, it can use the object detection algorithm to determine the human body contained in the video frame frame by frame. Subsequently, the identified human body can be tracked to form the movement trajectory of the human body. For the movement trajectories of different human bodies, three different types of feature sequences, namely, head feature sequence, human feature sequence, and clothing image sequence, and the quality information sequence corresponding to each feature sequence can be generated. By statistically analyzing the quality information in the quality information sequence, the representative features of each feature sequence can be determined from different types of feature sequences.

[0061] By matching the representative features with the underlying features in the database, the identity information of the user corresponding to the current motion trajectory can be determined. Later, when a user initiates a video editing request to the background through the video editing function provided by the application or applet, the background can obtain the user's identity information, and query the motion trajectory that matches the user's identity information from the previously processed motion trajectory, and perform video editing based on the motion trajectory, thereby providing the user with the edited video highlights. This ensures that the video highlights provided to the user mainly contain the user's video footage.

[0062] As can be seen from the above, the technical solution provided by one or more embodiments of the present disclosure can identify multiple different types of feature sequences related to the target object in the target video frames after determining multiple target video frames that characterize the motion trajectory of the target object from the target video. These different types of feature sequences can describe the target object from different angles. For each feature sequence, it may include features of corresponding types identified from each target video frame. In addition, each feature sequence has its own quality information sequence, and the quality information sequence can characterize the quality of each feature in the feature sequence. Subsequently, based on the quality information sequence, a representative feature can be determined for each feature sequence, and the representative feature can be used as the feature in the feature sequence that best reflects the characteristics of the target object. By providing representative features that meet the quality requirements, the recognition result of the target object is more accurate.

[0063] After determining the representative features of each feature sequence, each representative feature can be matched with the corresponding type of base database features in turn. The base database features can have their own identifiers. By matching the representative features with the base database features, the identifiers corresponding to each representative feature can be characterized by the matching results. In this way, by statistically analyzing the matching results of each representative feature, the target object can be identified.

[0064] It can be seen from the above description that the technical solution provided by one or more embodiments of the present disclosure can obtain multiple feature sequences that describe the target object from different angles by analyzing the motion trajectory of the target object. In combination with the quality feature sequence, the representative features of the feature sequence can be obtained, and feature matching can be performed based on the representative features in each feature sequence, so as to obtain a more accurate recognition result.

[0065] See also Figure 4 In one embodiment of the present disclosure, a device for identifying a target object is provided, the device comprising:

[0066] The video frame determination unit 100 is used to obtain a target video and determine a target video frame for representing a motion trajectory of a target object from the target video;

[0067] The feature sequence recognition unit 200 is used to recognize at least two feature sequences related to the target object in the target video frame, and different feature sequences respectively represent different types of features of the target object; wherein each feature sequence corresponds to a respective quality information sequence, and the quality information sequence is used to represent the quality of each feature in the corresponding feature sequence;

[0068] The object identification unit 300 is used to determine the respective representative features from each of the feature sequences according to the quality information sequence, match each of the representative features with the pre-stored underlying database features of the corresponding type, and identify the target object based on the matching results.

[0069] Among them, the specific processing logic of each functional module can be found in the description of the aforementioned method implementation method, which will not be repeated here.

[0070] Each unit described in the above embodiments may be implemented by a computer chip or a product having a certain function. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0071] For the convenience of description, the above devices are described in terms of functions and are described separately in various units. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0072] See also Figure 5An embodiment of the present disclosure further provides an electronic device, which includes a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the above-mentioned target object recognition method is implemented.

[0073] The present disclosure also provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the above-mentioned target object recognition method is implemented.

[0074] The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0075] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory, that is, implementing the method in the above method embodiment.

[0076] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0077] See also Figure 6 An embodiment of the present disclosure also provides a video generation method, which may include the following steps.

[0078] S11: In response to a user's video generation request, a plurality of target videos are acquired.

[0079] S31: Based on the above target object recognition method, the target object contained in each of the target videos is recognized.

[0080] S51: Determine a target object that matches the user from among the identified target objects.

[0081] S71: extracting to-be-processed video frames of a target object matching the user from the multiple target videos, and generating a video of the user based on the to-be-processed video frames.

[0082] In this embodiment, the user can send a video generation request to the server through the terminal device. The target video shot by the camera deployed in the offline scene can be stored in the server. After the server responds to the video generation request sent by the user's terminal device, it can obtain multiple target videos related to the user. Among them, the shooting period of the target video can overlap with the period of the user's activities in the offline scene. In addition, the offline scene covered by the target video can also overlap with the scene of the user's activities. In this way, it can be ensured that the user can be included in the video screen of the acquired target video.

[0083] After acquiring multiple target videos, for each target video, each target object contained in the target video can be identified in the manner described in steps S1 to S5, thereby obtaining the object identification of each target object. Subsequently, the target object that matches the user who initiated the video generation request can be determined from the multiple identified target objects. Specifically, when entering an offline scene, the user can complete registration and authorization in advance to obtain the user's identification and base library features. By comparing the object identifications of each identified target object, the target object corresponding to the user's identification can be obtained, and the target object with the user's identification can be used as the target object that matches the user. Subsequently, the video frame containing the target object can be extracted from the target video, and the extracted video frame can be edited to obtain the user's video highlights.

[0084] It can be seen that through the above-mentioned video generation method, video frames containing the user can be accurately extracted from the many target videos shot, and based on the extracted video frames, the user's video highlights can be automatically edited, realizing the intelligent generation process of video highlights.

[0085] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0086] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the embodiments of the device, equipment, and storage medium, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0087] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

[0088] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for identifying a target object, characterized in that: The method comprises: Acquire a target video, and determine a target video frame for representing a motion trajectory of a target object from the target video; In the target video frame, at least two feature sequences related to the target object are identified, and different feature sequences respectively characterize different types of features of the target object; wherein each feature sequence corresponds to a respective quality information sequence, and the quality information sequence is used to characterize the quality of each feature in the corresponding feature sequence; According to the quality information sequence, respective representative features are determined from each feature sequence, and each representative feature is matched with a pre-stored base database feature of a corresponding type, and the target object is identified based on the matching result.

2. The method according to claim 1, characterized in that Determining a target video frame for representing the motion trajectory of the target object from the target video includes: Identify each candidate object contained in the target video, and determine the body mass information of each candidate object in the target video; According to the quality level represented by the human body mass information, a target object to be identified is determined from each of the candidate objects, and a target video frame for representing the motion trajectory of the target object is determined from the target video.

3. The method according to claim 1 or 2, characterized in that: Each of the feature sequences contains multiple types of features; Determining respective representative features from each of the feature sequences comprises: For any target feature sequence of the at least two feature sequences, a target quality information sequence corresponding to the target feature sequence is acquired, and quality information of each type of feature in the target feature sequence is counted according to the target quality information sequence; Based on the statistical quality information, the type feature with the best quality is used as the representative feature of the target feature sequence; or based on the statistical quality information, the weight value of each type feature in the target feature sequence is determined, and based on the weight value, each type feature in the target feature sequence is weightedly fused to obtain the fused representative feature.

4. The method according to claim 1 or 2, characterized in that: Matching each of the representative features with pre-stored base database features of the corresponding type includes: For any target representative feature among the representative features, determine a target base database feature that is of the same type as the target representative feature from pre-stored base database features; The similarities between the target representative feature and each of the target base database features are calculated respectively, and based on the similarities and a preset threshold value corresponding to the type of the target representative feature, a matching result of the target representative feature in the pre-stored base database features is determined.

5. The method according to claim 4, characterized in that Determining the matching result of the target representative feature in the pre-stored base database features includes: Determine the highest similarity among the calculated similarities, and when the highest similarity is greater than or equal to the preset threshold, use the target base database feature corresponding to the highest similarity as the base database feature that matches the target representative feature; Among them, different types of target representation features have different corresponding preset thresholds.

6. The method according to claim 1, characterized in that Each of the pre-stored base database features has its own standard identifier; identifying the target object based on the matching result includes: For the same target object, if each of the representative features of the target object has a respective matching base database feature, the standard identifiers of the respective matching base database features are counted; The target object is identified according to the statistically obtained standard identifiers.

7. The method according to claim 6, characterized in that The method further comprises: For the same target object, in response to a matching result that any representative feature related to the target object does not match the base database feature, the target object is determined to be an object that is not pre-registered.

8. The method according to claim 1, characterized in that The pre-stored base features are generated in the following way: Obtain user's image information that meets preset rules; Identify at least two preset types of type features related to the user from the image information, and use the identified type features as the base library features of the user; wherein the at least two preset types correspond to the types represented by the at least two feature sequences; The base database feature of the user is bound to the standard identifier of the user.

9. The method according to claim 8, characterized in that Obtaining user image information that meets preset rules includes: Acquire current image information of a user, and identify body mass information of the user in the current image information; When the quality level represented by the human body quality information reaches the quality level defined by a preset rule, the current image information is regarded as image information that complies with the preset rule.

10. The method according to claim 1, characterized in that The different feature sequences include at least one of a head portrait feature sequence, a body feature sequence, and a clothing icon feature sequence.

11. A video generation method, characterized in that: The method comprises: In response to a user's video generation request, obtaining multiple target videos; Based on the target object recognition method according to claim 1, identifying the target object contained in each of the target videos; Determining a target object that matches the user from among the identified target objects; A to-be-processed video frame of a target object matching the user is extracted from the multiple target videos, and a video of the user is generated based on the to-be-processed video frame.

12. A target object recognition device, characterized in that: The device comprises: A video frame determination unit, used to acquire a target video and determine a target video frame for representing a motion trajectory of a target object from the target video; A feature sequence recognition unit, used to recognize at least two feature sequences related to the target object in the target video frame, where different feature sequences respectively represent different types of features of the target object; wherein each feature sequence corresponds to a respective quality information sequence, and the quality information sequence is used to represent the quality of each feature in the corresponding feature sequence; The object identification unit is used to determine the respective representative features from each of the feature sequences according to the quality information sequence, match each of the representative features with the pre-stored underlying database features of the corresponding type, and identify the target object based on the matching results.

13. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the method according to any one of claims 1 to 10 is implemented.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images