Target model establishing method and video processing equipment

Abstract

The embodiment of the invention provides a target model establishment method and video processing equipment, and relates to the technical field of security and protection. The method comprises the steps of recording first information whenever the first information used for indicating a first target and detecting a first image group to which a video frame of the first target belongs is received, searching video data of the first image group from recorded video data and analyzing the video data to obtain image data whenever the first image group meets a preset modeling condition, and sending the image data to a server; and establishing respective models of all the first targets in the first image group by using the image data obtained by the same analysis. By applying the embodiment of the invention, the frequency of analyzing the video data of the same image group can be reduced, the modeling time is shortened, and the efficiency of establishing the target model is further improved.

Description

Technical Field

[0001] The present application relates to the field of security technology, and in particular to a target model establishment method and video processing equipment. Background Art

[0002] As big model technology is increasingly used in the security industry, such as high-precision personnel recognition and vehicle or non-motor vehicle target detection based on videos or images, the realization of these intelligent functions all rely on the generation, storage and retrieval system of massive model data.

[0003] In related technologies, the features of the targets detected in the video images are generally extracted through a pre-trained model, and the model data of each target is established based on the extracted features. The process of establishing the model data can be regarded as the process of converting the features of the target into data that can be processed by the computer. In layman's terms, it is to convert the target features into a digital expression that can be understood by the computer. In different scenarios, there may be different requirements for target modeling due to different purposes or needs of target detection, which makes the data obtained by modeling different. For example: Common target features include: multimodal features (features that integrate two or more different modal information such as images, text, audio, depth maps, etc.), unimodal features (color features, texture features, shape features, local features, depth features, etc.) or behavioral features, etc., which will not be explained one by one here.

[0004] When building a model for each target, it is usually necessary to decode the GOP (Group of Pictures) corresponding to each target and then build a model for that target based on the decoded image data. However, this solution requires decoding a complete GOP before building a model for each target. If there are many video frames in a GOP, multiple frames must be decoded continuously to complete the decoding of the GOP of one target before decoding the GOP of the next target. This places high demands on the decoder's performance. If the decoder's performance is poor, decoding may be stuck, resulting in a low modeling rate. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide a method for establishing a target model and a video processing device to increase the speed of establishing the target model. The specific technical solution is as follows:

[0006] In a first aspect of an embodiment of the present application, a method for establishing a target model is provided, the method comprising:

[0007] recording, whenever first information indicating a first target and a first image group to which a video frame in which the first target is detected belongs, wherein the first target is at least one target detected from all video frames of the first image group, and the first information is generated after the first target is detected from a recorded video segment;

[0008] Whenever a first image group meets a preset modeling condition, searching for video data of the first image group from recorded video data and parsing to obtain image data, wherein the first image group is an image group in a recorded video clip;

[0009] The image data obtained by the same analysis is used to establish respective models of all first targets in the first image group.

[0010] In a possible implementation manner, searching for video data of the first image group from the recorded video data and parsing to obtain image data includes:

[0011] Determining, from among the generalized key frames of the first image group, an optimal generalized key frame for each of the first targets in the first image group, and determining the optimal generalized key frame ranked last in the first image group as the optimal generalized key frame for the target corresponding to the first image group; wherein the optimal generalized key frame is a video frame with the highest score among the generalized key frames of the first image group, the score of the video frame is positively correlated with the amount of information about the first target contained in the video frame, and the generalized key frame is a key frame of the first image group and a video frame whose temporal interval with the key frame is less than a preset interval threshold, or the generalized key frame is a key frame of the first image and a video frame whose number of frames between the key frame and the key frame is less than a preset frame number threshold;

[0012] Searching for video data of the first image group from the recorded video data, and starting with a key frame of the video data and ending with the target optimal generalized key frame, parsing the key frame, the target optimal generalized key frame, and all video frames between the key frame and the target optimal generalized key frame to obtain image data of each video frame;

[0013] The establishing of respective models of all first targets in the first image group using the image data of each video frame obtained from the same parsing includes:

[0014] Models of all first objects in the first image group are established using the image data of the optimal generalized key frame of each first object.

[0015] In a possible implementation, the using the image data of the optimal generalized key frame of each first object to establish the model of the first object itself includes:

[0016] If the score of the optimal generalized keyframe of the first object is higher than a preset score threshold, establishing a model of the first object using the image data of the optimal generalized keyframe of the first object;

[0017] The method further comprises:

[0018] If the score of the optimal generalized key frame of the first target is not higher than the preset score threshold, calculating the score difference between the optimal video frame of the first target and the optimal generalized key frame of the first target as the score difference of the first target; wherein the optimal video frame of the first target is the video frame with the highest score for the first target in the first image group;

[0019] If the score difference of the first target is lower than a preset difference threshold, a model of the first target is established using image data of the optimal generalized key frame of the first target.

[0020] In one possible implementation, the method further includes:

[0021] If the score difference is not lower than the preset difference threshold, the model of the first target is established using the image data of the optimal video frame.

[0022] In one possible implementation,

[0023] The preset scoring threshold is set according to the category of the first target, and the preset scoring threshold is pre-calculated in the following way:

[0024] Sort the modeled objects in order from latest to earliest to obtain an object sequence;

[0025] Searching, in order from front to back, for a preset number of targets of the same category as the first target in the object sequence as second targets;

[0026] Obtaining scores of the video frames used when building the model of each second target, and calculating a mean of the obtained scores;

[0027] The mean is determined as the scoring threshold, or the mean increased by a preset ratio is determined as the scoring threshold.

[0028] According to a second aspect of the embodiments of the present application, a video processing device is provided, wherein the video processing device is configured to:

[0029] recording, whenever first information indicating a first target and a first image group to which a video frame in which the first target is detected belongs, wherein the first target is at least one target detected from all video frames of the first image group, and the first information is generated after the first target is detected from a recorded video segment;

[0030] Whenever a first image group meets a preset modeling condition, searching for video data of the first image group from recorded video data and parsing to obtain image data, wherein the first image group is an image group in a recorded video clip;

[0031] The image data obtained by the same analysis is used to establish respective models of all first targets in the first image group.

[0032] In one possible implementation,

[0033] The video processing device searches for video data of the first image group from recorded video data and parses the video data to obtain image data, including:

[0034] Determining, from among the generalized key frames of the first image group, an optimal generalized key frame for each of the first targets in the first image group, and determining the optimal generalized key frame ranked last in the first image group as the optimal generalized key frame for the target corresponding to the first image group; wherein the optimal generalized key frame is a video frame with the highest score among the generalized key frames of the first image group, the score of the video frame is positively correlated with the amount of information about the first target contained in the video frame, and the generalized key frame is a key frame of the first image group and a video frame whose temporal interval with the key frame is less than a preset interval threshold, or the generalized key frame is a key frame of the first image and a video frame whose number of frames between the key frame and the key frame is less than a preset frame number threshold;

[0035] Searching for video data of the first image group from the recorded video data, and starting with a key frame of the video data and ending with the target optimal generalized key frame, parsing the key frame, the target optimal generalized key frame, and all video frames between the key frame and the target optimal generalized key frame to obtain image data of each video frame;

[0036] The establishing of respective models of all first targets in the first image group using the image data of each video frame obtained from the same parsing includes:

[0037] Models of all first objects in the first image group are established using the image data of the optimal generalized key frame of each first object.

[0038] In a possible implementation, the video processing device uses the image data of the optimal generalized key frame of each first object to establish a model of the first object itself, including:

[0039] If the score of the optimal generalized keyframe is higher than a preset score threshold, establishing a model of the first target using the image data of the optimal generalized keyframe;

[0040] The method further comprises:

[0041] If the score of the optimal generalized key frame is not higher than the preset score threshold, calculating the score difference between the optimal video frame and the optimal generalized key frame, wherein the optimal video frame is the video frame with the highest score in the first image group;

[0042] If the score difference is lower than a preset difference threshold, the model of the first target is established using the image data of the optimal generalized key frame.

[0043] In a possible implementation, the video processing device is further configured to, if the score difference is not lower than the preset difference threshold, establish a model of the first target using the image data of the optimal video frame.

[0044] In one possible implementation,

[0045] The preset scoring threshold is set according to the category of the first target, and the preset scoring threshold is pre-calculated in the following way:

[0046] Sort the modeled objects in order from latest to earliest to obtain an object sequence;

[0047] Searching, in order from front to back, for a preset number of targets of the same category as the first target in the object sequence as second targets;

[0048] Obtaining scores of the video frames used when building the model of each second target, and calculating a mean of the obtained scores;

[0049] The mean is determined as the scoring threshold, or the mean increased by a preset ratio is determined as the scoring threshold.

[0050] Beneficial effects of the embodiments of the present application:

[0051] An embodiment of the present application provides a target model establishment method and video processing device. Whenever first information indicating a first video frame, a first target, and a first image group to which the first video frame belongs is received, the first information is recorded accordingly. When the first image group meets the preset modeling conditions, the video data of the first image group is parsed to obtain image data, thereby using the image data parsed at the same time to establish respective models of all first targets in the first image group, without having to parse the video data of the image group corresponding to each first target separately. This reduces the number of times the video data of the same image group is parsed, shortens the modeling time, and thereby improves the efficiency of target model establishment.

[0052] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0054] Figure 1 A first schematic diagram of the target modeling method provided in an embodiment of the present application;

[0055] Figure 2 A second schematic diagram of the target modeling method provided in an embodiment of the present application;

[0056] Figure 3 An example diagram of an image group and a generalized key frame provided in an embodiment of the present application;

[0057] Figure 4 A third schematic diagram of the target modeling method provided in an embodiment of the present application;

[0058] Figure 5 A schematic diagram of a method for calculating a preset scoring threshold provided in an embodiment of the present application;

[0059] Figure 6 A flowchart of the target modeling method provided in an embodiment of the present application;

[0060] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of this application.

[0062] To more clearly illustrate the target modeling method provided in this application, the following are explanations of the relevant terms involved in this article:

[0063] Private frame: A private information frame added by the camera to the bitstream, containing target information such as target frame, target type, etc.

[0064] I-frame: also known as key frame or intra-coded frame, is a complete image frame that can be decoded independently without relying on information from other frames. It is similar to a static image and can be regarded as a reference point in a video sequence.

[0065] P-frames, also known as forward-predictive frames, are generated based on the previous I-frame or P-frame. P-frames store the image changes compared to the previous frame, so P-frames generally offer better compression than I-frames. Decoding a P-frame requires first decoding the I-frame or P-frame on which it depends, and then reconstructing the current frame based on this information.

[0066] In order to improve the speed of establishing a target model, the first aspect of the embodiment of the present application provides a method for establishing a target model, such as Figure 1 The first schematic diagram of the target model establishment method provided in the embodiment of the present application is shown, and the method includes the following steps:

[0067] Step S10, whenever first information indicating a first target and a first image group to which a video frame in which the first target is detected belongs is received, recording the first information;

[0068] The first target is at least one target detected from all video frames of the first image group, and the first information is generated after the first target is detected from the first video frame;

[0069] Step S20: whenever the first image group meets the preset modeling condition, searching for video data of the first image group from the recorded video data and parsing to obtain image data;

[0070] The first image group is an image group in a recorded video clip;

[0071] Step S30 : establishing respective models of all first objects in the first image group using the image data obtained from the same analysis.

[0072] By adopting the embodiment of the present application, whenever the first information for indicating the first target and the first image group to which the video frame in which the first target is detected is received, the first information will be recorded accordingly. When the first image group meets the preset modeling conditions, the video data of the first image group is parsed to obtain image data, thereby using the image data parsed at the same time to establish respective models of all the first targets in the first image group, without having to parse the video data of the image group corresponding to each first target separately, thereby reducing the number of times the video data of the same image group is parsed, shortening the modeling time, and thereby improving the efficiency of target model building.

[0073] In step S10, since the first information is used to indicate the first video frame, the first target, and the first image group to which the first video frame belongs, recording the first information actually records the association between the first target and the first image group. For example, if first information 1 is currently received, indicating target 1 and first image group 1, then first information 1 is recorded. First information 1 can reflect the correspondence between first target 1 and first image group 1 to which the video frame in which first target 1 is detected belongs. Therefore, what is actually recorded when recording the first information is the association between each first target and the image group to which the video frame containing the first target belongs. It is understood that the same first information is used to indicate one first image group and at least one first target.

[0074] In the above-mentioned step S20, the preset modeling condition may be that the first targets indicated by a continuous preset first number of first messages received within a preset time are not associated with the same first image group, or that the time from the moment when the first information indicating the same first image group was first recorded is equal to a preset time threshold, or that the first targets indicated by a continuous preset second number of first messages received are not associated with the same first image, or other custom conditions. All of these are possible, and the embodiments of the present application are not limited to this.

[0075] Since the first information indicating the first target and the first image group is recorded in step S10, and the video data of the first image group that meets the preset modeling conditions is searched from the recorded video data and parsed to obtain image data in step S20, each first target in the first image group can be modeled in step S30 using the image data obtained by the same parsing in step S20. It should be understood that each first target in the first image group refers to a target detected from a video frame in the first image group.

[0076] For example, assuming that, through step S10 above, first information 1 indicating target 1 and image group 1, target 3 and image group 1, and target 4 and image group 1 are recorded, and first information 2 indicating target 2 and image group 2 are recorded, when image group 1 meets the preset modeling conditions, the video data of image group 1 is acquired and parsed to obtain the corresponding image data 1 in image group 1. In this way, the models of targets 1, 3, and 4 can be established using image data 1. For ease of description, establishing a model will be referred to as modeling in the following text.

[0077] It is understood that in step S30, when the target is modeled using the image data obtained from the same parsing, for each first target, the image data of the optimal frame of the first target in the first image group is used to model the first target. However, when decoding (i.e., parsing) the video data of the first image group, the decoding depends on the key frames of the first image group. If the number of frames between the optimal frame and the key frame is greater, more frames need to be decoded, and vice versa, fewer frames need to be decoded.

[0078] If the first image group can only detect one first target, and the number of frames between the optimal frame of the first target and the key frame is small, but if all video frames of the first image group are decoded, it is useless to decode the video frames after the optimal frame of the first target in the first image group, which wastes computing resources.

[0079] Therefore, in order to reduce the waste of computing resources, for each first target in the first image group, the video frame with the highest score can be selected from the video frames in the first image group that are closer to the key frame in the time domain, and the video frame with the highest score can be used for modeling.

[0080] Based on this, in a possible implementation, as Figure 2 FIG2 is a second schematic diagram of a target modeling method provided in an embodiment of the present application, the method comprising the following steps:

[0081] Step S10, whenever first information indicating a first target and a first image group to which a video frame in which the first target is detected belongs is received, recording the first information;

[0082] Step S201: Whenever the first image group satisfies a preset modeling condition, an optimal generalized keyframe is determined for each first target in the first image group from among the generalized keyframes in the first image group, and the last optimal generalized keyframe in the first image group is determined as the optimal generalized keyframe for the target in the first image group.

[0083] Step S202: Searching for video data of a first image group from the recorded video data, and parsing the key frame, the target optimal generalized key frame, and all video frames between the key frame and the target optimal generalized key frame, starting with the key frame of the video data and ending with the target optimal generalized key frame, to obtain image data of each video frame;

[0084] Step S31, using the image data of the optimal generalized key frame of each first object to establish the models of all first objects in the first image group;

[0085] The above step S10 is referred to above and will not be described again here. Steps S201 to S202 are Figure 1 The specific detailed steps of step S20, step S31 Figure 1 Specific refinement steps of step S30.

[0086] In step S201, the score of the video frame is positively correlated with the amount of information about the first target contained in the video frame. A generalized key frame may refer to a key frame of the first image group and a video frame whose time interval with the key frame is less than a preset interval threshold, or may refer to a key frame of the first image and a video frame whose number of frames between the key frame and the key frame is less than a preset frame number threshold. For example, Figure 3 The following is an example diagram of an image group and a generalized key frame provided by an embodiment of the present application. Assuming that the image group includes an I frame and multiple p frames, namely, p1 frame, p2 frame, p3 frame, p4 frame, and p5 frame, the preset interval threshold is 30 milliseconds, and the time interval between two adjacent frames is 10 milliseconds, then the generalized key frames are I frame, p1 frame, p2 frame, and p3 frame. If for target 1, the video frame with the highest score among I frame, p1 frame, p2 frame, and p3 frame is p1 frame, then p1 frame is used as the optimal generalized key frame. In another example, still using Figure 3 For example, assuming the image group shown includes an I frame and multiple P frames, namely, P1, P2, P3, P4, and P5, and the preset frame number threshold is 4, then the generalized key frames are I, P1, P2, and P3. If, for target 1, the highest-scoring video frame among I, P1, P2, and P3 is P1, then P1 is selected as the optimal generalized key frame.

[0087] In the above step S201, after determining the target optimal generalized key frame with the last priority, during decoding, it is only necessary to parse the key frame, the target optimal generalized key frame, and all video frames between the key frame and the target generalized optimal key frame. At this time, the image data obtained by parsing contains the optimal generalized key frame required for modeling each first target in the first image group, and then in step S31, the model of each first target itself can be established using the optimal generalized key frame corresponding to each first target.

[0088] By adopting the method of the embodiment of the present application, since the generalized key frame is the key frame of the first image group and the video frame whose time domain interval with the key frame is less than a preset interval threshold, or the key frame of the first image and the video frame whose number of frames between the key frame and the key frame is less than a preset frame number threshold, the video frame with the highest score is determined as the optimal generalized key frame in the generalized key frames, and the image data of the optimal generalized key frame is used to model the first target. This can reduce the number of decoded frames, reduce the decoding calculation complexity, thereby improving the decoding speed, and at the same time reduce the waste of resources during decoding.

[0089] However, since the accuracy of the model is positively correlated with the score of the video frame selected during modeling, if only the decoding complexity is considered, the accuracy of the model may be low. Based on this, in a possible implementation, Figure 4 FIG. 3 is a third schematic diagram of a target modeling method provided in an embodiment of the present application, the method comprising the following steps:

[0090] Step S10, whenever first information indicating a first target and a first image group to which a video frame in which the first target is detected belongs is received, recording the first information;

[0091] Step S201: Whenever the first image group satisfies a preset modeling condition, an optimal generalized keyframe is determined for each first target in the first image group from among the generalized keyframes in the first image group, and the last optimal generalized keyframe in the first image group is determined as the optimal generalized keyframe for the target in the first image group.

[0092] Step S202: Searching for video data of a first image group from the recorded video data, and parsing the key frame, the target optimal generalized key frame, and all video frames between the key frame and the target optimal generalized key frame, starting with the key frame of the video data and ending with the target optimal generalized key frame, to obtain image data of each video frame;

[0093] Step S311: if the score of the optimal generalized key frame of the first object is higher than a preset score threshold, a model of the first object is established using the image data of the optimal generalized key frame of the first object;

[0094] Step S40: If the score of the optimal generalized key frame of the first target is not higher than the preset score threshold, the score difference between the optimal video frame of the first target and the optimal generalized key frame of the first target is calculated as the score difference of the first target;

[0095] Among them, the optimal video frame of the first target can refer to the video frame with the highest score for the first target among all video frames in the first image group, or it can refer to the video frame with the highest score for the first target among the video frames extracted from the first image group according to a preset frame extraction rule.

[0096] Step S50 : If the score difference of the first target is lower than a preset difference threshold, a model of the first target is established using the image data of the optimal generalized key frame of the first target.

[0097] Among them, steps S10 to S202 refer to the above and are not repeated here. Step S311 is Figure 2 Specific implementation steps of step S31.

[0098] In the above step S311, the preset scoring threshold may be set based on actual experience, or may be calculated according to a preset threshold calculation method. The calculation process of the preset scoring threshold is described below and will not be repeated here.

[0099] It is understandable that within the same first image group, the optimal generalized keyframes for different first targets may be the same video frame or different video frames. Similarly, the optimal video frames for different first targets may be the same video frame or different video frames. If the score of the first target in a video frame is higher than a preset score threshold, it can be considered that the video frame contains sufficient information about the first target, and the accuracy of the model established using the image data of the video frame is high. Therefore, the model of the first target can be directly established using the image data of the video frame. Conversely, the accuracy of the model established using the image data of the video frame is considered low, and the difference between the score of the optimal video frame of the first target in the first image group and the score of the optimal generalized keyframe of the first target needs to be calculated. If the score difference is less than the preset difference threshold, it can be considered that the accuracy of the model of the first target established using the optimal generalized keyframe of the first target is not much different from the accuracy of the model of the first target established using the optimal video frame of the first target, and the model of the first target can be established using the optimal generalized keyframe of the first target.

[0100] In the above step S40, the preset difference threshold is determined based on actual experience, and the embodiment of the present application is not limited to this.

[0101] By adopting the embodiment of the present application, when the score of the optimal generalized key frame of the first target is greater than a preset score threshold, and when the score of the optimal generalized key frame of the first target is not greater than the preset score threshold and the difference between the score of the generalized key frame of the first target and the score of the optimal video frame of the first target is less than the preset difference threshold, the optimal generalized key frame of the first target is used for modeling, which can reduce the decoding calculation complexity, improve the decoding speed, and thus improve the target modeling rate.

[0102] It can be understood that if the score difference of the first target is not less than the preset difference threshold, it can be considered that the accuracy of the model established using the optimal generalized key frame of the first target is quite different from the accuracy of the model established using the optimal video frame of the first target. The model of the first target should be established using the image data of the one with the highest score for the first target between the optimal generalized key frame of the first target and the optimal video frame of the first target to improve the accuracy of the established model.

[0103] The following is a detailed description of the process of calculating the preset scoring threshold:

[0104] It is understood that the preset scoring threshold is set according to the category of the first object, and the preset scoring thresholds for first objects of different categories may be the same or different. For example, the preset scoring threshold for an object of the animal category is different from the preset scoring threshold for an object of the car category.

[0105] like Figure 5 FIG. 1 is a schematic diagram of a method for calculating a preset scoring threshold provided in an embodiment of the present application, comprising the following steps:

[0106] Step S501, sorting the modeled objects in order from latest to earliest to obtain an object sequence;

[0107] Step S502: searching the object sequence for a preset number of objects of the same category as the first object in descending order as the second object;

[0108] Step S503, obtaining scores of the video frames used when building the models of the second targets, and calculating the average of the obtained scores;

[0109] Step S504: determining the mean as the scoring threshold of the category to which the first target belongs, or determining the mean increased by a preset ratio as the scoring threshold of the category to which the first target belongs.

[0110] Among them, in the above step S501, sorting the modeled objects in order from late to early may refer to sorting the modeled objects in order from late to early according to the time of modeling of each object, or may refer to sorting the modeled objects in order from late to early according to the time when the modeled objects were detected. The embodiments of the present application are not limited to this.

[0111] The preset number is determined based on actual experience, and may be, for example, 10, 20, or other values, and is not limited in the embodiments of the present application. The preset ratio is also determined based on actual experience, and may be, for example, 5%, 20%, or other values, and is not limited in the embodiments of the present application.

[0112] For example, assuming that the first target is a sedan and the category of the first target is car, then search for targets of the car category among the modeled objects, such as off-road vehicles and pickup trucks. If the score of the video frame used to build the off-road vehicle model is a, and the score of the video frame used to build the pickup truck model is b, then the score threshold can be (a+b) / 2 or [(a+b) / 2](1+c), where the value range of c is [0, 1].

[0113] According to an embodiment of the present application, a preset number of targets of the same category as the first target are searched among the modeled objects as second targets, scores of the video frames used to establish the models of each second target are obtained, and the average of the obtained scores is calculated. The average is determined as the scoring threshold, or the average increased by a preset proportion is determined as the scoring threshold. In this way, the determined scoring threshold is more stable and closer to the overall true threshold, and has higher reliability.

[0114] The target modeling method provided by the embodiment of the present application is described below with reference to specific embodiments. Figure 6 Shown is a flow chart of the target modeling method provided in an embodiment of the present application.

[0115] Step S601, when receiving first information, recording first information indicating a first target and a first image group (corresponding to the aforementioned step S10);

[0116] Step S602, parsing the video data of the image group that meets the preset modeling conditions to obtain image data (corresponding to the aforementioned step S20);

[0117] Step S603: for a first object in the first image group, determine whether the optimal generalized keyframe score of the first object is higher than a preset threshold; if so, execute step S604; otherwise, execute step S605;

[0118] Step S604, modeling using the optimal generalized keyframe (corresponding to the aforementioned step S311);

[0119] Step S605 , calculating the score difference between the optimal video frame and the optimal generalized key frame (corresponding to the aforementioned step S40 );

[0120] Step S606: Determine whether the score difference is higher than a threshold. If the score difference is higher than the threshold, execute step S607; otherwise, execute step S608.

[0121] Step S607, modeling using the optimal video frame;

[0122] Step S608: Modeling using the optimal generalized key frame (corresponding to the aforementioned step S50).

[0123] Step S609, after building a model for a first object in the first image group, determining whether there is a first object in the first image group that has not been modeled, if so, returning to step S603, if not, executing step S6010;

[0124] Step S6010, determining whether there is still an image group that meets the preset modeling conditions; if so, returning to step S602, if not, ending.

[0125] Corresponding to the aforementioned first aspect, the second aspect of the embodiment of the present application provides a video processing device, which may be a video recording device or other device used for video processing.

[0126] The video processing device is used for:

[0127] recording, whenever first information indicating a first target and a first image group to which a video frame in which the first target is detected belongs, wherein the first target is at least one target detected from all video frames of the first image group, and the first information is generated after the first target is detected from a recorded video segment;

[0128] Whenever a first image group meets a preset modeling condition, searching for video data of the first image group from recorded video data and parsing to obtain image data, wherein the first image group is an image group in a recorded video clip;

[0129] The image data obtained by the same analysis is used to establish respective models of all first targets in the first image group.

[0130] In one possible implementation,

[0131] The video processing device searches for video data of the first image group from recorded video data and parses the video data to obtain image data, including:

[0132] Determining, from among the generalized key frames of the first image group, an optimal generalized key frame for each of the first targets in the first image group, and determining the optimal generalized key frame ranked last in the first image group as the optimal generalized key frame for the target corresponding to the first image group; wherein the optimal generalized key frame is a video frame with the highest score among the generalized key frames of the first image group, the score of the video frame is positively correlated with the amount of information about the first target contained in the video frame, and the generalized key frame is a key frame of the first image group and a video frame whose temporal interval with the key frame is less than a preset interval threshold, or the generalized key frame is a key frame of the first image and a video frame whose number of frames between the key frame and the key frame is less than a preset frame number threshold;

[0133] Searching for video data of the first image group from the recorded video data, and starting with a key frame of the video data and ending with the target optimal generalized key frame, parsing the key frame, the target optimal generalized key frame, and all video frames between the key frame and the target optimal generalized key frame to obtain image data of each video frame;

[0134] The establishing of respective models of all first targets in the first image group using the image data of each video frame obtained from the same parsing includes:

[0135] Models of all first objects in the first image group are established using the image data of the optimal generalized key frame of each first object.

[0136] In one possible implementation,

[0137] The video processing device uses the image data of the optimal generalized key frame of each first object to establish a model of the first object itself, including:

[0138] If the score of the optimal generalized keyframe is higher than a preset score threshold, establishing a model of the first target using the image data of the optimal generalized keyframe;

[0139] The method further comprises:

[0140] If the score of the optimal generalized key frame is not higher than the preset score threshold, calculating the score difference between the optimal video frame and the optimal generalized key frame, wherein the optimal video frame is the video frame with the highest score in the first image group;

[0141] If the score difference is lower than a preset difference threshold, the model of the first target is established using the image data of the optimal generalized key frame.

[0142] In a possible implementation, the video processing device is further configured to, if the score difference is not lower than the preset difference threshold, establish a model of the first target using the image data of the optimal video frame.

[0143] In a possible implementation, the preset scoring threshold is set according to the category of the first target, and the preset scoring threshold is pre-calculated in the following manner:

[0144] Sort the modeled objects in order from latest to earliest to obtain an object sequence;

[0145] Searching, in order from front to back, for a preset number of targets of the same category as the first target in the object sequence as second targets;

[0146] Obtaining scores of the video frames used when building the model of each second target, and calculating a mean of the obtained scores;

[0147] The mean is determined as the scoring threshold, or the mean increased by a preset ratio is determined as the scoring threshold.

[0148] According to a third aspect of the present application, a target modeling device is provided, the device comprising:

[0149] a recording module, configured to record, upon receiving first information indicating a first target and a first image group to which a video frame in which the first target is detected belongs, the first information, wherein the first target is at least one target detected from all video frames of the first image group, and the first information is generated after the first target is detected from a recorded video segment;

[0150] a parsing module, configured to search for video data of the first image group from the recorded video data and parse the video data to obtain image data whenever the first image group satisfies a preset modeling condition, wherein the first image group is an image group in the recorded video clip;

[0151] The building module is used to build the models of all the first targets in the first image group by using the image data obtained by the same analysis.

[0152] In a possible implementation, the parsing module is specifically configured to:

[0153] Determining, from among the generalized key frames of the first image group, an optimal generalized key frame for each of the first targets in the first image group, and determining the optimal generalized key frame ranked last in the first image group as the optimal generalized key frame for the target corresponding to the first image group; wherein the optimal generalized key frame is a video frame with the highest score among the generalized key frames of the first image group, the score of the video frame is positively correlated with the amount of information about the first target contained in the video frame, and the generalized key frame is a key frame of the first image group and a video frame whose temporal interval with the key frame is less than a preset interval threshold, or the generalized key frame is a key frame of the first image and a video frame whose number of frames between the key frame and the key frame is less than a preset frame number threshold;

[0154] Searching for video data of the first image group from the recorded video data, and starting with a key frame of the video data and ending with the target optimal generalized key frame, parsing the key frame, the target optimal generalized key frame, and all video frames between the key frame and the target optimal generalized key frame to obtain image data of each video frame;

[0155] The establishing of respective models of all first targets in the first image group using the image data of each video frame obtained from the same parsing includes:

[0156] Models of all first objects in the first image group are established using the image data of the optimal generalized key frame of each first object.

[0157] In one possible implementation,

[0158] The step of establishing a model of the first object itself by using the image data of the optimal generalized key frame of each first object includes:

[0159] If the score of the optimal generalized keyframe of the first object is higher than a preset score threshold, establishing a model of the first object using the image data of the optimal generalized keyframe of the first object;

[0160] The method further comprises:

[0161] If the score of the optimal generalized key frame of the first target is not higher than the preset score threshold, calculating the score difference between the optimal video frame of the first target and the optimal generalized key frame of the first target as the score difference of the first target; wherein the optimal video frame of the first target is the video frame with the highest score for the first target in the first image group;

[0162] If the score difference of the first target is lower than a preset difference threshold, a model of the first target is established using image data of the optimal generalized key frame of the first target.

[0163] In one possible embodiment, the device is further used for:

[0164] If the score difference is not lower than the preset difference threshold, the model of the first target is established using the image data of the optimal video frame.

[0165] In one possible implementation,

[0166] The preset scoring threshold is set according to the category of the first target, and the preset scoring threshold is pre-calculated in the following way:

[0167] Sort the modeled objects in order from latest to earliest to obtain an object sequence;

[0168] Searching, in order from front to back, for a preset number of targets of the same category as the first target in the object sequence as second targets;

[0169] Obtaining scores of the video frames used when building the model of each second target, and calculating a mean of the obtained scores;

[0170] The mean is determined as the scoring threshold of the category to which the first target belongs, or the mean increased by a preset proportion is determined as the scoring threshold of the category to which the first target belongs.

[0171] The present application also provides an electronic device, such as Figure 7 Shown, including:

[0172] Memory 701, used for storing computer programs;

[0173] The processor 702 is configured to execute the program stored in the memory 701, and implement the following steps:

[0174] Whenever first information sent for a first video frame and a first object is received, correspondingly recording the image group to which the first object and the first video frame belong, wherein the first video frame is any video frame in the recorded video clip, the first object is any object appearing in the first video frame, and the first information is generated after the first object is analyzed from the first video frame;

[0175] Whenever a first image group satisfies a preset modeling condition, searching for and parsing video data of the first image group from recorded video data to obtain image data of each video frame in the first image group, wherein the first image group is any image group in the recorded video clip;

[0176] For each first target corresponding to the first image group, a model of the first target is established using the image data obtained by analysis.

[0177] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0178] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0179] In another embodiment provided in the present application, a computer-readable storage medium is further provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned target model establishment methods are implemented.

[0180] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any target model building method in the above embodiments.

[0181] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state drive (SSD).

[0182] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0183] Each embodiment in this specification is described in a related manner. Similar portions between the embodiments can be referenced to each other. Each embodiment focuses on the differences from other embodiments. In particular, the video processing device embodiment is generally similar to the method embodiment, so its description is relatively simple. For related portions, reference can be made to the description of the method embodiment.

[0184] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A method for establishing a target model, characterized in that: The method comprises: recording, whenever first information indicating a first target and a first image group to which a video frame in which the first target is detected belongs, wherein the first target is at least one target detected from all video frames of the first image group, and the first information is generated after the first target is detected from a recorded video segment; Whenever a first image group meets a preset modeling condition, searching for video data of the first image group from recorded video data and parsing to obtain image data, wherein the first image group is an image group in a recorded video clip; The image data obtained by the same analysis is used to establish respective models of all first targets in the first image group.

2. The method according to claim 1, characterized in that The step of searching for the video data of the first image group from the recorded video data and parsing the video data to obtain the image data includes: Determining, from among the generalized key frames of the first image group, an optimal generalized key frame for each of the first targets in the first image group, and determining the optimal generalized key frame ranked last in the first image group as the optimal generalized key frame for the target corresponding to the first image group; wherein the optimal generalized key frame is a video frame with the highest score among the generalized key frames of the first image group, the score of the video frame is positively correlated with the amount of information about the first target contained in the video frame, and the generalized key frame is a key frame of the first image group and a video frame whose temporal interval with the key frame is less than a preset interval threshold, or the generalized key frame is a key frame of the first image and a video frame whose number of frames between the key frame and the key frame is less than a preset frame number threshold; Searching for video data of the first image group from the recorded video data, and starting with a key frame of the video data and ending with the target optimal generalized key frame, parsing the key frame, the target optimal generalized key frame, and all video frames between the key frame and the target optimal generalized key frame to obtain image data of each video frame; The establishing of respective models of all first targets in the first image group using the image data of each video frame obtained from the same parsing includes: Models of all first objects in the first image group are established using the image data of the optimal generalized key frame of each first object.

3. The method according to claim 2, characterized in that The step of establishing a model of the first object itself by using the image data of the optimal generalized key frame of each first object includes: If the score of the optimal generalized keyframe of the first object is higher than a preset score threshold, establishing a model of the first object using the image data of the optimal generalized keyframe of the first object; The method further comprises: If the score of the optimal generalized key frame of the first target is not higher than the preset score threshold, calculating the score difference between the optimal video frame of the first target and the optimal generalized key frame of the first target as the score difference of the first target; wherein the optimal video frame of the first target is the video frame with the highest score for the first target in the first image group; If the score difference of the first target is lower than a preset difference threshold, a model of the first target is established using image data of the optimal generalized key frame of the first target.

4. The method according to claim 3, characterized in that The method further comprises: If the score difference is not lower than the preset difference threshold, the model of the first target is established using the image data of the optimal video frame.

5. The method according to claim 3 or 4, characterized in that The preset scoring threshold is set according to the category of the first target, and the preset scoring threshold is pre-calculated in the following way: Sort the modeled objects in order from latest to earliest to obtain an object sequence; Searching, in order from front to back, for a preset number of targets of the same category as the first target in the object sequence as second targets; Obtaining scores of the video frames used when building the model of each second target, and calculating a mean of the obtained scores; The mean is determined as the scoring threshold of the category to which the first target belongs, or the mean increased by a preset proportion is determined as the scoring threshold of the category to which the first target belongs.

6. A video processing device, characterized in that: The video processing device is used for: recording, whenever first information indicating a first target and a first image group to which a video frame in which the first target is detected belongs, wherein the first target is at least one target detected from all video frames of the first image group, and the first information is generated after the first target is detected from a recorded video segment; Whenever a first image group meets a preset modeling condition, searching for video data of the first image group from recorded video data and parsing to obtain image data, wherein the first image group is an image group in a recorded video clip; The image data obtained by the same analysis is used to establish respective models of all first targets in the first image group.

7. The video processing device according to claim 6, characterized in that The video processing device searches for video data of the first image group from recorded video data and parses the video data to obtain image data, including: Determining, from among the generalized key frames of the first image group, an optimal generalized key frame for each of the first targets in the first image group, and determining the optimal generalized key frame ranked last in the first image group as the optimal generalized key frame for the target corresponding to the first image group; wherein the optimal generalized key frame is a video frame with the highest score among the generalized key frames of the first image group, the score of the video frame is positively correlated with the amount of information about the first target contained in the video frame, and the generalized key frame is a key frame of the first image group and a video frame whose temporal interval with the key frame is less than a preset interval threshold, or the generalized key frame is a key frame of the first image and a video frame whose number of frames between the key frame and the key frame is less than a preset frame number threshold; Searching for video data of the first image group from the recorded video data, and starting with a key frame of the video data and ending with the target optimal generalized key frame, parsing the key frame, the target optimal generalized key frame, and all video frames between the key frame and the target optimal generalized key frame to obtain image data of each video frame; The establishing of respective models of all first targets in the first image group using the image data of each video frame obtained from the same parsing includes: Models of all first objects in the first image group are established using the image data of the optimal generalized key frame of each first object.

8. The video processing device according to claim 7, characterized in that The video processing device uses the image data of the optimal generalized key frame of each first object to establish a model of the first object itself, including: If the score of the optimal generalized keyframe is higher than a preset score threshold, establishing a model of the first target using the image data of the optimal generalized keyframe; The video processing device is further configured to: If the score of the optimal generalized key frame is not higher than the preset score threshold, calculating the score difference between the optimal video frame and the optimal generalized key frame, wherein the optimal video frame is the video frame with the highest score in the first image group; If the score difference is lower than a preset difference threshold, the model of the first target is established using the image data of the optimal generalized key frame.

9. The video processing device according to claim 8, characterized in that The video processing device is further configured to, if the score difference is not lower than the preset difference threshold, establish a model of the first target using the image data of the optimal video frame.

10. The video processing device according to claim 8 or 9, characterized in that: The preset scoring threshold is set according to the category of the first target, and the preset scoring threshold is pre-calculated in the following way: Sort the modeled objects in order from latest to earliest to obtain an object sequence; Searching, in order from front to back, for a preset number of targets of the same category as the first target in the object sequence as second targets; Obtaining scores of the video frames used when building the model of each second target, and calculating a mean of the obtained scores; The mean is determined as the scoring threshold, or the mean increased by a preset ratio is determined as the scoring threshold.

Images