Data generation method and device, equipment, medium and product

By automatically analyzing the target parts and object information in the video clips and generating target data, the problems of difficult and cost of manual labeling in the prior art are solved, and efficient and low-cost detection of package violence is achieved.

CN119048950APending Publication Date: 2024-11-29BEIJING JINGDONG YUANSHENG TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411017391.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

When detecting package violence, the prior art relies on users to view surveillance videos to have visual fatigue problems, and deep learning-based methods require high-quality annotation data, which is costly and difficult to obtain sufficient annotation data.

Method used

By determining the target video clip from the to-process video, using the target part information in the preset state, combining the location detection model and the object detection model, the video frames are automatically analyzed and the target data is generated, and the dependence on manual annotation is reduced.

Benefits of technology

It achieves improving detection efficiency without increasing costs, automating video analysis, generating target data containing violent behavior, reducing the difficulty and cost of obtaining sample data.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048950A_ABST
    Figure CN119048950A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a data generation method and device, equipment, a medium and a product. The method comprises the following steps: determining at least one target video clip from a to-be-processed video; for the at least one target video clip, determining position information of a to-be-processed object in each target video frame in the target video clip according to part information of a target part in the target video clip; and according to multiple pieces of position information associated with the target video clip, fusing the to-be-processed object into the target video clip to obtain target data. According to the technical scheme provided by the embodiment of the invention, the convenience and accuracy of generating the target data corresponding to the preset behavior are improved, and the effect of determining the cost of the target data is further reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of data processing technology, and in particular, to a data generation method, device, equipment, medium and product. Background Art

[0002] With the development of e-commerce, more and more users will obtain corresponding items online. After obtaining the items, they can be packaged and sorted. During the packaging and sorting process, there may be violent behavior, and the losses caused by violent behavior are also very large. Based on this, violent behavior detection can be performed on packages to achieve the effect of reducing costs.

[0003] Currently, the presence of violent behavior can be determined using at least one of the following methods:

[0004] The first method is to set up monitoring devices in the sorting area, and the supervising user can determine whether there is any violent behavior by viewing the monitoring video collected by the monitoring device.

[0005] The second method: Use a deep learning-based target detection algorithm to detect a moving package, and then analyze the moving coordinates of the highest point of the package to determine whether it is a violent act.

[0006] When implementing the technical solution based on the above method, the inventor found the following problems:

[0007] The first method mainly relies on users to view surveillance videos, which has the problem of high dependence on users. Furthermore, long-term viewing of surveillance videos will cause certain visual fatigue, which makes it impossible to effectively determine whether there is violence. The second method based on deep learning to determine whether there is violence is highly dependent on high-quality labeled data. However, the video data corresponding to violent behavior is difficult to obtain (mainly because violent behavior does not occur frequently, it is necessary to manually find video clips of violent operations from massive video data, and then analyze and process the video clips to obtain labeled sample data). In other words, the cost of labeling data is high. Summary of the invention

[0008] The present invention provides a data generation method, device, equipment, medium and product, so as to improve the efficiency of determining target data without increasing the cost.

[0009] In a first aspect, an embodiment of the present invention provides a data generation method, the method comprising:

[0010] Determine at least one target video segment from the video to be processed, wherein the target video segment includes a plurality of target video frames, the target video frames include a target part, and the target part is in a preset state;

[0011] For the at least one target video segment, determining position information of the object to be processed in each target video frame in the target video segment according to the position information of the target part in the target video segment;

[0012] According to a plurality of position information associated with the target video segment, the object to be processed is merged into the target video segment to obtain target data.

[0013] Furthermore, before determining at least one target video segment from the video to be processed, the method further includes:

[0014] A video to be processed captured by at least one video capture device in a target area is obtained, wherein the target area is an area for sorting items and / or packages to be sorted.

[0015] Further, the step of determining at least one target video segment from the video to be processed includes:

[0016] For at least one to-be-processed video frame in the to-be-processed video, when it is determined that the to-be-processed video frame includes a target part and the target part is in a preset state, obtaining a preset number of to-be-checked video frames before and after the to-be-processed video frame;

[0017] If the target part is included in the video frames to be checked and the target part is in the preset state, the video frames to be checked and the video frames to be processed are used as target video frames to obtain the target video segment.

[0018] Furthermore, the method further comprises:

[0019] Using a to-be-processed video frame or a to-be-checked video frame that does not include the target part or the target part is not in the preset state as a first video frame;

[0020] According to the first video frame and the total number of frames of the target video segment, a next video frame to be processed is determined to determine whether it includes a target part and whether the target part is in a preset state.

[0021] Further, the determining, based on the first video frame and the total number of frames of the target video segment, whether the next video frame to be processed includes the target part and whether the target part is in a preset state comprises:

[0022] If there are multiple first video frames after the to-be-processed video frame, the next video frame of the to-be-checked video frame that is the latest in time sequence is used as the start frame;

[0023] According to the total number of frames and the starting frame, it is determined whether the next video frame to be processed includes a target part and whether the target part is in a preset state.

[0024] Furthermore, the method further comprises:

[0025] If the target part is not included in the to-be-processed video frame or the target part is not in the preset state, determining whether the next to-be-processed video frame of the to-be-processed video frame includes the target part and whether the target part is in the preset state; or,

[0026] If the target part is not included in the video frame to be checked or the target part is not in the preset state, it is determined whether the next video frame of the video frame to be processed includes the target part and whether the target part is in the preset state.

[0027] Furthermore, the target video frame also includes a target object, and the target part is a local part of the target object.

[0028] Furthermore, whether the to-be-processed video frame includes the target object, the target part, and whether the target part is in a preset state is determined by:

[0029] Inputting the to-be-processed video frame into a pre-trained part detection model to obtain a part detection result;

[0030] When the classification identifier and the part information of the part detection result meet the preset conditions, determining that the to-be-processed video frame includes the target part and the target part is in the preset state; and

[0031] The video frame to be processed is input into a pre-trained object detection model for object recognition, and when the recognition result is a target result, it is determined that the video frame to be processed includes the target object.

[0032] Furthermore, determining the position information of the object to be processed in each target video frame in the target video segment according to the position information of the target part in the target video segment includes:

[0033] Determining the center point information of the target part according to the part information of the first video frame of the target video segment;

[0034] Determine a throwing path of the object to be processed according to the size information of the object to be processed, the preset throwing information and the center point information;

[0035] Based on the throwing path, position information of the object to be processed in each target video frame in the target video segment is determined.

[0036] Furthermore, determining the position information of the object to be processed in each target video frame in the target video segment based on the throwing path includes:

[0037] Points are evenly selected on the throwing path according to the total number of frames of the target video clip to obtain position information of the object to be processed in the target video frame.

[0038] Furthermore, the object to be processed is determined based on the following method:

[0039] Analyzing and processing a plurality of to-be-detected video frames based on a pre-trained sorting object detection model to output an object image including the to-be-processed object;

[0040] The object image is input into a pre-trained image processing model for refinement processing to obtain an object to be processed with refined features.

[0041] Furthermore, after the object to be processed is merged into the target video segment, the method includes:

[0042] For any two video frames in the target video clip, blurring the objects to be processed in the two video frames to obtain blurred images;

[0043] The target data is expanded according to the blurred image.

[0044] In a second aspect, an embodiment of the present invention further provides a data generating device, the device comprising:

[0045] A video segment determination module, used to determine at least one target video segment from the video to be processed, wherein the target video segment includes a plurality of target video frames, the target video frames include a target part and the target part is in a preset state;

[0046] a position information determination module, configured to determine, for the at least one target video segment, position information of the object to be processed in each target video frame in the target video segment according to the position information of the target part in the target video segment;

[0047] The target data generating module is used to fuse the object to be processed into the target video segment according to a plurality of position information associated with the target video segment to obtain target data.

[0048] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0049] one or more processors;

[0050] a storage device for storing one or more programs,

[0051] When the one or more programs are executed by the one or more processors, the one or more processors implement the data generation method as described in any one of the embodiments of the present invention.

[0052] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute the data generation method as described in any one of the embodiments of the present invention.

[0053] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, is used to execute the data generation method as described in any one of the embodiments of the present invention.

[0054] The technical solution of the embodiment of the present invention determines a target video segment including multiple target video frames from the video to be processed, and the target video frame includes a target part and the target part is in a preset state, and determines the position information of the object to be processed in each target video frame in the target video segment according to the part information of the target part in the target video segment, and then merges the object to be processed into the target video segment according to multiple position information associated with the target video segment to obtain target data. This solves the problem in the prior art that it is necessary to manually find video segments of violent operations from massive video data, and then analyze and process the video segments to obtain labeled sample data, and there are problems such as high difficulty and high cost in obtaining sample data. This realizes automatic analysis and processing of the video to be processed, and then reconstructs target data including violent behavior, thereby improving the convenience and low cost of determining target data. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required to describe the embodiments. Obviously, the drawings introduced are only drawings of a part of the embodiments to be described in the present invention, rather than all the drawings. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0056] Figure 1 A schematic diagram of a data generation method provided by an embodiment of the present invention;

[0057] Figure 2 A schematic diagram of a data generation method provided by an embodiment of the present invention;

[0058] Figure 3 A schematic diagram of a data generation method provided by an embodiment of the present invention;

[0059] Figure 4 A flow chart of a data generation method provided by an embodiment of the present invention

[0060] Figure 5 A schematic diagram of a monitoring video frame and an object to be processed obtained after analyzing and processing the monitoring video frame provided by an embodiment of the present invention;

[0061] Figure 6 A schematic diagram of the structure of a data generating device provided by an embodiment of the present invention;

[0062] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0064] Before introducing the technical solutions shared by the embodiments of the present invention, an exemplary description of the application scenarios can be given. The solution provided by the embodiments of the present invention can be used to determine sample data (target data) showing violent behavior on the object to be processed, and the subsequent use of the sample data (target data) can be determined based on actual needs. Optionally, a neural network model for detecting violent behavior can be trained based on the sample data (target data). The specific usage is not limited in this embodiment, as long as the sample data is determined by the method provided by the embodiments of the present invention, it is within its protection scope.

[0065] In this embodiment, the violent behavior is mainly the throwing behavior of the object to be processed. The object to be processed is mainly the articles or packages to be acted on. The violent behavior can be generated in various scenarios, for example, the various scenarios can be loading and unloading scenarios, behavior detection scenarios on the transportation line, behavior detection scenarios in the sorting process, behavior detection scenarios in the distribution process, etc. Correspondingly, the violent behavior is the throwing behavior of the object to be processed in the above scenarios.

[0066] For different application scenarios, the method of determining sample data is the same, and the application scenario of sorting is used as an example for explanation.

[0067] Usually, various actions of the processed object are mostly completed by the user's hand operation, so the part detection model can be a model for detecting hand parts. Furthermore, if there are hands in the image, but there is no sorting behavior, there may be no violent behavior. Therefore, the trained part detection model not only needs to determine whether the image includes the target part, but also needs to determine the state information of the target part. Next, we will introduce how to obtain the part detection model.

[0068] The part detection model may be a neural network model, and optionally, the neural network model may be a YoLoV8 model. In order to obtain a better balance, multiple models of YoLoV8 may be tested, and finally, the M model is selected as the final part detection model.

[0069] It should be noted that different scenarios have different degrees of difficulty in object detection and provide different computing power support. Different models can be selected according to the specific scenario.

[0070] Before obtaining the part detection model, sample data involved in training the part detection model may be obtained first.

[0071] Usually, violent acts against the treated object are usually carried out by the hands. In order to make the target data more realistic, it is necessary to generate data of violent acts near the hand part. Therefore, a large number of sample images with annotations of hand parts and human body can be obtained from public and authorized academic data sets and used as training samples. The hands in the training samples can be in a state of holding an object, an empty state, a state where the hand is in contact with oneself, and a state where the hand is in contact with others. Based on the above four states, they can be classified to obtain a label for each training sample. Optionally, the label in the empty state is set as the first category, and the labels in other states (the state of holding an object, the state of contact with oneself, and the state of contact with others) are set as the second category.

[0072] For each training sample, the image corresponding to the current training sample is input into the part detection model to be trained. The part detection model can output the corresponding discrimination result. Based on the discrimination result and the label, the loss value can be determined, and the model parameters in the part detection model to be trained can be corrected according to the loss value. The model obtained when the loss function in the part detection model to be trained converges is used as the final part detection model that can be used. In other words, the part detection model can output whether the image includes the hand part and the state information of the hand part.

[0073] It should also be noted that, in actual applications, if only the hand part is included but not the user, it may not be possible to further determine whether there is violent behavior. Therefore, a user detection model can also be trained. The user detection model is used to analyze and process the input image to determine whether the input image includes the user. In this embodiment, the part detection model and the user detection model can be made to work together to determine whether the input image includes the user, the target part, and the state information of the target part. The state information at this time includes that the hand part is in an empty state or other states.

[0074] On the basis of the above technical solution, it is also necessary to obtain data corresponding to the object to be processed, that is, it is necessary to obtain a lot of sample data including the package image. At this time, the package image is an image including the object to be processed. Then, based on the sample data, a detection model for the object to be processed is trained to analyze and process the collected image, identify the object to be processed, and then fuse the object to be processed with the hand image to obtain the final target data.

[0075] In this embodiment, data including the object to be processed can be collected based on at least two methods. The first method is to select images with packages or similar packages labeled from a public and authorized dataset as a training dataset. Another method is to manually label a small number of images including packages as a training set, and use a semi-supervised method to label the data. First, use a labeling tool to manually label some packages, and then use the labeled data to detect packages on unlabeled visual data. Manually correct the prediction results of the model, and the corrected data is used to continue training the model. The above process is repeated multiple times until the accuracy of the model's detection effect on the package reaches a preset accuracy threshold.

[0076] It can be understood that the object detection model to be processed is obtained based on the training sample obtained. It should be noted that the object detection model to be processed can identify the object to be processed in the video frame to be processed, that is, an object image including the object to be processed with the background removed can be obtained. In order to make the edge and feature of the object to be processed in the object image more refined, the object image can be input into the image processing model to obtain an image in which the features of the object to be processed in the object image are refined, and the image obtained at this time is used as the object to be processed.

[0077] After the above preparations are completed, the following method can be used to generate target data containing certain behaviors.

[0078] Figure 1A flow chart of a data generation method provided in an embodiment of the present invention is applicable to situations where target data for generating preset behaviors for an object to be processed is generated. The method can be executed by a data generation device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device, and the electronic device can be a mobile terminal, a PC, or a server, etc.

[0079] like Figure 1 As shown, the method includes:

[0080] S110: Determine at least one target video segment from the video to be processed.

[0081] Among them, the video to be processed can be a surveillance video of a certain area. The area can be an area for sorting items, an area for packing items, or an area for loading and unloading goods, etc. The specific area is not limited in this embodiment, as long as the video to be processed in the area can participate in subsequent processing. The target video segment is a video segment determined from the video frame to be processed, and the video clips in the video segment are continuous in time sequence. The target video segment includes multiple target video frames, and the target video frame includes a target part and the target part is in a preset state. The target part can be a preset hand part. The preset state can be a state in which the hand part is in an idle state. The number of target video segments can include one or more, and the specific number is finally determined according to the screening conditions.

[0082] It should also be noted that the target video segment includes a plurality of target video frames, each of which includes a target part, and the target part is in a preset state.

[0083] In this embodiment, before determining at least one target video segment from the video to be processed, the method further includes: obtaining the video to be processed captured by at least one video capture device in a target area, wherein the target area is an area where a preset action is performed on the items to be processed and / or the packages to be processed.

[0084] It can be understood that before obtaining the target video clip, multiple videos to be processed can be obtained first. Multiple video acquisition devices can be deployed in the target area, for example, the target area can be any area where there are sorting behaviors, packaging behaviors, loading and unloading behaviors, and the transmission of items on the assembly line requires user participation. Optionally, the video acquisition device can be a camera to collect surveillance video based on the video acquisition device. In the specific application process, the surveillance video corresponding to each video acquisition device can be retrieved and used as the video to be processed. It is also possible to intercept the video to be processed corresponding to the operation time from the surveillance video.

[0085] S120: For the at least one target video segment, determine, according to the location information of the target location in the target video segment, location information of the object to be processed in each target video frame in the target video segment.

[0086] It should be noted that the processing method for each target video segment is the same, and in this embodiment, the processing of one target video segment is taken as an example for description.

[0087] The part information may be the contour information of the target part in the target video frame, or the circumscribed rectangle information, etc. For each target video clip, the objects to be processed may be the same or different, and a processing object may be randomly selected from the object library and fused into the target video frame. The position information may be the fusion position information of the processing object fused into the target video frame.

[0088] Specifically, an object image corresponding to the object to be processed is obtained, and the location information of the target location in the target video clip is determined. According to the location information and the object image of the object to be processed, the fusion location of the object to be processed in each target video frame, that is, the location information, can be determined.

[0089] It should be noted that the position information of the object to be processed determined above is calculated based on the behavior of throwing objects in a real environment such as horizontal throwing and high throwing in simulated violent behavior.

[0090] S130 : According to a plurality of position information associated with the target video segment, merge the object to be processed into the target video segment to obtain target data.

[0091] It should be noted that for each target video segment, each target video frame has a position information, so a target video segment can include multiple position information. After the object image corresponding to the object to be processed is fused to each target video frame of the target video segment, a sample video corresponding to the simulated violent behavior can be obtained, and this sample video can be used as target data.

[0092] That is to say, the target data for throwing the object to be processed can be generated in the above manner.

[0093] In this embodiment, after the target data is obtained, the violent behavior detection model can be trained based on the target data, and can also be used for other purposes, the specific use of which is not limited in this embodiment.

[0094] The technical solution provided by the embodiment of the present invention can determine a target video segment including a target part and the target part is in a preset state from a video frame to be processed. Next, the position information of the object to be processed fused into the target video frame can be determined based on the position information of the target part in the target video segment, and then the object image of the object to be processed is fused into the target video frame based on the position information to obtain the target data, which solves the problem that there are few samples of specific behaviors in the prior art, resulting in difficulty and high cost in obtaining sample data, and realizes that target data containing specific behaviors can be efficiently generated under the premise of low cost, thereby improving the convenience and low cost of determining target data.

[0095] Figure 2 A flow chart of a data generation method provided in an embodiment of the present invention can further refine how to determine the target video segment on the basis of the aforementioned embodiment. The specific implementation method can refer to the detailed description of this embodiment, wherein the technical terms that are the same as or corresponding to the aforementioned embodiment are not repeated in this embodiment.

[0096] like Figure 2 As shown, the method includes:

[0097] S210: Obtain a video to be processed, wherein the video to be processed includes a plurality of video frames to be processed.

[0098] It can be understood that: the video to be processed includes multiple video frames to be processed, and each video frame to be processed can be analyzed and processed in turn to determine the target video segment based on the analysis and processing result.

[0099] S220: Input the video frame to be processed into a pre-trained part detection model to obtain a part detection result.

[0100] It can be understood that after the video frame to be processed is input into the part detection model, the part detection model can determine whether the video frame to be processed includes the target part, and the part status of the target part.

[0101] That is to say, the part detection result includes two types of information, one is whether the target part classification mark is included, and the other is the state information of the target part. The training of the part detection model can refer to the above explanation, which will not be repeated in this embodiment.

[0102] It should also be noted that the part detection result may also include coordinate information of the rectangular frame corresponding to the target part.

[0103] S230: When the classification identification and state information of the part detection result meet preset conditions, it is determined that the video frame to be processed includes the target part and the target part is in a preset state.

[0104] The preset condition may include that the classification mark is consistent with the preset classification mark, and the state information is consistent with the preset state. The preset classification mark may be a classification mark including the target part, and the preset state may be a state where the hand part is in an idle state.

[0105] It can be understood that: after the part detection model outputs the part detection result, it can be confirmed whether the classification mark in the part detection result is consistent with the preset classification mark, and whether the state information is consistent with the preset state. If both of the preset conditions are met, it is determined that the video frame to be processed includes the target part, and the target part is in the preset state.

[0106] S240: Input the video frame to be processed into a pre-trained object detection model for object recognition, and when the recognition result is a target result, determine that the video frame to be processed includes the target object.

[0107] The object detection model is pre-trained and is used to determine whether a target object is included in a video frame to be processed. The target object generally refers to a user.

[0108] Specifically, the video frame to be processed can be input into a pre-trained object detection model. The object detection model can analyze and process the video frame, and then output a recognition result. If the recognition result is a target result, and optionally, the target result includes a target object, it means that the video frame to be processed includes the target object.

[0109] It should also be noted that if the to-be-processed video frame does not include the target object, the processing will continue with the next video frame of the to-be-processed video frame.

[0110] In other words, only when both S230 and S240 meet the conditions can S250 and steps after S250 be executed, and as long as one of the above conditions is not met, the next to-be-processed video frame of the to-be-processed video frame is obtained.

[0111] S250: Obtain a preset number of video frames to be checked before and after the video frame to be processed.

[0112] The preset number of frames may be one frame or multiple frames, and the specific number may be set by the user according to actual needs. The video frames to be processed within the preset number of frames before and after the video frame to be processed may be used as the video frames to be checked.

[0113] S260: If the target part is included in all the video frames to be checked and the target part is in a preset state, the video frames to be checked and the video frames to be processed are used as target video frames to obtain a target video segment.

[0114] In this embodiment, for the video frame to be verified, a step of determining whether the video frame to be verified includes a target object, a target part, and whether the target part is in a preset state can be performed based on a part detection model. If all corresponding conditions are met, it means that the video frame to be verified can be used as a target video frame.

[0115] If all the video frames to be checked meet the above conditions, the video frames to be checked and the video frames to be processed may be sorted in time sequence to obtain target video segments that can be used later.

[0116] Of course, if there is a to-be-checked video frame that does not meet the condition, the other to-be-checked video frames will not be processed, and the next to-be-processed video frame will be obtained, and the steps of determining the target video segment in this embodiment will be repeated.

[0117] Based on the above technical solution, there may be a situation where the target part is not included in the video frame to be processed or the video frame to be checked, the target part is not in a preset state, or the target object is not included in the video frame to be processed. In this case, the video frame to be processed or the video frame to be checked that meets the above situation can be used as the first video frame. The advantage of determining the first video frame is that it is convenient to subsequently determine the next video frame to be processed, thereby improving the efficiency of determining the target video segment.

[0118] Optionally, a video frame to be processed or a video frame to be checked that does not include the target part or the target part is not in the preset state is used as the first video frame; based on the total number of frames of the first video frame and the target video segment, determine the next video frame to be processed to determine whether it includes the target part and whether the target part is in the preset state.

[0119] It can be understood that the video frame to be processed or the video frame to be checked that does not meet the preset condition can be used as the first video frame. The next video frame to be processed can be determined based on the total number of frames corresponding to the first video frame and the target video segment. In this embodiment, the method of determining whether the next video frame to be processed includes the target part and whether the target part is in a preset state based on the first video frame and the total number of frames of the target video segment can be:

[0120] If there are multiple first video frames after the video frame to be processed, the next video frame of the video frame to be checked that is the last in time sequence is used as the starting frame; based on the total number of frames and the starting frame, determine whether the next video frame to be processed includes the target part and whether the target part is in a preset state.

[0121] It can be understood that: from the perspective of the acquisition timing of the video frames to be checked and the video frames to be processed, the video frame that is the last in the timing can be used as the first video frame. The next video frame of the first video frame is used as the starting frame. Count half of the total number of frames backward from the starting frame, and use the video frame obtained at this time as the next video frame to be processed. The advantage of this setting is that the target video frames in the target video segment are all video frames that meet the preset conditions, and the first video frame no longer meets the preset conditions. At this time, when the next video frame is used as the video frame to be processed, the target video segment still cannot be determined. Therefore, this method can be used to improve the efficiency of determining the target video segment.

[0122] On the basis of the above technical solution, the method for determining the next video frame to be processed may also be: if the video frame to be processed does not include the target part or the target part is not in the preset state, then determining whether the next video frame to be processed of the video frame to be processed includes the target part and whether the target part is in the preset state; or,

[0123] If the target part is not included in the video frame to be checked or the target part is not in the preset state, it is determined whether the next video frame of the video frame to be processed includes the target part and whether the target part is in the preset state.

[0124] It can be understood that once it is detected that there are video frames in the video frames to be verified that do not meet the preset conditions, other video frames to be verified in the video segment will no longer be processed, and the next video frame of the video frame to be processed will be directly obtained, and the step of determining the target video segment will be repeated as the video frame to be processed.

[0125] The technical solution provided by the embodiment of the present invention can use the above method to determine the target video segment, thereby facilitating the subsequent determination of target data. The above method can improve the efficiency of determining the target video segment, thereby also improving the efficiency of determining the target data.

[0126] Figure 3 A flow chart of a data generation method provided in an embodiment of the present invention can, on the basis of the aforementioned embodiment, further refine: "determine the position information of the object to be processed in each target video frame in the target video segment according to the position information of the target part in the target video segment". The specific implementation method can refer to the detailed description of this embodiment, wherein the technical terms that are the same as or corresponding to the above-mentioned embodiment will not be repeated in this embodiment.

[0127] like Figure 3 As shown, the method includes:

[0128] S310: Determine at least one target video segment from the video to be processed.

[0129] S320: Determine the center point information of the target part according to the part information of the first video frame of the target video segment.

[0130] It can be understood that after determining the target video segment, the position information of the first video frame of the target part in the target video frequency band can be obtained. A rectangular frame corresponding to the target part can be determined, and the rectangular frame can be represented in the form of four coordinates. The four coordinates can be the four vertices of the rectangular frame. Correspondingly, the position information can be the coordinate information of the four vertices. The center point information can be the center point of the four vertices. It can also be that after determining the position information of the hand part, the coordinates of the palm are determined, and this coordinate is used as the center point information of the target part.

[0131] S330: Determine a throwing path of the object to be processed according to the size information of the object to be processed, the preset throwing information and the center point information.

[0132] The size information of the object to be processed may be the length and width of the object to be processed. The preset throwing information may include two methods: horizontal throwing and high throwing. The center point information may be the throwing starting point of horizontal throwing or high throwing. The length of the throwing path is usually a preset multiple of the width of the object to be processed.

[0133] It can be understood that a trajectory can be generated in the target video clip, the starting point of the trajectory can be the center point of the hand part, and the high throw or horizontal throw can be simulated to obtain the throwing path of the object to be processed.

[0134] On the basis of the above technical solution, the object to be processed is determined based on the following method: a plurality of video frames to be detected are analyzed and processed based on a pre-trained sorting object detection model to output the object to be processed in the video frames to be detected.

[0135] It can be understood that the video to be processed in the park can be obtained. Each video frame in the video to be processed can be used as a video frame to be detected. The sorting object detection model can be a SAM model, which can process the input video frame to be detected. The sorting object detection model can analyze and process the video frame to be detected, obtain the maximum connected area of ​​the object to be processed, and obtain the final image corresponding to the object to be processed. The above method can be used to analyze and process all video frames to be detected to obtain the objects to be processed in all video frames to be detected in the video to be processed. Based on the above method, a set including all objects to be processed can be obtained.

[0136] In this embodiment, the advantage of determining a set of objects to be processed is that the violent behavior data corresponding to various objects to be processed can be simulated, thereby improving the richness of the target data.

[0137] S340: Based on the throwing path, determine the position information of the object to be processed in each target video frame in the target video segment.

[0138] It can be understood that after obtaining the throwing path, points can be evenly selected on the throwing path according to the total number of target video frames in the target video clip to obtain multiple trajectory points corresponding to the total number of frames. Each trajectory point can be used as the position information of the object to be processed in the corresponding target video frame.

[0139] For example, the target video clip includes 10 target video frames, and 10 points can be evenly taken from the throwing path to obtain 10 pieces of position information. Based on the position information, the position information of the object to be processed in the corresponding target video frame can be determined.

[0140] S350: According to a plurality of position information associated with the target video segment, the object to be processed is merged into the target video segment to obtain target data.

[0141] It can be understood that after determining the position information of the object to be processed in each target video frame, the image corresponding to the object to be processed can be fused into the corresponding target video frame according to the position information, and the corresponding target video segment after fusion is used as the target data.

[0142] It should be noted that the target data can be obtained by directly fusing the image corresponding to the object to be processed with the target video clip, but the realism is low. This is because the image corresponding to the object to be processed is static, while the throwing speed corresponding to violent operation behavior in the real world is generally fast, which will produce motion blur. Therefore, deep learning can be used to train neural network parameters to generate motion blur data.

[0143] Optionally, based on the throwing path, determining the position information of the object to be processed in each target video frame in the target video segment includes: evenly selecting points on the throwing path according to the total number of frames of the target video segment to obtain the position information of the object to be processed in the target video frame.

[0144] That is, the number of uniformly selected points can be determined according to the total number of frames of the target video segment. Points are uniformly selected on the throwing path according to the number of uniformly selected points to obtain the position information of the object to be processed in each target video frame in the target video segment.

[0145] After the object to be processed is merged into the target video segment, the method includes: for any two target video frames in the target video segment, blurring the object to be processed in the two target video frames to obtain a blurred image; and expanding the target data according to the blurred image.

[0146] It can be understood that the objects to be sorted in two adjacent target video frames in the target video segment can be blurred to obtain a blurred image, and the target data can be expanded with the blurred image.

[0147] The technical solution provided by the embodiment of the present invention can determine the object image corresponding to the object to be processed, as well as the position information of the object to be processed in each target video frame in the target video segment, and then fill the object to be processed into the corresponding target video frame according to the position information to obtain the target data.

[0148] As an alternative embodiment of the above embodiment, Figure 4 The present invention provides a data generation method flow chart.

[0149] Usually, violent acts are usually performed by hands. In order to make the generated package data (target data) more realistic, the package throwing can be generated in the hand part. The first step can be to detect the human hand area. Specifically, the hand is located by a target detection algorithm based on deep learning. The deep learning model requires a large amount of annotated data. For cost considerations, manual annotation needs to be minimized. Therefore, images of annotated hands and objects can be selected from a public and authorized database and used as a training data set.

[0150] Train the part detection model (hand detection model). After obtaining the labeled data, train the target detection model based on deep learning. Considering the balance between detection accuracy and computational complexity, the more common target detection algorithm YoLoV8 can be used to detect hands. In order to achieve a better balance, the solution provided in the embodiment of the present invention tests multiple models of YoLoV8 and selects the M model as the final target detection model. The difficulty of detecting objects (users) and packages (objects to be processed) in different scenes varies, and the computing power support provided is also different. Different models can be selected according to specific scenarios. Specifically, the hand part attributes can be generally divided into four categories: hands holding objects, empty state, hands in contact with oneself, and hands in contact with others. The 4 categories can be changed to 2 categories: empty, others (hands holding objects, hands in contact with oneself, and hands in contact with others). Then calculate the minimum circumscribed rectangle of the labeled quadrilateral and use it as the true value of the YOLO detection for training.

[0151] Determine the target video segment. After the hand detection model training is completed, the surveillance video in the target area (park) can be selected. Each video frame to be processed in the surveillance video (video to be processed) is used as the input of the hand detection model to determine whether the video frame to be processed includes the hand part and whether the hand part is in an empty state. If the above conditions are met and the video frame to be processed includes the user, the video frames of the 5 frames before and after the video frame to be processed can be obtained. The video frame obtained at this time is used as the video frame to be verified. If the video frames to be verified include the target object, the target part, and the target part is in an empty state, the video segment consisting of these 11 video frames to be verified can be used as the target video segment.

[0152] It should be noted that the number of target video frames in the target video segment is set according to actual needs, and its specific number is not limited in this embodiment. The reason and benefit of using the above method to determine the target video segment is that the video frames in which the throwing behavior actually occurs can be avoided, and the target data can be generated based on the video frames in which the throwing behavior does not occur.

[0153] Package data collection (data collection of objects to be processed) and package detection model training (detection model of objects to be processed). The deep learning-based target detection algorithm is used to detect packages (objects to be processed) as the source data for generating packages. The deep learning model requires a large amount of labeled data, and there are two ways to collect data. The first method is to select pictures of packages or similar packages (such as suitcases) labeled in a public and authorized dataset as a training dataset. The second method can be to manually label a small number of images of packages as a training set and design a semi-supervised method to label the data. You can use the labeling tool to manually label some packages, and then use the labeled data to detect packages on unlabeled visual data. After that, the model's prediction results are manually corrected, and the corrected data is used to continue training the model. This process is repeated several times until the model's detection accuracy for the package reaches a certain threshold, such as 95%.

[0154] The data source of the object to be processed is obtained based on the object detection model. After the package detection model (object detection model) is trained, it can be used to test the model on the park surveillance video to obtain many rectangular boxes of detected packages. At this time, the rectangular boxes are in the background. If the rectangular box is directly integrated into the target video clip, there will be a problem that the rectangular box and the pasted video frame are inconsistent and not realistic enough. You can use the SAM (image processing model) large model to segment and obtain a more refined package. For example, see Figure 5 The surveillance video frame on the left is analyzed and processed based on the package detection model (the object detection model to be processed) to obtain a rectangular frame including the package, that is, the object image mentioned above, such as Figure 5The object image can be further input into the SAM model (image processing model) to analyze and process the data in the rectangular frame of the object image to obtain the best Figure 5 The image of the object to be processed is shown in the lower right corner. At this time, the image of the object to be processed with clear edge contours and refined features is obtained.

[0155] Generate target data: Pasting the image corresponding to the object to be processed directly onto the target video clip is not realistic, because the image corresponding to the object to be processed is still or moves slowly on a conveyor belt. In the real world, the throwing speed in violent operations is generally fast, which will produce motion blur. Therefore, the video frame can be blurred. The neural network parameters can be trained based on the deep learning method to generate motion blurred images, that is, blurred images, based on the trained neural network. The blurred images generated by the above method are more diverse and have stronger generalization ability.

[0156] Specifically, a trajectory is generated on the target video clip. The starting point of the trajectory is the center point of the hand in the first video frame. The length is generally 10 times the width of the package. There are two directions: horizontal and vertical upward to simulate flat throwing and high throwing. Points are evenly selected for the entire trajectory, and each point is used as the position information of the object to be processed in the target video frame. After the object to be processed is fused into the corresponding target video frame, it can be blurred. Optionally, for any two target video frames, the two target video frames can be input into the model obtained by the above training to obtain a blurred image. The target data can be expanded based on the blurred image.

[0157] The technical solution of the embodiment of the present invention determines a target video segment including multiple target video frames from the video to be processed, and the target video frame includes a target part and the target part is in a preset state, and determines the position information of the object to be processed in each target video frame in the target video segment according to the part information of the target part in the target video segment, and then merges the object to be processed into the target video segment according to multiple position information associated with the target video segment to obtain target data. This solves the problem in the prior art that it is necessary to manually find video segments of violent operations from massive video data, and then analyze and process the video segments to obtain labeled sample data, and there are problems such as high difficulty and high cost in obtaining sample data. This realizes automatic analysis and processing of the video to be processed, and then reconstructs target data including violent behavior, thereby improving the convenience and low cost of determining target data.

[0158] Figure 6 A schematic diagram of the structure of a data generating device provided by an embodiment of the present invention, the device comprises: a video segment determining module 410, a position information determining module 420 and a target data generating module 430.

[0159] Among them, the video segment determination module 410 is used to determine at least one target video segment from the video to be processed, wherein the target video segment includes multiple target video frames, the target video frames include target parts and the target parts are in a preset state; the position information determination module 420 is used to determine the position information of the object to be processed in each target video frame in the target video segment according to the part information of the target part in the target video segment for the at least one target video segment; the target data generation module 430 is used to merge the object to be processed into the target video segment based on multiple position information associated with the target video segment to obtain target data.

[0160] The technical solution of the embodiment of the present invention determines a target video segment including multiple target video frames from the video to be processed, and the target video frame includes a target part and the target part is in a preset state, and determines the position information of the object to be processed in each target video frame in the target video segment according to the part information of the target part in the target video segment, and then merges the object to be processed into the target video segment according to multiple position information associated with the target video segment to obtain target data. This solves the problem in the prior art that it is necessary to manually find video segments of violent operations from massive video data, and then analyze and process the video segments to obtain labeled sample data, and there are problems such as high difficulty and high cost in obtaining sample data. This realizes automatic analysis and processing of the video to be processed, and then reconstructs target data including violent behavior, thereby improving the convenience and low cost of determining target data.

[0161] Based on the above technical solution, the device also includes: a video acquisition module to be processed, which is used to obtain the video to be processed acquired by at least one video acquisition device in the target area, wherein the target area is an area for sorting items and / or packages to be sorted.

[0162] Based on the above technical solutions, the video segment determination module includes:

[0163] A video frame selection unit, configured to, for at least one video frame to be processed in the video to be processed, when it is determined that the video frame to be processed includes a target part and the target part is in a preset state, obtain a preset number of video frames to be checked before and after the video frame to be processed;

[0164] The video segment determining unit is used to take the video frames to be checked and the video frames to be processed as target video frames to obtain the target video segment if the target part is included in the video frames to be checked and the target part is in the preset state.

[0165] On the basis of the above technical solutions, the device further includes:

[0166] A first video frame determination module, configured to use a to-be-processed video frame or a to-be-checked video frame that does not include the target part or the target part is not in the preset state as a first video frame;

[0167] The next video frame determination module is used to determine whether the next video frame to be processed includes a target part and whether the target part is in a preset state according to the first video frame and the total number of frames of the target video segment.

[0168] On the basis of the above technical solutions, the next video frame determination module includes:

[0169] a start frame determining unit, configured to, if the number of first video frames after the video frame to be processed includes a plurality, set the next video frame of the video frame to be checked which is the last in time sequence as the start frame;

[0170] The next video frame determination unit is used to determine, based on the total number of frames and the start frame, whether the next video frame to be processed includes a target part and whether the target part is in a preset state.

[0171] On the basis of the above technical solutions, if the video frame to be processed does not include the target part or the target part is not in the preset state, then determine whether the next video frame to be processed of the video frame to be processed includes the target part and whether the target part is in the preset state; or,

[0172] If the target part is not included in the video frame to be checked or the target part is not in the preset state, it is determined whether the next video frame of the video frame to be processed includes the target part and whether the target part is in the preset state.

[0173] Based on the above technical solutions, the target video frame also includes a target object, and the target part is a local part of the target object.

[0174] On the basis of the above technical solutions, whether the to-be-processed video frame includes a target object, a target part, and whether the target part is in a preset state is determined by the following method:

[0175] Inputting the to-be-processed video frame into a pre-trained part detection model to obtain a part detection result;

[0176] When the classification identifier and the part information of the part detection result meet the preset conditions, determining that the to-be-processed video frame includes the target part and the target part is in the preset state; and

[0177] The video frame to be processed is input into a pre-trained object detection model for object recognition, and when the recognition result is a target result, it is determined that the video frame to be processed includes the target object.

[0178] On the basis of the above technical solutions, the location information determination module includes:

[0179] A center point information determining unit, configured to determine the center point information of the target part according to the part information of the first video frame of the target video segment;

[0180] A throwing path determining unit, used to determine the throwing path of the object to be processed according to the size information of the object to be processed, the preset throwing information and the center point information;

[0181] A position information determining unit is used to determine the position information of the object to be processed in each target video frame in the target video segment based on the throwing path.

[0182] On the basis of the above-mentioned technical solutions, the position information determination unit is used to evenly select points on the throwing path according to the total number of frames of the target video segment to obtain the position information of the object to be processed in the target video frame.

[0183] On the basis of the above technical solutions, the object to be processed is determined based on the following method:

[0184] A plurality of to-be-detected video frames are analyzed and processed based on a pre-trained sorting object detection model to output the to-be-processed objects in the to-be-detected video frames.

[0185] On the basis of the above-mentioned technical solutions, the method also includes: a blurred image determination module, which is used to blur the objects to be processed in any two video frames in the target video segment to obtain a blurred image; and a target data expansion module, which is used to expand the target data based on the blurred image.

[0186] The data generating device provided in the embodiment of the present invention can execute the data generating method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0187] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present invention.

[0188] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 7 A block diagram of an exemplary electronic device 50 suitable for implementing exemplary embodiments of the present invention is shown. Figure 7 The electronic device 50 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0189] like Figure 7 As shown, the electronic device 50 is in the form of a general computing device. The components of the electronic device 50 may include but are not limited to: one or more processors or processing units 501, a system memory 502, and a bus 503 connecting different system components (including the system memory 502 and the processing unit 501).

[0190] Bus 503 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0191] The electronic device 50 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 50, including volatile and non-volatile media, removable and non-removable media.

[0192] System memory 502 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 504 and / or cache memory 505. Electronic device 50 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 506 may be used to read and write non-removable, non-volatile magnetic media ( Figure 7 not shown, usually called a "hard drive"). Although Figure 7Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 503 via one or more data medium interfaces. The memory 502 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the embodiments of the present invention.

[0193] A program / utility 508 having a set (at least one) of program modules 507 may be stored, for example, in the memory 502, such program modules 507 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 507 generally perform the functions and / or methods of the embodiments described herein.

[0194] The electronic device 50 may also communicate with one or more external electronic devices 509 (e.g., keyboards, pointing devices, displays 510, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 50, and / or communicate with any device that enables the electronic device 50 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed through an input / output (I / O) interface 511. Furthermore, the electronic device 50 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 512. As shown, the network adapter 512 communicates with other modules of the electronic device 50 through the bus 503. It should be understood that although Figure 7 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 50, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0195] The processing unit 501 executes various functional applications and data processing by running the programs stored in the system memory 502, such as implementing the data generation method provided in the embodiment of the present invention.

[0196] An embodiment of the present invention also provides a storage medium containing computer executable instructions, which are used to execute the data generation method when executed by a computer processor.

[0197] The method includes:

[0198] Determine at least one target video segment from the video to be processed, wherein the target video segment includes a plurality of target video frames, the target video frames include a target part, and the target part is in a preset state;

[0199] For the at least one target video segment, determining position information of the object to be processed in each target video frame in the target video segment according to the position information of the target part in the target video segment;

[0200] According to a plurality of position information associated with the target video segment, the object to be processed is merged into the target video segment to obtain target data.

[0201] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0202] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0203] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0204] Computer program code for performing the operation of embodiments of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0205] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data generation method, characterized in that: include: Determine at least one target video segment from the video to be processed, wherein the target video segment includes a plurality of target video frames, the target video frames include a target part, and the target part is in a preset state; For the at least one target video segment, determining position information of the object to be processed in each target video frame in the target video segment according to the position information of the target part in the target video segment; According to a plurality of position information associated with the target video segment, the object to be processed is merged into the target video segment to obtain target data.

2. The method according to claim 1, characterized in that Before determining at least one target video segment from the video to be processed, the method further includes: A video to be processed captured by at least one video capture device in a target area is obtained, wherein the target area is an area where a preset action is performed on the items to be processed and / or the packages to be processed.

3. The method according to claim 1, characterized in that The step of determining at least one target video segment from the video to be processed includes: For at least one to-be-processed video frame in the to-be-processed video, when it is determined that the to-be-processed video frame includes a target part and the target part is in a preset state, obtaining a preset number of to-be-checked video frames before and after the to-be-processed video frame; If the target part is included in the video frames to be checked and the target part is in the preset state, the video frames to be checked and the video frames to be processed are used as target video frames to obtain the target video segment.

4. The method according to claim 3, characterized in that The method further comprises: Using a to-be-processed video frame or a to-be-checked video frame that does not include the target part or the target part is not in the preset state as a first video frame; According to the first video frame and the total number of frames of the target video segment, a next video frame to be processed is determined to determine whether it includes a target part and whether the target part is in a preset state.

5. The method according to claim 4, characterized in that The step of determining, based on the first video frame and the total number of frames of the target video segment, whether the next video frame to be processed includes a target part and whether the target part is in a preset state comprises: Taking the next video frame of the first video frame which is the last in time sequence as the starting frame; According to the total number of frames and the starting frame, it is determined whether the next video frame to be processed includes a target part and whether the target part is in a preset state.

6. The method according to claim 1, characterized in that The target video frame also includes a target object, and the target part is a local part of the target object.

7. The method according to claim 6, characterized in that Whether the video frame to be processed includes a target object, a target part, and whether the target part is in a preset state is determined in the following manner: Inputting the to-be-processed video frame into a pre-trained part detection model to obtain a part detection result; When the classification identifier and the part information of the part detection result meet the preset conditions, determining that the to-be-processed video frame includes the target part and the target part is in the preset state; as well as, The video frame to be processed is input into a pre-trained object detection model for object recognition, and when the recognition result is a target result, it is determined that the video frame to be processed includes the target object.

8. The method according to claim 1, characterized in that The step of determining the position information of the object to be processed in each target video frame in the target video segment according to the position information of the target part in the target video segment comprises: Determine the center point information of the target part according to the part information of the first video frame of the target video segment; Determine a throwing path of the object to be processed according to the size information of the object to be processed, the preset throwing information and the center point information; Based on the throwing path, position information of the object to be processed in each target video frame in the target video segment is determined.

9. The method according to claim 8, characterized in that The step of determining the position information of the object to be processed in each target video frame in the target video segment based on the throwing path includes: Points are evenly selected on the throwing path according to the total number of frames of the target video clip to obtain position information of the object to be processed in the target video frame.

10. The method according to claim 1, characterized in that The object to be processed is determined based on the following method: Analyzing and processing a plurality of to-be-detected video frames based on a pre-trained sorting object detection model to output an object image including the to-be-processed object; The object image is input into a pre-trained image processing model for refinement processing to obtain an object to be processed with refined features.

11. The method according to claim 1, characterized in that: After the object to be processed is merged into the target video segment, the method comprises: For any two target video frames in the target video segment, blurring the objects to be processed in the two target video frames to obtain blurred images; The target data is expanded according to the blurred image.

12. A data generating device, characterized in that: include: A video segment determination module, used to determine at least one target video segment from the video to be processed, wherein the target video segment includes a plurality of target video frames, the target video frames include a target part and the target part is in a preset state; a position information determination module, configured to determine, for the at least one target video segment, position information of the object to be processed in each target video frame in the target video segment according to the position information of the target part in the target video segment; The target data generating module is used to fuse the object to be processed into the target video segment according to a plurality of position information associated with the target video segment to obtain target data.

13. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the data generating method according to any one of claims 1 to 11.

14. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to perform the data generation method according to any one of claims 1 to 11 when executed by a computer processor.

15. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program is used to perform the data generation method according to any one of claims 1 to 11.