Data processing method, device, storage medium and electronic device
By decoding the target video stream to obtain motion vectors and combining the detection model to match the target association, the problem of high cost of apparent feature calculation in the prior art is solved, efficient multi-objective tracking and real-time video analysis are achieved, and hardware costs are reduced.
Patent Information
- Application Number
- CN202111337449.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-12
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-11-12
AI Technical Summary
In the fields of application such as transportation, the existing multi-objective tracking technology has high hardware requirements, increased costs, and high calculation costs, making it difficult to achieve real-time processing.
By decoding the target video stream, the motion vectors in the frame image are obtained as characterization information, the detection model is used to obtain the basic information and motion vectors of the target object, and the target association matching and trajectory confirmation are performed, which reduces computing power consumption and improves data processing efficiency.
It effectively reduces the computing power consumption of characterization information acquisition, improves the data processing efficiency during multi-target object tracking, reduces hardware costs, and realizes real-time video analysis in fields such as traffic monitoring.
Smart Images

Figure CN114170545B_ABST
Abstract
Description
Technical Field
[0001] Embodiments of the present invention relate to the field of data processing, and in particular, to a data processing method, device, storage medium, and electronic device. Background Art
[0002] With the development of society and the advancement of urbanization, urban populations have become denser, and more and more people like to travel and tour during festivals, which has led to the traffic, public safety, key area prevention and other related issues in urban construction gradually becoming hidden dangers in urban construction. At present, images and videos from camera monitoring can provide relevant information for urban traffic construction, sound the alarm for some dangers in advance, help relevant departments take preventive measures in advance, and provide a certain degree of safety guarantee for people's safe travel.
[0003] Obviously, real-time intelligent analysis of massive video data has a wide range of application scenarios, and video surveillance systems are widely used due to their high authenticity, accuracy, and real-time performance. Among them, obtaining the motion trajectory of specific target objects in the video through multi-target tracking technology (MOT) is the basis of many intelligent analysis applications.
[0004] Among them, the multi-target tracking technology (MOT) uses the strategy of detection first and then tracking. It obtains the coordinates of the target frame based on the depth detector, extracts the basic information of the tracked target, and uses the appearance feature algorithm to obtain the appearance features of the tracked object. The combined description of the basic information and the appearance features is used as the combined identifier for association matching.
[0005] However, in actual applications in areas such as transportation, a main control device often has to access dozens of videos, each of which contains dozens of targets. Real-time processing places very high demands on hardware. In order to control hardware costs, surface features are often not used, or more expensive hardware is used to calculate surface features.
[0006] From the above, it can be seen that the main problem in tracking and identifying target objects based on existing methods is that the computational cost of extracting the apparent features of the target objects is high. Summary of the invention
[0007] The main advantage of the present invention is that it provides a data processing method, device, storage medium and electronic device, which use the motion vector in the frame image as the representation information, decode the target video stream, and simultaneously obtain the motion vector of the target object, thereby reducing the computing power consumed in obtaining the representation information.
[0008] Another advantage of the present invention is that it provides a data processing method, device, storage medium and electronic device, which use motion vectors in frame images as representation information, decode the target video stream, and simultaneously obtain the form of the motion vector of the target object, and simultaneously obtain the motion trajectories of multiple target objects, thereby improving the data processing efficiency when tracking multiple target objects.
[0009] According to an embodiment of the present invention, there is provided a data processing method, comprising:
[0010] Decoding the target video stream to obtain a picture stream and a frame motion vector set, wherein the frame motion vector set contains representation information of all target objects;
[0011] Based on the detection model, determine at least one target object, and based on the picture stream and the frame motion vector set, use the detection model to obtain basic information and the representation information of the at least one target object;
[0012] Based on the basic information and the representation information, target association matching is performed on the target objects in different frame images to obtain a target matching group;
[0013] Based on the target matching group, association information between the target objects in the target matching group is confirmed.
[0014] In an exemplary embodiment, the method further comprises:
[0015] Based on the association information, a motion trajectory of the at least one target object is acquired.
[0016] In an exemplary embodiment, determining at least one target object based on a detection model, and acquiring basic information and the representation information of the at least one target object using the detection model based on the picture stream and the frame motion vector set includes:
[0017] Determining the at least one target object through a target detection frame of the detection model, wherein each target object corresponds to one target detection frame;
[0018] Based on the picture stream and the frame motion vector set, the detection model is used to obtain the basic information and the representation information of the target object in the target detection box in each frame image.
[0019] In an exemplary embodiment, based on the target matching group, confirming association information between the target objects in the target matching group includes:
[0020] Performing similarity scoring on the target matching group to obtain a target similarity value;
[0021] A first threshold is set, and when the target similarity value reaches the first threshold, target association processing is performed on the target objects in the target matching group to obtain an association value of the target matching group;
[0022] A second threshold is set, and based on the relationship between the association value and the second threshold, association information between the target objects in the target matching group is confirmed.
[0023] In an exemplary embodiment, the confirming the association information between the target objects in the target matching group based on the relationship between the association value and the second threshold includes:
[0024] When the association value of the target matching group reaches the second threshold, it is confirmed that the association information indicates that the target objects in the target matching group are the same target object;
[0025] When the association value of the target matching group does not reach the second threshold, it is confirmed that the association information indicates that the target objects in the target matching group are not the same target object.
[0026] According to another embodiment of the present invention, there is provided a data processing device, comprising:
[0027] A first acquisition module is used to decode the target video stream to obtain a picture stream and a frame motion vector set, wherein the frame motion vector set contains representation information of all target objects;
[0028] a second acquisition module, configured to determine at least one target object based on a detection model, and acquire basic information and the representation information of the at least one target object using the detection model based on the picture stream and the frame motion vector set;
[0029] A matching module, configured to perform target association matching on the target objects in different frame images based on the basic information and the representation information to obtain a target matching group;
[0030] The association information confirmation module is used to confirm the association information between the target objects in the target matching group based on the target matching group.
[0031] In an exemplary embodiment, the second acquisition module includes:
[0032] A target object confirmation unit, configured to determine the at least one target object through a target detection frame of the detection model, wherein each target object corresponds to one target detection frame;
[0033] An information acquisition unit is used to acquire the basic information and the representation information of the target object in the target detection box in each frame image by using the detection model based on the picture stream and the frame motion vector set.
[0034] In an exemplary embodiment, the association information confirmation module includes:
[0035] A similarity scoring unit, used to perform similarity scoring on the target matching group to obtain a target similarity value;
[0036] an association processing unit, configured to set a first threshold value, and when the target similarity value reaches the first threshold value, perform target association processing on the target objects in the target matching group to obtain an association value of the target matching group;
[0037] The association information confirmation unit is used to set a second threshold and confirm the association information between the target objects in the target matching group based on the relationship between the association value and the second threshold.
[0038] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.
[0039] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 is a hardware structure block diagram of a mobile terminal according to a data processing method of an embodiment of the present invention;
[0041] Figure 2 is a flow chart of a data processing method according to an embodiment of the present invention;
[0042] Figure 3 is a structural block diagram of a data processing device according to an embodiment of the present invention. DETAILED DESCRIPTION
[0043] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.
[0044] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.
[0045] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a detection method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.
[0046] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as a computer program corresponding to a detection method in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0047] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0048] From the above invention background, it can be seen that in order to obtain the motion trajectory of a specific target object in video data through multi-target tracking technology (MOT), it is necessary to use an appearance feature algorithm to obtain the appearance features of the tracked object. However, the real-time processing of a large number of videos and tracking of a large number of targets by the appearance feature algorithm has very high hardware requirements. In order to control hardware costs, appearance features are often not used, or more expensive hardware is used to run the appearance feature algorithm.
[0049] In order to better solve the above technical problems, the present invention discloses a data processing method, device, storage medium and electronic device, which will be described in detail one by one in the following embodiments.
[0050] See also Figure 2 , Figure 2 A flowchart of a data processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps:
[0051] S202, decoding the target video stream to obtain a picture stream and a frame motion vector set, wherein the frame motion vector set contains representation information of all target objects;
[0052] According to one embodiment of the present invention, in step S202, the target video stream may be a video stream obtained by an external video recording device that may contain the motion trajectory of the target object, that is, the motion trajectory of the target object may be contained in the target video stream, or may not be contained in the target video stream. The target object may be a person, a car (including various types of vehicles such as motor vehicles and non-motor vehicles), or an object. In the embodiment provided by the invention, the target object refers to a movable dynamic aggregate, and the movement mode may be active movement or passive movement. The above is an example of the target object and is not intended to be limiting. Correspondingly, the target video stream may be a video stream of a movable dynamic aggregate containing the above people, cars, objects, and or a combination of the above.
[0053] According to an embodiment of the present invention, preferably, in step S202, decoding the target video stream can be specifically implemented as decoding the video stream using ffmpeg, wherein obtaining a frame motion vector set refers to a motion vector set extracted from a bitstream when decoding the video stream, i.e., a set of Motion Vectors, wherein the frame motion vector set specifically corresponds to a set of motion vectors presented in each frame of an image in a picture stream, and in the embodiment provided by the present invention, the frame motion vector set includes representation information of all target objects in each frame. In other words, the frame motion vector set obtained in this step includes motion vectors of all target objects.
[0054] It is known from the prior art that in common video coding protocols such as H.264 and MPEG4, the coded bitstream consists of I, P, and B frames. Among them, the I frame is a complete frame, and only the intra-frame coding is required. The P frame contains the motion vector (MV) and motion compensation information, and the original data is restored in combination with one or more forward reference frames during decoding. The B frame is a data frame that relies on forward and backward bidirectional reference frame coding. That is to say, the P frame in the video encoding and decoding process contains the motion vector, that is, the motion vector information, without the need for additional calculation.
[0055] According to an embodiment of the present invention, the data processing method further includes:
[0056] S204: determining at least one target object based on the detection model, and acquiring basic information and the representation information of the at least one target object using the detection model based on the picture stream and the frame motion vector set;
[0057] According to an embodiment of the present invention, the step S204 further includes:
[0058] S2041, determining the at least one target object through the target detection frame of the detection model, wherein each target object corresponds to one target detection frame;
[0059] S2042. Based on the picture stream and the frame motion vector set, use the detection model to obtain the basic information and the representation information of the target object in the target detection box in each frame image.
[0060] According to an embodiment of the present invention, preferably, the detection model is a deep learning model. For example, the Yolov5 network model, wherein the algorithm used in the Yolov5 network model is a representative algorithm in the single-stage target detection algorithm, which converts the detection into a regression idea and completes the target positioning and classification at one time. According to some other embodiments of the present invention, other network models that can achieve the same detection function can also be applied, which is only for example and not for limitation.
[0061] According to an embodiment of the present invention, optionally, in each frame of image, the detection model may correspond to a target detection frame for the at least one target object in each frame of image.
[0062] According to one embodiment of the present invention, the step S2042 is specifically: according to the target detection frame, using the detection model to obtain the basic information of the target object under the target detection frame in each frame image; according to the target detection frame, using the detection model to obtain the representation information of the target object under the target detection frame in each frame image.
[0063] The basic information of the target object includes information such as the position, size and dimension of the target object. The representation information of the target object is the motion vector information obtained when the target object is decoded in the video stream, and the representation information of the target object comes from the frame motion vector set obtained by decoding the target video stream.
[0064] According to an embodiment of the present invention, optionally, based on the target detection frame, the detection model is used to obtain basic information of the target object in each frame image, specifically, using the center coordinates, scale information, shape ratio and other information of the target detection frame, and obtaining the basic information of the target object under the target detection frame through the detection model, that is, information such as the position, size and dimensions of the target object.
[0065] Among them, according to the target detection frame, the detection model is used to obtain the representation information of the target object in each frame image, that is, the motion vector, wherein the number of the motion vectors of the target object is related to the target detection frame, for example, in the frame-by-frame image, the number of motion vectors of the car category is more than the number of motion vectors of the person category, and the number of motion vectors of the bus category is more than the number of motion vectors of the battery vehicle or bicycle category. When pedestrians walk or vehicles drive in different postures, their internal motion vectors will also be different. The embodiment of the present invention utilizes the distinguishability of the motion vectors of these different types of target objects and uses the motion vector as the representation information of the target object.
[0066] It should be noted that early research on tracking target objects was mainly based on template matching technology and simple component models. After that, target detection has made great progress, and the appearance features of objects have gradually shifted from global representation to local representation, and various local feature descriptors have emerged, such as SIFT feature descriptors, shape context feature descriptors, HOG feature descriptors, and LBP feature descriptors. In the MOT task, the use of the above feature description algorithms has a high computational cost and a large time-consuming overhead, which increases the cost of practical applications.
[0067] The embodiment provided by the present invention utilizes the feature that the decoded information of the video stream naturally contains the motion vector, and extracts the motion vector information contained therein while decoding the video stream, and uses it as the representation information of the object. In this way, the computing power is reduced and the data processing speed is improved.
[0068] According to an embodiment of the present invention, the data processing method further includes:
[0069] S206, performing target association matching on the target objects in different frame images based on the basic information and the representation information to obtain a target matching group;
[0070] In step S2061, the target matching group includes a plurality of target objects from different frame images for target association matching, and there is no limit to the number of the target matching groups, which may be one or more.
[0071] According to an embodiment of the present invention, the data processing method further includes:
[0072] S208: Based on the target matching group, confirm the association information between the target objects in the target matching group;
[0073] Wherein the step S208 further comprises:
[0074] S2081, performing similarity scoring on the target matching group to obtain a target similarity value;
[0075] S2082, setting a first threshold, and when the target similarity value reaches the first threshold, performing target association processing on the target objects in the target matching group to obtain an association value of the target matching group;
[0076] S2083: Set a second threshold, and confirm association information between the target objects in the target matching group based on a relationship between the association value and the second threshold.
[0077] According to an embodiment of the present invention, optionally, the different frame images may be adjacent frame images. In some other embodiments of the present invention, the different frame images may also be non-adjacent frame images within a certain time period.
[0078] According to an embodiment of the present invention, based on the relationship between the association value and the second threshold, confirming the association information between the target objects in the target matching group includes:
[0079] When the target similarity value of the target matching group reaches the first threshold, it proves that the target objects in the target matching group have a high similarity, but at this time, the multiple target objects in the target matching group may be the same target object or different target objects. Therefore, it is necessary to perform the target association processing on the target objects in the target matching group with a high similarity in the target matching group, and confirm whether the target objects in the target matching group are the same based on the obtained association value.
[0080] According to an embodiment of the present invention, when the target similarity value of the target matching group does not reach the first threshold, it proves that the similarity of the target objects in the target matching group is not high, and the target association processing is no longer performed on the target objects in the target matching group.
[0081] According to one embodiment of the present invention, preferably, the target association processing can be implemented as Hungarian matching algorithm association processing. In other embodiments, other similar algorithms can also be used for target association processing. As long as the algorithm can achieve the same function, it can be applied to the present invention and is not limited here.
[0082] According to one embodiment of the present invention, step S2083 can be specifically implemented as follows: when the association value of the target matching group reaches the second threshold, it is confirmed that the association information is that the target objects in the target matching group are the same target object; when the association value of the target matching group does not reach the second threshold, it is confirmed that the association information is that the target objects in the target matching group are not the same target object.
[0083] According to an embodiment of the present invention, the method further comprises:
[0084] S210: Acquire a motion trajectory of the at least one target object based on the association information.
[0085] According to an embodiment of the present invention, step S210 can be specifically implemented as follows: according to the association information confirmed in step S208, that is, when the association information is that the target objects in the target matching group are the same target object, the motion trajectory of the target objects in the target matching group is obtained; when the association information is that the target objects in the target matching group are not the same target object, the motion trajectory of the target objects in the target matching group is not obtained. The motion trajectory is the motion trajectory of the target object to be tracked, and since the number of corresponding target matching groups is not limited, the motion trajectory of the target object can be the motion trajectory of one target object or the motion trajectory of multiple target objects. That is, there is no limit to the number of target objects to be tracked. In other words, the embodiment provided by the present invention can be applicable to both scenarios of single target object tracking and scenarios of multi-target object tracking.
[0086] It should be noted that for each target object to be tracked, its motion vector is obtained based on the target detection frame, and the stationary or moving state of the target object will affect the generation of its motion vector. In addition, the motion vectors of target objects of different categories are very different in appearance. The same category of target objects will affect the size and distribution of the motion vector due to different angles, postures, etc., and on this basis, the apparent features of people and cars can be distinguished.
[0087] In addition, in the application scenarios of multi-channel video streams and multiple target tracking, the motion vector information already existing in the video bitstream is extracted as the apparent feature, that is, the representation information, and its calculation speed is much faster than the calculation speed of traditional apparent features such as HOG and SIFT.
[0088] The method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, or by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.
[0089] In the present embodiment, a data processing device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.
[0090] According to another embodiment of the present invention, Figure 3 , a data processing device is provided, comprising:
[0091] A first acquisition module 30 is used to decode the target video stream to obtain a picture stream and a frame motion vector set, wherein the frame motion vector set contains representation information of all target objects;
[0092] A second acquisition module 40 is used to determine at least one target object based on the detection model, and based on the picture stream and the frame motion vector set, use the detection model to obtain basic information and the representation information of the at least one target object; a matching module 50 is used to perform target association matching on the target objects in different frame images based on the basic information and the representation information to obtain a target matching group;
[0093] The association information confirmation module 60 is used to confirm the association information between the target objects in the target matching group based on the target matching group.
[0094] According to one embodiment of the present invention, the target video stream may be a video stream obtained by an external video recording device that may contain the motion trajectory of the target object, that is, the motion trajectory of the target object may be contained in the target video stream, or may not be contained in the target video stream. The target object may be a person, a vehicle (including various types of vehicles such as motor vehicles and non-motor vehicles), or an object. In the embodiment provided by the invention, the target object refers to a movable dynamic aggregate, and the movement mode may be active movement or passive movement. The above is an example of the target object and is not intended to be limiting. Correspondingly, the target video stream may be a video stream of a movable dynamic aggregate containing the above people, vehicles, objects, and / or combinations thereof.
[0095] According to an embodiment of the present invention, preferably, decoding the target video stream can be specifically implemented as decoding the video stream using ffmpeg, wherein obtaining a frame motion vector set refers to a motion vector set extracted from a bitstream when decoding the video stream, i.e., a set of Motion Vectors, wherein the frame motion vector set specifically corresponds to a set of motion vectors presented in each frame of an image in a picture stream, and in the embodiment provided by the present invention, the frame motion vector set includes representation information of all target objects in each frame. In other words, the frame motion vector set obtained in this step includes motion vectors of all target objects.
[0096] It is known from the prior art that in common video coding protocols such as H.264 and MPEG4, the coded bitstream consists of I, P, and B frames. Among them, the I frame is a complete frame, and only the intra-frame coding is required. The P frame contains the motion vector (MV) and motion compensation information, and the original data is restored in combination with one or more forward reference frames during decoding. The B frame is a data frame that relies on forward and backward bidirectional reference frame coding. That is to say, the P frame in the video encoding and decoding process contains the motion vector, that is, the motion vector information, without the need for additional calculation.
[0097] The second acquisition module 40 further includes:
[0098] A target object confirmation unit 41 is used to determine the at least one target object through the target detection frame of the detection model, wherein each target object corresponds to one target detection frame;
[0099] The information acquisition unit 42 acquires the basic information and the representation information of the target object in the target detection frame in each frame image by using the detection model based on the picture stream and the frame motion vector set.
[0100] According to an embodiment of the present invention, preferably, the detection model is a deep learning model. For example, the Yolov5 network model, wherein the algorithm used in the Yolov5 network model is a representative algorithm in the single-stage target detection algorithm, which converts the detection into a regression idea and completes the target positioning and classification at one time. According to some other embodiments of the present invention, other network models that can achieve the same detection function can also be applied, which is only for example and not for limitation.
[0101] According to an embodiment of the present invention, optionally, in each frame of image, the detection model may correspond to a target detection frame for the at least one target object in each frame of image.
[0102] The basic information of the target object includes information such as the position, size and dimension of the target object. The representation information of the target object is the motion vector information obtained when the target object is decoded in the video stream, and the representation information of the target object comes from the frame motion vector set obtained by decoding the target video stream.
[0103] According to an embodiment of the present invention, optionally, based on the target detection frame, the detection model is used to obtain basic information of the target object in each frame image, specifically, using the center coordinates, scale information, shape ratio and other information of the target detection frame, and obtaining the basic information of the target object under the target detection frame through the detection model, that is, information such as the position, size and dimensions of the target object.
[0104] Among them, according to the target detection frame, the detection model is used to obtain the representation information of the target object in each frame image, that is, the motion vector, wherein the number of the motion vectors of the target object is related to the target detection frame, for example, in the frame-by-frame image, the number of motion vectors of the car category is more than the number of motion vectors of the person category, and the number of motion vectors of the bus category is more than the number of motion vectors of the battery vehicle or bicycle category. When pedestrians walk or vehicles drive in different postures, their internal motion vectors will also be different. The embodiment of the present invention utilizes the distinguishability of the motion vectors of these different types of target objects and uses the motion vector as the representation information of the target object.
[0105] According to an embodiment of the present invention, the target matching group includes a plurality of target objects from different frame images for target association matching, and the number of the target matching groups is not limited, and may be one or more.
[0106] The associated information confirmation module 60 further includes:
[0107] A similarity scoring unit 61 is used to perform similarity scoring on the target matching group to obtain a target similarity value;
[0108] An association processing unit 62 is used to set a first threshold value. When the target similarity value reaches the first threshold value, target association processing is performed on the target objects in the target matching group to obtain an association value of the target matching group.
[0109] The association information confirmation unit 63 is used to set a second threshold and confirm the association information between the target objects in the target matching group based on the relationship between the association value and the second threshold.
[0110] According to an embodiment of the present invention, optionally, the different frame images may be adjacent frame images. In some other embodiments of the present invention, the different frame images may also be non-adjacent frame images within a certain time period.
[0111] According to an embodiment of the present invention, when the target similarity value of the target matching group reaches the first threshold, it proves that the target objects in the target matching group have a high similarity, but at this time, the multiple target objects in the target matching group may be the same target object or different target objects. Therefore, it is necessary to perform the target association processing on the target objects in the target matching group with a higher similarity in the target matching group, and confirm whether the target objects in the target matching group are the same based on the obtained association value.
[0112] According to an embodiment of the present invention, when the target similarity value of the target matching group does not reach the first threshold, it proves that the similarity of the target objects in the target matching group is not high, and the target association processing is no longer performed on the target objects in the target matching group.
[0113] According to one embodiment of the present invention, preferably, the target association processing can be implemented as Hungarian matching algorithm association processing. In other embodiments, other similar algorithms can also be used for target association processing. As long as the algorithm can achieve the same function, it can be applied to the present invention and is not limited here.
[0114] According to one embodiment of the present invention, when the association value of the target matching group reaches the second threshold, the association information is confirmed that the target objects in the target matching group are the same target object; when the association value of the target matching group does not reach the second threshold, the association information is confirmed that the target objects in the target matching group are not the same target object.
[0115] According to an embodiment of the present invention, the data processing device further comprises:
[0116] The third acquisition module 70 is configured to acquire a motion trajectory of the at least one target object based on the associated information.
[0117] According to an embodiment of the present invention, when the associated information is that the target objects in the target matching group are the same target object, the motion trajectory of the target objects in the target matching group is obtained; when the associated information is that the target objects in the target matching group are not the same target object, the motion trajectory of the target objects in the target matching group is not obtained. The motion trajectory is the motion trajectory of the target object that needs to be tracked. Since the number of corresponding target matching groups is not limited, the motion trajectory of the target object can be the motion trajectory of one target object or the motion trajectory of multiple target objects. That is, there is no limit to the number of target objects that need to be tracked. In other words, the embodiment provided by the present invention can be applicable to both scenarios of single target object tracking and scenarios of multi-target object tracking.
[0118] It should be noted that for each target object to be tracked, its motion vector is obtained based on the target detection frame, and the stationary or moving state of the target object will affect the generation of its motion vector. In addition, the motion vectors of target objects of different categories are very different in appearance. The same category of target objects will affect the size and distribution of the motion vector due to different angles, postures, etc., and on this basis, the apparent features of people and cars can be distinguished.
[0119] In addition, in the application scenarios of multi-channel video streams and multiple target tracking, the motion vector information already existing in the video bitstream is extracted as the apparent feature, that is, the representation information, and its calculation speed is much faster than the calculation speed of traditional apparent features such as HOG and SIFT.
[0120] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.
[0121] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.
[0122] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.
[0123] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.
[0124] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.
[0125] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.
[0126] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A data processing method, It is characterized in that include: Decoding the target video stream to obtain a picture stream and a frame motion vector set, wherein the frame motion vector set contains representation information of all target objects; Based on the detection model, determine at least one target object, and based on the picture stream and the frame motion vector set, use the detection model to obtain basic information and the representation information of the at least one target object; Based on the basic information and the representation information, target association matching is performed on the target objects in different frame images to obtain a target matching group; Based on the target matching group, confirming association information between the target objects in the target matching group; The at least one target object is determined based on the detection model, and based on the picture stream and the frame motion vector set, basic information and the representation information of the at least one target object are obtained by using the detection model: the at least one target object is determined by a target detection frame of the detection model, wherein each target object corresponds to one target detection frame; Based on the picture stream and the frame motion vector set, using the detection model to obtain the basic information and the representation information of the target object under the target detection frame in each frame image; The method of confirming the association information between the target objects in the target matching group based on the target matching group includes: performing a similarity score on the target matching group to obtain a target similarity value; setting a first threshold, and when the target similarity value reaches the first threshold, performing target association processing on the target objects in the target matching group to obtain the association value of the target matching group; setting a second threshold, and confirming the association information between the target objects in the target matching group based on the relationship between the association value and the second threshold.
2. The method according to claim 1, It is characterized in that Also includes: Based on the association information, a motion trajectory of the at least one target object is acquired.
3. The method according to claim 1, It is characterized in that The confirming the association information between the target objects in the target matching group based on the relationship between the association value and the second threshold value includes: When the association value of the target matching group reaches the second threshold, it is confirmed that the association information indicates that the target objects in the target matching group are the same target object; When the association value of the target matching group does not reach the second threshold, it is confirmed that the association information indicates that the target objects in the target matching group are not the same target object.
4. A data processing device, It is characterized in that include: A first acquisition module is used to decode the target video stream to obtain a picture stream and a frame motion vector set, wherein the frame motion vector set contains representation information of all target objects; a second acquisition module, configured to determine at least one target object based on a detection model, and acquire basic information and the representation information of the at least one target object using the detection model based on the picture stream and the frame motion vector set; A matching module, configured to perform target association matching on the target objects in different frame images based on the basic information and the representation information to obtain a target matching group; An association information confirmation module, used to confirm association information between the target objects in the target matching group based on the target matching group; The second acquisition module includes: a target object confirmation unit, used to determine the at least one target object through the target detection frame of the detection model, wherein each target object corresponds to one target detection frame; an information acquisition unit, used to acquire the basic information and the representation information of the target object under the target detection frame in each frame image by using the detection model based on the picture stream and the frame motion vector set; The association information confirmation module includes: a similarity scoring unit, which is used to perform similarity scoring on the target matching group to obtain a target similarity value; an association processing unit, which is used to set a first threshold. When the target similarity value reaches the first threshold, target association processing is performed on the target objects in the target matching group to obtain the association value of the target matching group; and an association information confirmation unit, which is used to set a second threshold. Based on the relationship between the association value and the second threshold, the association information between the target objects in the target matching group is confirmed.
5. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 3 when executed.
6. An electronic device comprising a memory and a processor, It is characterized in that A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 3.
