Position parameter optimization method, trajectory generation method, equipment, medium and product

Abstract

The present application provides a position parameter optimization method, a trajectory generation method, a device, a medium, and a product, which relate to the field of target detection technology. The method includes: obtaining the initial position parameters of the moving target in the i-th frame image in the target time series data and the acquisition time deviation of the i-th frame image; generating the position parameter features of the i-th frame image based on the initial position parameters, the acquisition time deviation, and the encoded latent variables of the i-1-th frame image; obtaining the optimized position parameters of the moving target in the i-th frame image and the deviation between the position parameter features of the i-th frame image and the i-1-th frame image based on the position parameter features and the decoded latent variables of the i-1-th frame image; obtaining the smoothed position parameters of the moving target in the i-1-th frame image based on the position parameter features, the decoded latent variables of the i-2-th frame image, and the deviation. The present application can optimize the position parameters of the moving target and improve the continuity of the position parameters of the moving target at adjacent moments.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a position parameter optimization method, trajectory generation method, device, medium and product. Background Art

[0002] Object tracking or detection is one of the basic tasks in the field of deep learning, specifically outputting the bounding box parameters of several moving objects in each frame of the image.

[0003] Existing target tracking or detection algorithms determine the bounding box parameters of a moving target based on the positional features of the moving target in an image at each moment when detecting the position of the moving target.

[0004] However, since the existing solutions do not take into account that the movement of the moving target is continuous in time sequence, the accuracy of detecting the position parameters of the moving target separately at each moment is poor, resulting in drastic changes in the position parameters of the moving target between adjacent moments. Summary of the Invention

[0005] The purpose of the present invention is to provide a position parameter optimization method, trajectory generation method, equipment, medium and product to address the deficiencies in the above-mentioned prior art, so as to optimize the position parameters of the moving target and improve the continuity of the position parameters of the moving target at adjacent moments.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of the present application are as follows:

[0007] In a first aspect, an embodiment of the present application provides a method for optimizing position parameters of a moving target, the method comprising:

[0008] Obtaining the initial position parameters of the moving target in the i-th frame image in the target time series data and the acquisition time deviation of the i-th frame image, where the acquisition time deviation represents the deviation between the actual acquisition time of the image and the corresponding standard acquisition time, where i is a positive integer greater than or equal to 3;

[0009] generating a position parameter feature of the i-th frame image based on the initial position parameter, the acquisition time deviation, and the encoding latent variable of the i-1-th frame image, wherein the encoding latent variable of the i-1-th frame image is determined based on the encoding latent variable of the i-2-th frame image, the initial position parameter of the i-1-th frame image, and the acquisition time deviation;

[0010] Obtaining, based on the position parameter features of the i-th frame image and the decoded latent variables of the i-1-th frame image, an optimized position parameter of a moving object in the i-th frame image, and a deviation between the position parameter features of the i-th frame image and the i-1-th frame image, wherein the decoded latent variables of the i-1-th frame image are determined based on the decoded latent variables of the i-2-th frame image and the position parameter features of the i-1-th frame image;

[0011] Based on the position parameter features of the i-th frame image, the decoded latent variables of the i-2-th frame image, and the deviation between the position parameter features of the i-th frame image and the i-1-th frame image, the smooth position parameters of the moving target in the i-1-th frame image are obtained.

[0012] Optionally, the method further includes:

[0013] Obtaining a deviation between the position parameter features of the i-th frame image and the position parameter features of the i+1-th frame image based on the position parameter features of the i-th frame image and the decoded latent variables of the i-1-th frame image;

[0014] Based on the position parameter features of the i-th frame image, the decoded latent variables of the i-th frame image, and the deviation between the position parameter features of the i-th frame image and the i+1-th frame image, the predicted position parameters of the moving target in the i+1-th frame image are obtained.

[0015] Optionally, the step of obtaining the acquisition time deviation of the i-th frame image in the target time series data further includes:

[0016] Obtaining a standard interval of the target time series data;

[0017] The collection time deviation is normalized according to the standard interval to obtain a normalized collection time deviation.

[0018] Optionally, the acquiring of initial position parameters of the moving target in the i-th frame image in the target time series data includes:

[0019] If the normalized acquisition time deviation of the i-th frame image is not within the target deviation range, the predicted position parameters of the moving target in the i-th frame image obtained based on the i-1-th frame image are used as the initial position parameters of the moving target in the i-th frame image.

[0020] Optionally, the encoding latent variable and the decoding latent variable are parameters of a target neural network model;

[0021] The target neural network model is obtained through the following training steps:

[0022] Acquire training data, wherein the training data includes initial position parameters and target position parameters of the moving target in each frame of sample image of the sample time series data, and acquisition time deviation of each frame of sample image;

[0023] According to the initial position parameters corresponding to each frame of sample image and the acquisition time deviation, an initial neural network model is used to output training position parameters;

[0024] Constructing an optimization loss function and a smoothing loss function based on the training position parameters and the real position parameters corresponding to each frame of sample image;

[0025] Obtaining a joint loss function of each frame sample image based on the optimization loss function and the smoothing loss function;

[0026] Obtaining a total loss function of the sample time series data based on the joint loss function of the sample images of each frame;

[0027] The initial neural network model is updated based on the total loss function until the model converges to obtain the target neural network model.

[0028] Optionally, the encoding latent variable and the decoding latent variable are parameters of a target neural network model;

[0029] The target neural network model is obtained through the following training steps:

[0030] Acquire training data, wherein the training data includes initial position parameters and actual position parameters of the moving target in each frame of sample image of the sample time series data, and acquisition time deviation of each frame of sample image;

[0031] According to the initial position parameters corresponding to each frame of sample image and the acquisition time deviation, an initial neural network model is used to output training position parameters;

[0032] Based on the training position parameters and the real position parameters corresponding to each frame of sample image, an optimization loss function, a smoothing loss function and a prediction loss function are respectively constructed;

[0033] Obtaining a joint loss function of each frame sample image based on the optimization loss function, the smoothing loss function and the prediction loss function;

[0034] Obtaining a total loss function of the sample time series data based on the joint loss function of the sample images of each frame;

[0035] The initial neural network model is updated based on the total loss function until the model converges to obtain the target neural network model.

[0036] In a second aspect, an embodiment of the present application further provides a method for generating a trajectory of a moving target, the method comprising:

[0037] generating a motion trajectory of the moving target based on the optimized position parameter and the smoothed position parameter obtained by the method of the first aspect; or

[0038] Based on the optimized position parameters, the smoothed position parameters and the predicted position parameters obtained by the method of the first aspect, a motion trajectory of the moving target is generated.

[0039] In a third aspect, an embodiment of the present application further provides an electronic device comprising a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the method described in any one of the first aspects, or to implement the method described in the second aspect.

[0040] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the method described in any one of the first aspects, or implements the method described in the second aspect.

[0041] In a fifth aspect, an embodiment of the present application further provides a computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the method described in any one of the first aspects, or implements the method described in the second aspect.

[0042] The beneficial effects of this application are:

[0043] The present application provides a position parameter optimization method, a trajectory generation method, an apparatus, a medium, and a product. For the deviation of the initial position parameters and the acquisition time of the moving target in the i-th frame image, the optimized position parameters of the moving target in the i-th frame image and the smoothed position parameters of the moving target in the i-1th frame image are determined according to the deviation between the encoded latent variables of the i-1th frame image pair, the decoded latent variables of the i-1th frame image, the decoded latent variables of the i-2th frame image, and the position parameter features of the i-th frame image and the i-1th frame image. Since the deviation between the encoded latent variables, the decoded latent variables, and the position parameter features of each frame fully considers the temporal continuity of the movement of the moving target, the accuracy of the position parameters of the moving target in the i-th frame image and the i-1th frame image is improved, the drastic change of the position parameters of the moving target in the i-1th frame image and the i-1th frame image is reduced, and the motion position change of the moving target in the i-1th frame image and the i-1th frame image is made more continuous and smooth. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 Schematic diagram of the process of the motion target position parameter optimization method provided in the embodiment of the application Figure 1 ;

[0046] Figure 2 Schematic diagram of actual collection time and standard collection time;

[0047] Figure 3 Schematic diagram of the process of the motion target position parameter optimization method provided in the embodiment of the application Figure 2 ;

[0048] Figure 4 A block diagram of the principle of the position parameter optimization model provided in the embodiment of the present application;

[0049] Figure 5 Schematic diagram of the process of the motion target position parameter optimization method provided in the embodiment of the application Figure 3 ;

[0050] Figure 6 Schematic diagram of the target neural network model training steps provided in the embodiment of this application Figure 1 ;

[0051] Figure 7 Schematic diagram of the target neural network model training steps provided in the embodiment of this application Figure 2 ;

[0052] Figure 8 Schematic diagram of the structure of the position parameter optimization model provided in the embodiment of this application Figure 1 ;

[0053] Figure 9 Schematic diagram of the structure of the position parameter optimization model provided in the embodiment of this application Figure 2 ;

[0054] Figure 10 A schematic diagram of the structure of a target position encoder provided in an embodiment of the present application;

[0055] Figure 11 A schematic diagram of the structure of the target filter decoder provided in an embodiment of the present application;

[0056] Figure 12 A schematic diagram of the structure of a target smoothing decoder or a target prediction decoder provided in an embodiment of the present application;

[0057] Figure 13 Schematic diagram before and after motion trajectory optimization;

[0058] Figure 14 Schematic diagram of the process of the motion target position parameter optimization method provided in the embodiment of the application Figure 4 ;

[0059] Figure 15 A schematic diagram of the structure of a moving target position parameter optimization device provided in an embodiment of the present application;

[0060] Figure 16 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0062] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without creative work are within the scope of protection of the present application.

[0063] In addition, the terms "first," "second," and the like in the description and claims of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatus.

[0064] It should be noted that, in the absence of conflict, the features in the embodiments of this application can be combined with each other.

[0065] Existing target tracking or detection algorithms determine the bounding box parameters of a moving target based on the positional features of the moving target in an image at each moment when detecting the position of the moving target.

[0066] However, since the existing solutions do not take into account that the movement of the moving target is continuous in time sequence, the position parameters of the moving target change dramatically between adjacent moments, and the accuracy of the detected position parameters of the moving target is poor.

[0067] There are numerous traditional parameter optimization methods, such as those based on special geometric curves (such as Bezier curves and cubic splines), time-domain filtering methods (such as windowed median smoothing, windowed mean smoothing, recursive filtering, and Kalman filtering), and frequency-domain filtering methods (such as low-pass filtering). These traditional parameter optimization methods are unable to analyze the motion pattern (stationary, uniform motion, uniformly accelerated motion, etc.) of an object from dynamic data and determine a reasonable parameter estimation scheme. Some algorithms, such as median smoothing and low-pass filtering, are generally only suitable for low-speed motion scenarios. When an object moves at high speeds, these algorithms cannot respond promptly, resulting in low sensitivity and high latency. While other algorithms, such as recursive filtering and Kalman filtering, can define motion patterns, they require prior information about the signal. If the object's motion does not conform to this prior information, the algorithm will suffer from reduced accuracy or even failure. Adaptive filtering algorithms dynamically adjust filter coefficients without requiring prior information about the signal, but they require a certain amount of time to update the coefficients, which increases latency.

[0068] In addition to the above-mentioned traditional parameter optimization methods, the parameter optimization algorithm based on neural networks can also optimize the target motion parameters. However, it can only filter the results of the current frame, but cannot optimize the position parameters of the moving target in the historical frame, and still cannot reduce the drastic changes in the position parameters of the moving target between adjacent frames.

[0069] Based on this, the embodiments of the present application intend to provide a position parameter optimization method, a trajectory generation method, an apparatus, a medium and a product. For the initial position parameters and the acquisition time deviation of the moving target in the i-th frame image, the optimized position parameters of the moving target in the i-th frame image and the smoothed position parameters of the moving target in the i-1th frame image are determined according to the deviation between the encoded latent variables of the i-1th frame image pair, the decoded latent variables of the i-1th frame image, the decoded latent variables of the i-2th frame image and the position parameter characteristics of the i-1th frame image. Since the deviation between the encoded latent variables, the decoded latent variables and the position parameter characteristics of each frame fully considers the temporal continuity of the movement of the moving target, the accuracy of the position parameters of the moving target in the i-th frame image and the i-1th frame image is improved, the drastic changes in the position parameters of the moving target in the i-1th frame image and the i-1th frame image are reduced, and the motion position changes of the moving target in the i-1th frame image and the i-1th frame image are more continuous and smooth.

[0070] Please refer to Figure 1 , which is a flow chart of the method for optimizing the position parameters of a moving target provided in an embodiment of the present application Figure 1 ,like Figure 1 As shown, the method may include:

[0071] S11: Obtain the initial position parameters of the moving target in the i-th frame image in the target time series data and the acquisition time deviation of the i-th frame image. The acquisition time deviation represents the deviation between the actual acquisition time of the image and its corresponding standard acquisition time. i is a positive integer greater than or equal to 3.

[0072] In this embodiment, the target time series data includes multiple consecutive frames of images. The target time series data is a series of image data generated by the continuous motion of a moving target over a time series. The i-th frame of the target time series data is extracted, and the moving target in the i-th frame is detected using a preset external detection algorithm to determine the initial position parameters of the moving target in the i-th frame.

[0073] Please refer to Figure 2 , is a schematic diagram of the actual collection time and the standard collection time, as shown Figure 2 As shown, based on the deviation between the actual acquisition time and the standard acquisition time of the i-th frame image extracted from the target time series data, the acquisition time deviation of the i-th frame image is determined, and the acquisition time deviation between the actual acquisition time and the standard acquisition time of the i-th frame image is Δt(t).

[0074] For example, an external detection algorithm is used to perform target detection on the current frame image, a bounding box of a moving target in the current frame image is determined, and initial position parameters are determined according to the bounding box parameters.

[0075] The bounding box parameters include: the center coordinates of the bounding box (c x ,c y ) and the width and height (w,h) of the bounding box, convert the bounding box parameters to initial position parameters (x,y,s,r).

[0076] in, W and H are the width and height of the current frame image. After the change, the values ​​of the initial position parameters (x, y, s, r) are normalized to the range of [0, 1].

[0077] S12: Generate position parameter features of the i-th frame image based on the initial position parameters, the acquisition time deviation and the encoded latent variables of the i-1-th frame image.

[0078] In this embodiment, the encoding latent variable of the i-1th frame image is determined based on the encoding latent variable of the i-2th frame image, the initial position parameter of the i-1th frame image and the acquisition time deviation. Specifically, the encoding latent variable of the i-1th frame image is the motion change state of the moving target from the 1st frame image to the i-1th frame image determined by the target position encoder.

[0079] The target position encoder is used to position encode the initial position parameters, acquisition time deviation and the encoded latent variables of the i-1th frame image to determine the position parameter features of the i-th frame image. The position parameter features of the i-th frame image are used to represent the characteristics of the position change of the moving target in the i-th frame image over time.

[0080] S13: Based on the position parameter features of the i-th frame image and the decoded latent variables of the i-1-th frame image, the optimized position parameters of the moving target in the i-th frame image and the deviation between the position parameter features of the i-th frame image and the i-1-th frame image are obtained.

[0081] In this embodiment, the decoding latent variables of the i-1th frame are determined based on the decoding latent variables of the i-2th frame image and the position parameter characteristics of the i-1th frame image. Specifically, the decoding latent variables of the i-1th frame image are the motion change state of the moving target from the 1st frame image to the i-1th frame image determined by the target filtering decoder. The target filtering decoder is used to filter and decode the position parameter characteristics of the i-1th frame image and the decoding latent variables of the i-1th frame image to generate the optimized position parameters of the moving target in the i-1th frame image.

[0082] The target filter decoder is also used to determine the motion change state of the moving target from the 1st frame image to the i-1th frame image recorded in the decoding latent variable of the i-1th frame image, determine the motion parameters of the moving target in the i-1th frame image, and determine the deviation between the position parameter characteristics of the i-th frame image and the i-1th frame image based on the position parameter characteristics of the i-th frame image and the motion parameters of the moving target in the i-1th frame image.

[0083] S14: Based on the position parameter features of the i-th frame image, the decoded latent variables of the i-2-th frame image, and the deviation between the position parameter features of the i-th frame image and the i-1-th frame image, obtain the smooth position parameters of the moving target in the i-1-th frame image.

[0084] In this embodiment, the smoothed position parameters of the moving object in the in-th image frame are obtained based on the position parameter features of the i-th image frame, the decoded latent variables of the in-1-th image frame, and the deviation between the position parameter features of the (i-n+1)-th image frame and the in-th image frame, where n is greater than or equal to 1.

[0085] Specifically, according to the value of n, multiple interconnected target smoothing decoders are set, wherein the first target smoothing decoder is connected to the target filtering decoder, and the target filtering decoder provides the first target smoothing decoder with the deviation between the position parameter features of the i-th frame image and the i-1-th frame image; each target smoothing decoder provides the next target smoothing decoder with the deviation between the position parameter features of the in-th frame image and the in-1-th frame image.

[0086] The decoding latent variable of the in-1th frame image is the motion change state of the moving target from the 1st frame image to the in-1th frame image determined by the target smoothing decoder. The target smoothing decoder is used to smooth the position parameter features of the i-th frame image and the deviation between the position parameter features of the i-n+1th frame image and the in-th frame image according to the decoding latent variable of the in-1th frame image, and generate the smoothed position parameters of the moving target in the in-th frame image.

[0087] For example, if n=1, the single target smoothing decoder generates the smoothing position parameter of the i-1th frame image based on the position parameter feature of the i-th frame image, the decoding latent variable of the i-2th frame image, and the deviation between the position parameter features of the i-th frame image and the i-1th frame image.

[0088] If n=2, the first target smoothing decoder generates the smoothing position parameters of the i-1th frame image based on the position parameter features of the i-th frame image, the decoding latent variables of the i-2th frame image, and the deviation between the position parameter features of the i-th frame image and the i-1th frame image; the second target smoothing decoder generates the smoothing position parameters of the i-2th frame image based on the position parameter features of the i-th frame image, the decoding latent variables of the i-3th frame image, and the deviation between the position parameter features of the i-1th frame image and the i-2th frame image.

[0089] If n=3, the first target smoothing decoder generates the smoothing position parameters of the i-1th frame image based on the position parameter features of the i-th frame image, the decoding latent variables of the i-2th frame image, and the deviation between the position parameter features of the i-th frame image and the i-1th frame image; the second target smoothing decoder generates the smoothing position parameters of the i-2th frame image based on the position parameter features of the i-th frame image, the decoding latent variables of the i-3th frame image, and the deviation between the position parameter features of the i-1th frame image and the i-2th frame image; the third target smoothing decoder generates the smoothing position parameters of the i-3th frame image based on the position parameter features of the i-th frame image, the decoding latent variables of the i-4th frame image, and the deviation between the position parameter features of the i-2th frame image and the i-3th frame image.

[0090] It should be noted that, in the smoothed position parameters from the i-1th frame image to the inth frame image output by multiple target smoothing decoders, the deviation between the position parameter features of the i-n+1th frame image and the inth frame image input is greater than the deviation between the position parameter features of the inth frame image and the in-1th frame image input, and the deviation information between the position parameter features is accumulated. Therefore, the smoothed position parameters of the inth frame image are more accurate than the smoothed position parameters of the in-1th frame image, that is, the higher the delay of the historical frame, the higher the accuracy of the smoothed position parameters.

[0091] The method for optimizing the position parameters of a moving target provided in the above embodiment determines the optimized position parameters of the moving target in the i-th frame image and the smoothed position parameters of the moving target in the i-th frame image based on the deviation between the initial position parameters and the acquisition time of the moving target in the i-th frame image, the encoded latent variables of the i-1-th frame image, the decoded latent variables of the i-2-th frame image, and the deviation between the position parameter features of the i-th frame image and the i-1-th frame image. Since the deviation between the encoded latent variables, the decoded latent variables, and the position parameter features of each frame fully considers the temporal continuity of the movement of the moving target, the accuracy of the position parameters of the moving target in the i-th frame image and the i-1-th frame image is improved, the drastic changes in the position parameters of the moving target in the i-1-th frame image and the i-th frame image are reduced, and the motion position changes of the moving target in the i-1-th frame image and the i-th frame image are made more continuous and smooth.

[0092] Based on the above embodiment, the present application also provides a method for optimizing the position parameters of a moving target. Figure 3 , which is a flow chart of the method for optimizing the position parameters of a moving target provided in an embodiment of the present application Figure 2 ,like Figure 3 As shown, the method may further include:

[0093] S21: Based on the position parameter features of the i-th frame image and the decoded latent variables of the i-1-th frame image, obtain the deviation between the position parameter features of the i-th frame image and the i+1-th frame image.

[0094] In this embodiment, the target filter decoder is also used to determine the motion change state of the moving target from the 1st frame image to the i-1th frame image recorded in the decoding latent variable of the i-1th frame image, determine the motion parameters of the moving target in the i+1th frame image, and determine the deviation between the position parameter characteristics of the i-th frame image and the i+1th frame image based on the position parameter characteristics of the i-th frame image and the motion parameters of the moving target in the i+1th frame image.

[0095] S22: Based on the position parameter features of the i-th frame image, the decoded latent variables of the i-th frame image, and the deviation between the position parameter features of the i-th frame image and the i+1-th frame image, obtain the predicted position parameters of the moving target in the i+1-th frame image.

[0096] In this embodiment, the smoothed position parameters of the moving object in the i+mth image frame are obtained based on the position parameter features of the i-th image frame, the decoded latent variables of the i+m-1th image frame, and the deviation between the position parameter features of the i+m-1th image frame and the i+mth image frame, where m is greater than or equal to 1.

[0097] Specifically, according to the value of m, multiple interconnected prediction smoothing decoders are set, wherein the first target prediction decoder is connected to the target filtering decoder, and the target filtering decoder provides the first target prediction decoder with the deviation between the position parameter features of the i-th frame image and the i+1-th frame image; each target prediction decoder provides the next target prediction decoder with the deviation between the position parameter features of the i+m-th frame image and the i+m+1-th frame image.

[0098] The decoded latent variable of the i+m-1th frame image is the motion change state of the moving target from the 1st frame image to the i+m-1th frame image determined by the target prediction decoder. The target prediction decoder is used to smooth the position parameter features of the i-th frame image and the deviation between the position parameter features of the i+m-1th frame image and the i+m-1th frame image according to the decoded latent variable of the i+m-1th frame image, and generate the smoothed position parameters of the moving target in the i+mth frame image.

[0099] For example, if m=1, the single target prediction decoder generates the smoothed position parameters of the i+1th frame image based on the position parameter features of the i-th frame image, the decoded latent variables of the i-th frame image, and the deviation between the position parameter features of the i-th frame image and the i+1th frame image.

[0100] Please refer to Figure 4 , is a principle block diagram of the target neural network model provided in the embodiment of the present application, such as Figure 4 As shown in the figure, the initial position parameter p(t) and the acquisition time deviation Δt(t) are used as the input of the current time t and input to the target position encoder E0(t). The output of the timing encoder E0(t) is connected to the target filter decoder D0, multiple target smoothing decoders D -1 、D -3 、D -3 and target prediction decoder D1.

[0101] Target filter decoder D0 is connected to target smoothing decoder D -1Provides the deviation between the position parameter features of the i-th frame image and the i-1-th frame image, the target filter decoder D0 is connected to the target prediction decoder D1 to provide the deviation between the position parameter features of the i-th frame image and the i+1-th frame image; the target smoothing decoder D -1 、D -2 、D -3 Connected in sequence, the deviation between the position parameter features of adjacent frame images is provided to the subsequent target smoothing decoder. The filtering output of the target filter decoder D0 is the current frame position parameter. Target smoothing decoder D -1 、D -3 、D -3 The smooth outputs are the smooth position parameters of the first frame Smooth position parameters of the second previous frame and the smoothed position parameters of the third frame before The prediction output of the target prediction decoder D1 is the prediction position parameter of the first frame after

[0102] If the i-th frame image is the first frame image, that is, t=0, then only the outputs of the target filter decoder D0 and the target prediction decoder D1 are valid, and the outputs of the target smoothing decoder D -1 、D -2 、D -3 The output of is invalid; if the i-th frame image is the second frame image, that is, t = 1, then only the target filtering decoder D0 and the target smoothing decoder D -1 , the output of the target prediction decoder D1 is valid, the output of the target smoothing decoder D -2 、D -3 The output of is invalid; if the i-th frame image is the third frame image, that is, t = 2, then only the target filtering decoder D0 and the target smoothing decoder D -1 , target smoothing decoder D -2 , the output of the target prediction decoder D1 is valid, the output of the target smoothing decoder D -3 The output is invalid.

[0103] It should be noted that the target position encoder, target filter decoder, target smoothing decoder and target prediction decoder in the embodiments of the present application all use the RNN of the GRU layer, where h is the latent variable of the GRU layer, which is used to infer the latent variable h(t) at the next moment based on the latent variable h(t-1) at the previous moment.

[0104] The following combination Figure 5 A possible implementation method for obtaining the acquisition time deviation is described.

[0105] Please refer to Figure 5 , which is a flow chart of the method for optimizing the position parameters of a moving target provided in an embodiment of the present application Figure 3,like Figure 5 As shown, the acquisition time deviation of the i-th frame image in the target time series data in the above S11 may also include:

[0106] S111: Obtain the standard interval of the target time series data.

[0107] S112: Normalizing the collection time deviation according to the standard interval to obtain a normalized collection time deviation.

[0108] In this embodiment, the standard interval is a pre-set fixed time interval for extracting each frame of image from the target time series data. Since the number of image frames played per second varies for different target time series data, the time interval between two adjacent frames of image in the different target time series data varies. Therefore, the time unit for calculating the initial acquisition time deviation of the i-th frame of image in the different target time series data is not uniform. Therefore, the standard interval is used to normalize the acquisition time deviation to obtain a normalized acquisition time deviation. Specifically, the normalized acquisition time deviation Δt(t) can be calculated based on the ratio of the acquisition time deviation to the standard interval.

[0109] In a possible implementation, the step of obtaining the initial position parameters of the moving target in the i-th frame image in the target time series data in S11 may include:

[0110] If the normalized acquisition time deviation of the i-th frame image is not within the target deviation range, the predicted position parameters of the moving target in the i-th frame image obtained based on the i-1-th frame image are used as the initial position parameters of the moving target in the i-th frame image.

[0111] In this embodiment, if the normalized acquisition time deviation is not within the target deviation range, it is determined that the time difference between the actually extracted i-th frame image and the previously extracted i-1-th frame image is too large, and there is an unextracted lost frame image before the i-th frame image. Generating optimized position parameters directly based on the initial position parameters of the target to be tracked in the i-th frame image is less accurate. In this case, a target prediction decoder can be used to generate predicted position parameters of the moving target in the i-th frame image based on the position parameter features of the i-1-th frame image, the decoded latent variables of the i-1-th frame image, and the deviation between the position parameter features of the i-1-th frame image and the i-th frame image. The predicted position parameters of the moving target in the i-th frame image are used as the initial position parameters of the lost frame image, and the acquisition time deviation of the lost frame image is determined to be 0.

[0112] For example, the target deviation range may be (-0.5, +0.5]. If the value exceeds the range, it is determined that there is a lost frame image before the acquired i-th frame image.

[0113] The motion target position parameter optimization method provided by the above embodiment can use the predicted position parameters output by the previous frame image as the initial position parameters of the lost frame image when there is frame loss in the extracted image, thereby avoiding the poor accuracy of the calculated optimized position parameters, smoothed position parameters and predicted position parameters due to frame loss. Using the predicted position parameters output by the previous frame image as the initial position parameters of the lost frame image can ensure the accuracy of the calculated optimized position parameters, smoothed position parameters and predicted position parameters.

[0114] The method for optimizing the position parameters of a moving target provided by the above embodiment is implemented based on a pre-trained target neural network model, which is composed of a target position encoder, a target filter decoder, and a target smoothing decoder.

[0115] The following combination Figure 6 A possible implementation method for training the target neural network model is described.

[0116] Please refer to Figure 6 , which is a flow diagram of the target neural network model training steps provided in the embodiment of the present application Figure 1 ,like Figure 6 As shown, the target neural network model training steps may include:

[0117] S31: Acquire training data, where the training data includes initial position parameters and target position parameters of the moving target in each frame of sample image of the sample time series data, and acquisition time deviation of each frame of sample image.

[0118] In this embodiment, sample time series data is acquired. The sample time series data is a series of sample image data generated by the continuous motion of a moving object over a time series. Multiple consecutive frames of sample images are extracted from the sample time series data. The capture time deviation of each frame of the sample image is determined based on the deviation between the sample capture time of each frame extracted from the sample time series data and the standard capture time.

[0119] A preset external detection algorithm is used to detect moving targets in each sample image frame, determine the initial position parameters of the moving targets in each sample image frame, and manually label the target position parameters of the moving targets in each sample image frame. The target position parameters are used to represent the actual position of the moving targets in each sample image frame. For example, the external detection algorithm can be the YoloV5 model and the unfiltered Deepsort algorithm.

[0120] S32: Based on the initial position parameters and acquisition time deviation corresponding to each frame of sample image, the initial neural network model is used to output the training position parameters.

[0121] In this embodiment, the initial position parameters and acquisition time deviation corresponding to each frame of sample image are used as inputs of the initial neural network model, and the training position parameters corresponding to each frame of sample image are output through the initial neural network model, wherein the training position parameters include the optimized position parameters of the moving target in each frame of sample image and the smoothed position parameters of the moving target in the previous frame of sample image corresponding to each frame of sample image.

[0122] In the example, the initial neural network model is a recurrent convolutional neural network (RNN), which is used to generate the output at the current moment and the latent variables at the current moment based on the input at the current moment and the latent variables at the previous moment. The latent variables at the current moment record the implicit change information from the input at the first moment to the input at the current moment, and the latent variables at the current moment participate in the calculation at the next moment; wherein, the output at the current moment is calculated by a first activation function based on the input at the current moment and the latent variables at the previous moment, the first activation function includes a first weight parameter, the latent variable at the current moment is an implicit parameter of the recurrent convolutional neural network, and is calculated by a second activation function based on the input at the current moment and the latent variables at the previous moment, and the second activation function includes a second weight parameter.

[0123] It should be noted that for the input at the first moment in the RNN, the hidden variable at the previous moment is a randomly initialized hidden variable or is zero.

[0124] In this embodiment, the initial neural network model includes multiple RNNs, corresponding to the initial position encoder, initial filter decoder, and initial smoothing decoder. The initial filter decoder is used to output the optimized position parameters of the moving target in each sample frame, and the initial smoothing decoder is used to output the smoothed position parameters of the moving target in the previous sample frame corresponding to each sample frame.

[0125] Specifically, in the initial position encoder, the input at the current moment includes the initial position parameters of the current frame sample image and the acquisition moment deviation, the latent variable at the previous moment is the sample coding latent variable of the previous frame sample image, the output at the current moment is the sample position parameter feature of the current frame sample image, the latent variable at the current moment is the sample coding latent variable of the current frame sample image, and the sample coding latent variable of the current frame sample image is used to represent the motion change state of the moving target learned by the initial position encoder from the first frame to the current frame.

[0126] In the initial filtering decoder, the input at the current moment includes the position parameter features of the current frame sample image, the latent variable at the previous moment is the decoded latent variable of the previous frame sample image, the output at the current moment is the optimized position parameter of the moving target in the current frame sample image and the deviation between the position parameter features of the current frame sample image and the previous frame sample image, the latent variable at the current moment is the decoded latent variable of the current frame sample image, and the decoded latent variable of the current frame sample image is used to represent the motion change state of the moving target from the first frame to the current frame learned by the initial filtering decoder.

[0127] In the initial smoothing decoder, since the initial smoothing decoder is used to optimize the position parameters of the moving target in the previous frame sample image, the current moment of the initial smoothing decoder is the previous moment corresponding to the current moment of the initial filtering decoder. The input at the current moment includes the position parameter features of the current frame sample image and the deviation between the position parameter features of the current frame sample image and the previous frame sample image. The latent variable at the previous moment is the decoded latent variable of the previous two frames of sample images. The output at the current moment is the smooth position parameter of the moving target in the previous frame sample image. The latent variable at the current moment is the decoded latent variable of the previous frame sample image. The decoded latent variable of the previous frame sample image is used to represent the motion change state of the moving target from the first frame to the previous frame learned by the initial smoothing decoder.

[0128] S33: Based on the training position parameters and the real position parameters corresponding to each frame sample image, an optimization loss function and a smoothing loss function are respectively constructed.

[0129] In this embodiment, an optimization loss function is constructed based on the optimized position parameters of the current frame sample image and the target position parameters of the current frame sample image, and a smoothing loss function is calculated based on the smoothing position parameters of the previous frame sample image and the target position parameters of the previous frame sample image.

[0130] S34: Obtain a joint loss function of each frame sample image based on the optimization loss function and the smoothing loss function.

[0131] In this embodiment, based on the optimization loss function, optimization weight, smoothing loss function and smoothing weight of each frame sample image, the optimization loss function and smoothing loss function of each frame sample image are weightedly calculated to obtain a joint loss function of each frame sample image.

[0132] S35: Obtain the total loss function of the sample time series data based on the joint loss function of each frame sample image.

[0133] In this embodiment, during each round of model training, the joint loss function of each frame of sample images is accumulated according to the number of sample images in each frame of the sample time series data to obtain the total loss function of the sample time series data.

[0134] S36: Update the initial neural network model based on the total loss function until the model converges to obtain the target neural network model.

[0135] In this embodiment, the first weight parameter and the second weight parameter in the first activation function and the second activation function in the initial position encoder, the initial filtering decoder and the initial smoothing decoder are optimized respectively according to the total loss function. After multiple rounds of training, the training is stopped when the total loss function of the multi-frame sample images meets the target convergence condition, and the target neural network model including the target position encoder, the target filtering decoder and the target smoothing decoder is obtained.

[0136] Among them, the encoding latent variable of the target position encoder serves as the hidden parameter of the target position encoder, and needs to be calculated according to the second activation function optimized according to the second weight parameter of the target position encoder; the decoding latent variable of the target filter decoder serves as the hidden parameter of the target filter decoder, and needs to be calculated according to the second activation function optimized according to the second weight parameter of the target filter decoder; the decoding latent variable of the target smooth decoder serves as the hidden parameter of the target smooth decoder, and needs to be calculated according to the second activation function optimized according to the second weight parameter of the target smooth decoder.

[0137] In a possible implementation, the position parameter optimization module further includes: a target prediction decoder. Figure 7 , which is a flow diagram of the target neural network model training steps provided in the embodiment of the present application Figure 2 ,like Figure 7 As shown, the target neural network model training steps may include:

[0138] S41: Acquire training data, where the training data includes initial position parameters and actual position parameters of the moving target in each frame of sample image of the sample time series data, and acquisition time deviation of each frame of sample image.

[0139] In this embodiment, the method for obtaining the training data can refer to the above S11, which will not be described in detail in this embodiment.

[0140] S42: Based on the initial position parameters and acquisition time deviation corresponding to each frame of sample image, the initial neural network model is used to output the training position parameters.

[0141] In this embodiment, the initial position parameters and acquisition time deviation corresponding to each frame of sample image are used as inputs of the initial neural network model, and the training position parameters corresponding to each frame of sample image are output through the initial neural network model, wherein the training position parameters include the optimized position parameters of the moving target in each frame of sample image, the smoothed position parameters of the moving target in the previous frame of sample image corresponding to each frame of sample image, and the predicted position parameters of the moving target in the next frame of sample image corresponding to each frame of sample image.

[0142] Among them, the initial neural network model also includes an initial prediction decoder, which adopts an RNN structure. Since the initial prediction decoder is used to predict the position parameters of the moving target in the next frame sample image, the current moment of the initial prediction decoder is the next frame corresponding to the current frame of the initial filtering decoder. The input at the current moment includes the position parameter features of the current frame sample image and the deviation between the position parameter features of the current frame sample image and the next frame sample image. The latent variable at the previous moment is the decoded latent variable of the current frame sample image. The output at the current moment is the predicted position parameter of the moving target in the next frame sample image. The latent variable at the current moment is the decoded latent variable of the next frame sample image. The decoded latent variable of the next frame sample image is used to represent the motion change state of the moving target from the first frame to the next frame learned by the initial prediction decoder.

[0143] S43: Based on the training position parameters and the true position parameters corresponding to each frame sample image, an optimization loss function, a smoothing loss function and a prediction loss function are respectively constructed.

[0144] In this embodiment, the construction of the optimization loss function and the smoothing loss function can refer to the above S13, which will not be described in detail in this embodiment. The prediction loss function is calculated based on the predicted position parameters of the next frame sample image and the target position parameters of the next frame sample image.

[0145] S44: Obtain a joint loss function of each frame sample image based on the optimization loss function, the smoothing loss function and the prediction loss function.

[0146] In this embodiment, based on the optimization loss function, optimization weight, smoothing loss function, smoothing weight, prediction loss function and prediction weight of each frame sample image, the optimization loss function, smoothing loss function and prediction loss function of each frame sample image are weightedly calculated to obtain the joint loss function of each frame sample image.

[0147] S45: Obtain the total loss function of the sample time series data based on the joint loss function of each frame sample image.

[0148] In this embodiment, during each round of model training, the joint loss function of each frame of sample images is accumulated according to the number of sample images in each frame of the sample time series data to obtain the total loss function of the sample time series data.

[0149] S46: Update the initial neural network model based on the total loss function until the model converges to obtain the target neural network model.

[0150] In this embodiment, the first weight parameter and the second weight parameter in the first activation function and the second activation function in the initial position encoder, the initial filtering decoder, the initial smoothing decoder and the initial prediction decoder are optimized respectively according to the total loss function. After multiple rounds of training, the training is stopped when the total loss function of the multi-frame sample image meets the target convergence condition, and the target neural network model including the target position encoder, the target filtering decoder, the target smoothing decoder and the target prediction decoder is obtained.

[0151] The decoded latent variables of the target prediction decoder serve as hidden parameters of the target smoothing decoder and need to be calculated according to the second activation function optimized according to the second weight parameter of the target prediction decoder.

[0152] In one possible implementation, see Figure 8 , which is a schematic diagram of the structure of the position parameter optimization model provided in the embodiment of the present application Figure 1 ,like Figure 8 As shown, the position parameter optimization model includes: a target position encoder, a target filtering decoder, a target smoothing decoder and a target prediction decoder.

[0153] Among them, the target position encoder generates the position parameter features of the i-th frame image based on the initial position parameters of the moving target in the i-th frame image, the acquisition time deviation of the i-th frame image and the encoding latent variables of the i-1-th frame image.

[0154] The target position encoder is respectively connected to the target filter decoder, the target smoothing decoder and the target prediction decoder. The target filter decoder obtains the optimized position parameters of the i-th frame image and the deviation between the position parameter features of the i-th frame image and the i-1-th frame image based on the position parameter features of the i-th frame image and the decoding latent variables of the i-1-th frame image.

[0155] The target filter decoder is connected to the target smoothing decoder to provide the target prediction decoder with the deviation between the position parameter features of the i-th frame image and the -1-th frame image. The target smoothing decoder obtains the smooth position parameters of the moving target in the i-1-th frame image based on the position parameter features of the i-th frame image, the decoding latent variables of the i-2-th frame image, and the deviation between the position parameter features of the i-th frame image and the i-1-th frame image.

[0156] The target filtering decoder is connected to the target prediction decoder to provide the target prediction decoder with the deviation between the position parameter features of the i-th frame image and the i+1-th frame image. The target smoothing decoder obtains the predicted position parameters of the moving target in the i+1-th frame image based on the position parameter features of the i-th frame image, the decoding latent variables of the i-th frame image, and the deviation between the position parameter features of the i-th frame image and the i+1-th frame image.

[0157] In another possible implementation, see Figure 9 , which is a schematic diagram of the structure of the position parameter optimization model provided in the embodiment of the present application Figure 2 ,like Figure 9 As shown, if it is necessary to output smoothing position parameters of multiple previous frames and predicted position parameters of multiple subsequent frames, multiple target smoothing decoders and multiple target prediction decoders are required.

[0158] Among them, multiple target smoothing decoders are connected to each other, and each target smoothing decoder provides the deviation between the position parameter features of adjacent frame images to the next target smoothing decoder; multiple target prediction decoders are connected to each other, and each target prediction decoder provides the deviation between the position parameter features of adjacent frame images to the next target prediction decoder.

[0159] For example, for the current frame sample image at time t, the optimized position parameters of the moving target in the current frame sample image are output by the position parameter optimization model based on the sample initial position parameters and sample time deviation (x, y, s, r, Δt) of the current frame sample image. Smooth position parameters of moving objects in the previous i frames of sample images and the predicted position parameters of the moving target in the j-frame sample image

[0160] Taking the initial neural network model outputting the optimized position parameters of the current frame, the smoothed position parameters of the first three frames, and the predicted position parameters of the next frame as an example, the loss function calculation formulas of the initial neural network model are:

[0161] The optimized loss function L0(t) is:

[0162]

[0163] Smooth loss function L -i (t) is:

[0164]

[0165]

[0166]

[0167] Prediction loss function L j (t) is:

[0168]

[0169] The joint loss function of the current frame is:

[0170]

[0171] in, is the optimized position parameter of the sample image of the current frame, is the target position parameter of the sample image of the current frame; is the smoothing position parameter of the sample image of the previous frame, is the target position parameter of the previous frame sample image; is the smoothing position parameter of the first two frames of sample images, is the target position parameter of the first two frames of sample images; is the smoothing position parameter of the first three frames of sample images, are the target position parameters of the first three frames of sample images; is the predicted position parameter of the next frame sample image, is the target position parameter of the next frame sample image. ||*|| is the norm, w -3 ,w -2 ,w -1 ,w0,w1 are weights, for example, w -3 =8,w -2 =4,w -1 =2,w0=1,w1=1 / 2.

[0172] Since the neural network model needs to be trained based on multiple consecutive frames of sample images, the total loss function of the consecutive multiple frames of sample images is:

[0173]

[0174] Here, n can be 1024 frames.

[0175] It should be noted that if the current frame sample image is the first frame sample image, the previous first frame, the previous second frame and the previous third frame are invalid outputs; if the current frame sample image is the second frame sample image, the previous second frame and the previous third frame are invalid outputs; if the current frame sample image is the third frame sample image, the previous third frame is an invalid output. For invalid outputs, their weight is 0.

[0176] For examples, please refer to Figure 10 , is a schematic diagram of the structure of the target position encoder provided in an embodiment of the present application, such as Figure 10 As shown in the figure, the target position encoder E0(t) uses an RNN, where the target position encoder E0(t) has two input recurrent convolutional layers, one of which is used to input the initial position parameters of the moving target in the i-th frame image, and the other is used to input the acquisition time of the i-th frame image. The inputs of the two input recurrent convolutional layers are merged by the connection layer Concat and then pass through two recurrent convolutional layers to output position parameter features. The encoded latent variables of the i-1-th frame image are not shown in the figure.

[0177] In some embodiments, the RNN can use a GRU layer (Gated Recurrent Unit) as a recurrent convolution layer. The two input recurrent convolution layers [GRU C4] are used to input four-channel features and output four-channel features respectively. After Concat synthesis and 8-channel features, they are output through [GRU C8, GRU C18] to output 16-channel position parameter features.

[0178] It should be noted that the initial position parameters (x, y, s, r) are four-channel features. Since the acquisition time deviation Δt(t) is a single-channel feature, it is necessary to repeatedly stack the acquisition time deviation Δt(t) to form a four-channel acquisition time deviation (Δt, Δt, Δt, Δt).

[0179] For examples, please refer to Figure 11 , is a schematic diagram of the structure of the target filter decoder provided in an embodiment of the present application, such as Figure 11 As shown, the target filter decoder D0 uses an RNN, wherein the target filter decoder D0 has an input recurrent convolution layer, at least one intermediate recurrent convolution layer, and three output recurrent convolution layers. The input recurrent convolution layer is used to input the position parameter features of the i-th frame image. The input recurrent convolution layer has three outputs, one of which passes through the intermediate recurrent convolution layer to the first output recurrent convolution layer and is used to output the optimized position parameters of the moving target in the i-th frame image. The second output passes through the second output recurrent convolution layer to output the deviation between the position parameter features of the i-th frame image and the i-1-th frame image as the first decoding feature. The third output passes through the third output recurrent convolution layer to output the deviation between the position parameter features of the i-th frame image and the i+1-th frame image as the second decoding feature. The decoding latent variable of the i-1-th frame image is not shown in the figure.

[0180] Among them, RNN can use GRU layer as recurrent convolution layer, the input recurrent convolution layer [GRU C32] is used to output 32-channel features, the two output recurrent convolution layers [GRU C16] are used to output two 16-channel features as the deviation between the position parameter features of the i-th frame image and the i-1-th frame image, and the deviation between the position parameter features of the i-th frame image and the i+1-th frame image, the middle recurrent convolution layer [GRU C16] and another output recurrent convolution layer [GRU C4] are used to output 4-channel features as the optimized position parameters of the moving target in the i-th frame image.

[0181] For examples, please refer to Figure 12 , is a schematic diagram of the structure of the target smoothing decoder or target prediction decoder provided in an embodiment of the present application, such as Figure 12 As shown, the target smooth decoder D -i (t) or target prediction decoder Dj (t) uses the same RNN structure, including a connection layer, two intermediate recurrent convolutional layers, and two output recurrent convolutional layers, where the target smoothing decoder D -i The connection layer of (t) is used to splice the position parameter features of the i-th frame image and the deviation between the position parameter features of the i-th frame image and the i-1-th frame image, and the target smoothing decoder D -i The output of the connection layer of (t) is connected to an output cycle convolution layer to output the deviation between the position parameter features of the i-1 frame image and the i-2 frame image as the decoding feature of the post decoder, and the target smoothing decoder D -i The other output of the connection layer of (t) passes through two intermediate recurrent convolutional layers to another output recurrent convolutional layer, which is used to output the smoothed position parameters of the moving object in the i-1th frame image. The decoded latent variables of the i-2th frame image are not shown in the figure.

[0182] Target prediction decoder D j The connection layer of (t) is used to splice the position parameter features of the i-th frame image and the deviation between the position parameter features of the i-th frame image and the i+1-th frame image, and the target prediction decoder D j (t) One output of the connection layer is connected to an output cycle convolution layer outputs the deviation between the position parameter features of the i+1 frame image and the i+2 frame image, the target prediction decoder D j The other output of the connection layer of (t) passes through two intermediate recurrent convolutional layers to another output recurrent convolutional layer, which is used to output the predicted position parameters of the moving object in the i+1th frame image. The decoded latent variables of the i-th frame image are not shown in the figure.

[0183] Among them, RNN can use GRU layer as the recurrent convolution layer, an output recurrent convolution layer [GRU C16] is used to output 16-channel features as the deviation between the position parameter features of adjacent frame images, and the intermediate recurrent convolution layer [GRU C32, GRUC16] is passed through another output recurrent convolution layer [GRU C4] to output 4-channel features as the smoothed position parameters of the i-1 frame image or the predicted position parameters of the i+1 frame image.

[0184] A possible implementation method of optimizing the position parameters of a moving target using the target neural network model is described below in conjunction with an embodiment and accompanying drawings.

[0185] Based on the method for optimizing the position parameters of a moving target disclosed in the above embodiment, this embodiment further provides a method for generating a trajectory of a moving target.

[0186] In a possible implementation, the motion trajectory of the moving target may be generated according to the optimized position parameters and the smoothed position parameters.

[0187] In this embodiment, for the multiple frames of images in the target time series data, an external detection algorithm is used to determine the initial position parameters of the moving target in the multiple frames of images. Based on the initial position parameters of the moving target in the multiple frames of images, the initial motion trajectory of the moving target on the target time series data can be generated.

[0188] After the motion target position parameter optimization method of the above-mentioned embodiment is used to generate the optimized position parameters of the motion target in the i-th frame image and the smoothed position parameters of the motion target in the in-th frame image, the initial position parameters of the motion target in the i-th frame image are replaced by the optimized position parameters of the motion target in the i-th frame image, and the initial position parameters of the motion target in the in-th frame image are replaced by the smoothed position parameters of the motion target in the in-th frame image, so as to generate the motion trajectory of the motion target from the in-th frame to the i-th frame based on the optimized position parameters and the smoothed position parameters, thereby realizing the optimization of the initial motion trajectory.

[0189] In a possible implementation, a motion trajectory of the moving target may be generated according to the optimized position parameters, the smoothed position parameters, and the predicted position parameters.

[0190] After the motion target position parameter optimization method of the above embodiment is used to generate the optimized position parameters of the motion target in the i-th frame image, the smoothed position parameters of the motion target in the in-th frame image, and the predicted position parameters of the motion target in the i+M-th frame image, the initial position parameters of the motion target in the i-th frame image are replaced by the optimized position parameters of the motion target in the i-th frame image, the initial position parameters of the motion target in the in-th frame image are replaced by the smoothed position parameters of the motion target in the in-th frame image, and the initial position parameters of the motion target in the i+m-th frame image are replaced by the predicted position parameters of the motion target in the i+m-th frame image, so as to generate the target motion trajectory of the motion target in the i-th frame based on the optimized position parameters, the smoothed position parameters, and the predicted position parameters, thereby realizing the optimization of the initial motion trajectory.

[0191] It should be noted that, in the i-th frame, the position parameters of the moving target in the i-th frame are the optimized position parameters of the i-th frame generated based on the initial position parameters of the moving target in the i-th frame image, and the position parameters of the moving target in the i-1th frame are the smoothed position parameters of the i-1th frame generated based on the initial position parameters of the moving target in the i-1th frame image; then, in the i+1th frame, the position parameters of the moving target in the i-1th frame are the smoothed position parameters of the i-1th frame generated based on the initial position parameters of the moving target in the i+1th frame image, and the position parameters of the moving target in the i-1th frame are the smoothed position parameters of the i-1th frame generated based on the initial position parameters of the moving target in the i+1th frame image, that is, the various position parameters at the corresponding moment of the previous frame image output are replaced according to the various position parameters output by the latest frame image.

[0192] For examples, please refer to Figure 13 , which is a schematic diagram before and after the motion trajectory optimization, as shown Figure 13 As shown, the motion trajectory generated by the position parameters obtained by the target neural network model provided by the embodiment of the present application is closest to the real motion trajectory compared with the motion trajectory generated by the external detection algorithm.

[0193] Please refer to Figure 14 , which is a flow chart of the method for optimizing the position parameters of a moving target provided in an embodiment of the present application Figure 4 ,like Figure 14 As shown, after generating the motion trajectory of the moving target according to the optimized position parameters, the smoothed position parameters and the predicted position parameters, the method may further include:

[0194] S41: Determine at least one target interpolation point between the i-th frame image and the (i-1)-th frame image, and / or between the i-th frame image and the (i+1)-th frame image.

[0195] S42: Calculating interpolation position parameters of the target to be tracked at at least one target interpolation point based on the optimized position parameters, the smoothed position parameters and / or the predicted position parameters, and the position of at least one target interpolation point.

[0196] S43: updating the motion trajectory of the moving target according to the optimized position parameters, the smoothed position parameters, the predicted position parameters and the interpolated position parameters.

[0197] In this embodiment, since the optimized position parameters of the i-th frame, the smoothed position parameters of the in-th frame, and the predicted position parameters of the i+m-th frame are the position parameters of the moving target at different times, the changes in the position parameters between adjacent frames are discrete. In order to improve the accuracy of the motion trajectory of the moving target, new position parameters need to be inserted between multiple frames to improve the continuity of the motion trajectory.

[0198] Specifically, at least one target interpolation point is determined between the i-th frame image and the i-1-th frame image, between the in-th frame image and the in-1-th frame image, between the i-th frame image and the i+1-th frame image, and between the i+m-th frame image and the i+m+1-th frame image. Based on the position of the target interpolation point, the position parameters of the two frame images adjacent to the target interpolation point are determined. Based on the position parameters of the two frame images adjacent to the target interpolation point, the interpolation position parameters of the moving target at the target interpolation point are calculated.

[0199] According to the optimized position parameters of the i-th frame, the smoothed position parameters of the in-th frame, the one-inch position parameters of the i+m-th frame and multiple interpolated position parameters, the motion trajectory of the moving target is updated to improve the accuracy and continuity of the motion trajectory of the moving target.

[0200] For example, the target interpolation point is σ, and the moment before the target interpolation point is The next moment The interpolated motion parameters

[0201] Based on the above embodiment, the present application also provides a moving target position parameter optimization device. Figure 15 , is a structural diagram of a moving target position parameter optimization device provided in an embodiment of the present application, such as Figure 15 As shown, the device includes:

[0202] An image parameter acquisition module 11 is configured to acquire the initial position parameters of the moving target in the i-th frame image in the target time series data and the acquisition time deviation of the i-th frame image, where the acquisition time deviation represents the deviation between the actual acquisition time of the image and its corresponding standard acquisition time, and i is a positive integer greater than or equal to 3;

[0203] a position parameter feature generating module 12 for generating a position parameter feature of the i-th frame image based on the initial position parameter, the acquisition time deviation, and the encoding latent variable of the i-1-th frame image, wherein the encoding latent variable of the i-1-th frame image is determined based on the encoding latent variable of the i-2-th frame image, the initial position parameter of the i-1-th frame image, and the acquisition time deviation;

[0204] a filtering and decoding module 13 for obtaining optimized position parameters of a moving target in the i-th image frame based on the position parameter features of the i-th image frame and the decoded latent variables of the i-1-th image frame, as well as a deviation between the position parameter features of the i-th image frame and the i-1-th image frame, wherein the decoded latent variables of the i-1-th frame are determined based on the decoded latent variables of the i-2-th image frame and the position parameter features of the i-1-th image frame;

[0205] The smoothing decoding module 14 is used to obtain the smooth position parameters of the moving target in the i-1th frame image based on the position parameter characteristics of the i-th frame image, the decoding latent variables of the i-2th frame image, and the deviation between the position parameter characteristics of the i-th frame image and the i-1th frame image.

[0206] Optionally, the device further includes: a prediction decoding module;

[0207] The smoothing decoding module 14 is further configured to obtain a deviation between the position parameter features of the i-th frame image and the i+1-th frame image based on the position parameter features of the i-th frame image and the decoding latent variables of the i-1-th frame image;

[0208] The prediction decoding module is used to obtain the predicted position parameters of the moving target in the i+1 frame image based on the position parameter features of the i-th frame image, the decoding latent variables of the i-th frame image, and the deviation between the position parameter features of the i-th frame image and the i+1-th frame image.

[0209] Optionally, the image parameter acquisition module 11 further includes:

[0210] An interval acquisition unit, used to obtain the standard interval of target time series data;

[0211] The normalization calculation unit is used to normalize the collection time deviation according to the standard interval to obtain the normalized collection time deviation.

[0212] Optionally, the image parameter acquisition module 11 is specifically used to use the predicted position parameters of the moving target in the i-th frame image obtained based on the i-1-th frame image as the initial position parameters of the moving target in the i-th frame image if the acquisition time deviation after normalization of the i-th frame image is not within the target deviation range.

[0213] Optionally, the encoding latent variables and the decoding latent variables are parameters of the target neural network model;

[0214] The target neural network model is trained through the following modules;

[0215] The training data acquisition module is used to acquire training data, which includes the initial position parameters and target position parameters of the moving target in each frame of the sample time series data, as well as the acquisition time deviation of each frame of the sample image;

[0216] The model output module is used to output the training position parameters using the initial neural network model according to the initial position parameters and acquisition time deviation corresponding to each frame sample image;

[0217] A loss function calculation module is used to construct an optimization loss function and a smoothing loss function based on the training position parameters and the real position parameters corresponding to each frame sample image;

[0218] A loss function joint module is used to obtain a joint loss function of each frame sample image based on the optimization loss function and the smoothing loss function;

[0219] The loss function summary module is used to obtain the total loss function of the sample time series data based on the joint loss function of each frame sample image;

[0220] The model update module is used to update the initial neural network model based on the total loss function until the model converges to obtain the target neural network model.

[0221] Optionally, the encoding latent variables and the decoding latent variables are parameters of the target neural network model;

[0222] The target neural network model is obtained through the following modules;

[0223] The training data acquisition module is used to acquire training data, which includes the initial position parameters and the actual position parameters of the moving target in each frame of the sample time series data, as well as the acquisition time deviation of each frame of the sample image;

[0224] The model output module is used to output the training position parameters using the initial neural network model according to the initial position parameters and acquisition time deviation corresponding to each frame sample image;

[0225] The loss function calculation module is also used to construct the optimization loss function, the smoothing loss function and the prediction loss function respectively based on the training position parameters and the real position parameters corresponding to each frame sample image;

[0226] The loss function joint module is further used to obtain a joint loss function of each frame sample image based on the optimization loss function, the smoothing loss function and the prediction loss function;

[0227] The loss function summary module is used to obtain the total loss function of the sample time series data based on the joint loss function of each frame sample image;

[0228] The model update module is used to update the initial neural network model based on the total loss function until the model converges to obtain the target neural network model.

[0229] In one possible implementation, the present application further provides a moving target trajectory generation device, the device comprising:

[0230] The trajectory generation module is used to generate the motion trajectory of the moving target based on the optimized position parameters and smoothed position parameters obtained by the method of the first aspect; or to generate the motion trajectory of the moving target based on the optimized position parameters, smoothed position parameters and predicted position parameters obtained by the method of the first aspect.

[0231] The above-mentioned device is used to execute the method provided in the above-mentioned embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0232] The above modules can be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more microprocessors, or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented by scheduling program code through a processing element, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0233] Please refer to Figure 16 , is a schematic diagram of an electronic device provided in an embodiment of the present application, such as Figure 16 As shown, the electronic device 100 includes: a memory 101, a processor 102 and a computer program stored in the memory 101, and the processor 102 executes the computer program to implement the method for optimizing the position parameters of a moving target of any of the above embodiments, or to implement the method for generating the trajectory of a moving target of the above embodiments.

[0234] In one possible implementation, an embodiment of the present application also provides a computer-readable storage medium having a computer program / instruction stored thereon. When the computer program / instruction is executed by a processor, the method for optimizing the position parameters of a moving target of any of the above embodiments is implemented, or the method for generating the trajectory of a moving target of the above embodiments is implemented.

[0235] In one possible implementation, an embodiment of the present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the method for optimizing the position parameters of a moving target of any of the above-mentioned embodiments, or implements the method for generating the trajectory of a moving target of the above-mentioned embodiments.

[0236] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0237] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0238] In addition, the functional units in various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or hardware plus software functional units.

[0239] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (English: Read-Only Memory, abbreviated: ROM), a random access memory (English: Random Access Memory, abbreviated: RAM), a magnetic disk or an optical disk, and other media that can store program code.

[0240] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A method for optimizing the position parameters of a moving target, characterized in that: include: Obtaining the initial position parameters of the moving target in the i-th frame image in the target time series data and the acquisition time deviation of the i-th frame image, where the acquisition time deviation represents the deviation between the actual acquisition time of the image and the corresponding standard acquisition time, where i is a positive integer greater than or equal to 3; generating a position parameter feature of the i-th frame image based on the initial position parameter, the acquisition time deviation, and the encoding latent variable of the i-1-th frame image, wherein the encoding latent variable of the i-1-th frame image is determined based on the encoding latent variable of the i-2-th frame image, the initial position parameter of the i-1-th frame image, and the acquisition time deviation; Obtaining, based on the position parameter features of the i-th frame image and the decoded latent variables of the i-1-th frame image, an optimized position parameter of a moving object in the i-th frame image, and a deviation between the position parameter features of the i-th frame image and the i-1-th frame image, wherein the decoded latent variables of the i-1-th frame image are determined based on the decoded latent variables of the i-2-th frame image and the position parameter features of the i-1-th frame image; Based on the position parameter features of the i-th frame image, the decoded latent variables of the i-2-th frame image, and the deviation between the position parameter features of the i-th frame image and the i-1-th frame image, the smooth position parameters of the moving target in the i-1-th frame image are obtained.

2. The method according to claim 1, wherein The method further comprises: Obtaining a deviation between the position parameter features of the i-th frame image and the position parameter features of the i+1-th frame image based on the position parameter features of the i-th frame image and the decoded latent variables of the i-1-th frame image; Based on the position parameter features of the i-th frame image, the decoded latent variables of the i-th frame image, and the deviation between the position parameter features of the i-th frame image and the i+1-th frame image, the predicted position parameters of the moving target in the i+1-th frame image are obtained.

3. The method according to claim 2, wherein The step of obtaining the acquisition time deviation of the i-th frame image in the target time series data further includes: Obtaining a standard interval of the target time series data; The collection time deviation is normalized according to the standard interval to obtain a normalized collection time deviation.

4. The method according to claim 3, wherein The step of obtaining the initial position parameters of the moving target in the i-th frame image in the target time series data includes: If the normalized acquisition time deviation of the i-th frame image is not within the target deviation range, the predicted position parameters of the moving target in the i-th frame image obtained based on the i-1-th frame image are used as the initial position parameters of the moving target in the i-th frame image.

5. The method according to claim 1, wherein The encoding latent variables and the decoding latent variables are parameters of the target neural network model; The target neural network model is obtained through the following training steps: Acquire training data, wherein the training data includes initial position parameters and target position parameters of the moving target in each frame of sample image of the sample time series data, and acquisition time deviation of each frame of sample image; According to the initial position parameters corresponding to each frame of sample image and the acquisition time deviation, an initial neural network model is used to output training position parameters; Constructing an optimization loss function and a smoothing loss function based on the training position parameters and the real position parameters corresponding to each frame of sample image; Obtaining a joint loss function of each frame sample image based on the optimization loss function and the smoothing loss function; Obtaining a total loss function of the sample time series data based on the joint loss function of the sample images of each frame; The initial neural network model is updated based on the total loss function until the model converges to obtain the target neural network model.

6. The method according to any one of claims 2 to 4, wherein: The encoding latent variables and the decoding latent variables are parameters of the target neural network model; The target neural network model is obtained through the following training steps: Acquire training data, wherein the training data includes initial position parameters and actual position parameters of the moving target in each frame of sample image of the sample time series data, and acquisition time deviation of each frame of sample image; According to the initial position parameters corresponding to each frame of sample image and the acquisition time deviation, an initial neural network model is used to output training position parameters; Based on the training position parameters and the real position parameters corresponding to each frame of sample image, an optimization loss function, a smoothing loss function and a prediction loss function are respectively constructed; Obtaining a joint loss function of each frame sample image based on the optimization loss function, the smoothing loss function and the prediction loss function; Obtaining a total loss function of the sample time series data based on the joint loss function of the sample images of each frame; The initial neural network model is updated based on the total loss function until the model converges to obtain the target neural network model.

7. A method for generating a trajectory of a moving target, characterized in that: The method comprises: generating a motion trajectory of the moving target based on the optimized position parameter and the smoothed position parameter obtained by the method according to any one of claims 1 to 4; or Based on the optimized position parameters, the smoothed position parameters and the predicted position parameters obtained by the method according to any one of claims 2 to 4, a motion trajectory of the moving target is generated.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, wherein: The processor executes the computer program to implement the method according to any one of claims 1 to 6, or implements the method according to claim 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented, or the method according to claim 7 is implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented, or the method according to claim 7 is implemented.

Images