Mobile target detection method, storage medium, electronic device, system and vehicle

Abstract

The present application relates to a mobile target detection method, storage medium, electronic device, system and vehicle. The mobile target detection method comprises: generating or providing at least one predicted target in a detection frame image; determining the mobile target in the detection frame image based on feature comparison between the predicted target in the detection frame image and the predicted targets in multiple associated frame images. The present application improves the accuracy of mobile target detection and can adapt to more complex application scenarios.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to a moving target detection method, storage medium, electronic device, system and vehicle. Background Art

[0002] In order to improve vehicle safety, more and more vehicle manufacturers are beginning to equip their vehicles with moving target detection systems. The moving target detection system can detect moving targets around the vehicle to further save video data, trigger the vehicle to start warning functions or notify the owner, etc., to provide safety protection for vehicles in the parking state. However, most of the current moving target detection methods are traditional target detection methods, such as background subtraction, frame difference or optical flow methods, which are not accurate and prone to missed detection or false detection. Summary of the invention

[0003] The embodiments of the present application provide a mobile target detection method, storage medium, electronic device, system and vehicle, which improve the accuracy of mobile target detection and can adapt to more complex application scenarios to at least partially solve the above-mentioned technical problems.

[0004] To achieve the above-mentioned purpose, according to the first aspect of the present application, a mobile target detection method is provided, including: generating or providing at least one predicted target in a detection frame image; determining the mobile target in the detection frame image based on feature comparison between the predicted target in the detection frame image and the predicted targets in multiple associated frame images.

[0005] Optionally, the feature comparison includes position feature comparison and / or image feature comparison.

[0006] Optionally, the associated frame image is a frame image data that is earlier or later than the detection frame image among multiple frames of image data of a video data.

[0007] Optionally, determining the moving target in the detection frame image based on feature comparison between the predicted target in the detection frame image and the predicted targets in multiple associated frame images includes: performing feature comparison between the predicted target in the detection frame image and the predicted targets in multiple associated frame images having different inter-frame intervals.

[0008] Optionally, generating or providing at least one predicted target in a detection frame image includes: in response to the input of the video data, inputting the video data into a target detection model so that the target detection model outputs predicted target information for each frame of image data in the video data; wherein the predicted target information includes the predicted target in the image data.

[0009] Optionally, determining the moving target in the detection frame image based on feature comparison between the predicted target in the detection frame image and the predicted targets in multiple associated frame images includes: for the image data of the i-th frame in the video data, determining the moving target information of the image data of the i-th frame based on the predicted target information of the image data of the i-th frame and the predicted target information of at least one frame of image data before and / or after the i-th frame; determining the moving target in the image data of the i-th frame based on the moving target information of the image data of the i-th frame; wherein the detection frame image is the image data of the i-th frame, and the associated frame image is at least one frame of image data before and / or after the i-th frame; the moving target information includes the predicted target whose position is moved in the image data; and i is a positive integer.

[0010] Optionally, determining the moving target information of the image data of the i-th frame based on the predicted target information of the image data of the i-th frame and the predicted target information of at least one frame of image data before and / or after the i-th frame includes: determining m first target information of the image data of the i-th frame based on the predicted target information of the image data of the i-th frame and the predicted target information of the image data of m frames before the i-th frame; determining m second target information of the image data of the i-th frame based on the predicted target information of the image data of the i-th frame and the predicted target information of the image data of m frames after the i-th frame; determining the moving target information of the image data of the i-th frame based on the m first target information and m second target information of the image data of the i-th frame; wherein m is a positive integer.

[0011] Optionally, determining m first target information of the image data of the i-th frame according to the predicted target information of the image data of the i-th frame and the predicted target information of the image data of m frames before the i-th frame comprises: determining m first target information of the image data of the i-th frame according to the predicted target information of the image data of the i-th frame and the predicted target information of the image data of the m frames before the i-th frame m The predicted target information of the image data of the frame is determined to determine the image data of the i-th frame relative to the i-th frame. m The method comprises: determining m second target information of the image data of the i-th frame according to the predicted target information of the image data of the i-th frame and the predicted target information of the image data of the m frames after the i-th frame, comprising: determining m second target information of the image data of the i-th frame according to the predicted target information of the image data of the i-th frame and the predicted target information of the image data of the m frames after the i-th frame. m The predicted target information of the image data of the frame is used to determine the image data of the i-th frame relative to the i+k-th frame. mThe second target information of the image data of the frame; wherein the k m Is a positive integer.

[0012] Optionally, determining the moving target information of the i-th frame of image data according to the m first target information and the m second target information of the i-th frame of image data comprises: determining the moving target information of the i-th frame of image data relative to the i-th frame of image data; m The first target information of the image data of the i-th frame, and the image data of the i-th frame relative to the i+k-th frame m The second target information of the image data of the frame is subjected to intersection processing to obtain a fused target information of the image data of the i-th frame; and the m fused target information of the image data of the i-th frame are subjected to union processing to obtain the moving target information of the image data of the i-th frame.

[0013] Optionally, determining the moving target information of the image data of the i-th frame based on the predicted target information of the image data of the i-th frame and the predicted target information of at least one frame of image data located before and / or after the i-th frame includes: constructing a predicted target matrix of two frames of image data based on the predicted target information of the image data of the i-th frame and the predicted target information of one frame of image data located before and / or after the i-th frame; performing feature similarity operation on each element in the predicted target matrix to obtain a target feature similarity matrix corresponding to the predicted target matrix; determining the moving target information of the image data of the i-th frame based on the target feature similarity matrix; wherein each element in the predicted target matrix includes two predicted targets belonging to different image data; and each element in the target feature similarity matrix indicates the target feature similarity between two predicted targets.

[0014] Optionally, performing a feature similarity operation on each element in the predicted target matrix to obtain a target feature similarity matrix corresponding to the predicted target matrix includes: performing a first similarity operation on each element in the predicted target matrix to obtain a motion feature similarity matrix corresponding to the predicted target matrix; performing a second similarity operation on each element in the predicted target matrix to obtain an image feature similarity matrix corresponding to the predicted target matrix; fusing the motion feature similarity matrix and the image feature similarity matrix to obtain the target feature similarity matrix; wherein each element in the motion feature similarity matrix indicates the motion feature similarity between two predicted targets; and each element in the image feature similarity matrix indicates the image feature similarity between two predicted targets.

[0015] Optionally, performing a second similarity operation on each element in the predicted target matrix to obtain an image feature similarity matrix corresponding to the predicted target matrix includes: for each element in the predicted target matrix, respectively extracting corresponding local data from two frames of the image data to construct a local image matrix corresponding to the predicted target matrix; performing feature extraction processing on each element in the local image matrix to construct a local feature matrix corresponding to the local image matrix; performing the second similarity operation on each element in the local feature matrix to obtain the image feature similarity matrix; wherein each element in the local image matrix includes the local data corresponding to the two predicted targets in the image data to which they belong; and each element in the local feature matrix includes the image features of each of the two predicted targets.

[0016] Optionally, determining the moving target in the image data of the i-th frame based on the moving target information of the image data of the i-th frame includes: taking the predicted target that meets preset conditions in the moving target information of the image data of the i-th frame as the moving target in the image data of the i-th frame; wherein the preset conditions include at least one of the following: a moving distance of the predicted target is greater than a preset distance, and a target category of the predicted target belongs to a preset category.

[0017] According to a second aspect of the present application, a non-temporary computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the moving target detection method as described above is implemented.

[0018] According to a third aspect of the present application, an electronic device is provided, comprising: a memory on which a computer program is stored; and a processor for executing the computer program in the memory to implement the mobile target detection method as described above.

[0019] According to the fourth aspect of the present application, a mobile target detection system is provided, including: a mobile target detection module, used to: generate or provide at least one predicted target in a detection frame image; determine the mobile target in the detection frame image based on feature comparison between the predicted target in the detection frame image and the predicted targets in multiple associated frame images.

[0020] According to a fifth aspect of the present application, a vehicle is provided, the vehicle comprising the electronic device as described above, or comprising the moving target detection system as described above.

[0021] The benefits of this application are: it improves the accuracy of moving target detection and can adapt to more complex application scenarios.

[0022] More specifically, some embodiments of the present application may produce the following specific beneficial effects:

[0023] The mobile target detection method provided in the embodiment of the present application improves the accuracy of mobile target detection and can adapt to more complex application scenarios. For example, when the embodiment of the present application is applied to a vehicle scenario, it can adapt to complex indoor and / or outdoor parking scenarios and reduce the missed detection rate and false detection rate of mobile targets.

[0024] The embodiment of the present application matches each frame of image data with the previous / following frame of image data based on the output of the target detection model, that is, for each frame of image data, the moving target is determined by combining the frame of image data and at least one frame of image data before and / or after the frame of image data, which can reduce the target detection ghosting problem caused by target occlusion or removal of occlusion. The moving target detection method provided in the embodiment of the present application determines the moving target for each frame of image data by combining the frame of image data and the image data before and after the frame of image data, further reducing the target detection ghosting problem.

[0025] The embodiment of the present application matches different target movement speeds by setting and adjusting the inter-frame interval. When the inter-frame interval is larger, targets moving at a slower speed can be detected more accurately. When the inter-frame interval is smaller, targets moving at a faster speed can be detected more accurately, thereby achieving full detection of targets moving at various speeds in the image data.

[0026] The mobile target detection method provided in the embodiment of the present application determines the mobile target by performing feature similarity calculation on the predicted targets in two frames of image data. The feature similarity calculation considers the image features of the predicted target in addition to the motion features of the predicted target, making the similarity calculation between the predicted targets more accurate, helping to improve the accuracy of mobile target detection and further reducing the missed detection rate and false detection rate of mobile targets.

[0027] Other features and advantages of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without creative work.

[0029] In order to more completely understand the present application and its beneficial effects, the following description will be given in conjunction with the accompanying drawings, wherein the same figure numbers represent the same parts in the following description.

[0030] Figure 1 is a flow chart of a moving target detection method provided in an embodiment of the present application;

[0031] Figure 2 is a training diagram of a target detection model provided in an embodiment of the present application;

[0032] Figure 3 is a schematic diagram of a moving target detection process provided by an embodiment of the present application;

[0033] Figure 4 is a block diagram of an electronic device provided in an embodiment of the present application;

[0034] Figure 5 is a block diagram of another electronic device provided in an embodiment of the present application;

[0035] Figure 6 is a schematic diagram of a mobile target detection system provided in an embodiment of the present application;

[0036] Figure 7 It is a flowchart of another moving target detection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0038] According to a first aspect of the present application, an embodiment of the present application provides a moving target detection method.

[0039] See also Figures 1 to 7 ,in, Figure 7 : is a flow chart of a moving target detection method provided by an embodiment of the present application. The moving target detection method may include the following steps:

[0040] Step S710: Generate or provide at least one predicted target in a detection frame image;

[0041] Step S720: Determine the moving target in the detection frame image based on feature comparison between the predicted target in the detection frame image and the predicted targets in a plurality of associated frame images.

[0042] Optionally, the associated frame image is a frame image data that is earlier or later than the detection frame image among multiple frames of image data of a video data.

[0043] Optionally, the feature comparison includes position feature comparison and / or image feature comparison.

[0044] Optionally, the above step S720 includes: performing feature comparison between the predicted target in the detection frame image and the predicted targets in a plurality of associated frame images with different inter-frame intervals.

[0045] The above step S710 can be implemented by step S110 in the following embodiment, and the above step S720 can be implemented by steps S120 and S130 in the following embodiment. For example, the above detection frame image can be implemented as the i-th frame image data in the following embodiment, and the above associated frame image can be implemented as at least one frame image data located before and / or after the i-th frame in the following embodiment. For the introduction of steps S710 and S720, please refer to the following embodiment, which will not be repeated here.

[0046] See also Figure 1 , Figure 1 : is a flow chart of a moving target detection method provided by an embodiment of the present application. The moving target detection method may include the following steps:

[0047] Step S110: In response to the input of video data, the video data is input into the target detection model, so that the target detection model outputs predicted target information of each frame of image data in the video data;

[0048] Step S120: for the i-th frame of image data in the video data, determining the moving target information of the i-th frame of image data according to the predicted target information of the i-th frame of image data and the predicted target information of at least one frame of image data before and / or after the i-th frame;

[0049] Step S130: determining a moving target in the i-th frame of image data according to the moving target information of the i-th frame of image data.

[0050] The video data includes at least one frame of image data. When the embodiment of the present application is applied to a vehicle scene, the video data may be collected by a device with a video acquisition function carried by the vehicle, such as a driving recorder, a panoramic image camera and / or other sensors, which may indicate the surrounding environment of the vehicle. Optionally, the embodiment of the present application may be applied to a vehicle in a parked state, that is, the vehicle is stationary, so that the video data input in step S110 refers to the video data collected by the vehicle in the parked state.

[0051] In response to the input of video data, the embodiment of the present application inputs the video data into the target detection model, and performs target detection on each frame of image data in the video data through the target detection model, so that the target detection model outputs the predicted target information of each frame of image data. The embodiment of the present application does not limit the specific model type of the target detection model. Optionally, the target detection model is a deep learning model, for example, single-stage target detection models such as YOLO or SSD (Single Shot MultiBox Detector), two-stage target detection models such as R-CNN (Region-based Convolutional Neural Networks) and Faster R-CNN, DETR (Detection Transformer) based on Transformer architecture, etc. In actual applications, the target detection model whose calculation amount and accuracy match the deployment subject can be specifically selected in combination with the application requirements. For example, when the embodiment of the present application is applied to a vehicle scenario, a lightweight target detection model can be selected to facilitate vehicle-side deployment.

[0052] Before the target detection model is applied, the target detection model can be trained and tuned, and after the accuracy of the target detection model meets the requirements, the target detection model and its parameters are packaged and deployed. Optionally, when the embodiment of the present application is applied to a vehicle scenario, the training of the target detection model can be performed on other devices, such as computer devices such as servers.

[0053] In order to make the prediction results of the target detection model more accurate, Figure 2 As shown, the embodiment of the present application constructs a training data set and a validation data set, trains the target detection model through the training data set, adjusts the parameters of the target detection model, until the test results of the target detection model on the validation data set meet the accuracy requirements of subsequent use, and solidifies and saves the target detection model and its parameters. Among them, in order to improve the robustness and accuracy of the target detection model, such as Figure 2 As shown in the figure, when constructing the training dataset and the validation dataset, it is necessary to collect as many sample categories as possible, and then clean, transform, pre-label, and manually label each sample data. For example, when constructing the training dataset and the validation dataset, you can set the sample target category mv_class and the sample scene category mv_scene.

[0054]

[0055]

[0056] Wherein, p is the number of sample target categories, and p is a positive integer, and the sample target categories include but are not limited to: people, small cars, large cars, electric vehicles, motorcycles, etc.; q is the number of sample scene categories, and q is a positive integer, and the sample scene categories include but are not limited to: time scenes, location scenes, lighting scenes, target density scenes, etc., or the sample scene categories include but are not limited to: roadside parking scenes, parking lot parking scenes, and garage parking scenes, etc.

[0057] In step S110, the target detection model (such as the trained target detection model) processes the video data to obtain the predicted target information of each frame of image data in the video data. The predicted target information includes the predicted targets in the image data, for example, people, animals, small cars, large cars, electric cars, motorcycles, etc. in the image data. Of course, the predicted target information may also include other contents, such as the predicted probability of each predicted target, the predicted features of each predicted target, etc.

[0058] For each frame of image data in the video data, the embodiment of the present application combines the predicted target information of the frame of image data and the predicted target information of at least one frame of image data before and / or after the frame of image data to determine the moving target information of the frame of image data. For example, for the i-th frame of image data in the video data, the moving target information of the i-th frame of image data is determined based on the predicted target information of the i-th frame of image data and the predicted target information of at least one frame of image data before and / or after the i-th frame; i is a positive integer. For each frame of image data, the embodiment of the present application combines the frame of image data and at least one frame of image data before and / or after it to determine the moving target, which can reduce the problem of target detection ghosting caused by phenomena such as target occlusion or removal of occlusion.

[0059] Among them, for the i-th frame image data in the video data, the embodiment of the present application can determine the moving target information of the i-th frame image data according to the predicted target information of the i-th frame image data and the predicted target information of at least one frame image data before the i-th frame image data; or, the moving target information of the i-th frame image data can be determined according to the predicted target information of the i-th frame image data and the predicted target information of at least one frame image data after the i-th frame image data; or, the moving target information of the i-th frame image data can be determined according to the predicted target information of the i-th frame image data, the predicted target information of at least one frame image data before the i-th frame image data, and the predicted target information of at least one frame image data after the i-th frame image data. For example, for the i-th frame image data in the video data, the moving target information of the i-th frame image data is determined according to the predicted target information of the i-th frame image data, the predicted target information of the x-frame image data before the i-frame image data, and the predicted target information of the y-frame image data after the i-frame image data; x and y can be positive integers with the same value or positive integers with different values.

[0060] The moving target information of the image data includes predicted targets with position movement in the image data. In an embodiment of the present application, the predicted targets with position movement may be determined by comparing the positions of each predicted target in the predicted target information of two image data; or, the predicted targets with position movement may be determined by calculating the similarity and / or spatial position of each predicted target in the predicted target information of two image data. For other descriptions of the process of determining the moving target information, please refer to the following embodiments, which will not be elaborated here.

[0061] In step S130, for the i-th frame of image data, the moving target in the i-th frame of image data is determined according to the moving target information of the i-th frame of image data. Optionally, for each frame of image data, the predicted target included in the moving target information of the frame of image data can be directly used as the moving target in the frame of image data; or the predicted target included in the moving target information of the frame of image data can be screened to determine the moving target in the frame of image data.

[0062] Based on this, optionally, the above step S130 includes: taking the predicted target that meets the preset condition in the moving target information of the i-th frame image data as the moving target in the i-th frame image data. The preset condition includes at least one of the following: the moving distance of the predicted target is greater than the preset distance, and the target category of the predicted target belongs to the preset category. By setting the preset condition to filter the predicted targets included in the moving target information, the detection of the moving target can be made more accurate and meet the personalized needs of the user.

[0063] In some embodiments, the above step S130 further includes: executing a preset task according to the moving target in each frame of image data. The embodiment of the present application does not limit the specific task content of the preset task. Optionally, the preset task includes an alarm task. For example, when the embodiment of the present application is applied to a vehicle scene, if there is a moving target, a vehicle alarm mechanism such as honking can be triggered; or, the preset task includes a notification task. For example, when the embodiment of the present application is applied to a vehicle scene, if there is a moving target, a notification message can be sent to the owner to indicate the moving target and its location, etc.; or, the preset task includes a recording task. For example, when the embodiment of the present application is applied to a vehicle scene, if there is a moving target, a vehicle recording function can be triggered to record the movement of the moving target and save the recorded video.

[0064] In summary, the mobile target detection method provided by the embodiment of the present application responds to the input of video data, inputs the video data into the target detection model, and outputs the predicted target information of each frame of image data in the video data after processing by the target detection model; then, for each frame of image data, the mobile target information of the frame of image data is determined according to the predicted target information of the frame of image data and the predicted target information of at least one frame of image data before and / or after the frame; then, the mobile target in the frame of image data is determined according to the mobile target information of each frame of image data. The embodiment of the present application improves the accuracy of mobile target detection by introducing a target detection model to detect mobile targets, and can adapt to more complex application scenarios. For example, when the embodiment of the present application is applied to vehicle scenarios, it can adapt to complex indoor and / or outdoor parking scenarios and reduce the missed detection rate and false detection rate of mobile targets. Furthermore, the embodiment of the present application matches each frame of image data with the previous / following frame of image data based on the output of the target detection model, that is, for each frame of image data, the moving target is determined by combining the frame of image data and at least one frame of image data before and / or after it, which can reduce the target detection ghosting problem caused by target occlusion or removal of occlusion. In addition, when the embodiment of the present application is applied to a vehicle scene, the embodiment of the present application can be implemented as a mobile target detection method alone, or as a mobile target detection part based on video data in a multi-sensor fusion detection framework when the vehicle performance is matched.

[0065] In some embodiments, the above step S120 may include the following steps:

[0066] Step S121: determining m first target information of the i-th frame image data according to the predicted target information of the i-th frame image data and the predicted target information of the m-frame image data before the i-th frame;

[0067] Step S122: determining m second target information of the i-th frame image data according to the predicted target information of the i-th frame image data and the predicted target information of the m-frame image data located after the i-th frame;

[0068] Step S123: Determine the moving target information of the i-th frame image data according to the m first target information and the m second target information of the i-th frame image data.

[0069] The embodiment of the present application can simultaneously combine m frames of image data before the current frame and m frames of image data after the current frame to further reduce the problem of target detection ghosting, where m is a positive integer.

[0070] Wherein, when m is 1, the image data of a frame before the i-th frame and the image data of a frame after the i-th frame may refer to the image data of the adjacent frames of the i-th frame, or may refer to the image data with a certain inter-frame interval before the i-th frame. Similarly, the inter-frame intervals between the m-frame image data before the i-th frame and the i-th frame image data are k1 to k m The inter-frame intervals between the m frames of image data after the i-th frame and the i-th frame of image data are k1 to k m , where k1 to k m The value of any inter-frame interval in can be 1 or a positive integer greater than 1, and k1 to k m The intervals between any two frames are not the same.

[0071] In step S121, based on the predicted target information of the i-th frame image data and the predicted target information of the image data of a frame before the i-th frame, a first target information of the i-th frame image data can be determined, and thus, based on the predicted target information of the i-th frame image data and the predicted target information of the m-frame image data before the i-th frame, m first target information of the i-th frame image data can be determined. The first target information includes a predicted target in the image data that is moved relative to the image data before it.

[0072] In step S122, based on the predicted target information of the i-th frame image data and the predicted target information of a frame image data after the i-th frame, a second target information of the i-th frame image data can be determined, and thus, based on the predicted target information of the i-th frame image data and the predicted target information of m frames image data after the i-th frame, m second target information of the i-th frame image data can be determined. The second target information includes a predicted target in the image data that is moved relative to the image data after it.

[0073] For example, the inter-frame interval is k. m For example, km is a positive integer, the above step S121 includes: according to the prediction target information of the i-th frame image data, and the ik-th frame image data m The predicted target information of the frame image data determines the i-th frame image data relative to the ik-th frame image data. m The first target information of the frame image data includes the first target information of the i-th frame image data relative to the ik-th frame image data. m The above step S121 includes: according to the prediction target information of the i-th frame image data, and the i+k-th frame image data, m The predicted target information of the frame image data determines the i-th frame image data relative to the i+k-th frame image data. m The second target information of the frame image data includes the second target information of the i-th frame image data corresponding to the i+k-th frame image data. m There is a predicted target whose position moves due to frame image data.

[0074] Based on the processing results of the above steps S121 and S122, in step S123, the moving target information of the i-th frame image data is determined according to the m first target information and the m second target information of the i-th frame image data. Optionally, the m first target information and the m second target information can be processed by union to obtain the moving target information; or, the m first target information and the m second target information can be processed by intersection to obtain the moving target information; or, a first target information and a second target information can be processed by intersection first, and then the results of the m intersection processing can be processed by union to obtain the moving target information; or, a first target information and a second target information can be processed by union first, and then the results of the m union processing can be processed by intersection to obtain the moving target information, which is not limited in the embodiment of the present application. Of course, in practical applications, other methods can also be used to obtain the moving target information, for example, a preset number of predicted targets can be selected from the m first target information and the m second target information to form the moving target information, etc.

[0075] In some embodiments, taking the combination of intersection processing and union processing as an example, the above step S123 includes: m The first target information of the frame image data, and the i-th frame image data relative to the i+k-th frame image data m The second target information of the frame image data is subjected to intersection processing to obtain a fused target information of the i-th frame image data; and the m fused target information of the i-th frame image data is subjected to union processing to obtain the moving target information of the i-th frame image data.

[0076] That is to say, the embodiment of the present application fuses the first target information and the second target information according to the inter-frame interval, intersects the first target information and the second target information with the same corresponding inter-frame interval, obtains the fused target information under each inter-frame interval, and then obtains the union of the fused target information under each inter-frame interval to obtain the moving target information. Since the corresponding inter-frame interval is smaller when the target moves faster, and the corresponding inter-frame interval is larger when the target moves slower, the embodiment of the present application can accurately detect moving targets with different moving speeds by setting different inter-frame intervals.

[0077] For example, Figure 3 As shown, the target detection model outputs the predicted target information of each frame of image data in the video data. For the i-th frame of image data, according to the predicted target information of the i-th frame of image data and the predicted target information of the i-k1-th frame of image data, the first target information of the i-th frame of image data relative to the i-k1-th frame of image data is determined; according to the predicted target information of the i-th frame of image data and the predicted target information of the i+k1-th frame of image data, the second target information of the i-th frame of image data relative to the i+k1-th frame of image data is determined; ...; according to the predicted target information of the i-th frame of image data and the predicted target information of the i+k1-th frame of image data, the second target information of the i-th frame of image data relative to the i+k1-th frame of image data is determined. m The predicted target information of the frame image data determines the i-th frame image data relative to the ik-th frame image data. m The first target information of the frame image data; the predicted target information according to the i-th frame image data, and the i+k-th m The predicted target information of the frame image data determines the i-th frame image data relative to the i+k-th frame image data. m Then, the first target information of the i-th frame image data relative to the i-k1-th frame image data and the second target information of the i-th frame image data relative to the i+k1-th frame image data are subjected to intersection processing to obtain the fused target information of the i-th frame image data at the inter-frame interval k1; ...; the second target information of the i-th frame image data relative to the ik-th frame image data is ... m The first target information of the frame image data, and the i-th frame image data relative to the i+k-th frame image data m The second target information of the frame image data is processed by intersection, and the i-th frame image data is obtained at the frame interval k m Then, the i-th frame image data is processed in the interval k1 to k m The fused target information under is processed by union to obtain the moving target information of the i-th frame image data, so as to further determine the moving target in the i-th frame image data. For example, the predicted target that meets the preset conditions in the moving target information of the i-th frame image data is used as the moving target in the i-th frame image data.

[0078] In practical applications, if only one interframe interval is set, such as the interframe interval km , then the above step S123 includes: m The first target information of the frame image data, and the i-th frame image data relative to the i+k-th frame image data m The second target information of the frame image data is processed by intersection to obtain the moving target information of the i-th frame image data. Of course, in this case, other methods can also be used to determine the moving target information, such as performing union processing to obtain the moving target information, which is not limited in the embodiment of the present application.

[0079] In summary, the moving target detection method provided by the embodiment of the present application determines the moving target for each frame of image data by combining the frame of image data and the image data before and after the frame of image data, thereby further reducing the problem of target detection ghosting. In addition, the embodiment of the present application sets and adjusts the inter-frame interval to match different target moving speeds. When the inter-frame interval is large, it can more accurately detect targets with slower moving speeds, and when the inter-frame interval is small, it can more accurately detect targets with faster moving speeds, thereby achieving full detection of targets with various moving speeds in the image data.

[0080] In some embodiments, the above step S120 may include the following steps:

[0081] Step S12A: constructing a prediction target matrix of two frames of image data according to the prediction target information of the i-th frame of image data and the prediction target information of a frame of image data before and / or after the i-th frame;

[0082] Step S12B: performing feature similarity calculation on each element in the prediction target matrix to obtain a target feature similarity matrix corresponding to the prediction target matrix;

[0083] Step S12C: Determine the moving target information of the i-th frame image data according to the target feature similarity matrix.

[0084] Each element in the prediction target matrix includes two prediction targets belonging to different image data. m The prediction target information of the frame image data is constructed, and the prediction target matrix of two frames of image data is constructed as an example. The prediction target matrix can be shown as follows:

[0085]

[0086] Among them, the ik m The number of predicted targets in the frame image data is w, that is, the ikth mThe prediction target information of the frame image data includes w prediction targets, where w is a positive integer; the number of prediction targets in the i-th frame image data is v, that is, the prediction target information of the i-th frame image data includes v prediction targets, where v is a positive integer; the element anchor_data in the above prediction target matrix w,v Including the ik m The w-th predicted target in the frame image data and the v-th predicted target in the i-th frame image data, or in other words, the element anchor_data in the above prediction target matrix w,v By the ik m A data pair consisting of the w-th prediction target in the frame image data and the v-th prediction target in the i-th frame image data.

[0087] In the above step S12B, for each element in the prediction target matrix, a feature similarity operation can be performed on the two prediction targets included in the element to obtain the target feature similarity between the two prediction targets; according to the target feature similarities corresponding to each element in the prediction target matrix, a target feature similarity matrix can be constructed. Among them, each element in the target feature similarity matrix indicates the target feature similarity between the two prediction targets.

[0088] The embodiment of the present application does not limit the specific method of feature similarity calculation. Optionally, the feature similarity calculation includes motion feature similarity calculation and / or image feature similarity calculation. The embodiment of the present application takes the similarity calculation of motion features and the similarity calculation of image features as an example. Optionally, the above step S12B includes the following steps:

[0089] Step S12B1: performing a first similarity operation on each element in the prediction target matrix to obtain a motion feature similarity matrix corresponding to the prediction target matrix;

[0090] Step S12B2: performing a second similarity operation on each element in the prediction target matrix to obtain an image feature similarity matrix corresponding to the prediction target matrix;

[0091] Step S12B3: Fuse the motion feature similarity matrix and the image feature similarity matrix to obtain a target feature similarity matrix.

[0092] Each element in the motion feature similarity matrix indicates the motion feature similarity between two predicted targets. The first similarity operation is the similarity operation of the motion features, which can be achieved by performing a distance measurement on the two predicted targets. The embodiment of the present application does not limit the specific method of the first similarity operation. Optionally, the first similarity operation is based on any of the following distance metrics: Manhattan distance metric, Euclidean distance metric, IoU (Intersection over Union) distance metric, CIoU (Complete-IoU) distance metric, DIoU (Distance-IoU) distance metric, GIoU (Generalized Intersection over Union) distance metric, and EIoU (Efficient-IoU) distance metric, etc.

[0093] To the i-th frame image data and the ik-th m Taking the first similarity operation for each element in the prediction target matrix of the frame image data as an example, the following motion feature similarity matrix can be obtained:

[0094]

[0095] Among them, the element distance_data in the above motion feature similarity matrix w,v Instructions m The motion feature similarity between the w-th predicted target in the frame image data and the v-th predicted target in the i-th frame image data.

[0096] Each element in the image feature similarity matrix indicates the image feature similarity between the two prediction targets. The second similarity operation is the similarity operation of the image features, which can be achieved by measuring the feature similarity of the two prediction targets. Based on this, it is first necessary to perform feature extraction processing on each prediction target, and then perform the second similarity operation based on the result of the feature extraction. Therefore, optionally, the above step S12B2 includes: for each element in the prediction target matrix, respectively extracting the corresponding local data from the two frames of image data, and constructing a local image matrix corresponding to the prediction target matrix; performing feature extraction processing on each element in the local image matrix, and constructing a local feature matrix corresponding to the local image matrix; performing a second similarity operation on each element in the local feature matrix to obtain an image feature similarity matrix.

[0097] Since each element in the prediction target matrix includes two prediction targets belonging to different image data, for each prediction target in each element, local data corresponding to the prediction target can be obtained from the image data to which it belongs, and a local image matrix can be constructed according to the local data corresponding to each prediction target in each element in the prediction target matrix. Each element in the local image matrix includes local data corresponding to the two prediction targets in the image data to which they belong.

[0098] To construct the i-th frame image data and the ik-th m Taking the local image matrix of frame image data as an example, the local image matrix can be shown as follows:

[0099]

[0100] Among them, the element image_data in the above local image matrix w,v Including the ik m The wth predicted target in the frame image data is the ikth m The local data corresponding to the frame image data, and the local data corresponding to the v-th predicted target in the i-th frame image data, or in other words, the element image_data in the above local image matrix w,v By the ik m The data pair is composed of the local data corresponding to the wth prediction target in the frame image data and the local data corresponding to the vth prediction target in the i-th frame image data. The local data corresponding to the prediction target in the image data to which it belongs can be cut from the image data to which it belongs according to the position information of the prediction target.

[0101] Afterwards, feature extraction is performed on the local data corresponding to each prediction target in each element in the local image matrix to obtain the image features of each prediction target. According to the image features of each prediction target in each element in the local image matrix, a local feature matrix can be constructed. Among them, each element in the local feature matrix includes the image features of two prediction targets respectively. The embodiment of the present application does not limit the specific method of feature extraction. Optionally, feature extraction is based on any one of the following: HOG (Histogram of Oriented Gradients), SURF (Speeded-Up Robust Features), LBP (Local Binary Patterns) and HAAR (Haar-like features), etc.

[0102] To construct the i-th frame image data and the ik-th mTaking the local feature matrix of frame image data as an example, the local feature matrix can be shown as follows:

[0103]

[0104] Among them, the element feature_data in the above local feature matrix w,v Including the ik m The image features of the w-th predicted target in the frame image data, and the image features of the v-th predicted target in the i-th frame image data, or in other words, the element feature_data in the above local feature matrix w,v By the ik m The image features of the w-th prediction target in the frame image data and the image features of the v-th prediction target in the i-th frame image data constitute a data pair. m The image features of the w-th prediction target in the frame image data are extracted from the local data corresponding to the w-th prediction target, and the image features of the v-th prediction target in the i-th frame image data are extracted from the local data corresponding to the v-th prediction target.

[0105] It can be seen from the above embodiment that each element in the image feature similarity matrix indicates the image feature similarity between two prediction targets. Therefore, by performing a second similarity operation on each element in the local feature matrix, the image feature similarity matrix can be obtained. The embodiment of the present application does not limit the specific method of the second similarity operation. Optionally, the second similarity operation is based on any of the following distance metrics: L1 distance metric, L2 distance metric, and cosine distance metric, etc.

[0106] To construct the i-th frame image data and the ik-th m Taking the image feature similarity matrix of frame image data as an example, the image feature similarity matrix can be shown as follows:

[0107]

[0108] Among them, the element fea_dis_data in the above image feature similarity matrix w,v Instructions m The image feature similarity between the w-th predicted target in the frame image data and the v-th predicted target in the i-th frame image data is calculated by performing the feature_data operation on the element in the local feature matrix. w,v Perform the second similarity operation to obtain the element fea_dis_data in the image feature similarity matrix w,v .

[0109] In the above step S12B3, the motion feature similarity matrix and the image feature similarity matrix of the two frames of image data are fused to obtain the target feature similarity matrix of the two frames of image data. The embodiment of the present application does not limit the specific method of fusion processing. Optionally, the fusion processing includes any of the following methods: average processing, weighted average processing, and harmonic average processing.

[0110] To the i-th frame image data and the ik-th m The motion feature similarity matrix of the frame image data and the image feature similarity matrix are weighted averaged to obtain the i-th frame image data and the ik-th frame image data. m Taking the target feature similarity matrix of frame image data as an example, the target feature similarity matrix can be shown as follows:

[0111]

[0112] Among them, α refers to the i-th frame image data and the ik-th frame image data. m The weight value of the motion feature similarity matrix of the frame image data, β refers to the weight value of the motion feature similarity matrix of the i-th frame image data and the ik-th frame image data. m The weight value of the image feature similarity matrix of the frame image data.

[0113] In the above step S12C, the moving target information of the i-th frame image data can be determined according to the target feature similarity matrix of the two image data. The predicted targets in the two image data can be matched based on the target feature similarity matrix, and the predicted targets that are not successfully matched in the i-th frame image data are constructed as the moving target information. The embodiment of the present application does not limit the specific matching method. Optionally, the matching of the predicted targets can be performed based on any of the following methods: KM matching algorithm and Hungarian matching algorithm.

[0114] According to the i-th frame image data and the ik-th m The target feature similarity matrix of the frame image data is used to determine the moving target information of the i-th frame image data as an example. The moving target information can be as follows:

[0115]

[0116] Among them, the i-th frame image data is relative to the ik-th frame image data. m The number of predicted targets that are not successfully matched to the frame image data is s1, and s1 is a positive integer.

[0117] In the case where the moving target information of the current frame is determined by combining the image data of a frame before the current frame and the image data of a frame after the current frame, taking the current frame as the i-th frame as an example, the moving target information of the current frame can be determined based on the image data of the i-th frame and the i-th frame. mThe target feature similarity matrix of the frame image data determines the first target information of the i-th frame image data; based on the i-th frame image data and the i+k-th frame image data m The target feature similarity matrix of the frame image data determines the second target information of the i-th frame image data; the first target information no_match_left i1 and the second target information no_match_right i1 It can be as follows:

[0118]

[0119]

[0120] Then, the first target information and the second target information are subjected to intersection processing to obtain the moving target information of the i-th frame image data, and the moving target information can be shown as follows:

[0121]

[0122] On the basis of combining the image data of a frame before the current frame and the image data of a frame after the current frame, if multiple inter-frame intervals are set, the above-mentioned intersection processing of the first target information and the second target information can be called the fused target information of the i-th frame image data, and the fused target information of the i-th frame image data under multiple inter-frame intervals can be processed by union to obtain the moving target information of the i-th frame image data. Among them, the fused target information of the i-th frame image data under multiple inter-frame intervals is no_match im And the moving target information no_match of the i-th frame image data i It can be as follows:

[0123]

[0124]

[0125] In summary, the mobile target detection method provided in the embodiment of the present application determines the mobile target by performing feature similarity calculation on the predicted target in two frames of image data. Among them, the feature similarity calculation considers the image features of the predicted target in addition to the motion features of the predicted target, so that the similarity calculation between the predicted targets is more accurate, which helps to improve the accuracy of mobile target detection and further reduce the missed detection rate and false detection rate of mobile targets.

[0126] According to the second aspect of the present application, an embodiment of the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements the steps of the above-mentioned moving target detection method when the program is executed by a processor. The non-transitory computer-readable storage medium has all the beneficial effects of the above-mentioned moving target detection method, and the present application will not repeat them here.

[0127] According to a third aspect of the present application, an embodiment of the present application further provides an electronic device, such as Figure 4 As shown, it includes: a memory 402 and a processor 401, the memory 402 stores a computer program; the processor 401 is used to execute the computer program in the memory 402 to implement the steps of the above-mentioned moving target detection method. The electronic device 400 has all the beneficial effects of the above-mentioned moving target detection method, and this application will not repeat them here.

[0128] See also Figure 5 , the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 to a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0129] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The electronic device 500 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 5 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0130] In particular, according to some embodiments of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of some embodiments of the present application are executed.

[0131] It should be noted that the computer-readable medium described in some embodiments of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above, which is not specifically limited in the present application. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0132] In some embodiments of the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device. In some embodiments of the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0133] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: generates or provides at least one predicted target in a detection frame image; determines the moving target in the detection frame image based on the feature comparison between the predicted target in the detection frame image and the predicted targets in multiple associated frame images; wherein the associated frame image is a frame of image data that is earlier or later than the detection frame image in multiple frames of image data of a video data; the feature comparison includes position feature comparison and / or image feature comparison.

[0134] Computer program code for performing the operations of some embodiments of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a portion of a code, which contains one or more executable instructions for implementing a specified logical function.

[0136] It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures.

[0137] For example, two boxes shown in succession may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of boxes in the block diagram and / or flow chart, can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0138] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0139] According to a fourth aspect of the present application, an embodiment of the present application further provides a moving target detection system. Figure 6 As shown, the moving target detection system includes: a moving target detection module 602.

[0140] The moving target detection module 602 is used to: generate or provide at least one predicted target in a detection frame image; and determine the moving target in the detection frame image based on feature comparison between the predicted target in the detection frame image and the predicted targets in multiple associated frame images.

[0141] Optionally, the associated frame image is a frame image data that is earlier or later than the detection frame image among multiple frames of image data of the video data. Based on this, optionally, as Figure 6 As shown, the moving object detection system further includes a video acquisition module 601 for acquiring video data, which can be implemented as a device with a video acquisition function, such as a camera.

[0142] Optionally, the feature comparison includes position feature comparison and / or image feature comparison.

[0143] Alternatively, if Figure 6 As shown, the moving target detection system further includes a task execution module 603, which is used to execute a preset task according to the moving target in each frame of the image data.

[0144] For the functions and beneficial effects of each module in the mobile target detection system, please refer to the introduction of the above mobile target detection method, which will not be elaborated here.

[0145] According to a fifth aspect of the present application, an embodiment of the present application further provides a vehicle, including an electronic device or a mobile target detection system, which can be used to perform the above-mentioned mobile target detection method. The vehicle has all the beneficial effects of the above-mentioned electronic device or mobile target detection system, which will not be described in detail in this application.

[0146] The vehicle may be a fuel vehicle, a plug-in hybrid vehicle or a new energy vehicle, etc., and this application does not make any specific limitation on this.

[0147] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0148] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0149] The embodiments, implementation methods and related technical features of the present application can be combined and replaced with each other without conflict.

[0150] The above are only preferred embodiments of the present application and do not constitute any form of limitation to the present application. Although the descriptions of various embodiments in the embodiments of the present application have different focuses, for parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application are still within the scope of the technical solution of the present application.

Claims

1. A moving target detection method, characterized in that: include: Generate or provide at least one predicted target in a detection frame image; Determine the moving target in the detection frame image according to feature comparison between the predicted target in the detection frame image and the predicted targets in a plurality of associated frame images; Wherein, the associated frame image is a frame image data earlier or later than the detection frame image among multiple frames image data of a video data; The generating or providing at least one predicted target in a detection frame image includes: In response to the input of the video data, the video data is input into a target detection model so that the target detection model outputs predicted target information of the image data of each frame in the video data; the predicted target information includes the predicted target in the image data; The detection frame image is the image data of the i-th frame, the multiple associated frame images are multiple frames of image data located before and / or after the i-th frame, and i is a positive integer; the determining the moving target in the detection frame image according to the feature comparison between the predicted target in the detection frame image and the predicted targets in the multiple associated frame images includes: Determine moving target information of the image data of the i-th frame according to the predicted target information of the image data of the i-th frame and the predicted target information of at least one frame of the image data before and / or after the i-th frame; the moving target information includes the predicted target whose position is moved in the image data; Determine the moving target in the image data of the i-th frame according to the moving target information of the image data of the i-th frame; Wherein, determining the moving target information of the image data of the i-th frame according to the predicted target information of the image data of the i-th frame and the predicted target information of at least one frame of image data before and / or after the i-th frame includes: Constructing a prediction target matrix of the two frames of image data according to the prediction target information of the i-th frame of image data and the prediction target information of the one frame of image data before and / or after the i-th frame; each element in the prediction target matrix includes two prediction targets belonging to different image data; Performing feature similarity calculation on each element in the predicted target matrix to obtain a target feature similarity matrix corresponding to the predicted target matrix; each element in the target feature similarity matrix indicates the target feature similarity between two predicted targets; Determining the moving target information of the image data of the i-th frame according to the target feature similarity matrix; The step of performing a feature similarity operation on each element in the prediction target matrix to obtain a target feature similarity matrix corresponding to the prediction target matrix includes: Performing a first similarity operation on each element in the prediction target matrix to obtain a motion feature similarity matrix corresponding to the prediction target matrix; each element in the motion feature similarity matrix indicates the motion feature similarity between two prediction targets; Performing a second similarity operation on each element in the prediction target matrix to obtain an image feature similarity matrix corresponding to the prediction target matrix; each element in the image feature similarity matrix indicates the image feature similarity between two prediction targets; Fusion of the motion feature similarity matrix and the image feature similarity matrix to obtain the target feature similarity matrix; The performing of a second similarity operation on each element in the prediction target matrix to obtain an image feature similarity matrix corresponding to the prediction target matrix includes: For each element in the prediction target matrix, corresponding local data are respectively extracted from the two frames of image data to construct a local image matrix corresponding to the prediction target matrix; each element in the local image matrix includes the local data corresponding to the two prediction targets in the image data to which they belong; Performing feature extraction processing on each element in the local image matrix to construct a local feature matrix corresponding to the local image matrix; each element in the local feature matrix includes image features of two prediction targets respectively; The second similarity operation is performed on each element in the local feature matrix to obtain the image feature similarity matrix.

2. The method according to claim 1, characterized in that The determining the moving target in the detection frame image according to the feature comparison between the predicted target in the detection frame image and the predicted targets in a plurality of associated frame images comprises: The feature comparison is performed between the predicted target in the detection frame image and the predicted targets in a plurality of associated frame images having different inter-frame intervals.

3. The method according to claim 1, characterized in that The determining, according to the predicted target information of the image data of the i-th frame and the predicted target information of at least one frame of image data before and / or after the i-th frame, moving target information of the image data of the i-th frame comprises: Determine m first target information of the image data of the i-th frame according to the predicted target information of the image data of the i-th frame and the predicted target information of the image data of m frames before the i-th frame; Determine m second target information of the image data of the i-th frame according to the predicted target information of the image data of the i-th frame and the predicted target information of the image data of m frames after the i-th frame; Determine the moving target information of the i-th frame of image data according to the m first target information and the m second target information of the i-th frame of image data; Wherein, m is a positive integer.

4. The method according to claim 3, characterized in that The determining m first target information of the image data of the i-th frame according to the predicted target information of the image data of the i-th frame and the predicted target information of the image data of m frames before the i-th frame comprises: According to the predicted target information of the image data of the i-th frame, and the i-th frame m The predicted target information of the image data of the frame is determined to determine the image data of the i-th frame relative to the i-th frame. m the first target information of the image data of the frame; The determining m second target information of the image data of the i-th frame according to the predicted target information of the image data of the i-th frame and the predicted target information of the image data of m frames after the i-th frame comprises: According to the predicted target information of the image data of the i-th frame, and the i+k-th frame m The predicted target information of the image data of the frame is used to determine the image data of the i-th frame relative to the i+k-th frame. m the second target information of the image data; Among them, the k m Is a positive integer.

5. The method according to claim 4, characterized in that The determining the moving target information of the i-th frame of image data according to the m first target information and the m second target information of the i-th frame of image data comprises: The image data of the i-th frame is relative to the ik-th frame. m The first target information of the image data of the i-th frame, and the image data of the i-th frame relative to the i+k-th frame m Performing intersection processing on the second target information of the image data of the i-th frame to obtain a fused target information of the image data of the i-th frame; The m pieces of fused target information of the i-th frame of image data are processed by union to obtain the moving target information of the i-th frame of image data.

6. The method according to any one of claims 1 to 5, characterized in that: The step of determining the moving target in the i-th frame of image data according to the moving target information of the i-th frame of image data comprises: The predicted target that meets a preset condition in the moving target information of the image data of the i-th frame is used as the moving target in the image data of the i-th frame; The preset condition includes at least one of the following: the moving distance of the predicted target is greater than a preset distance, and the target category of the predicted target belongs to a preset category.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the moving target detection method according to any one of claims 1 to 6 is implemented.

8. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the moving target detection method according to any one of claims 1 to 6.

9. A moving target detection system, characterized in that: include: A moving target detection module is used to: generate or provide at least one predicted target in a detection frame image; Determine the moving target in the detection frame image according to feature comparison between the predicted target in the detection frame image and the predicted targets in a plurality of associated frame images; Wherein, the associated frame image is a frame image data earlier or later than the detection frame image among multiple frames image data of a video data; Wherein, the moving target detection module is also used for: In response to the input of the video data, the video data is input into a target detection model so that the target detection model outputs predicted target information of the image data of each frame in the video data; the predicted target information includes the predicted target in the image data; Wherein, the detection frame image is the image data of the i-th frame, the multiple associated frame images are multiple frames of image data located before and / or after the i-th frame, and i is a positive integer; the moving target detection module is further used for: Determine moving target information of the image data of the i-th frame according to the predicted target information of the image data of the i-th frame and the predicted target information of at least one frame of the image data before and / or after the i-th frame; the moving target information includes the predicted target whose position is moved in the image data; Determine the moving target in the image data of the i-th frame according to the moving target information of the image data of the i-th frame; Wherein, the moving target detection module is also used for: Constructing a prediction target matrix of the two frames of image data according to the prediction target information of the i-th frame of image data and the prediction target information of the one frame of image data before and / or after the i-th frame; each element in the prediction target matrix includes two prediction targets belonging to different image data; Performing feature similarity calculation on each element in the predicted target matrix to obtain a target feature similarity matrix corresponding to the predicted target matrix; each element in the target feature similarity matrix indicates the target feature similarity between two predicted targets; Determining the moving target information of the image data of the i-th frame according to the target feature similarity matrix; Wherein, the moving target detection module is also used for: Performing a first similarity operation on each element in the prediction target matrix to obtain a motion feature similarity matrix corresponding to the prediction target matrix; each element in the motion feature similarity matrix indicates the motion feature similarity between two prediction targets; Performing a second similarity operation on each element in the prediction target matrix to obtain an image feature similarity matrix corresponding to the prediction target matrix; each element in the image feature similarity matrix indicates the image feature similarity between two prediction targets; Fusion of the motion feature similarity matrix and the image feature similarity matrix to obtain the target feature similarity matrix; Wherein, the moving target detection module is also used for: For each element in the prediction target matrix, corresponding local data are respectively extracted from the two frames of image data to construct a local image matrix corresponding to the prediction target matrix; each element in the local image matrix includes the local data corresponding to the two prediction targets in the image data to which they belong; Performing feature extraction processing on each element in the local image matrix to construct a local feature matrix corresponding to the local image matrix; each element in the local feature matrix includes image features of two prediction targets respectively; The second similarity operation is performed on each element in the local feature matrix to obtain the image feature similarity matrix.

10. A vehicle, characterized in that: The vehicle comprises the electronic device as claimed in claim 8, or comprises the moving object detection system as claimed in claim 9.

Images