An object detection method and apparatus

Abstract

The embodiment of the present disclosure discloses an object detection method and device, wherein the object detection method comprises: determining key frames and non-key frames in each video frame in a to-be-detected video based on features of each video frame; determining a detection result of a target key frame in the key frames by performing feature aggregation on object features of the target key frame and object features of other key frames, and determining a detection result of the non-key frames by correcting object features of the non-key frames by applying object features of the key frames. Wherein, by performing feature aggregation on object features of the key frames and object features of other key frames, the feature quality of the key frames is enhanced, the accuracy of object detection on the key frames is improved, and based on the corrected object features of the non-key frames, the object features of the non-key frames are corrected, and the accuracy of detecting the non-key frames is improved accordingly. Therefore, the scheme of the present disclosure can improve the accuracy of object detection.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer vision, and in particular, to an object detection method and device. BACKGROUND

[0002] In application scenarios such as assisted driving or autonomous driving, sometimes video object detection technology is needed, which is usually used to determine the category and position of objects contained in each video frame in a video. The commonly used video object detection technology is to sequentially apply an image detection algorithm to each video frame to achieve video object detection.

[0003] However, due to motion blur or video defocus, etc., there are often quality-degraded video frames in a video, resulting in low accuracy when video object detection is performed by using the prior art. SUMMARY

[0004] To solve the above technical problems, the present disclosure is proposed. Embodiments of the present disclosure provide an object detection method and device.

[0005] According to an aspect of an embodiment of the present disclosure, an object detection method is provided, comprising:

[0006] determining key frames and non-key frames in each video frame in a to-be-detected video based on features of the video frames;

[0007] determining a detection result of a target key frame in the key frames by aggregating object features of the target key frame and object features of other key frames;

[0008] determining a detection result of the non-key frames by correcting object features of the non-key frames by applying object features of the key frames.

[0009] According to another aspect of an embodiment of the present disclosure, an object detection device is provided, comprising:

[0010] a video frame determination module configured to determine key frames and non-key frames in each video frame in a to-be-detected video based on features of the video frames;

[0011] a key frame detection module configured to determine a detection result of a target key frame in the key frames by aggregating object features of the target key frame and object features of other key frames;

[0012] a non-key frame detection module configured to determine a detection result of the non-key frames by correcting object features of the non-key frames by applying object features of the key frames.

[0013] According to a further aspect of the embodiments of the present disclosure, a computer readable storage medium is provided, which stores a computer program for executing the object detection method according to any of the embodiments of the present disclosure.

[0014] According to a further aspect of the embodiments of the present disclosure, an electronic device is provided, which comprises:

[0015] a processor;

[0016] a memory for storing instructions executable by the processor;

[0017] the processor is configured to read the executable instructions from the memory and execute the instructions to implement the object detection method according to any of the embodiments of the present disclosure.

[0018] Based on the object detection method and device, the computer readable storage medium and the electronic device provided by the embodiments of the present disclosure, when performing object detection, first, each video frame included in the video to be detected is divided into key frames and non-key frames, and then different methods are used to perform object detection on the key frames and the non-key frames. Wherein, when performing object detection on the key frames, the feature quality of the key frames is enhanced by aggregating the object features of the key frames and the object features of other key frames, and the accuracy of object detection on the key frames is improved. In addition, when the non-key frames are detected by the scheme provided by the embodiments of the present disclosure, the object features of the non-key frames are corrected based on the object features of the key frames, and then the object features of the non-key frames are corrected based on the corrected object features of the non-key frames, and the accuracy of detecting the non-key frames is improved accordingly, thereby improving the accuracy of object detection.

[0019] Therefore, compared with the prior art of using image detection algorithm on each video frame, the scheme provided by the embodiments of the present disclosure can improve the accuracy of object detection. BRIEF DESCRIPTION OF DRAWINGS

[0020] The above and other objects, features and advantages of the present application will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which:

[0021] Figure 1 is a schematic diagram of a device to which the present disclosure is applicable.

[0022] Figure 2 is a flowchart of an object detection method provided by an exemplary embodiment of the present disclosure.

[0023] Figure 3 is a flowchart of an object detection method provided by another example embodiment of the present disclosure.

[0024] Figure 4 is a flowchart of an object detection method provided by another example embodiment of the present disclosure.

[0025] Figure 5 is a flowchart of an object detection method provided by another example embodiment of the present disclosure.

[0026] Figure 6 is a scene example diagram of an object detection method provided by an example embodiment of the present disclosure.

[0027] Figure 7 is an example diagram of a network model in an object detection method provided by an example embodiment of the present disclosure.

[0028] Figure 8 is a flowchart of an object detection method provided by another example embodiment of the present disclosure.

[0029] Figure 9 is a flowchart of an object detection method provided by another example embodiment of the present disclosure.

[0030] Figure 10 is a structural diagram of an object detection device provided by another example embodiment of the present disclosure.

[0031] Figure 11 is a structural diagram of an object detection device provided by another example embodiment of the present disclosure.

[0032] Figure 12 is a structural diagram of an electronic device provided by an example embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] Hereinafter, example embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, and not all embodiments of the present disclosure. It should be understood that the present disclosure is not limited to the example embodiments described herein.

[0034] It should be noted that: unless otherwise specified, the relative arrangement, numerical expressions and values of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0035] Those skilled in the art can understand that the terms "first", "second" and the like in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, and do not represent any specific technical meaning, nor do they represent the inherent logical order between them.

[0036] It should also be understood that, in the embodiments of the present disclosure, “multiple” can refer to two or more, and “at least one” can refer to one, two or more.

[0037] It should also be understood that, for any component, data or structure mentioned in the embodiments of the present disclosure, one or more can be generally understood without explicit limitation or in the context of the opposite implication.

[0038] In addition, the term “and / or” in the present disclosure is only to describe the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. In addition, the character “ / ” in the present disclosure generally represents an “or” relationship between the front and rear associated objects.

[0039] It should also be understood that the description of the embodiments of the present disclosure focuses on the differences between the embodiments, and the same or similar parts can be referred to each other, and for the sake of brevity, will not be repeated.

[0040] At the same time, it should be understood that, for the sake of brevity, the size of each part shown in the drawings is not drawn in accordance with the actual proportional relationship.

[0041] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application or uses.

[0042] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail, but should be considered part of the specification where appropriate.

[0043] It should be noted that: similar signs and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0044] The embodiments of the present disclosure can be applied to terminal devices, computer systems, servers and other electronic devices, which can operate with many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with terminal devices, computer systems, servers and other electronic devices include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, mainframe computer systems and distributed cloud computing technology environments including any of the above systems, etc.

[0045] Electronic devices such as terminal devices, computer systems, servers, and the like can be described in the general context of computer system-executable instructions, such as program modules, being executed by a computer system. Generally, program modules can include routines, programs, objects, components, logic, data structures, and the like that perform particular tasks or implement particular abstract data types. Computer systems / server can be practiced in distributed cloud computing environments with remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules can be located in local and / or remote computer system storage media including memory storage devices.

[0046] SUMMARY

[0047] In application scenarios such as assisted driving or autonomous driving, sometimes video object detection technology needs to be applied, which is usually used to determine the category and location of objects contained in each video frame of a video. The video is usually composed of multiple still pictures, and each video frame usually refers to one of the multiple pictures.

[0048] The commonly used video object detection technology usually sequentially applies an image detection algorithm to each video frame to achieve video object detection. Specifically, in the prior art, the video is first divided into multiple video frames, and then an image detection algorithm (such as an image semantic segmentation method) is applied to each video frame to determine the category and location of objects contained in each video frame, thereby achieving object detection.

[0049] However, due to reasons such as motion blur or video defocus, sometimes there are quality-degraded video frames in the video. The quality-degraded video frames can have problems such as picture blurring or partial area being occluded, which causes the image detection algorithm to often fail to accurately detect the quality-degraded video frames. Therefore, if the quality-degraded video frames are detected according to the current scheme, there is a problem of low detection accuracy.

[0050] Exemplary system

[0051] The embodiments of the present disclosure can be applied to application scenarios that need to detect objects in a video, which can include application scenarios such as assisted driving or autonomous driving.

[0052] For example, in the application scenarios of assisted driving or autonomous driving, a vehicle-mounted camera of a vehicle can capture a video of a road where the vehicle is located, and then the scheme provided by the embodiments of the present disclosure can be used to detect objects in the video and identify the category and location of the objects contained in the video. Further, the driving state can also be adjusted according to the identified result, thereby providing a reference for assisted driving or autonomous driving.

[0053] Figure 1 is a scenario to which the present disclosure is applicable. As shown inFigure 1 As shown, the embodiments of the present disclosure are applied in an application scenario of assisted driving or automatic driving, wherein the device 100 for object detection based on the scheme provided by the embodiments of the present disclosure can be connected with the vehicle-mounted camera 200.

[0054] The connection mode of the device 100 for object detection and the vehicle-mounted camera 200 can include multiple modes. In one of the possible modes, the device 100 can be electrically connected with the vehicle-mounted camera 200, for example, the device 100 is a car machine device, in which case the device 100 can be electrically connected with the vehicle-mounted camera 200. In another possible mode, the connection mode of the device 100 and the vehicle-mounted camera 200 can also be network connection.

[0055] The vehicle-mounted camera 200 can shoot a video and transmit the video to the device 100. After obtaining the video, the device 100 performs the object detection method provided by the embodiments of the present disclosure to determine the category and position of the object included in the video.

[0056] In the application scenario of assisted driving or automatic driving, the device 100 can be a car machine device (for example, a vehicle-mounted intelligent terminal), and the video shot by the vehicle-mounted camera 200 can include a road on which the vehicle travels, in which case, after obtaining the video, the device 100 can determine the category and position of the object contained in the road on which the vehicle travels by performing object detection on the video, and determine whether the driving state of the vehicle needs to be adjusted accordingly.

[0057] For example, if the category and position of the object contained in the video indicate that there is a pedestrian in front of the vehicle, the vehicle can be slowed down or turned to avoid collision with the pedestrian.

[0058] In one possible example, the device 100 can include a video acquisition module 101, a feature extraction module 102, a key frame judgment module 103, a key frame detection module 104, and a non-key frame detection module 105.

[0059] The video acquisition module 101 can be used to acquire a video to be detected and determine each video frame contained in the video to be detected.

[0060] The feature extraction module 102 can extract the features of each video frame. When extracting the features, the feature extraction module 102 can use a convolutional neural network, etc.

[0061] The key frame determination module 103 can determine whether each video frame is a key frame according to the features of each video frame extracted by the feature extraction module 102, thereby distinguishing the key frames and non-key frames in each video frame. Generally, the key frame determination module 103 distinguishes the key frames and non-key frames according to the degree of change in the content of each video frame. In addition, after distinguishing the key frames and non-key frames in each video frame, the key frame determination module 103 can transmit the key frames to the key frame detection module 104 and transmit the non-key frames to the non-key frame detection module 105.

[0062] After obtaining the key frames, the key frame detection module 104 can enhance the feature quality of the key frames by aggregating the object features of the key frames with the object features of other key frames, to obtain a higher-precision object detection result.

[0063] After obtaining the non-key frames, the non-key frame detection module 105 can correct the object features of the non-key frames based on the object features of the key frames, and determine the object detection result of the non-key frames after the correction.

[0064] Exemplary method

[0065] Figure 2 FIG. 1 is a flowchart of an object detection method provided by an exemplary embodiment of the present application. The embodiment can be applied on an electronic device, such as a vehicle, as shown in FIG. 2. Figure 2 As shown in FIG. 1, the method comprises the following steps:

[0066] In step 201, based on the features of each video frame in a to-be-detected video, key frames and non-key frames in the video frames are determined.

[0067] The to-be-detected video is captured by an imaging device. In the application scenario of assisted driving or automatic driving, the imaging device can be a vehicle-mounted camera. Of course, in other application scenarios, other types of imaging devices can also be used.

[0068] After the imaging device captures the to-be-detected video, the to-be-detected video can be transmitted to an electronic device for object detection. The electronic device can extract the features of each video frame and distinguish the key frames and non-key frames in each video frame according to the features of each video frame.

[0069] In the scheme provided in the embodiments of the present disclosure, if each video frame in the to-be-detected video is sorted according to the generation time, and the video frame generated earlier has a higher sorting position. In this case, the difference between a certain key frame and the features of the previous key frame is usually large, while the difference between a non-key frame and the features of the key frame corresponding to the non-key frame is usually small. The key frame corresponding to the non-key frame refers to the key frame generated before the non-key frame and closest to the non-key frame in terms of generation time.

[0070] In step S202, the detection result of the target key frame is determined by performing feature aggregation on the object features of the target key frame and the object features of other key frames.

[0071] The object features refer to the features of an object. When performing object detection on a video frame, a bounding box can be set in the video frame, and the area of the bounding box can contain an object in the video frame. In this case, the content contained in the bounding box can be regarded as an object, and the features of the area of the bounding box are the object features.

[0072] In the scheme provided in the embodiments of the present disclosure, the other key frames are usually key frames that have completed object detection. If the object detection is performed on each video frame according to the order of the generation time of each video frame, that is, the earlier the generation time of a video frame, the earlier the time of performing object detection on the video frame, in this case, the other key frames refer to key frames whose generation time is earlier than that of the target key frame.

[0073] The features of the other key frames and the target key frame are quite different, but the objects contained in the other key frames can have intersections with the objects contained in the target key frame, that is, the objects contained in the target key frame can exist in the other key frames. If object A is contained in both the other key frame and the target key frame, it can be considered that the other key frame and the target key frame record object A at different times.

[0074] In this case, performing feature aggregation on the object features of the target key frame by using the object features of the other key frame can enhance the quality of the object features of the target key frame, and facilitate obtaining an object detection result with high accuracy.

[0075] In step S203, the detection result of the non-key frame is determined by correcting the object features of the non-key frame by using the object features of the key frame.

[0076] The difference between the non-key frame and the key frame corresponding to the non-key frame is small. In this case, the object features of the non-key frame can be corrected by using the object features of the key frame, and object detection is performed on the corrected non-key frame, which can ensure the accuracy of object detection on the non-key frame.

[0077] When performing object detection on a video by using the prior art, an image detection algorithm is usually applied to each video frame to obtain an object detection result. In this case, if there is a video frame with poor quality, the accuracy of detection is usually low.

[0078] When the object detection is performed by the scheme provided in the embodiments of the present disclosure, each video frame included in the video to be detected is distinguished as a key frame and a non-key frame, and different manners are used to perform the object detection on the key frame and the non-key frame. When the object detection is performed on the key frame, the feature quality of the key frame is enhanced by aggregating the object features of the key frame and the object features of other key frames, and the accuracy of the object detection on the key frame is improved. In addition, when the scheme provided in the embodiments of the present disclosure is used to perform the object detection on the non-key frame, the object features of the non-key frame are corrected by using the object features of the key frame, and then the object features of the non-key frame are corrected based on the corrected object features of the non-key frame, and the accuracy of the detection on the non-key frame is improved accordingly, so that the accuracy of the object detection is improved.

[0079] Therefore, compared with the prior art in which the image detection algorithm is used on each video frame, the scheme provided in the embodiments of the present disclosure can improve the accuracy of the object detection.

[0080] In one example, each video frame included in a certain video to be detected is sorted according to the generation time, and the video frame generated earlier is arranged in the front in the sorting. The first video frame is a key frame, the second video frame to the ninth video frame are non-key frames, the tenth video frame is affected by the video defocus, and the content displayed by the tenth video frame is relatively blurred. In this case, the feature difference between the tenth video frame and the first video frame is large, and the tenth video frame is also a key frame.

[0081] If the object detection is performed according to the prior art, the category or position of the object included in the tenth video frame cannot be accurately determined due to the blurred content of the tenth video frame, and the accuracy of the detection is low.

[0082] When the object detection is performed on the tenth video frame by the scheme provided in the embodiments of the present disclosure, the object features of the tenth video frame and the object features of the first video frame are aggregated, and the category and type of the object included in the tenth video frame are determined by using the aggregated object features. By aggregating the features, the quality of the object features of the tenth video frame is enhanced, which helps to improve the accuracy of the object detection on the tenth video frame.

[0083] Further, in the scheme provided in the embodiments of the present disclosure, each video frame in the video to be detected is divided into a key frame and a non-key frame, and different detection methods are used on the key frame and the non-key frame. When the object detection is performed on the non-key frame, the object features of the non-key frame are corrected by using the object features of the key frame. The correction process of the object features often requires a small amount of calculation, and therefore, when the object detection is performed by the scheme provided in the embodiments of the present disclosure, the accuracy of the object detection can be improved, and the efficiency of the detection can be ensured.

[0084] In the above embodiment, the operation of determining the key frame and the non-key frame in each video frame is provided. If the video frame is the first frame of the video to be detected, the video frame is determined as the key frame, and the first frame is the earliest generated video frame in the video to be detected.

[0085] In addition, referring to Figure 3 In another exemplary embodiment of the present disclosure, if the video frame is other than the first frame, the key frame and the non-key frame in each video frame can be determined by the following steps based on the above-described Figure 2

[0086] Step S2011, determining the difference value between the features of the video frame and the features of the first key frame.

[0087] The first key frame is the key frame in the key frame set closest to the current time in terms of generation time. For example, if each video frame contained in a certain video to be detected is sorted according to the generation time, and the video frame generated earlier is arranged at the front in the sorting, and the fifth video frame is a key frame, when determining whether the sixth video frame is a key frame, the first key frame is the fifth video frame.

[0088] Step S2012, determining whether the difference value is greater than the difference threshold value, if yes, performing the operation of step S2013, if no, performing the operation of step S2014.

[0089] Step S2013, if the difference value is greater than the difference threshold value, determining the video frame as the key frame.

[0090] That is, if the feature difference between the video frame and the first key frame is large, the video frame is a key frame.

[0091] Step S2014, if the difference value is not greater than the difference threshold value, determining the video frame as the non-key frame.

[0092] That is, if the feature difference between the video frame and the first key frame is small, the video frame is a non-key frame.

[0093] ​In the scheme provided in the embodiments of the present disclosure, the difference value between the feature of the video frame and the feature of the first key frame can be determined by a neural network model supporting pseudo label technology. The neural network model can be trained in a self-supervised manner using the currently distinguished key frames and non-key frames, and the key frames and non-key frames can be distinguished by the trained neural network model. In this case, the difference threshold can be the detection loss obtained by the neural network model. Specifically, when detecting whether a video frame is a key frame, the neural network model can determine the detection loss of the video frame based on the last key frame. If the detection loss is greater than the difference threshold, the video frame is assigned a corresponding pseudo label, and the video frame is determined as a key frame.

[0094] Through the scheme provided in this embodiment of the present disclosure, the key frames and non-key frames in each video frame can be distinguished, which facilitates subsequent object detection using different methods for key frames and non-key frames respectively, so as to improve the accuracy of object detection while ensuring the efficiency of object detection.

[0095] Referring to Figure 4 In another exemplary embodiment of the present disclosure, the detection result of the target key frame can be determined by the following steps based on the above Figure 2 The detection result of the target key frame can be determined by the following steps based on the above

[0096] In step S2021, the object feature of the target key frame and the spatial position of the object feature are determined.

[0097] In step S2022, the short-term memory is updated based on the confidence of the object feature of the target key frame and the spatial position of the object feature.

[0098] The short-term memory is used to store the object features and the spatial positions of the object features of no more than M key frames, and M is a predetermined positive integer.

[0099] When updating the short-term memory, the update can be based on the confidence of the object feature and the spatial position. In this case, after determining the confidence corresponding to each object feature and its spatial position, each object feature and its spatial position can be sorted in descending order of confidence, and the first n1 object features and their spatial positions in the sorted order are updated to the short-term memory, where n1 is a predetermined positive integer. Alternatively, the confidence of each object feature and its spatial position can be compared with a predetermined confidence threshold, and the object features and their spatial positions greater than the confidence threshold are updated to the short-term memory.

[0100] In step S2023, the long-term memory is updated based on the object features dequeued from the short-term memory, and the long-term memory is used to store the object features of the other key frames.

[0101] In the scheme provided in the embodiments of the present disclosure, the short-term memory is a stack that stores at most M object features and spatial positions of key frames. If the object feature of the current key frame is stored in the short-term memory, the object feature at the top of the short-term memory is popped out. Based on the popped-out object feature, the long-term memory can be updated. Through the update of the long-term memory, the long-term memory can store object features of other key frames.

[0102] In the long-term memory, the length is not limited, and the long-term memory can store object features of multiple key frames. In addition, when the long-term memory is updated based on the popped-out object feature, the popped-out object feature can be stored in the long-term memory.

[0103] Alternatively, when the long-term memory is updated, the popped-out object features can be sorted according to the order from large to small of the confidence of each popped-out object feature, and the first n2 object features in the sorting are stored in the long-term memory, where n2 is a preset positive integer. In this case, since the confidence of the object feature in the long-term memory is high, if the object feature of the target key frame is aggregated by using the object feature in the long-term memory, the effect of improving the quality of the object feature of the target key frame is better, thereby further improving the accuracy of object detection.

[0104] In step S2024, the object feature of the target key frame is aggregated based on the object feature in the long-term memory and the object feature and the spatial position of the object feature stored in the short-term memory.

[0105] In the short-term memory, the generation time of the video frame corresponding to the object feature is close to the generation time of the target key frame, and the same object can exist in the two video frames, and the positions of the object in the two video frames are also close. In this case, when the object feature of the target key frame is aggregated, the object feature stored in the short-term memory and the spatial position of the object feature can be combined, so that the appearance information and the position information of the object feature stored in the short-term memory are fully utilized.

[0106] In addition, the interval between the generation time of the video frame corresponding to the object feature stored in the long-term memory and the generation time of the target key frame is large, and even if the same object exists in the two video frames, the positions of the object in the two video frames have a large interval. In this case, when the object feature of the target key frame is aggregated, the object feature in the long-term memory is used, and the spatial position of the object feature in the long-term memory is no longer considered.

[0107] In step S2025, a detection result of the target key frame is determined based on the object feature of the target key frame after the aggregation.

[0108] In this step, the object features of the target key frame after feature aggregation can be classified and regressed, so as to obtain the detection result of the target key frame.

[0109] By the scheme provided by this embodiment of the present disclosure, the object features of the target key frame can be aggregated by using the object features of other key frames, so as to improve the quality of the object features of the target key frame. Then, the object detection of the target key frame is performed by using the object features after quality improvement, so as to improve the detection efficiency of the target key frame.

[0110] In the above step S2021, the operation of determining the object features of the target key frame is disclosed. In this operation, the initial object features of the target key frame can be extracted, and the initial object features are taken as the object features of the target key frame. The operation of extracting the initial object features of the target key frame can refer to the current feature extraction technology, which will not be described here.

[0111] In addition, in order to improve the accuracy of object detection, refer to Figure 5 In another exemplary embodiment of the present disclosure, the object features of the target key frame can be determined by the following steps based on the above Figure 2 The object features of the target key frame are determined by the following steps based on the above

[0112] Step S3021, extracting a plurality of initial object features of the target key frame.

[0113] Step S3022, calculating the similarity between the plurality of initial object features of the target key frame and the features of each region in the target key frame.

[0114] In the target key frame, a plurality of regions are included. If the similarity between a certain initial object feature and the feature of a certain region is high, it indicates that the possibility of the region being the position of the object feature is greater.

[0115] Step S3023, determining that the initial object feature with the similarity meeting the first preset condition is the corrected object feature.

[0116] The first preset condition can be various forms of conditions. In a feasible design, the initial object feature with the similarity greater than the first similarity threshold value can be determined as the initial object feature meeting the first preset condition. Alternatively, in another feasible design, n3 initial object features can be determined, the similarities of the n3 initial object features are greater than the similarities of other initial object features, and the n3 initial object features are the initial object features meeting the first preset condition, wherein n3 is a preset positive integer.

[0117] Step S3024: Based on the similarity between the corrected object features and the features of each region in the target keyframe, the corrected object features are corrected again until the number of corrections reaches a preset number.

[0118] When the modified object features are modified again through this step, the similarity between the modified object features and the features of each region in the target keyframe can be determined, and the object features whose similarity meets the first preset condition can be selected. Then, the selected object features are modified again until the number of modifications reaches the preset number.

[0119] Step S3025: After the number of corrections reaches a preset number, the object features that meet the second preset conditions are determined as the object features of the target keyframe.

[0120] The second preset condition can take various forms. In one feasible design, after the number of corrections reaches a preset number, the object features with a similarity greater than a second similarity threshold among the corrected object features are determined to be object features that meet the second preset condition; or, in another feasible design, n4 object features can be selected from the corrected object features, and the similarity of these n4 object features is greater than the similarity of the corrected object features. These n4 object features are the initial object features that meet the second preset condition, where n4 is a preset positive integer.

[0121] In this embodiment of the present disclosure, after extracting multiple initial object features from the target keyframe, the initial object features are repeatedly refined based on the similarity between each initial object feature and the features of each region in the target keyframe. The refined object features can be referred to as refined object features, wherein the refined object features have a high similarity to the features of the regions in the target keyframe.

[0122] The initial object features in the target keyframe may contain noise, which can interfere with object detection. For example, see... Figure 6 The diagram shows a target keyframe containing vehicles traveling on a road and trees growing along the roadside. The extracted region containing a certain initial object feature of this target keyframe is shown below. Figure 6 The dashed bounding box in the image shows that it includes not only vehicles but also parts of trees. For this initial object feature, the tree portion is considered noise.

[0123] The scheme for determining object features of target keyframes provided in the above embodiments of this disclosure can obtain detailed object features. For Figure 6For example, as shown in the solid boundary box in the figure, fine object features can be obtained by removing the noise caused by the trees, reducing the interference of the noise, and thus improving the accuracy of object detection.

[0124] In a possible implementation, in the scheme of the embodiments of the present disclosure, the modification of the initial object features can be implemented by a network model comprising multiple decoders.

[0125] Referring to Figure 7 As shown in the schematic diagram, in a network model, six decoders are connected in sequence. After extracting the multiple initial object features of the target key frame, the initial object features can be transmitted to the first-level decoder. The first-level decoder calculates the similarity between the multiple initial object features of the target key frame and the features of each region in the target key frame, wherein the similarity can be determined by attention calculation. After determining the similarity, the initial object features that meet the first preset condition are taken as the modified object features, and the modified object features are transmitted to the second-level decoder. The second-level decoder modifies the modified object features again.

[0126] After the modification of the second-level decoder is completed, the object features modified by the second-level decoder are transmitted to the third-level decoder. The third-level decoder, the fourth-level decoder, the fifth-level decoder, and the sixth-level decoder modify the received object features in sequence. After the modification of the sixth-level decoder is completed, the object features that meet the second preset condition are output, and the object features that meet the second preset condition are the object features of the target key frame.

[0127] In this case, the preset number of times is 6, that is, after the initial object features are modified for 6 times, the object features of the target key frame are obtained.

[0128] In addition, in the scheme provided by the embodiments of the present disclosure, the initial object features can also be modified by other network models, which are not limited by the embodiments of the present disclosure.

[0129] In the above embodiments of the present disclosure, the operation of feature aggregation of the object features of the target key frame based on the object features in the long-term memory and the object features and the spatial positions of the object features stored in the short-term memory is disclosed. Referring to Figure 8 In another exemplary embodiment of the present disclosure, the feature aggregation can be performed by the following steps based on the above Figure 2 embodiments shown in the figure.

[0130] Step S4021, selecting T object features from the long-term memory.

[0131] T is a preset positive integer, in this step, T object features can be selected from the long-term memory, or the confidence of the T object features selected from the long-term memory is higher than that of other object features.

[0132] In step S4022, based on the cosine similarity between the T object features and the object features in the short-term memory and the object features of the target key frame respectively, the first weight of the T object features for the object features in the short-term memory and the object features of the target key frame respectively is determined.

[0133] In this step, first, the cosine similarity between the T objects and the object features in the short-term memory is calculated, and the cosine similarity between the T objects and the object features of the target key frame is calculated, and then based on the cosine similarity, the first weight of the T object features for the object features in the short-term memory and the first weight of the T object features for the object features of the target key frame are determined. Generally, the higher the cosine similarity, the higher the first weight.

[0134] In step S4023, based on the first weight, the weighted sum of the T object features and the object features in the short-term memory and the object features of the target key frame respectively is calculated.

[0135] The weighted sum of the T object features and the object features in the short-term memory is the first object feature, and the weighted sum of the T object features and the object features of the target key frame is the second object feature.

[0136] This step is used to enhance the features of the object features in the target key frame and the object features in the short-term memory based on the T object features in the long-term memory. The first object feature is the object feature after feature enhancement of the object feature in the short-term memory, and the second object feature is the object feature after feature enhancement of the object feature in the target key frame.

[0137] In step S4024, based on the first object feature, the spatial position of the first object feature stored in the short-term memory, the second object feature and the spatial position of the second object feature, the apparent similarity and the geometric similarity between the first object feature and the second object feature are determined.

[0138] The object features and the spatial positions of the object features are stored in the short-term memory, and the first object feature is the object feature obtained after enhancement of the object feature stored in the short-term memory. In this case, the spatial position of the first object feature is the same as that of the object feature before enhancement, so the spatial position of the object feature stored in the short-term memory can be considered as the spatial position of the corresponding first object feature.

[0139] This step is used to continue to enhance the object features (i.e., the second object features) of the target key frame by the enhanced object features (i.e., the first object features) in the short-term memory. The appearance similarity between different object features is used to measure the similarity between the appearances of different object features, and the geometric similarity between different object features is used to measure the similarity between the spatial positions of different object features.

[0140] In an example, the appearance similarity and the geometric similarity between the first object features and the second object features can be determined based on a relation network model (i.e., a RelationNet model) in the embodiments of the present disclosure. Of course, the appearance similarity and the geometric similarity between the first object features and the second object features can also be determined in other manners, which are not limited in the embodiments of the present disclosure.

[0141] Step S4025, determining a second weight of the first object features for the second object features based on the appearance similarity and the geometric similarity between the first object features and the second object features.

[0142] Through this step, the second weight can be determined in combination with the appearance similarity and the geometric similarity. Generally, the higher the appearance similarity and the geometric similarity between the first object features and the second object features, the higher the second weight.

[0143] Step S4026, calculating a weighted sum between the first object features and the second object features based on the second weight.

[0144] The weighted sum between the first object features and the second object features is the object features of the target key frame after feature aggregation.

[0145] Through the scheme provided by the embodiments of the present disclosure, the object features in the long-term memory and the object features and the spatial positions of the object features stored in the short-term memory can be used to perform feature aggregation on the object features of the target key frame, so as to enhance the quality of the object features of the target key frame, and facilitate to improve the accuracy of object detection on the target key frame.

[0146] In the scheme provided in the above embodiments, the operation of correcting the object features of the non-key frame by applying the object features of the key frame to determine the detection result of the non-key frame is provided. Referring to Figure 9 In another example embodiment of the present disclosure, the operation can be implemented by the following steps based on the embodiments shown in the above Figure 2

[0147] Step S2031, determining a candidate region of the non-key frame based on the detection result of the first video frame.

[0148] ​The time difference between the generation time of the first video frame and the generation time of the non-key frame is within a preset range, that is, the generation time of the first video frame is close to the generation time of the non-key frame. In a possible design, the video frames are sorted in the order of generation time from early to late, and the first video frame can be the last video frame of the non-key frame.

[0149] Since the generation time of the first video frame is close to the generation time of the non-key frame, the positions of the same object in the first video frame and the non-key frame are close, and therefore, the candidate region of the non-key frame can be determined based on the detection result of the first video frame. Specifically, the region in which the object in the first video frame is located can be taken as the candidate region of the non-key frame, or the region in which the object in the first video frame is located can be slightly expanded, and the expanded region can be taken as the candidate region of the non-key frame.

[0150] Step S2032: Extracting object features in the candidate region of the non-key frame.

[0151] Step S2033: Correcting the object features in the candidate region based on the object features of the key frame to obtain a feature correction result.

[0152] In the above embodiments of the present disclosure, the operation of obtaining fine object features through steps S3021 to S3025 is disclosed. In order to improve the accuracy of object detection, the object features of the key frame applied in step S2033 can include fine object features, that is, the object features in the candidate region are corrected based on the fine object features of the key frame.

[0153] Step S2034: Determining a detection result of the non-key frame based on the feature correction result.

[0154] In this step, the feature correction result can be classified and regressed to obtain the detection result of the non-key frame.

[0155] Through the scheme provided in the embodiments of the present disclosure, the object features of the non-key frame can be corrected based on the object features of the key frame, the quality of the object features of the non-key frame is improved, and the accuracy of object detection of the non-key frame is improved accordingly.

[0156] In step S2033, the operation of correcting the object features in the candidate region based on the object features of the key frame to obtain a feature correction result is disclosed, and the operation can be implemented through the following steps:

[0157] First, a similarity between the object feature of the key frame and the object feature in the candidate region is calculated; then, the object feature in the candidate region that meets the third preset condition is determined as a third object feature according to the similarity; and the third object feature and the object feature of the key frame are fused to obtain the feature correction result.

[0158] The third object feature is an object feature with a high similarity to the object feature of the key frame. In a feasible implementation, among the object features included in the candidate region, the object feature with a similarity greater than a third similarity threshold value is the object feature meeting the third preset condition. Alternatively, in another feasible design, n5 object features can be determined from the object features included in the candidate region, the n5 object features have a similarity greater than the similarity of other object features in the candidate region, and the n5 object features are the object features meeting the third preset condition, where n5 is a preset positive integer.

[0159] Through the above steps, the object feature in the candidate region can be corrected to obtain the feature correction result. Further, the amount of calculation required for correcting the object feature of the non-key frame is less than the amount of calculation required for feature aggregation of the object feature, and therefore, the scheme provided in the embodiments of the present disclosure can improve the accuracy of object detection of the non-key frame while ensuring the detection efficiency.

[0160] Exemplary device

[0161] Figure 10 is a structural diagram of an object detection device provided by an exemplary embodiment of the present disclosure. The object detection device can be arranged in an electronic device such as a terminal device or a server, or on an object such as a vehicle, and can perform the object detection method of any one of the embodiments of the present disclosure. As shown in the figure, the object detection device of the embodiment includes a video frame determination module 201, a key frame detection module 202, and a non-key frame detection module 203. Figure 10

[0162] The video frame determination module 201 is configured to determine key frames and non-key frames in each video frame in a video to be detected based on features of the video frames.

[0163] The key frame detection module 202 is configured to determine a detection result of a target key frame by performing feature aggregation on an object feature of the target key frame and object features of other key frames determined by the video frame determination module 201.

[0164] ​The non-key frame detection module 203 is configured to correct the object features of the non-key frame determined by the video frame determination module 201 by applying the object features of the key frame determined by the video frame determination module 201, and determine a detection result of the non-key frame.

[0165] In some embodiments, referring to FIG. 2, an example diagram of a structure of an object detection apparatus is shown, the video frame determination module 201 can include: Figure 11

[0166] The first determination unit 2011 is configured to determine the video frame as the key frame if the video frame is a first frame of the video to be detected, the first frame being the earliest generated video frame in the video to be detected.

[0167] The difference value determination unit 2012 is configured to determine a difference value between the features of the video frame and the features of a first key frame if the video frame is other than the first frame, the first key frame being a key frame in the set of key frames that is closest in time to the current time;

[0168] The second determination unit 2013 is configured to determine the video frame as the key frame if the difference value is greater than a difference threshold.

[0169] The third determination unit 2014 is configured to determine the video frame as the non-key frame if the difference value is not greater than the difference threshold.

[0170] In some embodiments, referring to FIG. 2, an example diagram of a structure of an object detection apparatus is shown, the key frame detection module 202 can include: Figure 11

[0171] The fourth determination unit 2021 is configured to determine the object features of the target key frame and the spatial positions of the object features.

[0172] The first updating unit 2022 is configured to update a short-term memory based on a confidence of the object features of the target key frame and the spatial positions of the object features determined by the fourth determination unit 2021, the short-term memory being configured to store object features and spatial positions of no more than M key frames, M being a preset positive integer.

[0173] The second updating unit 2023 is configured to update a long-term memory based on object features popped from the short-term memory when the short-term memory is updated by the first updating unit 2022, the long-term memory being configured to store the object features of the other key frames.

[0174] ​​The feature aggregation unit 2024 is configured to perform feature aggregation on the object features of the target key frame based on the object features in the long-term memory updated by the second updating unit 2023 and the object features stored in the short-term memory updated by the first updating unit 2022 and the spatial positions of the object features.

[0175] The first result determination unit 2025 is configured to determine the detection result of the target key frame based on the object features of the target key frame after feature aggregation determined by the feature aggregation unit 2024.

[0176] In some embodiments, the fourth determination unit 2021 in the object detection apparatus comprises:

[0177] The extraction unit is configured to extract a plurality of initial object features of the target key frame.

[0178] The first similarity calculation unit is configured to calculate the similarity between each of the plurality of initial object features of the target key frame extracted by the extraction unit and the features of each region in the target key frame.

[0179] The correction unit is configured to determine, based on the similarity between each of the plurality of initial object features of the target key frame calculated by the first similarity calculation unit and the features of each region in the target key frame, the initial object features that meet a first preset condition as corrected object features, and to perform re-correction on the corrected object features based on the similarity between the corrected object features and the features of each region in the target key frame until the number of correction reaches a preset number.

[0180] The first feature determination unit is configured to determine, after the number of correction of the correction unit reaches the preset number, the object features that meet a second preset condition as the object features of the target key frame.

[0181] In some embodiments, the feature aggregation unit 2024 in the object detection apparatus comprises:

[0182] The feature selection unit is configured to select T object features from the long-term memory.

[0183] The first weight calculation unit is configured to determine, based on the cosine similarity between each of the T object features selected by the feature selection unit and the object features in the short-term memory and the object features of the target key frame, the first weight of each of the T object features with respect to the object features in the short-term memory and the object features of the target key frame.

[0184] a first weighted sum calculation unit configured to calculate a weighted sum between the T object features and the object features in the short-term memory and the object features of the target key frame based on the first weights calculated by the first weight calculation unit, wherein the weighted sum between the T object features and the object features in the short-term memory is a first object feature and the weighted sum between the T object features and the object features of the target key frame is a second object feature;

[0185] a similarity determination unit configured to determine the apparent similarity and the geometric similarity between the first object feature and the second object feature based on the first object feature determined by the weighted sum calculation unit, the spatial position of the first object feature stored in the short-term memory, the second object feature determined by the weighted sum calculation unit and the spatial position of the second object feature;

[0186] a second weight calculation unit configured to determine a second weight of the first object feature for the second object feature based on the apparent similarity and the geometric similarity between the first object feature and the second object feature calculated by the similarity determination unit;

[0187] a second weighted sum calculation unit configured to calculate a weighted sum between the first object feature and the second object feature based on the second weight, wherein the weighted sum between the first object feature and the second object feature is the object feature of the target key frame after feature aggregation.

[0188] In some embodiments, referring to the example diagram of the structure of an object detection device shown in FIG. 2, the non-key frame detection module 203 includes: Figure 11

[0189] a candidate region determination unit 2031 configured to determine a candidate region of the non-key frame based on the detection result of the first video frame, wherein the time difference between the generation time of the first video frame and the generation time of the non-key frame is within a preset range;

[0190] a feature extraction unit 2032 configured to extract object features in the candidate region of the non-key frame determined by the candidate region determination unit 2031;

[0191] a feature correction unit 2033 configured to correct the object features in the candidate region extracted by the feature extraction unit 2032 based on the object features of the key frame to obtain a feature correction result;

[0192] a second result determination unit 2034 configured to determine the detection result of the non-key frame based on the feature correction result obtained by the feature correction unit 2033.

[0193] ​In some embodiments, the feature correction unit 2033 in the object detection apparatus includes:

[0194] a second similarity calculation unit configured to calculate a similarity between the object feature of the key frame and the object feature in the candidate region;

[0195] a second feature determination unit configured to determine, based on the similarity calculated by the second similarity calculation unit, the object feature in the candidate region as a third object feature if the similarity meets a third preset condition;

[0196] a correction result acquisition unit configured to perform fusion processing on the third object feature determined by the second feature determination unit and the object feature of the key frame to obtain the feature correction result.

[0197] An exemplary electronic device

[0198] Hereinafter, an electronic device according to an embodiment of the present application will be described with reference to Figure 12 The electronic device can be an electronic device including Figure 1 the device 100 for object detection shown in FIG. 1, or in another exemplary embodiment of the present disclosure, the electronic device can be an electronic device including Figure 1 the device 100 for object detection and the vehicle-mounted camera 200 shown in FIG. 2. Of course, the electronic device can also be in other forms, which are not limited in the present disclosure.

[0199] Figure 12 FIG. 3 illustrates a block diagram of an electronic device according to an embodiment of the present application.

[0200] As shown in Figure 12 FIG. 4, the electronic device 11 includes one or more processors 111 and a memory 112.

[0201] The processor 111 can be a central processing unit (CPU) or other forms of processing units having data processing and / or instruction execution capabilities, and can control other components in the electronic device 11 to perform desired functions.

[0202] The memory 112 can include one or more computer program products that can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory, for example, can include random access memory (RAM), cache memory, and / or the like. The non-volatile memory, for example, can include read-only memory (ROM), hard disk drives, solid-state drives, and / or the like. The computer-readable storage media can store one or more computer program instructions executable by the processor 111 to implement the object detection method of various embodiments of the present application described above and / or other desired functions. Various contents such as video frames, object features of video frames, and spatial positions of object features can also be stored in the computer-readable storage media.

[0203] In one example, the electronic device 11 can further include an input device 113 and an output device 114, which are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0204] In addition, the input device 113 can further include, for example, a keyboard, a mouse, and / or the like.

[0205] The output device 114 can output various information including the determined object detection result to the outside. The output device 114 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and / or the like.

[0206] Of course, in order to simplify, Figure 12 Only some of the components of the electronic device 11 related to the present application are shown in FIG. 1, and components such as buses, input / output interfaces, and / or the like are omitted. In addition, the electronic device 12 can include any other appropriate components according to the specific application.

[0207] Example computer program product and computer-readable storage medium

[0208] In addition to the above-described method and device, an embodiment of the present application can be a computer program product including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the object detection method according to various embodiments of the present application described in the above "Example Method" section of the specification.

[0209] The computer program product can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. The embodiments of the present application are not limited by the programming languages used to implement the methods.

[0210] In addition, the embodiments of the present application can also be a computer readable storage medium, which stores computer program instructions, and the computer program instructions, when executed by a processor, cause the processor to perform the steps in the object detection method according to various embodiments of the present application described in the above “Exemplary Method” section of the specification.

[0211] The computer readable storage medium can take the form of one or more combinations of any type of readable media. The readable media can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any combination thereof. More specific examples (a non-exhaustive list) of the readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0212] The basic principles of the present application are described above in combination with specific embodiments, but it should be noted that the advantages, advantages, effects and the like mentioned in the present application are only examples and are not limiting, and these advantages, advantages, effects and the like cannot be considered as the must-have of each embodiment of the present application. In addition, the above-mentioned specific details are only for the purpose of example and for the purpose of understanding, and the above-mentioned details do not limit the present application to the must-use of the above-mentioned specific details to realize.

[0213] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0214] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0215] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0216] The above description has been given for illustrative and descriptive purposes. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A method for object detection, comprising: determining key frames and non-key frames in each video frame in a video to be detected based on features of the video frame; determining a detection result of a target key frame in the key frames by aggregating object features of the target key frame and object features of other key frames; determining a detection result of a non-key frame by correcting object features of the non-key frame using object features of the key frames; wherein the determining the detection result of the target key frame by aggregating object features of the target key frame and object features of other key frames comprises: determining object features of the target key frame and spatial positions of the object features; after determining confidence degrees corresponding to each of the object features and the spatial positions of the object features, respectively, sorting each of the object features and the spatial positions of the object features in descending order of the confidence degrees, and updating a first n1 object features and corresponding spatial positions in the sorting to a short-term memory, wherein n1 is a preset positive integer; or comparing the confidence degrees of each of the object features and the spatial positions of the object features with a preset confidence threshold, and updating object features and spatial positions of the object features greater than the confidence threshold to the short-term memory, wherein the short-term memory is used to store object features and spatial positions of the object features of no more than M key frames, the object features stored in the short-term memory correspond to video frames generated at a time close to a generation time of the target key frame, and M is a preset positive integer; updating a long-term memory based on object features dequeued from the short-term memory, the long-term memory being used to store object features of the other key frames, the object features stored in the long-term memory corresponding to video frames generated at a time interval from the generation time of the target key frame; aggregating object features of the target key frame based on object features in the long-term memory and the object features and spatial positions of the object features stored in the short-term memory; determining the detection result of the target key frame based on the object features of the target key frame after the aggregation.

2. The method according to claim 1, wherein, The determining the key frames and non-key frames in each video frame in the video to be detected based on features of the video frame comprises: if the video frame is a first frame of the video to be detected, determining the video frame as the key frame, the first frame being a video frame generated earliest in the video to be detected; if the video frame is other than the first frame, determining a difference value between features of the video frame and features of a first key frame, the first key frame being a key frame in a key frame set closest in generation time to a current time; if the difference value is greater than a difference threshold, determining the video frame as the key frame; if the difference value is not greater than the difference threshold, determining the video frame as the non-key frame.

3. The method of claim 1, wherein, The determining the object features of the target key frame comprises: extracting a plurality of initial object features of the target key frame; calculate similarities between the initial object features of the target key frame and features of regions in the target key frame respectively; determine the initial object features that meet the first preset condition as corrected object features; based on the similarities between the corrected object features and the features of regions in the target key frame, correct the corrected object features again until the number of corrections reaches a preset number; after the number of corrections reaches the preset number, determine object features that meet a second preset condition as object features of the target key frame.

4. The method of claim 1, wherein, the feature aggregation of the object features of the target key frame based on the object features in the long-term memory, the object features stored in the short-term memory and spatial positions of the object features, comprises: select T object features from the long-term memory; based on cosine similarities between the T object features and the object features in the short-term memory and the object features of the target key frame respectively, determine first weights of the T object features for the object features in the short-term memory and the object features of the target key frame respectively; based on the first weights, calculate weighted sums between the T object features and the object features in the short-term memory and the object features of the target key frame respectively, wherein the weighted sum between the T object features and the object features in the short-term memory is a first object feature, and the weighted sum between the T object features and the object features of the target key frame is a second object feature; based on the first object feature, a spatial position of the first object feature stored in the short-term memory, the second object feature and a spatial position of the second object feature, determine apparent similarity and geometric similarity between the first object feature and the second object feature; based on the apparent similarity and the geometric similarity between the first object feature and the second object feature, determine a second weight of the first object feature for the second object feature; based on the second weight, calculate a weighted sum between the first object feature and the second object feature, wherein the weighted sum between the first object feature and the second object feature is an object feature of the target key frame after feature aggregation.

5. The method according to any one of claims 1 to 4, wherein, the determination of the detection result of the non-key frame by correcting the object features of the non-key frame based on the object features of the key frame, comprises: based on a detection result of a first video frame, determine a candidate region of the non-key frame, wherein a time difference between a generation time of the first video frame and a generation time of the non-key frame is within a preset range; extract object features in the candidate region of the non-key frame; correct the object features in the candidate region based on the object features of the key frame to obtain a feature correction result; based on the feature correction result, determine the detection result of the non-key frame.

6. The method of claim 5, wherein, the correction of the object features in the candidate region based on the object features of the key frame to obtain a feature correction result, comprises: calculate similarities between the object features of the key frame and the object features in the candidate region; determine an object feature in the candidate region as a third object feature when the similarity meets a third preset condition; perform fusion processing on the third object feature and the object feature of the key frame to obtain the feature correction result. 7.An object detection apparatus, comprising: a video frame determination module configured to determine key frames and non-key frames in each video frame based on features of the video frames in a video to be detected; a key frame detection module configured to determine a detection result of a target key frame by performing feature aggregation on object features of the target key frame and object features of other key frames determined by the video frame determination module; a non-key frame detection module configured to determine a detection result of a non-key frame by correcting object features of the non-key frame determined by the video frame determination module using object features of the key frames determined by the video frame determination module; wherein the key frame detection module is specifically configured to: determine the object features of the target key frame and spatial positions of the object features; sort each of the object features and the spatial positions of the object features in descending order of confidence after determining confidence of each of the object features and the spatial positions of the object features, and update the first n1 object features and the corresponding spatial positions in a short-term memory in the order, where n1 is a preset positive integer; or compare the confidence of each of the object features and the spatial positions of the object features with a preset confidence threshold, and update the object features and the spatial positions of the object features that are greater than the confidence threshold into the short-term memory, wherein the short-term memory is configured to store object features and spatial positions of the object features of no more than M key frames, the object features stored in the short-term memory correspond to video frames generated at a time close to a generation time of the target key frame, and M is a preset positive integer; update a long-term memory based on object features dequeued from the short-term memory, the long-term memory being configured to store object features of the other key frames, and the object features stored in the long-term memory correspond to video frames generated at a time interval from the generation time of the target key frame; perform feature aggregation on the object features of the target key frame based on the object features in the long-term memory and the object features and the spatial positions of the object features stored in the short-term memory; and determine the detection result of the target key frame based on the object features of the target key frame after the feature aggregation. 8.A computer readable storage medium, the storage medium storing a computer program, the computer program being configured to execute the object detection method of any one of claims 1-6. 9.An electronic device, comprising: a processor; a memory configured to store executable instructions for the processor; the processor configured to read the executable instructions from the memory and execute the instructions to implement the object detection method of any one of claims 1-6.

Images