Small target detection method, device and equipment based on optical flow and storage medium

Abstract

The invention provides a small target detection method and device based on optical flow, equipment and a storage medium, and relates to the technical field of target detection, and the method comprises the steps: determining an optical flow image according to a current frame image and a historical frame image of a target region, stacking the current frame image and the optical flow image to obtain a target image, and carrying out the detection of a small target according to the target image, and performing target detection by adopting a preset target detection model to obtain target detection information of the current frame image. According to the method, the time continuity information of the current frame image and the historical frame image can be determined, so that the target detection accuracy in a dynamic environment is improved.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a small target detection method, device, equipment and storage medium based on optical flow. Background Art

[0002] Small targets usually refer to targets with small area, low resolution and unclear feature information in the image. The detection of small targets has important application value in remote sensing monitoring, video surveillance, medical image analysis and other fields. In the field of image processing, especially target detection, the detection of small targets has always been a challenging problem.

[0003] Currently, image pyramid or feature pyramid techniques are used to enhance the feature representation of small objects. These techniques extract image features at different scales so that small objects can obtain sufficient feature information at a certain scale.

[0004] However, in complex backgrounds, since the target and the background are similar, there are no obvious features to distinguish them, which makes it difficult to effectively extract and identify the target, resulting in the target being unable to be detected or the recognition effect of the target is poor. Summary of the invention

[0005] The purpose of this application is to address the deficiencies in the above-mentioned prior art and provide a small target detection method, device, equipment and storage medium based on optical flow, which can determine the time continuity information of the current frame image and the historical frame images, thereby improving the accuracy of target detection in a dynamic environment.

[0006] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0007] In a first aspect, an embodiment of the present application provides a small target detection method based on optical flow, the method comprising:

[0008] Determine an optical flow image according to a current frame image and a historical frame image of a target area;

[0009] Stacking the current frame image and the optical flow image to obtain a target image;

[0010] According to the target image, a preset target detection model is used to perform target detection to obtain target detection information of the current frame image.

[0011] Optionally, determining the optical flow image according to the current frame image and the historical frame images of the target area includes:

[0012] According to a preset image resolution corresponding to the current frame image, optical flow calculation is performed on the current frame image and the historical frame image to obtain optical flow information corresponding to each pixel point in the current frame image at the preset image resolution;

[0013] The optical flow image is generated according to the optical flow information corresponding to each pixel point.

[0014] Optionally, performing optical flow calculation on the current frame image and the historical frame image according to a preset image resolution corresponding to the current frame image to obtain optical flow information corresponding to each pixel point in the current frame image at the preset image resolution includes:

[0015] According to the preset image resolution, a preset optical flow algorithm is used to perform optical flow calculation on the current frame image and the historical frame image to obtain optical flow information corresponding to each pixel at the preset image resolution;

[0016] Alternatively, according to the preset image resolution, a preset optical flow prediction model is used to perform optical flow calculation on the current frame image and the historical frame image to obtain optical flow information corresponding to each pixel at the preset image resolution.

[0017] Optionally, before stacking the current frame image and the optical flow image to obtain a target image, the method further includes:

[0018] Normalizing the optical flow information corresponding to each pixel in the optical flow image;

[0019] The stacking of the current frame image and the optical flow image to obtain a target image includes:

[0020] The current frame image and the normalized optical flow image are stacked to obtain the target image.

[0021] Optionally, normalizing the optical flow information corresponding to each pixel in the optical flow image includes:

[0022] The optical flow information of each pixel in the optical flow image in the first direction and the optical flow information in the second direction are normalized respectively by using a preset color coding method.

[0023] Optionally, stacking the current frame image and the optical flow image to obtain a target image includes:

[0024] The multiple color channels of the current frame image and the two optical flow channels of the optical flow image are stacked to obtain the target image having the multiple color channels and the two optical flow channels.

[0025] Optionally, the target detection information includes: information of at least two detection frames; and the method further includes:

[0026] According to the information of the at least two detection frames, a non-maximum suppression algorithm is used to perform a deduplication process on the at least two detection frames to obtain a target detection frame;

[0027] According to the position information of the target detection frame and the category information corresponding to the target detection frame, the position of the target object in the current frame image and the category to which the target object belongs are determined respectively.

[0028] In a second aspect, another embodiment of the present application provides a small target detection device based on optical flow, the device comprising:

[0029] A determination module, used to determine an optical flow image based on a current frame image and a historical frame image of a target area;

[0030] A stacking module, used for stacking the current frame image and the optical flow image to obtain a target image;

[0031] The detection module is used to perform target detection based on the target image using a preset target detection model to obtain target detection information of the current frame image.

[0032] In the third aspect, another embodiment of the present application provides a computer device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the storage medium communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the small target detection method based on optical flow as described in any of the first aspects above.

[0033] In a fourth aspect, another embodiment of the present application provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the small target detection method based on optical flow as described in any one of the above-mentioned first aspects are executed.

[0034] The beneficial effects of this application are:

[0035] The present application provides a small target detection method, device, equipment and storage medium based on optical flow. According to the current frame image and the historical frame image of the target area, an optical flow image is determined, the current frame image and the optical flow image are stacked to obtain the target image, and according to the target image, a preset target detection model is used to perform target detection to obtain the target detection information of the current frame image. The present application can determine the time continuity information of the current frame image and the historical frame image through the optical flow image, which helps to improve the detection accuracy when the target moves quickly or there is occlusion between frames. By effectively utilizing the advantages of optical flow and deep learning through the preset target detection model, the accuracy of target detection in dynamic environments can be improved, and it is also applicable to various dynamic scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A schematic diagram of a small target detection method based on optical flow provided in an embodiment of the present application;

[0038] Figure 2 A schematic diagram of a process for determining an optical flow image in a small target detection method based on optical flow provided in an embodiment of the present application;

[0039] Figure 3 A schematic diagram of a process for determining a target image in another small target detection method based on optical flow provided in an embodiment of the present application;

[0040] Figure 4 A schematic diagram of a flow chart of determining the category to which a target object belongs in a small target detection method based on optical flow provided in an embodiment of the application;

[0041] Figure 5 A schematic diagram of the structure of a small target detection device based on optical flow provided in an embodiment of the present application;

[0042] Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of order, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0044] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0045] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0046] Small targets usually refer to targets with small area, low resolution and unclear feature information in the image. The detection of small targets has important application value in remote sensing monitoring, video surveillance, medical image analysis and other fields. In the field of image processing, especially in the field of target detection, the detection of small targets has always been a challenging problem. To this end, this application provides a small target detection method based on optical flow. The method provided in the embodiment of this application is described below in conjunction with multiple figures. Figure 1 A flowchart of a small target detection method based on optical flow is provided in an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0047] Step 101: Determine an optical flow image according to a current frame image and historical frame images of a target area.

[0048] The target area can be any area with small targets, for example, any scene such as the sky, water surface, road, etc. The small target can be a target with a small area, low resolution, and unclear feature information. The current frame image is the frame image being displayed or processed, and the historical frame image is the image corresponding to the previous frame of the current frame image. The optical flow image is used to represent the motion pattern of the target in the image. The optical flow is a vector field, and each vector represents the moving direction and speed of a pixel point in the image between consecutive frames.

[0049] Optionally, the detected feature points in the current frame image of the target area and the detected feature points in the historical frame image are matched, and for each matched pair of detected feature points, the optical flow vector from the historical frame to the current frame is determined, and all the optical flow vectors are aggregated to form an optical flow image. The detected feature points may be corner points, edge points, etc., which are not limited in the embodiments of the present application.

[0050] Step 102: stack the current frame image and the optical flow image to obtain a target image.

[0051] The stacking is performed until the current frame image and the optical flow image are merged into a multi-channel image. For example, the current frame image is a three-channel image, the optical flow image is a two-channel image, and the obtained target image is a five-channel image.

[0052] Optionally, the dimension of the current frame image and the dimension of the optical flow image are determined respectively, and the current frame image and the optical flow image are stacked according to the dimensions to obtain a multi-dimensional target image.

[0053] Step 103: According to the target image, a preset target detection model is used to perform target detection to obtain target detection information of the current frame image.

[0054] Among them, the target detection model can be determined from the deep learning model according to the dimension of the target image, for example, it can be one of the Faster R-CNN (Region-based Convolutional Networks) model, the YOLO (You Only LookOnce) model, and the SSD (Single Shot MultiBox Detector) model. The target detection information may include information such as the target position, the target category, and the target confidence. The target position includes: the horizontal coordinate of the center point of the bounding box, the vertical coordinate of the center point of the bounding box, the width of the bounding box, and the height of the bounding box. The target category is used to indicate the classification of the target, for example, a water cup, a mobile phone, a mouse, etc. The target confidence is used to indicate the degree of confidence of the algorithm in the target detection task that the detected target belongs to a specific category.

[0055] Optionally, the preset target detection model's image requirements preprocess the target image, input the preprocessed target image into the preset target detection model, and the preset target detection model determines the position, category and confidence of the target image based on the processed target image.

[0056] In an embodiment of the present application, an optical flow image is determined based on a current frame image and a historical frame image of a target area, the current frame image and the optical flow image are stacked to obtain a target image, and a preset target detection model is used to perform target detection based on the target image to obtain target detection information of the current frame image. The present application can determine the temporal continuity information of the current frame image and the historical frame image through the optical flow image, which helps to improve the accuracy of detection when the target moves quickly or when occlusion occurs between frames. By effectively utilizing the advantages of optical flow and deep learning through a preset target detection model, the accuracy of target detection in a dynamic environment can be improved, and it is also applicable to various dynamic scenes.

[0057] Based on the above embodiments, the present application also provides a process for determining an optical flow image in a small target detection method based on optical flow. Figure 2 A schematic diagram of a process for determining an optical flow image in a small target detection method based on optical flow provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, in the above step 102, the current frame image and the optical flow image are stacked to obtain a target image, including:

[0058] Step 201: perform optical flow calculation on the current frame image and the historical frame images according to the preset image resolution corresponding to the current frame image, and obtain the optical flow information corresponding to each pixel point in the current frame image at the preset image resolution.

[0059] The preset image resolution may be the maximum resolution of the current frame image and the historical frame image, for example, the original image resolution. The larger the preset image resolution, the more accurate the detection of small objects.

[0060] Optionally, the detected feature points in the current frame image of the preset image resolution of the target area and the detected feature points in the historical frame images of the preset image resolution are matched, and for each matched pair of detected feature points, the optical flow information corresponding to each pixel in the current frame image at the preset image resolution is obtained. The optical flow information is the motion vector of each pixel in the current frame image between consecutive frames. In the specific calculation of the optical flow information, the optical flow calculation can also be performed on the current frame images and historical frame images of multiple scales to obtain the optical flow information corresponding to each pixel in the current frame image at different scales. The optical flow information corresponding to each pixel in the current frame image at different scales is superimposed to obtain the target optical flow information.

[0061] Step 201: Generate an optical flow image according to the optical flow information corresponding to each pixel.

[0062] Optionally, the optical flow information corresponding to each pixel is gathered, and an optical flow image is generated according to the optical flow information set.

[0063] In the embodiment of the present application, according to the preset image resolution corresponding to the current frame image, the optical flow calculation is performed on the current frame image and the historical frame image, the optical flow information corresponding to each pixel in the current frame image at the preset image resolution is obtained, and an optical flow image is generated. It is ensured that there are more details in the generated optical flow image, and the accuracy of target detection is improved.

[0064] On the basis of the above embodiments, the present application further provides a process for determining optical flow information in a small target detection method based on optical flow. In the above step 201, according to the preset image resolution corresponding to the current frame image, the optical flow calculation is performed on the current frame image and the historical frame image to obtain the optical flow information corresponding to each pixel in the current frame image at the preset image resolution, including:

[0065] According to the preset image resolution, the preset optical flow algorithm is used to calculate the optical flow of the current frame image and the historical frame image to obtain the optical flow information corresponding to each pixel at the preset image resolution.

[0066] Among them, the preset optical flow algorithm can be a Farneback algorithm, a Lucas-Kanade algorithm, a Horn-Schunck algorithm, or the like.

[0067] Optionally, based on the preset image resolution, the corresponding preset optical flow algorithm is determined, feature points are detected in the current frame image, points corresponding to the feature points in the current frame image are found in the historical frame images, and for each matching pair of feature points, their optical flow vectors on the image plane are estimated using the selected optical flow algorithm, and the optical flow information corresponding to each pixel point is determined based on the optical flow vector.

[0068] Alternatively, according to the preset image resolution, a preset optical flow prediction model is used to perform optical flow calculation on the current frame image and the historical frame images to obtain optical flow information corresponding to each pixel at the preset image resolution.

[0069] Among them, the preset optical flow prediction model can be a DeepFlow model, a PWC-Net (Pyramid, Warping, and Cost Volume) model, etc., and the embodiments of the present application are not limited to this.

[0070] Optionally, according to the preset image resolution, a corresponding preset optical flow prediction model is determined, and the current frame image and the historical frame image are used as inputs of the preset optical flow prediction model to obtain optical flow information corresponding to each pixel point at the preset image resolution.

[0071] In the embodiment of the present application, according to the preset optical flow algorithm or the preset optical flow prediction model, the optical flow calculation is performed on the current frame image and the historical frame image to obtain the optical flow information corresponding to each pixel at the preset image resolution. The present application can better capture subtle motion changes, help to more accurately identify and distinguish obscured objects, thereby improving the ability of the optical flow algorithm to handle occlusion situations.

[0072] Based on the above embodiment, the present application also provides another process of determining a target image in a small target detection method based on optical flow. Figure 3 A schematic diagram of a process for determining a target image in another small target detection method based on optical flow provided in an embodiment of the present application. Before stacking the current frame image and the optical flow image to obtain the target image in the above step 102, the method further includes:

[0073] Step 301: Normalize the optical flow information corresponding to each pixel in the optical flow image.

[0074] The normalization is used to normalize the velocity components in each direction of the optical flow vector corresponding to each pixel in the optical flow image.

[0075] Optionally, velocity components of multiple dimensions corresponding to optical flow information corresponding to each pixel in the optical flow image are determined, and the velocity components of multiple dimensions are normalized respectively to obtain normalized results of optical flow information corresponding to each pixel in the optical flow image.

[0076] As in step 102 above, the current frame image and the optical flow image are stacked to obtain a target image, including:

[0077] Step 302: stack the current frame image and the normalized optical flow image to obtain a target image.

[0078] Optionally, the dimension of the current frame image and the dimension of the normalized optical flow image are determined respectively, and the current frame image and the normalized optical flow image are stacked according to the dimensions to obtain a multi-dimensional target image.

[0079] In an embodiment of the present application, the optical flow information corresponding to each pixel in the optical flow image is normalized, and the current frame image and the normalized optical flow image are stacked to obtain a target image. The normalized optical flow information of the present application is easier to visualize on the image in the form of color mapping, making the motion trend more obvious. Normalization can reduce the impact of extreme values ​​on the algorithm and improve the robustness of the algorithm to outliers.

[0080] Based on the above embodiment, the present application further provides a process for normalizing optical flow information in a small target detection method based on optical flow. In the above step 301, the optical flow information corresponding to each pixel in the optical flow image is normalized, including:

[0081] The optical flow information of each pixel in the optical flow image in the first direction and the optical flow information in the second direction are normalized respectively by using a preset color coding method.

[0082] The preset color encoding method is used to convert the optical flow vector into a color image.

[0083] Optionally, the optical flow information of each pixel in the optical flow image in the first direction and the optical flow information in the second direction are normalized respectively, and mapped to the range of [0,1] according to the optical flow information of each pixel, and then converted into the hue in the hue, saturation, and brightness model, and the HSV value is converted into the RGB (Red, Green, Blue) value for display on the image.

[0084] For example, for the optical flow in the first direction, a maximum absolute value is intercepted, and values ​​greater than the absolute value are set to 1; for values ​​less than or equal to the absolute value, they are normalized by dividing by the absolute value, and the normalized values ​​are within the range of [-1,1]. For the optical flow in the second direction, the same operation as the first direction is adopted. For example, the maximum absolute value can be set to 20. The first direction can be the horizontal direction, and the second direction can be the vertical direction.

[0085] In an embodiment of the present application, a preset color coding method is used to normalize the optical flow information in the first direction and the optical flow information in the second direction of each pixel in the optical flow image, respectively. The value of the optical flow vector can be mapped within a fixed range, so that the optical flow data in different scenes and different motion speeds can have a unified scale, which is convenient for comparison and analysis.

[0086] Based on the above embodiment, the present application also provides a process for determining a target image in a third small target detection method based on optical flow. In the above step 102, the current frame image and the optical flow image are stacked to obtain the target image, including:

[0087] The multiple color channels of the current frame image and the two optical flow channels of the optical flow image are stacked to obtain a target image with multiple color channels and two optical flow channels.

[0088] Among them, the color channel of the current frame image is used to indicate the color composition of the current frame image. The multiple color channels can be red, green, blue channels, grayscale channels, cyan, magenta, yellow, and black channels, etc., which are specifically determined according to the current frame image. The two optical flow channels of the optical flow image refer to the horizontal direction channel of the optical flow and the vertical direction channel of the optical flow.

[0089] For example, the current frame image has three color channels and the optical flow image has two optical flow channels. The three color channels of the current frame image are stacked with the two optical flow channels of the optical flow image to obtain a target image with three color channels and two optical flow channels, that is, the target image has five channels.

[0090] In an embodiment of the present application, multiple color channels of the current frame image and two optical flow channels of the optical flow image are stacked to obtain a target image with multiple color channels and two optical flow channels. The color information and motion information can be combined together to improve the efficiency and accuracy of subsequent image processing.

[0091] On the basis of the above embodiment, the target detection information includes: information of at least two detection frames; for this purpose, the embodiment of the present application also provides a process for determining the category to which the target object belongs in the small target detection method based on optical flow, Figure 4 A schematic diagram of a flow chart for determining the category to which a target object belongs in a small target detection method based on optical flow provided in an embodiment of the application. Based on the above steps 101 to 103, the method further includes:

[0092] Step 401: Based on information of at least two detection frames, a non-maximum suppression algorithm is used to perform deduplication processing on the at least two detection frames to obtain a target detection frame.

[0093] Among them, the non-maximum suppression algorithm is used to determine the detection box with the highest confidence from multiple detection boxes, and remove other detection boxes that highly overlap with the high-confidence detection box.

[0094] Optionally, the detection frames are sorted according to the confidence of at least two detection frames, the detection frame with the highest confidence is determined from the sorted detection frames and used as the retained detection frame, the overlapping area between the remaining detection frames and the retained detection frames is calculated, and an overlapping threshold is set. When the overlap between a detection frame and the retained detection frame is greater than the overlapping threshold, the detection frame is removed until all detection frames are compared, thereby determining the target detection frame.

[0095] Step 402: According to the position information of the target detection frame and the category information corresponding to the target detection frame, the position of the target object in the current frame image and the category to which the target object belongs are determined respectively.

[0096] Among them, the category information corresponding to the target detection frame is used to distinguish the object category of the small target in the target detection frame, such as distinguishing between cats and dogs, or cars and motorcycles.

[0097] Optionally, the position information of each detection box is extracted from the result output by the target detection model, such as the coordinates of the upper left corner and the lower right corner of the rectangular box. The category information of each detection box is extracted from the result output by the target detection model, such as the category index or the category name. According to the position information of the target detection box, the category to which the target object belongs is determined according to the category information corresponding to the target detection box.

[0098] For example, when the information of the target detection frame is determined to be: [x1, y1, x2, y2] = [50, 50, 200, 200], confidence = 0.95, category information = 0, and the category corresponding to 0 in the category information is data. That is to say, the position of the target object is [x1, y1, x2, y2] = [50, 50, 200, 200], and the category to which the target object belongs is mouse.

[0099] Based on the same inventive concept, an optical flow-based small target detection device corresponding to the optical flow-based small target detection method is also provided in the embodiments of the present application. Since the principle of solving the problem by the device in the embodiments of the present application is similar to the above-mentioned optical flow-based small target detection method in the embodiments of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0100] Figure 5 A schematic diagram of the structure of a small target detection device based on optical flow provided in an embodiment of the present application, the device includes:

[0101] A determination module 501 is used to determine an optical flow image according to a current frame image and a historical frame image of a target area;

[0102] A stacking module 502 is used to stack the current frame image and the optical flow image to obtain a target image;

[0103] The detection module 503 is used to perform target detection based on the target image using a preset target detection model to obtain target detection information of the current frame image.

[0104] Optionally, the determination module 501 is specifically used to: perform optical flow calculation on the current frame image and the historical frame image according to a preset image resolution corresponding to the current frame image, to obtain optical flow information corresponding to each pixel point in the current frame image at the preset image resolution;

[0105] An optical flow image is generated based on the optical flow information corresponding to each pixel.

[0106] Optionally, the determination module 501 is specifically used to: perform optical flow calculation on the current frame image and the historical frame image according to the preset image resolution using a preset optical flow algorithm to obtain optical flow information corresponding to each pixel point at the preset image resolution;

[0107] Alternatively, according to the preset image resolution, a preset optical flow prediction model is used to perform optical flow calculation on the current frame image and the historical frame images to obtain optical flow information corresponding to each pixel at the preset image resolution.

[0108] Optionally, the stacking module 502 is further used to: normalize the optical flow information corresponding to each pixel in the optical flow image;

[0109] Optionally, the stacking module 502 is specifically used to stack the current frame image and the normalized optical flow image to obtain a target image.

[0110] Optionally, the stacking module 502 is specifically used to: normalize the optical flow information of each pixel in the optical flow image in the first direction and the optical flow information in the second direction respectively by using a preset color coding method.

[0111] Optionally, the stacking module 502 is specifically used to stack the multiple color channels of the current frame image and the two optical flow channels of the optical flow image to obtain a target image with multiple color channels and two optical flow channels.

[0112] Optionally, the target detection information includes: information of at least two detection frames; the device further includes: a processing module, specifically used to: according to the information of the at least two detection frames, use a non-maximum suppression algorithm to perform deduplication processing on the at least two detection frames to obtain a target detection frame;

[0113] According to the position information of the target detection frame and the category information corresponding to the target detection frame, the position of the target object in the current frame image and the category to which the target object belongs are determined respectively.

[0114] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0115] The present application also provides a computer device, Figure 6 A schematic diagram of the structure of a computer device provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the computing device includes: a processor 601, a memory 602, and optionally, a bus 603. The memory 602 stores machine-readable instructions executable by the processor 601. When the computer device is running, the processor 601 communicates with the memory 602 via the bus 603. When the machine-readable instructions are executed by the processor 601, the steps of the above-mentioned small target detection method based on optical flow are executed.

[0116] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned small target detection method based on optical flow are executed.

[0117] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the method embodiment, and will not be repeated in this application. In the several embodiments provided in this application, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0118] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or part of the technical solution that contributes to the prior art or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), disk or optical disk and other media that can store program code.

[0119] The above are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be covered by the protection scope of the present application.

Claims

1. A small target detection method based on optical flow, characterized in that: The method comprises: Determine an optical flow image according to a current frame image and a historical frame image of a target area; Stacking the current frame image and the optical flow image to obtain a target image; According to the target image, a preset target detection model is used to perform target detection to obtain target detection information of the current frame image.

2. The method according to claim 1, characterized in that The step of determining the optical flow image according to the current frame image and the historical frame images of the target area includes: According to a preset image resolution corresponding to the current frame image, optical flow calculation is performed on the current frame image and the historical frame image to obtain optical flow information corresponding to each pixel point in the current frame image at the preset image resolution; The optical flow image is generated according to the optical flow information corresponding to each pixel point.

3. The method according to claim 2, characterized in that The performing optical flow calculation on the current frame image and the historical frame image according to the preset image resolution corresponding to the current frame image to obtain the optical flow information corresponding to each pixel point in the current frame image at the preset image resolution includes: According to the preset image resolution, a preset optical flow algorithm is used to perform optical flow calculation on the current frame image and the historical frame image to obtain optical flow information corresponding to each pixel at the preset image resolution; or According to the preset image resolution, a preset optical flow prediction model is used to perform optical flow calculation on the current frame image and the historical frame image to obtain optical flow information corresponding to each pixel at the preset image resolution.

4. The method according to claim 1, characterized in that Before stacking the current frame image and the optical flow image to obtain the target image, the method further includes: Normalizing the optical flow information corresponding to each pixel in the optical flow image; The stacking of the current frame image and the optical flow image to obtain a target image includes: The current frame image and the normalized optical flow image are stacked to obtain the target image.

5. The method according to claim 4, characterized in that The normalizing the optical flow information corresponding to each pixel in the optical flow image includes: The optical flow information of each pixel in the optical flow image in the first direction and the optical flow information in the second direction are normalized respectively by using a preset color coding method.

6. The method according to claim 1, characterized in that The stacking of the current frame image and the optical flow image to obtain a target image includes: The multiple color channels of the current frame image and the two optical flow channels of the optical flow image are stacked to obtain the target image having the multiple color channels and the two optical flow channels.

7. The method according to claim 1, characterized in that The target detection information includes: information of at least two detection frames; the method further includes: According to the information of the at least two detection frames, a non-maximum suppression algorithm is used to perform a deduplication process on the at least two detection frames to obtain a target detection frame; According to the position information of the target detection frame and the category information corresponding to the target detection frame, the position of the target object in the current frame image and the category to which the target object belongs are determined respectively.

8. A small target detection device based on optical flow, characterized in that: The device comprises: A determination module, used to determine an optical flow image based on a current frame image and a historical frame image of a target area; A stacking module, used for stacking the current frame image and the optical flow image to obtain a target image; The detection module is used to perform target detection based on the target image using a preset target detection model to obtain target detection information of the current frame image.

9. A computer device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor executes the machine-readable instructions to perform the steps of the small target detection method based on optical flow as described in any one of claims 1 to 8.

10. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, executes the steps of the small target detection method based on optical flow as claimed in any one of claims 1 to 8.

Images