Target detection method, electronic equipment and computer readable storage medium

Abstract

The invention discloses a target detection method, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring a high-resolution current image; inputting the current image into the extraction network to obtain a density image corresponding to the current image; determining a target area in the current image from the density image according to the density distribution of each pixel point in the density image; and performing target detection processing on the target area in the current image to obtain target information in the current image. Therefore, the target detection precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a target detection method, an electronic device and a computer-readable storage medium. Background Art

[0002] In the field of image processing, object detection has achieved remarkable results. Object detection can detect vehicles in traffic scenes and floating waste on the surface of rivers.

[0003] For the management of floating waste on the river surface through target detection, the accuracy of target detection determines the management effect. At present, the floating waste on the river surface is detected by drones or remote sensing images. Due to problems such as shooting height, the image resolution used for target detection is small, resulting in low recognition accuracy of floating waste on the river surface. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a target detection method, an electronic device and a computer-readable storage medium, which can improve the accuracy of target detection.

[0005] To solve the above technical problems, the present application provides a target detection method, which includes: obtaining a high-resolution current image, wherein the current image includes drifting objects; inputting the current image into an extraction network to obtain a density image corresponding to the current image; determining a target area in the current image from the density image according to the density distribution of each pixel point in the density image; performing target detection processing on the target area in the current image to obtain target information in the current image.

[0006] In one embodiment, the extraction network includes at least two channels, each channel has a different scale, and the step of inputting the current image into the extraction network to obtain a density image corresponding to the current image includes: performing feature extraction on the current image using the at least two channels respectively to obtain scale features under each channel; and performing feature fusion processing on the scale features under each channel to obtain the density image.

[0007] In one embodiment, the step of determining the target area in the current image from the density image based on the density distribution of each pixel point in the density image includes: processing the density image using the maximum inter-class variance method to obtain an estimated value; and determining the area formed by the pixel points in the density image whose density values ​​are greater than the estimated value as the target area where the drifting object is located.

[0008] In one embodiment, the step of performing target detection processing on the target area in the current image to obtain the target information in the current image includes: inputting the current image into a residual network to obtain the target attribute information of each target area predicted by the residual network; merging the target areas in the current image according to the target attribute information of each target area to obtain the target information in the current image.

[0009] In one embodiment, the target attribute information includes target categories, and the step of merging the target areas in the current image according to the target attribute information of each target area to obtain the target information in the current image includes: classifying each target area according to the target category in each target area to obtain multiple category sets; traversing each category set; calculating the first intersection-and-union ratio between each target area and other target areas in the current image according to the category scores in each category set from large to small; merging two target areas whose first intersection-and-union ratio is greater than the first intersection-and-union ratio threshold to obtain the target information in the current image.

[0010] In one embodiment, the target information includes a first target frame formed by the area where the target is located in the current image. After the step of performing target detection processing on the target area in the current image to obtain the target information in the current image, the method further includes: in response to detecting the presence of a second target frame in the current tracking queue, calculating a second intersection-and-union ratio between the first target frame and an acquired prediction frame, the prediction frame being obtained based on prediction of the second target frame, the second target frame being obtained based on detection of historical images, and the acquisition time of the historical images being earlier than the acquisition time of the current image; performing target association processing according to the second intersection-and-union ratio between the first target frame and the prediction frame.

[0011] In one embodiment, the step of performing target association processing based on the second intersection-and-union ratio between the first target box and the prediction box includes: in response to the second intersection-and-union ratio being greater than or equal to a second intersection-and-union ratio threshold, fusing the first target box and the prediction box, and determining a new tracking queue based on the fused target box; in response to the second intersection-and-union ratio being less than a third intersection-and-union ratio threshold, adding the second target box to the current tracking queue, and the second intersection-and-union ratio threshold is greater than the third intersection-and-union ratio threshold.

[0012] In one embodiment, after the step of performing target detection processing on the target area in the current image to obtain the target information in the current image, the method further includes: in response to detecting that there is no second target frame in the current tracking queue, obtaining the queue number of the current tracking queue and the target frame number of the second target frame in the current tracking queue; in response to the queue number being less than or equal to the target frame number, performing target association processing according to the third intersection-and-union ratio between each second target frame and the first target frame in the current tracking queue.

[0013] In order to solve the above technical problems, the present application provides an electronic device, including a memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the above target detection method.

[0014] In order to solve the above technical problems, the present application provides a computer-readable storage medium including program data stored therein, and the program data is used to implement the above target detection method when executed by a processor.

[0015] In the above scheme, the acquired high-resolution current image is input into the extraction network to obtain the density image corresponding to the current image; the target area in the current image is determined from the density image according to the density distribution of each pixel in the density image; the target area in the current image is subjected to target detection processing to obtain the target information in the current image. On the one hand, the use of high-resolution images for target detection can improve the accuracy of target detection; on the other hand, by converting the high resolution into a density image, and then determining the target area according to the density distribution of each pixel in the density image, and then using the target area to determine the target information, the target detection efficiency is improved while retaining the image details. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0017] Figure 1 is a flowchart of an exemplary embodiment of a target detection method shown in the present application;

[0018] Figure 2 yes Figure 1 A simplified structural diagram of a target density extraction network in a target detection method is shown;

[0019] Figure 3 yes Figure 1FIG. 1 is a flow chart of an exemplary embodiment of step S130 in the target detection method;

[0020] Figure 4 yes Figure 3 The simplified structural diagram of the residual network yolov5 in the target detection method shown;

[0021] Figure 5 yes Figure 1 FIG. 1 is a flow chart of an exemplary embodiment of the target detection method after step S130; FIG.

[0022] Figure 6 is a mapping relationship diagram of the current image in the real world in the target detection method shown in this application;

[0023] Figure 7 is a block diagram of a target detection device shown in an exemplary embodiment of the present application;

[0024] Figure 8 It is a structural schematic diagram of an embodiment of an electronic device provided by the present application;

[0025] Fig. 9 It is a structural schematic diagram of an embodiment of a computer-readable storage medium provided by the present application. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It will be appreciated that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only some but not all structures related to the present application are shown in the drawings. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the art without making creative work are within the scope of protection of the present application.

[0027] First of all, it should be noted that river environment has always been an important part of environmental governance, but how to efficiently discover, manage and protect floating objects in rivers has become a problem today. At present, the use of drones or remote sensing images to detect floating waste on the surface of rivers has a low resolution for target detection due to problems such as shooting height, resulting in low accuracy in identifying floating waste on the surface of rivers.

[0028] Based on this, the present application proposes a target detection method, an electronic device and a computer-readable storage medium. Figure 1 , Figure 1 It is a flowchart of an exemplary embodiment of the target detection method shown in the present application.

[0029] The execution subject of the target detection method can be a terminal device or a server or other processing device, wherein the terminal device can be a user equipment (UE), a computer, a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The execution subject of the target detection method can also be a target detection device. In some possible implementations, the target detection method can be implemented by a processor calling a computer-readable instruction stored in a memory.

[0030] Specifically, the target detection method of this embodiment includes the following steps:

[0031] Step S110, obtaining a high-resolution current image.

[0032] The current image can be an image with a resolution of 4056x3040, 5280x3956, 8192x5460, etc.

[0033] In actual application scenarios, the target detection device may be a drone or a remote sensing device. The target detection device collects video streams in real time and uses the current video frame in the video stream as the current image.

[0034] Step S120, input the current image into the extraction network to obtain a density image corresponding to the current image.

[0035] In an embodiment of the present application, the target detection device inputs the current image into the extraction network to obtain a density image corresponding to the current image. The extraction network can be a lightweight target density extraction network, which can be understood as an image conversion network for converting the high-resolution current image into a density image. Exemplarily, the target detection device can perform feature extraction on the current image with at least two channels in the extraction network to obtain scale features under each channel; and perform feature fusion processing on the scale features under each channel to obtain the density image.

[0036] The detailed structure of the target density extraction network can be found in Figure 2 ,Depend on Figure 2It can be seen that the target density extraction network includes three layers of branches, that is, three channels. The three layers of branches are set with different convolution kernels so that the current image input therein can obtain multi-scale features under different receptive field ranges, and then determine the density image based on the multi-scale features. It should be noted that the first layer of branches includes 1 9x9 convolution layer and 3 7x7 convolution layers. The second layer of branches includes 1 7x7 convolution layer and 3 5x5 convolution layers. The third layer of branches includes 1 5x5 convolution layer and 3 3x3 convolution layers. Specifically, the target detection device performs downsampling processing on the high-resolution current image through the target density extraction network to obtain the downsampled current image, so as to reduce the memory occupation of the high-resolution image on the target density extraction network; respectively performs feature extraction processing of different scales on the downsampled current image through the first layer branch, the second layer branch and the third layer branch to obtain the first scale feature, the second scale feature and the third scale feature; then merges the first scale feature, the second scale feature and the third scale feature, and performs upsampling processing on the merged first scale feature, the second scale feature and the third scale feature to obtain the upsampled first scale feature, the upsampled second scale feature and the upsampled third scale feature; finally, the upsampled first scale feature, the upsampled second scale feature and the upsampled third scale feature are subjected to feature fusion processing to obtain a density image.

[0037] In other embodiments, the target detection device may pre-acquire the image conversion relationship between the high-resolution image and the density image, and then convert the acquired current image into the density image according to the image conversion relationship.

[0038] Step S130, determining a target area in the current image from the density image according to the density distribution of each pixel in the density image.

[0039] The target area can be understood as an image area in the current image where the target may exist. The target may be floating waste in the water surface of a river. In other embodiments, the target may also be a vehicle in a traffic scene.

[0040] As an example, when detecting floating waste on the surface of a river, the floating waste may be one or in pieces, so that there is a difference between the area where the floating waste is located and other areas. The target detection device can perform regional segmentation based on the density distribution difference between the target in the density image and other objects other than the target, and obtain the target area of ​​the target in the current image. Specifically, the target detection device performs image segmentation on the density image according to a pre-set grayscale threshold, and determines the area where the grayscale value of the pixel point in the density image is greater than or equal to the grayscale threshold as the target area; and determines the area where the grayscale value of the pixel point in the density image is less than the grayscale threshold as the background area. It should be noted that the pre-set grayscale threshold can be set manually.

[0041] Further, as another exemplary embodiment, the embodiment of the present application may use the maximum interclass variance method to determine the target area in the current image. Specifically, the target detection device uses the maximum interclass variance method to process the density image to obtain an estimated value; then the density image is segmented according to the estimated value to obtain the target area in the current image. It should be noted that the estimated value represents the maximum interclass variance between the target area and other areas in the density image. For the method of segmenting the density image according to the estimated value to obtain the target area in the current image, the target detection device can determine the pixel points in the density image whose grayscale value is greater than or equal to the estimated value as one category, and determine the pixel points in the density image whose grayscale value is less than the estimated value as another category, thereby obtaining the target area in the current image.

[0042] Furthermore, considering that there may be two overlapping target areas in the current image, the target detection device of the embodiment of the present application can calculate the overlap rate between the two overlapping target areas, and in response to the overlap rate being greater than the overlap rate threshold, the corresponding two target areas are merged.

[0043] Step S140, performing target detection processing on the target area in the current image to obtain target information in the current image.

[0044] The target information includes the target in the current image, the target position, and the target frame formed by the target area. It should be noted that the target detection device can convert the target area into the target frame through the connected domain extraction algorithm, that is, convert the target area into the target frame through the connected domain extraction algorithm.

[0045] As an example, the target detection device may input the current image into the target detection network to obtain the target information in the current image. The target detection network may be a yolov5 network, etc. As another example, the target detection device may obtain the feature similarity between the regional features of the target area and the preset regional features, and in response to the feature similarity being greater than a preset similarity threshold, the preset target information corresponding to the preset regional features is used as the target information of the corresponding target area.

[0046] It can be seen that the target detection method of the embodiment of the present application inputs the acquired high-resolution current image into the extraction network to obtain a density image corresponding to the current image; determines the target area in the current image from the density image according to the density distribution of each pixel in the density image; performs target detection processing on the target area in the current image to obtain the target information in the current image. On the one hand, the use of high-resolution images for target detection can improve the accuracy of target detection; on the other hand, by converting high resolution into a density image, and then determining the target area according to the density distribution of each pixel in the density image, and then using the target area to determine the target information, it is possible to improve the target detection efficiency while retaining the image details.

[0047] See also Figure 3 , Figure 3 yes Figure 1 The flowchart of an exemplary embodiment of step S130 in the target detection method is shown. Specifically, step S130 performs target detection processing on the target area in the current image, and the process of obtaining target information in the current image also includes the following steps:

[0048] Step S310, inputting the current image into the residual network to obtain target attribute information of each target area predicted by the residual network.

[0049] The target attribute information may include target category, target size, etc. The target size may include information such as the width and height of the target in the current image.

[0050] The residual network can be an improved yolov5 network, for details, please refer to Figure 4 ,Depend on Figure 4It can be seen that the residual network obtains the target category of the current image by processing the input current image, and fuses the semantic information of different stages, and can output the target size, such as the target width or height. It should be noted that the residual network in the embodiment of the present application is a trained residual network. It should be further noted that before training the residual network, it is necessary to preprocess the training image, specifically, to perform random rotation, scaling, flipping, cropping and other operations on the training image to facilitate fixing the size of the image to be trained in the input residual network; after that, the preprocessed image data to be trained can be normalized to speed up the convergence of the residual network.

[0051] Step S320 , merging the target areas in the current image according to the target attribute information of each target area to obtain the target information in the current image.

[0052] As an example, the target detection device can merge each target area according to the target category of the target in each target area to obtain the target information in the current image. Specifically, the target detection device can use the non-maximum suppression algorithm (NMS) for processing, but there are overlapping parts between the target areas, so when using the NMS algorithm, the two overlapping target areas need to be fused. The target detection device classifies each target area according to the target category in each target area to obtain multiple category sets; traverses each category set; calculates the first intersection-and-union ratio between each target area and other target areas in the current image according to the category scores in each category set from large to small; merges the two target areas whose first intersection-and-union ratio is greater than the first intersection-and-union ratio threshold to obtain the target information in the current image.

[0053] As another example, the target detection device can directly classify each target area according to the target category in each target area to obtain multiple category sets, and for each category set, determine whether there are overlapping target areas therein, and fuse the overlapping target areas in each category set to obtain the target information in the current image.

[0054] It should be noted that the target information in the embodiment of the present application may be the position information, size information, etc. of each category of targets in the current image.

[0055] It can be seen that the target detection method of the embodiment of the present application inputs the current image into the residual network to obtain the target attribute information of each target area predicted by the residual network; the target areas in the current image are merged according to the target attribute information of each target area to obtain the target information in the current image. The target attribute information prediction by the residual network can improve the prediction efficiency and accuracy, and enhance the recognition ability of small targets.

[0056] In order to achieve tracking and early warning of the detected target, based on the above embodiment, the embodiment of the present application can also track the detected target. Figure 5 , Figure 5 yes Figure 1 The flowchart of an exemplary embodiment after step S130 in the target detection method is shown. Specifically, after the target detection process is performed on the target area in the current image in step S130 to obtain the target information in the current image, the target detection method of the embodiment of the present application further includes the following steps:

[0057] Step S510, detecting whether there is a second target frame in the current tracking queue.

[0058] Step S520, in response to detecting the presence of a second target frame in the current tracking queue, calculate a second intersection-and-union ratio between the first target frame and the acquired prediction frame, the prediction frame is obtained based on the prediction of the second target frame, and the second target frame is obtained based on the detection of historical images, and the acquisition time of the historical images is earlier than the acquisition time of the current image.

[0059] The current tracking queue includes a second target frame, which is obtained based on the detection of the historical image, and the acquisition time of the historical image is earlier than the acquisition time of the current image. In other words, the target detection device acquires the historical image and the current image successively, and after performing target detection on the historical image, if there is a detected target, the tracking queue is determined according to the second target frame formed by the target detected from the historical image, that is, the current tracking queue, and then after acquiring the current image, the second target frame in the current tracking queue is updated according to the first target frame formed by the target detected in the current image to ensure the continuity of tracking.

[0060] The predicted frame is obtained based on the prediction of the second target frame. Exemplarily, the target detection device may use a Kalman filter to perform prediction processing on the second target frame to obtain the predicted frame.

[0061] In an embodiment of the present application, after the target detection device detects the target in the current image, it obtains the current tracking queue in real time and determines whether the current tracking queue is empty, that is, whether there is a second target frame that needs to be tracked based on the historical image. In response to detecting the presence of the second target frame in the current tracking queue, the second intersection-union ratio between the first target frame and the acquired prediction frame is calculated.

[0062] Step S530, performing target association processing according to the second intersection-over-union ratio between the first target frame and the predicted frame.

[0063] The target detection device determines the relationship between the second intersection-and-union ratio between the first target frame and the acquired prediction frame and the second intersection-and-union ratio threshold and the third intersection-and-union ratio threshold. In response to the second intersection-and-union ratio being greater than or equal to the second intersection-and-union ratio threshold, the first target frame and the prediction frame are fused, and a new tracking queue is determined based on the fused target frame; in response to the second intersection-and-union ratio being less than the third intersection-and-union ratio threshold, the second target frame is added to the current tracking queue; in response to the second intersection-and-union ratio being greater than or equal to the third intersection-and-union ratio threshold and less than the second intersection-and-union ratio threshold, no association processing is performed. Among them, the second intersection-and-union ratio threshold is greater than the third intersection-and-union ratio threshold. It should be noted that the Hungarian algorithm can be used to fuse the first target frame and the prediction frame.

[0064] Step S540 , in response to detecting that the second target frame does not exist in the current tracking queue, the queue number of the current tracking queue and the target frame number of the second target frame in the current tracking queue are obtained.

[0065] The number of target frames refers to the total number of second target frames in all current tracking queues in the target detection device.

[0066] When the target detection device determines that there is no second target frame in the current tracking queue, indicating that there is no second target frame associated with the first target frame, it obtains the queue number of the current tracking queue and the target frame number of the second target frame in the current tracking queue, and determines whether the queue number is less than or equal to the target frame number. If so, executes step S550.

[0067] Step S550 , in response to the number of queues being less than or equal to the number of target frames, performing target association processing according to the third intersection-and-union ratio between each second target frame and the first target frame in the current tracking queue.

[0068] If the target detection device determines that the number of queues is less than or equal to the number of target frames, target association processing is performed according to the third intersection-and-union ratio between each second target frame and the first target frame in the current tracking queue. Specifically, if the target detection device determines that the number of queues is less than or equal to the number of target frames, the third intersection-and-union ratio between the first target frame and each second target frame in the current tracking queue is calculated, and the second target frame corresponding to the maximum intersection-and-union ratio is strongly associated with the first target frame.

[0069] It should be noted that for the unassociated first target frame, an unassociated signal needs to be set for the unassociated first target frame, and the number of times the unassociated signal is set is recorded. If it is detected that the number of times the unassociated signal of the first target frame is set is greater than the preset number threshold, it is determined that the corresponding first target frame has been lost and no tracking processing is performed.

[0070] It can be seen that the target detection method of the embodiment of the present application detects whether there is a second target frame in the current tracking queue; in response to detecting that there is a second target frame in the current tracking queue, the second intersection and union ratio between the first target frame and the acquired prediction frame is calculated, the prediction frame is obtained based on the prediction of the second target frame, the second target frame is obtained based on the detection of the historical image, and the acquisition time of the historical image is earlier than the acquisition time of the current image; target association processing is performed according to the second intersection and union ratio between the first target frame and the prediction frame; in response to detecting that there is no second target frame in the current tracking queue, the queue number of the current tracking queue and the target frame number of the second target frame in the current tracking queue are obtained; in response to the queue number being less than or equal to the target frame number, target association processing is performed according to the third intersection and union ratio between each second target frame and the first target frame in the current tracking queue. Continuous tracking of the target can be achieved in this way.

[0071] On the basis of the above-mentioned embodiments, the embodiments of the present application can also obtain the continuous tracking time of the same target, determine whether the continuous tracking time is greater than the time threshold, and if so, report the corresponding target. For the method of obtaining the continuous tracking time of the same target, the area of ​​the target on the map can be compared with the preset area to determine, that is, the existence time of the area of ​​the target on the map in the preset area is obtained. It should be noted that the area information of the target on the map can be converted into positioning information (Global Positioning System, GPS). As an example, the target detection device obtains the continuous existence time of floating waste A in City B. If the continuous existence time of floating waste A in City B is greater than the time threshold, the corresponding management personnel of City B are notified to clean up the floating waste A. As another example, the target detection device obtains the residence time of vehicle C in road area D. If the residence time of vehicle C in road area D is greater than the time threshold, the corresponding management personnel of road area D are notified to rescue vehicle C.

[0072] It should be noted that when reporting a target, in order to accurately obtain the current location information of the target, the embodiment of the present application can also report the current location information of the target. For details on how to obtain the current location information, please refer to Figure 6 ,like Figure 6 As shown, the starting point a of the acquisition device is calibrated, and the height h1 of the target relative to the take-off point, the height h3 of the acquisition device relative to the starting point when acquiring the current image, the distance Pd from the center of the target to the center of the image, and the focal length of the acquisition device are obtained. The height h2=h3-h1 of the acquisition device relative to the target is obtained. Then, the offset Wd=h2 / (focal length)*Pd of the target position relative to the pixel center in the real world is obtained according to the trigonometric transformation. Finally, the offset Wd of the target position relative to the pixel center in the real world is processed in a preset size conversion method to obtain the current positioning information of the target (Global Positioning System, GPS).

[0073] Figure 7 FIG. 1 is a block diagram of a target detection device shown in an exemplary embodiment of the present application. Figure 7 As shown, the exemplary target detection device 700 includes: an acquisition module 710, an input module 720, a determination module 730 and a detection module 740. Specifically:

[0074] The acquisition module 710 is used to acquire a high-resolution current image.

[0075] The input module 720 is used to input the current image into the extraction network to obtain a density image corresponding to the current image.

[0076] The determination module 730 is used to determine the target area in the current image from the density image according to the density distribution of each pixel point in the density image.

[0077] The detection module 740 is used to perform target detection processing on the target area in the current image to obtain target information in the current image.

[0078] In this exemplary target detection device, the acquired high-resolution current image is input into the extraction network to obtain a density image corresponding to the current image; the target area in the current image is determined from the density image according to the density distribution of each pixel in the density image; the target area in the current image is subjected to target detection processing to obtain target information in the current image. On the one hand, the use of high-resolution images for target detection can improve the accuracy of target detection; on the other hand, by converting high resolution into a density image, and then determining the target area according to the density distribution of each pixel in the density image, and then using the target area to determine the target information, it is possible to improve the target detection efficiency while retaining the image details.

[0079] Among them, the functions of each module can be found in the target detection method embodiment, and will not be repeated here.

[0080] In order to implement the target detection method of the above embodiment, the present application proposes another electronic device, which is specifically referred to as Figure 8 , Figure 8 It is a structural schematic diagram of an embodiment of an electronic device provided by the present application.

[0081] The electronic device 800 includes a memory 801 and a processor 802 , wherein the memory 801 and the processor 802 are coupled.

[0082] The memory 801 is used to store program data, and the processor 802 is used to execute the program data to implement the target detection method of the above embodiment.

[0083] In this embodiment, the processor 802 may also be referred to as a CPU (Central Processing Unit). The processor 802 may be an integrated circuit chip having signal processing capabilities. The processor 802 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor 802 may also be any conventional processor, etc.

[0084] The present application also provides a computer-readable storage medium, such as Fig. 9 As shown, the computer-readable storage medium 900 is used to store program data 901. When the program data 901 is executed by the processor, it is used to implement the target detection method in the method embodiment of the present application.

[0085] The method involved in the target detection method embodiment of the present application, when implemented in the form of a software functional unit and sold or used as an independent product, can be stored in a device, such as a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the prior art that contributes to the technical solution or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0086] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for detecting floating objects in a river, characterized in that: The method comprises: Acquire a high-resolution current image, wherein the current image includes the drifting object; Inputting the current image into an extraction network to obtain a density image corresponding to the current image; Determining a target area in the current image from the density image according to density distribution of each pixel in the density image; Performing target detection processing on the target area in the current image to obtain target information in the current image.

2. The method for removing floating objects from a river according to claim 1, characterized in that: The extraction network includes at least two channels, and the scales of the channels are different. The step of inputting the current image into the extraction network to obtain a density image corresponding to the current image includes: Extracting features of the current image using the at least two channels respectively to obtain scale features of each channel; The scale features under each channel are subjected to feature fusion processing to obtain the density image.

3. The method for removing floating objects from a river according to claim 1, characterized in that: The step of determining the target area in the current image from the density image according to the density distribution of each pixel point in the density image comprises: The density image is processed by using the maximum inter-class variance method to obtain an estimated value; An area formed by pixel points in the density image whose density values ​​are greater than the estimated value is determined as a target area where the drifting object is located.

4. The method for removing floating objects from a river according to claim 1, characterized in that: The step of performing target detection processing on the target area in the current image to obtain target information in the current image includes: Inputting the current image into a residual network to obtain target attribute information of each target area predicted by the residual network; The target areas in the current image are merged according to the target attribute information of each target area to obtain the target information in the current image.

5. The method for removing floating objects from a river according to claim 4, characterized in that: The target attribute information includes a target category, and the step of merging the target areas in the current image according to the target attribute information of each target area to obtain the target information in the current image includes: Classify each target area according to the target category in each target area to obtain multiple category sets; Traverse each category collection; Calculate a first intersection-over-union ratio between each target region and other target regions in the current image according to the category scores in each category set from large to small; Perform region merging processing on two target regions whose first IoU ratio is greater than a first IoU ratio threshold to obtain target information in the current image.

6. The method for removing floating objects from a river according to claim 1, characterized in that: The target information includes a first target frame formed by a region where the target is located in the current image. After the step of performing target detection processing on the target region in the current image to obtain the target information in the current image, the method further includes: In response to detecting that there is a second target frame in the current tracking queue, calculating a second intersection-over-union ratio between the first target frame and an acquired prediction frame, the prediction frame being predicted based on the second target frame, the second target frame being acquired based on detecting a historical image, and the acquisition time of the historical image being earlier than the acquisition time of the current image; Target association processing is performed according to a second intersection-over-union ratio between the first target box and the predicted box.

7. The method for removing floating objects from a river according to claim 6, characterized in that: The step of performing target association processing according to the second intersection-and-union ratio between the first target frame and the prediction frame comprises: In response to the second IoU being greater than or equal to a second IoU threshold, fusing the first target frame and the predicted frame, and determining a new tracking queue based on the fused target frame; In response to the second IoU being less than a third IoU threshold, the second target frame is added to the current tracking queue, and the second IoU threshold is greater than the third IoU threshold.

8. The method for removing floating objects from a river according to claim 1, characterized in that: After the step of performing target detection processing on the target area in the current image to obtain target information in the current image, the method further includes: In response to detecting that the second target frame does not exist in the current tracking queue, acquiring the queue quantity of the current tracking queue and the target frame quantity of the second target frame in the current tracking queue; In response to the number of queues being less than or equal to the number of target frames, target association processing is performed according to a third intersection-and-union ratio between each second target frame and the first target frame in the current tracking queue.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: include: Program data is stored, and when the program data is executed by a processor, it is used to implement the method according to any one of claims 1 to 8.

Images