Ship detection method and device, electronic equipment and storage medium
By carrying a lightweight target detection network in satellites, using rotating bounding box detection, filter filtering and knowledge distillation technology, the problem of poor ship detection in the satellite computing environment is solved, and efficient and accurate ship identification.
Patent Information
- Application Number
- CN202311468608.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-06
- Publication Date
- 2025-02-14
AI Technical Summary
Traditional network models are not suitable for on-satellite computing environments, resulting in poor detection and identification of sea ships in satellites, especially in densely arranged ship detection.
A lightweight ship detection method is provided, which can detect the image of the target area by rotating bounding box, and use the target detection network obtained by filter filter filtering and knowledge distillation to identify the ship.
The rotation bounding box detection reduces background interference and improves the accuracy of ship feature extraction; and through the compression of network width and depth, a lightweight target detection network is obtained that is more suitable for on-star computing, achieving efficient ship Ship identification.
Smart Images
Figure CN119445389A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of optical remote sensing technology, and in particular to a ship detection method, device, electronic equipment and storage medium. Background Art
[0002] Due to the abundant marine resources, there are many ships operating at sea. Therefore, the inspection of ships at sea is an important part of marine management. Ship inspection plays an important role in maritime rescue work, port entry and exit management, and maintaining order in the sea area.
[0003] In the related technology, relevant detection modules (network models) are carried in satellites to detect and identify ships at sea.
[0004] However, unlike the central processing unit (CPU) deployed in ground servers, the computing power of onboard processing equipment is limited, and traditional network models are not suitable for onboard computing environments. Summary of the invention
[0005] The embodiments of the present application provide a ship detection method, device, electronic device and storage medium to provide a lightweight detection network for ships, which is adapted to an onboard computing environment.
[0006] In a first aspect, an embodiment of the present application provides a first ship detection method, the method comprising:
[0007] Capturing an image of a target area to obtain a captured image corresponding to the target area;
[0008] Performing rotation bounding box detection on the object in the acquired image to determine a rotation box of the object in the acquired image;
[0009] For any object rotation frame, the collected image and the angle information of the object rotation frame are input into a target detection network, and features of the object rotation frame are extracted through the target detection network to determine whether the object rotation frame corresponds to a ship;
[0010] The target detection network is obtained by filtering the pre-trained detection model and performing knowledge distillation on part or all of the filtered filters.
[0011] In the above scheme, since ships are mostly narrow rectangles after imaging, the intersection of the horizontal detection frame and the ship area is relatively small, and there are many invalid backgrounds, which may interfere with ship recognition, especially for densely arranged ships. The detection effect is poor. By performing a rotation bounding box detection on the collected image of the target area (such as the sea), the rotation box (object rotation box) containing the object in the collected image is determined. The object rotation box contains less background area, and the overlap between different boxes is small, so the ship feature extraction is better. In addition, the target detection network used is obtained by double processing by filtering the pre-trained detection model with filters (network width compression) and performing knowledge distillation (network depth optimization) on some or all of the filtered filters. Therefore, the lightweight target detection network with a streamlined structure is more suitable for on-board computing. By carrying the above target detection network in the satellite, feature extraction can be performed efficiently and effectively to identify ships in the image.
[0012] In some optional implementations, before inputting the collected image and the angle information of the object rotation frame into the target detection network, the method further includes:
[0013] Determining the bow direction of the object rotation frame;
[0014] Inputting the collected image and the angle information of the object rotation frame into the target detection network includes:
[0015] The collected image, the angle information of the object rotation frame and the bow direction are input into a target detection network.
[0016] In some optional implementations, determining the bow direction of the object rotation frame includes:
[0017] Performing grayscale processing on the object rotation frame to determine the grayscale value of each pixel in the object rotation frame;
[0018] The object rotation frame is segmented along the middle line of the long side to obtain two segmented rectangles, and the sum of the grayscale values of each segmented rectangle is determined respectively;
[0019] For any segmented rectangle, if the sum of the grayscale values of the segmented rectangle is greater than the sum of the grayscale values of another segmented rectangle, the opposite direction of the other segmented rectangle in the segmented rectangle is determined as the bow direction.
[0020] In some optional implementations, the pre-trained detection model is filtered by a filter, including:
[0021] For the pre-trained detection model, selecting a target filter from the pre-trained detection model, and absorbing knowledge of non-target filters through the target filter;
[0022] After pruning the non-target filter, the pre-trained detection model is fine-tuned to obtain a filtered filter.
[0023] In some optional implementations, whether the filtered filter performs knowledge distillation is determined in the following manner:
[0024] For any filtered filter, if the weight of the head convolution layer of the filtered filter is greater than the first preset weight, it is determined that the filtered filter does not perform knowledge distillation; otherwise, it is determined that the filtered filter performs knowledge distillation.
[0025] In some optional implementations, knowledge distillation is performed on some or all of the filtered filters, including:
[0026] For any filtered filter that is subjected to knowledge distillation, if the weight of the intermediate convolutional layer of the filtered filter is less than the second preset weight, performing head distillation and intermediate layer distillation on the filtered filter;
[0027] Otherwise, the filtered filter is subjected to head distillation.
[0028] In some optional implementations, the angle information of the object rotation frame includes coordinates of four corner points of the object rotation frame and the length and width of a minimum horizontal circumscribed rectangle of the object rotation frame.
[0029] In a second aspect, an embodiment of the present application provides a first ship detection device, the device comprising:
[0030] An image acquisition module is used to acquire an image of a target area to obtain an acquired image corresponding to the target area;
[0031] A detection frame module, used to perform rotation boundary frame detection on the object in the collected image, and determine the rotation frame of the object in the collected image;
[0032] a recognition module, for any object rotation frame, inputting the collected image and the angle information of the object rotation frame into a target detection network, performing feature extraction on the object rotation frame through the target detection network, and determining whether the object rotation frame corresponds to a ship;
[0033] The target detection network is obtained by filtering the pre-trained detection model and performing knowledge distillation on some or all of the filtered filters.
[0034] In a third aspect, an embodiment of the present application provides an electronic device, comprising at least one processor and at least one memory, wherein the memory stores a computer program, and when the program is executed by the processor, the processor executes any ship detection method described in the first aspect.
[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program executable by a processor, wherein when the program runs on the processor, the processor executes any of the ship detection methods described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0037] Figure 1 A schematic diagram of a flow chart of a first ship detection method provided in an embodiment of the present application;
[0038] Figure 2 A schematic diagram of a flow chart of a second ship detection method provided in an embodiment of the present application;
[0039] Figure 3 A schematic diagram of an object rotation frame along the middle line of the long side provided in an embodiment of the present application;
[0040] Figure 4 A schematic diagram of knowledge distillation provided in an embodiment of the present application;
[0041] Figure 5 A schematic diagram of the structure of a ship detection device provided in an embodiment of the present application;
[0042] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0043] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.
[0044] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, unless otherwise specified, "plurality" means two or more.
[0045] In the description of this application, it should be noted that, unless otherwise clearly specified and limited, the term "connection" should be understood in a broad sense, for example, it can be directly connected, or indirectly connected through an intermediate medium, or it can be the internal connection of two devices. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.
[0046] By carrying relevant detection modules (network models) in satellites, ships at sea can be detected and identified.
[0047] However, unlike the CPUs deployed in ground servers, the computing power of onboard processing devices is limited, and traditional network models are not suitable for onboard computing environments.
[0048] In view of this, an embodiment of the present application proposes a ship detection method, device, electronic device and storage medium, the method comprising: performing image acquisition on a target area to obtain a captured image corresponding to the target area; performing rotation boundary box detection on the object in the captured image to determine the object rotation frame in the captured image; for any object rotation frame, inputting the captured image and the angle information of the object rotation frame into a target detection network, performing feature extraction on the object rotation frame through the target detection network, and determining whether the object rotation frame corresponds to a ship; wherein the target detection network is obtained by filtering a pre-trained detection model and performing knowledge distillation on some or all of the filtered filters.
[0049] In the above scheme, since ships are mostly narrow rectangles after imaging, the intersection of the horizontal detection frame and the ship area is relatively small, and there are many invalid backgrounds, which may interfere with ship recognition, especially for densely arranged ships. The detection effect is poor. By performing a rotation bounding box detection on the collected image of the target area (such as the sea), the rotation box (object rotation box) containing the object in the collected image is determined. The object rotation box contains less background area, and the overlap between different boxes is small, so the ship feature extraction is better. In addition, the target detection network used is obtained by double processing by filtering the pre-trained detection model with filters (network width compression) and performing knowledge distillation (network depth optimization) on some or all of the filtered filters. Therefore, the lightweight target detection network with a streamlined structure is more suitable for on-board computing. By carrying the above target detection network in the satellite, feature extraction can be performed efficiently and effectively to identify ships in the image.
[0050] The following will be combined with the accompanying drawings and specific embodiments to describe in detail the technical solution of the present application and how the technical solution of the present application solves the above technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.
[0051] Figure 1 A schematic diagram of the process of the first ship detection method provided in the embodiment of the present application is shown as follows: Figure 1 As shown, the following steps are included:
[0052] Step S101: performing image acquisition on a target area to obtain an acquired image corresponding to the target area.
[0053] In this embodiment, image acquisition is performed for a target area (such as the sea) where ship detection is required. The acquired image may contain a ship, and object frame recognition and ship detection are required in the acquired image.
[0054] Step S102: performing rotation bounding box detection on the object in the acquired image to determine a rotation box of the object in the acquired image.
[0055] Since ships are mostly narrow rectangles after imaging, the intersection of the horizontal detection frame and the ship area is relatively small, and there are many invalid backgrounds, which may interfere with ship recognition, especially for densely arranged ships. By performing horizontal detection frame detection on the collected image, the two horizontal detection frames not only contain more background (such as the sea surface), but also intersect with each other, which will affect the accuracy of ship recognition.
[0056] Based on this, this embodiment adopts rotation bounding box detection to determine the rotation box (object rotation box) containing the object in the captured image.
[0057] By performing rotation bounding box detection on the collected image, the rotation box (object rotation box) containing the object in the collected image is determined. The object rotation box contains less background area and the overlap between different boxes is small, which can better extract ship features.
[0058] Step S103: For any object rotation frame, the collected image and the angle information of the object rotation frame are input into a target detection network, and features of the object rotation frame are extracted through the target detection network to determine whether the object rotation frame corresponds to a ship.
[0059] The target detection network is obtained by filtering the pre-trained detection model and performing knowledge distillation on some or all of the filtered filters.
[0060] Satellites operate in the air and are suitable for collecting images of the sea. However, the computing power of onboard processing equipment is limited, and traditional network models are not suitable for onboard computing environments.
[0061] Based on this, this embodiment performs lightweight processing on the pre-trained detection model.
[0062] In terms of network width compression, filter filtering (network width compression) is performed to reduce the complexity of deep convolutional neural networks;
[0063] Since the network structure of the filter is organized in layers, it is inevitable to execute the calculation operations of each layer of the network serially during the detection and recognition calculation process. The network depth is a key factor that determines the speed of network reasoning. Therefore, in terms of network depth compression, the number of network layers of the filter is effectively compressed through knowledge distillation.
[0064] The target detection network obtained by double processing of filtering the pre-trained detection model (network width compression) and knowledge distillation (network depth optimization) of some or all of the filtered filters has a compression rate increased by 75.39%, a speed increased by 30.77%, and a low recognition accuracy, making it more suitable for on-board computing.
[0065] In the above scheme, since ships are mostly narrow rectangles after imaging, the intersection of the horizontal detection frame and the ship area is relatively small, and there are many invalid backgrounds, which may interfere with ship recognition, especially for densely arranged ships. The detection effect is poor. By performing a rotation bounding box detection on the collected image of the target area (such as the sea), the rotation box (object rotation box) containing the object in the collected image is determined. The object rotation box contains less background area, and the overlap between different boxes is small, so the ship feature extraction is better. In addition, the target detection network used is obtained by double processing by filtering the pre-trained detection model with filters (network width compression) and performing knowledge distillation (network depth optimization) on some or all of the filtered filters. Therefore, the lightweight target detection network with a streamlined structure is more suitable for on-board computing. By carrying the above target detection network in the satellite, feature extraction can be performed efficiently and effectively to identify ships in the image.
[0066] In some optional implementations, the angle information of the object rotation frame includes coordinates of four corner points of the object rotation frame and the length and width of a minimum horizontal circumscribed rectangle of the object rotation frame.
[0067] In this embodiment, the object rotation box is a rectangular rotation box, which is rotated at a certain angle with the coordinate axis (x, y axis), and the angle information includes the coordinates of the lower left corner (bl), the upper left corner (tl), the lower right corner (br), and the upper right corner (tr) of the rectangular rotation box; and also includes the horizontal length and vertical width of the minimum horizontal circumscribed rectangle. The length and width of the minimum horizontal circumscribed rectangle can be used to correct the angle when the rotation box is almost parallel to the x, y axis.
[0068] Figure 2 A flow chart of a second ship detection method provided in an embodiment of the present application is shown as follows: Figure 2 As shown, the following steps are included:
[0069] Step S201: performing image acquisition on a target area to obtain an acquired image corresponding to the target area.
[0070] Step S202: performing rotation bounding box detection on the object in the acquired image to determine a rotation box of the object in the acquired image.
[0071] The specific implementation of steps S201 to S202 may refer to other embodiments and will not be described in detail here.
[0072] Step S203: Determine the bow direction of the object rotation frame.
[0073] During implementation, due to the use of rotating bounding box detection, a rotating frame containing the object (object rotating frame) is obtained. The object rotating frame contains less background area. When there is less background area interference in the object rotating frame, the direction of the object, that is, the direction of the bow, can be determined based on the characteristics that the bow accounts for a smaller proportion than the hull and stern.
[0074] Step S204: For any object rotation frame, the collected image, the angle information of the object rotation frame and the bow direction are input into a target detection network, and features of the object rotation frame are extracted through the target detection network to determine whether the object rotation frame corresponds to a ship.
[0075] Since the bow, hull and stern have different characteristics, the bow direction is also used as an input to the target detection network, which can identify ships more accurately.
[0076] In the above scheme, due to the use of rotating bounding box detection, a rotating box containing the object (object rotating box) is obtained. The object rotating box contains less background area. When there is less background area interference in the object rotating box, the direction of the object, that is, the direction of the bow, can be judged based on the characteristics that the bow accounts for a smaller proportion than the hull and stern. Since the bow, hull, and stern have different characteristics, the bow direction is also used as an input of the target detection network, and the target detection network can identify ships more accurately.
[0077] In some optional implementation manners, the above step S203 may be implemented by but not limited to the following methods:
[0078] Performing grayscale processing on the object rotation frame to determine the grayscale value of each pixel in the object rotation frame;
[0079] The object rotation frame is segmented along the middle line of the long side to obtain two segmented rectangles, and the sum of the grayscale values of each segmented rectangle is determined respectively;
[0080] For any segmented rectangle, if the sum of the grayscale values of the segmented rectangle is greater than the sum of the grayscale values of another segmented rectangle, the opposite direction of the other segmented rectangle in the segmented rectangle is determined as the bow direction.
[0081] As described above, when the object rotation frame has less background area interference, the direction of the object, that is, the bow direction, can be determined by combining the characteristics that the bow accounts for a smaller proportion than the hull and stern.
[0082] For ships, the bow is usually pointed, so the bow of the object rotation frame will have some background area. The bow and stern are usually straight and contain almost no background area, and the gray value of the background area is relatively large;
[0083] Based on this, this embodiment first determines the grayscale value of each pixel point in the object rotation frame; and divides the object rotation frame along the middle line of the long side (the ship is slender and the bow direction is on the long side) to obtain two segmented rectangles, and determines the sum of the grayscale values of each segmented rectangle respectively; the segmented rectangle with the larger sum of grayscale values is determined as the segmented rectangle where the bow is located, and the opposite direction of the other segmented rectangle in the segmented rectangle is determined as the bow direction.
[0084] The object rotation box is along the middle line of the long side. Figure 3 shown.
[0085] In some optional implementations, filtering the pre-trained detection model may be implemented by, but not limited to, the following methods:
[0086] For the pre-trained detection model, selecting a target filter from the pre-trained detection model, and absorbing knowledge of non-target filters through the target filter;
[0087] After pruning the non-target filter, the pre-trained detection model is fine-tuned to obtain a filtered filter.
[0088] In practice, there are some important filters and some unimportant filters in the pre-trained detection model. Therefore, the filters can be pruned. If these unimportant filters are directly pruned, it will be difficult to recover the model performance lost due to pruning by other means. And the pruning method requires multiple iterations, which consumes a lot of time and computing resources.
[0089] Based on this, after selecting the target filter from the pre-trained detection model, this embodiment does not directly perform iterative pruning, but absorbs the knowledge of the non-target filter through the target filter. Pruning is performed after the knowledge in the filter is absorbed, which can reduce the model performance loss. Due to the limitations of training techniques, the knowledge in the non-target filter may not be fully absorbed, and the pruned part still contains a small amount of knowledge, so fine-tuning is required to restore performance damage. The above method can efficiently train a lightweight target filter with less model damage.
[0090] This embodiment does not specifically limit the above-mentioned pre-trained detection model, such as the CenterNet (target detection) model. CenterNet is an anchor-free target detection algorithm. Compared with the traditional neural network algorithm, the anchor-free method does not need to generate a pre-selected box, nor does it consume additional resources to calculate the IOU (a function of the target detection algorithm performance) between the pre-selected box and the real bounding box. CenterNet is based on the target center point modeling, and directly regresses to obtain the target width and height, reducing the computational complexity of post-processing such as non-maximum suppression (NMS), and has the advantage of faster detection speed.
[0091] In some optional implementations, whether the filtered filter performs knowledge distillation is determined in the following manner:
[0092] For any filtered filter, if the weight of the head convolution layer of the filtered filter is greater than the first preset weight, it is determined that the filtered filter does not perform knowledge distillation; otherwise, it is determined that the filtered filter performs knowledge distillation.
[0093] In implementation, different filters have different convolutional layer weight distributions. If a filter head has a large convolutional layer weight, knowledge distillation on it will lose important parameters.
[0094] Based on this, this embodiment sets a first preset weight, and compares the head convolution layer weight of each filtered filter with the first preset weight; if the head convolution layer weight of the filtered filter is greater than the first preset weight, it means that the head convolution layer weight of the filter is large, and knowledge distillation will lose important parameters. Therefore, knowledge distillation is not performed on the filter; conversely, if the head convolution layer weight of the filtered filter is less than or equal to the first preset weight, it means that the head convolution layer weight of the filter is small, and knowledge distillation will not lose important parameters, and the number of layers and parameters can be compressed. Therefore, knowledge distillation is performed on the filter.
[0095] The above-mentioned first preset weight can be set according to the actual application scenario, and this embodiment does not make any specific limitation to this.
[0096] In some optional implementations, knowledge distillation is performed on some or all of the filtered filters, including:
[0097] For any filtered filter that is subjected to knowledge distillation, if the weight of the intermediate convolutional layer of the filtered filter is less than the second preset weight, performing head distillation and intermediate layer distillation on the filtered filter;
[0098] Otherwise, the filtered filter is subjected to head distillation.
[0099] As mentioned above, different filters have different convolutional layer weight distributions. For scenarios such as ship detection, there are more small and medium-sized targets, which account for a larger proportion than general mainstream data sets. They are usually assigned larger weights to low-level convolutional layers (due to the increase in the receptive field during the convolution calculation process, the obvious features of small targets will gradually be lost in high layers);
[0100] Based on this, this embodiment also sets a second preset weight to determine the weight of the intermediate convolution layer, and compares the intermediate convolution layer weight of the filter with the second preset weight; if the intermediate convolution layer weight of the filter is less than the second preset weight, it means that the intermediate convolution layer weight of the filter is small. The weight is mainly in the low layer, and the combination of head distillation and intermediate layer distillation is used, which will not lose many important parameters and has a better compression effect;
[0101] If the weight of the intermediate convolution layer of the filter is greater than or equal to the second preset weight, it means that the weight of the intermediate convolution layer of the filter is large, and performing intermediate layer distillation on it will lose important parameters. Therefore, only the head distillation is performed on the filter.
[0102] The second preset weight mentioned above can be set according to the actual application scenario, and this embodiment does not make any specific limitation to this.
[0103] In implementation, the specific principle of knowledge distillation can refer to the way the student network approaches the teacher network. Figure 4 As shown. The cross entropy loss function is constructed through the output of the teacher network and the output of the student network to help train the student network. The teacher network is a high-performance network that has been trained in advance. The learning of the student network includes two targets in the loss function: one is the cross entropy between the output category probability of the student network and the true value of the data label, which is called the hard target. The second is the cross entropy between the category probability output by the teacher network and the category probability output by the student network, which is called the soft target. The total loss function is the weighted sum between the soft and hard targets. By setting the weighting coefficient, the importance of the two cross entropy losses can be balanced. The performance of the student network finally trained is very similar to that of the teacher network, but in terms of complexity and computational complexity, the student network is much smaller than the teacher network.
[0104] The above scheme selects different knowledge distillation methods for different filters. Combined with the scenario of ship detection, it proposes a combination of head distillation and middle layer distillation to balance the two aspects of parameter loss and compression effect.
[0105] In some optional implementations, after performing knowledge distillation on the filter, the parameters of the filter may also be pruned to remove some redundant parameters.
[0106] In some optional implementations, after the filtered filters that are not subjected to knowledge distillation are determined in the above manner, in order to improve the compression effect of these filters, these filters may be processed in the following manner:
[0107] Based on cross-layer learning, parameters of the filtered filter are compressed.
[0108] Since the above-mentioned knowledge distillation method has certain requirements for weight distribution, when the head weight is increased, knowledge distillation (head distillation, or head distillation + middle layer distillation) cannot be performed, so it is impossible to compress the homology (redundant) parameters of this part of the filter through knowledge distillation. Based on this, this embodiment uses cross-layer learning (not limited to same-layer learning) to call the shallow part features of the teacher model (high-performance model) to guide the deep part features of the student model (the above-mentioned filter). Through cross-layer learning, the above-mentioned filter can efficiently obtain effective information and realize parameter compression of the filter.
[0109] like Figure 5 As shown, an embodiment of the present application provides a ship detection device 500, which includes:
[0110] The image acquisition module 501 is used to acquire an image of a target area to obtain an acquired image corresponding to the target area;
[0111] A detection frame module 502 is used to perform rotation bounding frame detection on the object in the collected image to determine a rotation frame of the object in the collected image;
[0112] The recognition module 503 is used for inputting the collected image and the angle information of the object rotation frame into a target detection network for any object rotation frame, performing feature extraction on the object rotation frame through the target detection network, and determining whether the object rotation frame corresponds to a ship;
[0113] The lightweight module 504 is used to obtain the target detection network by filtering the pre-trained detection model and performing knowledge distillation on part or all of the filtered filters.
[0114] In some optional implementations, the detection frame module 502 is further configured to:
[0115] Determining the bow direction of the object rotation frame;
[0116] The identification module 503 is specifically used for:
[0117] The collected image, the angle information of the object rotation frame and the bow direction are input into a target detection network.
[0118] In some optional implementations, the detection frame module 502 is specifically configured to:
[0119] Performing grayscale processing on the object rotation frame to determine the grayscale value of each pixel in the object rotation frame;
[0120] The object rotation frame is segmented along the middle line of the long side to obtain two segmented rectangles, and the sum of the grayscale values of each segmented rectangle is determined respectively;
[0121] For any segmented rectangle, if the sum of the grayscale values of the segmented rectangle is greater than the sum of the grayscale values of another segmented rectangle, the opposite direction of the other segmented rectangle in the segmented rectangle is determined as the bow direction.
[0122] In some optional implementations, the lightweight module 504 is specifically used for:
[0123] For the pre-trained detection model, selecting a target filter from the pre-trained detection model, and absorbing knowledge of non-target filters through the target filter;
[0124] After pruning the non-target filter, the pre-trained detection model is fine-tuned to obtain a filtered filter.
[0125] In some optional implementations, the lightweight module 504 is further used to:
[0126] For any filtered filter, if the weight of the head convolution layer of the filtered filter is greater than the first preset weight, it is determined that the filtered filter does not perform knowledge distillation; otherwise, it is determined that the filtered filter performs knowledge distillation.
[0127] In some optional implementations, the lightweight module 504 is specifically used for:
[0128] For any filtered filter that is subjected to knowledge distillation, if the weight of the intermediate convolutional layer of the filtered filter is less than the second preset weight, performing head distillation and intermediate layer distillation on the filtered filter;
[0129] Otherwise, the filtered filter is subjected to head distillation.
[0130] In some optional implementations, the angle information of the object rotation frame includes coordinates of four corner points of the object rotation frame and the length and width of a minimum horizontal circumscribed rectangle of the object rotation frame.
[0131] Since the device is the device in the method in the embodiment of the present application, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0132] Based on the same technical concept, the embodiment of the present application also provides an electronic device 600, such as Figure 6 As shown, it includes at least one processor 601 and a memory 602 connected to the at least one processor. The specific connection medium between the processor 601 and the memory 602 is not limited in the embodiment of the present application. Figure 6For example, the processor 601 and the memory 602 are connected via a bus 603. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0133] Among them, the processor 601 is the control center of the electronic device, which can use various interfaces and lines to connect various parts of the electronic device, and realize data processing by running or executing instructions stored in the memory 602 and calling data stored in the memory 602. Optionally, the processor 601 may include one or more processing units, and the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes the issuance of instructions. It is understandable that the above-mentioned modem processor may not be integrated into the processor 601. In some embodiments, the processor 601 and the memory 602 may be implemented on the same chip, and in some embodiments, they may also be implemented separately on independent chips.
[0134] The processor 601 may be a general-purpose processor, such as a CPU, a digital signal processor, an application-specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component, and may implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment of the ship detection method may be directly embodied as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.
[0135] The memory 602 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 602 may include at least one type of storage medium, such as a flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (Random Access Memory, RAM), a static random access memory (Static Random Access Memory, SRAM), a programmable read-only memory (Programmable Read Only Memory, PROM), a read-only memory (Read Only Memory, ROM), an electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), a magnetic memory, a disk, an optical disk, etc. The memory 602 is any other medium that can be used to carry or store a desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 602 in the embodiment of the present application can also be a circuit or any other device that can realize a storage function, for storing program instructions and / or data.
[0136] In the embodiment of the present application, the memory 602 stores a computer program. When the program is executed by the processor 601, the processor 601 executes:
[0137] Capturing an image of a target area to obtain a captured image corresponding to the target area;
[0138] Performing rotation bounding box detection on the object in the acquired image to determine a rotation box of the object in the acquired image;
[0139] For any object rotation frame, the collected image and the angle information of the object rotation frame are input into a target detection network, and features of the object rotation frame are extracted through the target detection network to determine whether the object rotation frame corresponds to a ship;
[0140] The target detection network is obtained by the processor 601 by filtering the pre-trained detection model and performing knowledge distillation on part or all of the filtered filters.
[0141] In some optional implementations, before inputting the collected image and the angle information of the object rotation frame into the target detection network, the processor 601 further executes:
[0142] Determining the bow direction of the object rotation frame;
[0143] The processor 601 specifically executes:
[0144] The collected image, the angle information of the object rotation frame and the bow direction are input into a target detection network.
[0145] In some optional implementations, the processor 601 specifically performs:
[0146] Performing grayscale processing on the object rotation frame to determine the grayscale value of each pixel in the object rotation frame;
[0147] The object rotation frame is segmented along the middle line of the long side to obtain two segmented rectangles, and the sum of the grayscale values of each segmented rectangle is determined respectively;
[0148] For any segmented rectangle, if the sum of the grayscale values of the segmented rectangle is greater than the sum of the grayscale values of another segmented rectangle, the opposite direction of the other segmented rectangle in the segmented rectangle is determined as the bow direction.
[0149] In some optional implementations, the processor 601 specifically performs:
[0150] For the pre-trained detection model, selecting a target filter from the pre-trained detection model, and absorbing knowledge of non-target filters through the target filter;
[0151] After pruning the non-target filter, the pre-trained detection model is fine-tuned to obtain a filtered filter.
[0152] In some optional implementations, the processor 601 further executes:
[0153] For any filtered filter, if the weight of the head convolution layer of the filtered filter is greater than the first preset weight, it is determined that the filtered filter does not perform knowledge distillation; otherwise, it is determined that the filtered filter performs knowledge distillation.
[0154] In some optional implementations, the processor 601 specifically performs:
[0155] For any filtered filter that is subjected to knowledge distillation, if the weight of the intermediate convolutional layer of the filtered filter is less than the second preset weight, performing head distillation and intermediate layer distillation on the filtered filter;
[0156] Otherwise, the filtered filter is subjected to head distillation.
[0157] In some optional implementations, the angle information of the object rotation frame includes coordinates of four corner points of the object rotation frame and the length and width of a minimum horizontal circumscribed rectangle of the object rotation frame.
[0158] Since the electronic device is the electronic device in the method in the embodiment of the present application, and the principle of solving the problem by the electronic device is similar to that of the method, the implementation of the electronic device can refer to the implementation of the method, and the repeated parts will not be repeated.
[0159] Based on the same technical concept, an embodiment of the present application also provides a computer-readable storage medium, which stores a computer program executable by a processor. When the program runs on the processor, the processor executes the steps of the above-mentioned ship detection method.
[0160] In some optional embodiments, various aspects of the ship detection method provided by the present application may also be implemented in the form of a program product, which includes computer-executable instructions. When the program product is run on a computer device, the computer-executable instructions are used to enable the computer device to execute the steps of the ship detection method according to various exemplary embodiments of the present application described above in this specification.
[0161] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0162] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0163] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0164] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0165] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0166] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A ship detection method, applied to a satellite, characterized in that: The method includes: Capturing an image of a target area to obtain a captured image corresponding to the target area; Performing rotation bounding box detection on the object in the acquired image to determine a rotation box of the object in the acquired image; For any object rotation frame, the collected image and the angle information of the object rotation frame are input into a target detection network, and features of the object rotation frame are extracted through the target detection network to determine whether the object rotation frame corresponds to a ship; The target detection network is obtained by filtering the pre-trained detection model and performing knowledge distillation on some or all of the filtered filters.
2. The method according to claim 1, characterized in that Before inputting the collected image and the angle information of the object rotation frame into the target detection network, the method further includes: Determining the bow direction of the object rotation frame; Inputting the collected image and the angle information of the object rotation frame into the target detection network includes: The collected image, the angle information of the object rotation frame and the bow direction are input into a target detection network.
3. The method according to claim 2, characterized in that Determining the bow direction of the rotation frame of the object includes: Performing grayscale processing on the object rotation frame to determine the grayscale value of each pixel in the object rotation frame; The object rotation frame is segmented along the middle line of the long side to obtain two segmented rectangles, and the sum of the grayscale values of each segmented rectangle is determined respectively; For any segmented rectangle, if the sum of the grayscale values of the segmented rectangle is greater than the sum of the grayscale values of another segmented rectangle, the opposite direction of the other segmented rectangle in the segmented rectangle is determined as the bow direction.
4. The method according to claim 1, characterized in that Filter the pre-trained detection model, including: For the pre-trained detection model, selecting a target filter from the pre-trained detection model, and absorbing knowledge of non-target filters through the target filter; After pruning the non-target filter, the pre-trained detection model is fine-tuned to obtain a filtered filter.
5. The method according to claim 1, characterized in that Determine whether the filtered filter undergoes knowledge distillation by: For any filtered filter, if the weight of the head convolution layer of the filtered filter is greater than the first preset weight, it is determined that the filtered filter does not perform knowledge distillation; otherwise, it is determined that the filtered filter performs knowledge distillation.
6. The method according to claim 1, characterized in that Perform knowledge distillation on some or all of the filtered filters, including: For any filtered filter that is subjected to knowledge distillation, if the weight of the intermediate convolutional layer of the filtered filter is less than the second preset weight, performing head distillation and intermediate layer distillation on the filtered filter; Otherwise, the filtered filter is subjected to head distillation.
7. The method according to any one of claims 1 to 6, characterized in that: The angle information of the object rotation frame includes the coordinates of the four corner points of the object rotation frame and the length and width of the minimum horizontal circumscribed rectangle of the object rotation frame.
8. A ship detection device, characterized in that: The device includes: An image acquisition module is used to acquire an image of a target area to obtain an acquired image corresponding to the target area; A detection frame module, used to perform rotation boundary frame detection on the object in the collected image, and determine the rotation frame of the object in the collected image; a recognition module, for any object rotation frame, inputting the collected image and the angle information of the object rotation frame into a target detection network, performing feature extraction on the object rotation frame through the target detection network, and determining whether the object rotation frame corresponds to a ship; The target detection network is obtained by filtering the pre-trained detection model and performing knowledge distillation on some or all of the filtered filters.
9. An electronic device, characterized in that: The method comprises at least one processor and at least one memory, wherein the memory stores a computer program, and when the program is executed by the processor, the processor executes the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: It stores a computer program executable by a computer, and when the program is run on the computer, the computer is enabled to execute the method according to any one of claims 1 to 7.