Small target detection method, device and equipment in night environment and storage medium

Abstract

The invention discloses a small target detection method, apparatus and device in a night environment, and a storage medium. The small target detection method in the night environment comprises the steps of obtaining a to-be-detected image; inputting the to-be-detected image into a trained detection model, performing convolution processing of a plurality of convolution layers, integrating extracted features through a full connection layer, and outputting an image detection result; wherein when a first convolution layer of the detection model performs convolution calculation, each local pixel block, covered by a convolution kernel, in a pixel region block of the to-be-detected image is adjusted according to the influence of a central pixel block of the pixel region block on other local pixel blocks, and the first convolution layer of the detection model performs convolution calculation; multiplying the convolution kernel to obtain a convolution result; and the central pixel block is set to be a fixed value 0 after being adjusted. By using the small target detection method in the night environment disclosed by the invention, the detection accuracy of the small target under the low light condition can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image detection technology, and in particular to a small target detection method, device, equipment and storage medium in a nighttime environment. Background Art

[0002] In the logistics industry, camera monitoring is a key technical means to ensure the quality and safety of transportation. Enterprises can effectively monitor the entire process of cargo loading and unloading, transportation and delivery through video information collected by camera monitoring to ensure compliance with logistics processes and prevent potential risks. Traditional manual monitoring methods are time-consuming and inefficient, and it is difficult to meet the needs of massive data processing. With the advancement of big data analysis and artificial intelligence technology, automated video analysis systems have emerged. They can perform image detection based on videos to identify abnormal behaviors such as cargo damage, improper operation or route deviation, so as to take timely measures to maintain the smoothness and efficiency of the logistics chain. However, in the logistics industry, logistics vehicles often run at night and use off-peak hours for efficient transportation. However, visibility is limited in low-light environments at night, and the images to be detected are noisy, and the accuracy of image detection methods for small targets is low. Summary of the invention

[0003] The present invention provides a small target detection method, device, equipment and storage medium in a nighttime environment, which are used to solve the technical problem of low detection accuracy of small targets under dark light conditions in the prior art.

[0004] In order to solve the above technical problems, in a first aspect, the present invention provides a small target detection method in a night environment, the method comprising:

[0005] Acquire the image to be detected;

[0006] The image to be detected is input into a trained detection model, and after convolution processing of several convolution layers, the extracted features are integrated by a fully connected layer, and an image detection result is output; wherein, when the first convolution layer of the detection model performs convolution calculation, each local pixel block in the pixel area block of the image to be detected covered by the convolution kernel is adjusted according to the influence of the central pixel block of the pixel area block on other local pixel blocks, and then multiplied with the convolution kernel to obtain the convolution result; the central pixel block is set to a fixed value of 0 after adjustment.

[0007] Optionally, each local pixel block in the pixel area block of the image to be detected covered by the convolution kernel is adjusted according to the influence of the central pixel block of the pixel area block and then multiplied with the convolution kernel to obtain a convolution result, including:

[0008] After calculating the difference between each local pixel block and the central fitness factor determined according to the central pixel block, each local pixel block and the central influence coefficient of the central pixel block, the convolution result is obtained by multiplying the convolution kernel. The corresponding convolution formula includes:

[0009] y=∑W i ·(X i -δ)

[0010] Among them, y is the convolution result, W i is the i-th element of the convolution kernel matrix, X i is the i-th local pixel block of the pixel area block, and δ is the central fitness factor.

[0011] Optionally, the central fitness factor determined according to the central pixel block, each of the local pixel blocks and the central influence coefficient of the central pixel block includes:

[0012] The calculation formula of the central fitness factor includes:

[0013]

[0014] The pixel area block size is 3×3, i is the i-th local pixel block of the pixel area block, X5 is the central pixel block, δ is the central fitness factor, ε is the central influence coefficient, ε∈[0,1].

[0015] Optionally, several convolutional layers of the detection model are divided into multiple convolutional units, and the convolutional kernel size, number of channels and step size of the convolutional layers contained in each convolutional unit are the same.

[0016] Optionally, the convolution units each include a single convolution layer and a residual unit formed by residual connections of at least two convolution layers, and the residual units are cascaded.

[0017] Optionally, the convolution layers contained in the convolution units connected sequentially have convolution kernels with increasing numbers of channels.

[0018] Optionally, the detection model includes a first convolution unit, a second convolution unit, a third convolution unit and a fourth convolution unit consisting of 36 convolution layers with a convolution kernel size of 3×3, and the residual unit in each convolution unit includes 2 convolution layers, wherein,

[0019] The first convolution unit includes two residual units, and the number of channels of the convolution kernel of each convolution layer is 64;

[0020] The second convolution unit includes 4 residual units, and the number of channels of the convolution kernel of each convolution layer is 128;

[0021] The third convolution unit includes 8 residual units, and the number of channels of the convolution kernel of each convolution layer is 256;

[0022] The fourth convolution unit includes 2 residual units, and the number of channels of the convolution kernel of each convolution layer is 512.

[0023] In a second aspect, the present invention provides a small target detection device in a nighttime environment, comprising a data acquisition module and a detection module;

[0024] The data acquisition module is used to acquire the image to be detected;

[0025] The detection module is used to input the image to be detected into a trained detection model, and after convolution processing of several convolution layers, the extracted features are integrated by a fully connected layer to output an image detection result; wherein, when the first convolution layer of the detection model performs convolution calculation, each local pixel block in the pixel area block of the image to be detected covered by the convolution kernel is adjusted according to the influence of the central pixel block of the pixel area block on other local pixel blocks, and then multiplied with the convolution kernel to obtain a convolution result; the central pixel block is set to a fixed value of 0 after adjustment.

[0026] In a third aspect, the present invention provides a small target detection device in a night environment, including a memory and a processor, wherein:

[0027] The memory is used to store computer programs;

[0028] The processor is used to read the program in the memory and execute the steps of the small target detection method in a night environment provided by the first aspect above.

[0029] In a fourth aspect, the present invention provides a computer-readable storage medium having a readable computer program stored thereon, which, when executed by a processor, implements the steps of the small target detection method in a night environment provided in the first aspect above.

[0030] Compared with the prior art, the present invention provides a method, device, equipment and storage medium for detecting small targets in a night environment, which has the following beneficial effects:

[0031] Acquire an image to be detected; input the image to be detected into a trained detection model, and after convolution processing of several convolution layers, the extracted features are integrated by a fully connected layer, and the image detection result is output; wherein, when the first convolution layer of the detection model performs convolution calculation, each local pixel block in the pixel area block of the image to be detected covered by the convolution kernel is adjusted according to the influence of the central pixel block of the pixel area block on other local pixel blocks, and then multiplied with the convolution kernel to obtain the convolution result; the central pixel block is set to a fixed value of 0 after adjustment. This method can effectively suppress the noise in the image to be detected by adjusting the local pixel blocks in the pixel area block by utilizing the influence of the central pixel block on the local pixel blocks, and improve the detection model's image recognition ability for small targets in low-light scenes at night, thereby improving the accuracy of the detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only part of the embodiments of the present invention, rather than all of the embodiments. For ordinary technicians in this field, without paying creative work, other drawings obtained based on these drawings all fall within the scope of protection of this application.

[0033] Figure 1 It is a flowchart of a small target detection method in a night environment provided by an embodiment of the present invention;

[0034] Figure 2 is a schematic diagram of a model structure of a detection model provided by an embodiment of the present invention;

[0035] Figure 3 It is a schematic diagram of the first convolution layer performing convolution calculation provided by an embodiment of the present invention;

[0036] Figure 4 is a structural schematic diagram of a small target detection device in a night environment provided by an embodiment of the present invention;

[0037] Figure 5 This is a schematic diagram of the structure of a small target detection device in a night environment provided by an embodiment of the present invention;

[0038] Figure 6 It is a structural schematic diagram of a computer-readable storage medium provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0040] In order to make the description of the disclosed content more detailed and complete, the following is an illustrative description of the implementation mode and specific examples of the present invention; however, this is not the only form of implementing or applying the specific embodiments of the present invention. The implementation mode covers the features of multiple specific embodiments and the method steps and their sequence for constructing and operating these specific embodiments. However, other specific embodiments can also be used to achieve the same or equal functions and step sequences. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0041] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.

[0042] In the description of the embodiments of the present application, "multiple" refers to two or more than two, and other quantifiers are similar and should be understood. The preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention. In addition, the embodiments of the present application and the features therein may be combined with each other if there is no conflict.

[0043] Example 1

[0044] like Figure 1 As shown, it is a flow chart of a small target detection method in a night environment provided by an embodiment of the present invention. The small target detection method in a night environment includes the following steps.

[0045] Step S101, obtaining an image to be detected;

[0046] In a specific embodiment, in low-light scenes at night, the onboard camera of a moving transport vehicle takes pictures, or the surveillance camera of a logistics site takes pictures of the site to obtain images to be detected. In order to adapt to the diversity of images to be detected, during the training of the detection model, the scene during the vehicle transportation process can be photographed from different angles, different times, different light intensities, etc., to obtain different types of image samples to be detected.

[0047] Step S102, input the image to be detected into the trained detection model, and after convolution processing of several convolution layers, the extracted features are integrated by the fully connected layer, and the image detection result is output; wherein, when the first convolution layer of the detection model performs convolution calculation, each local pixel block in the pixel area block of the image to be detected covered by the convolution kernel is adjusted according to the influence of the central pixel block of the pixel area block on other local pixel blocks, and then multiplied with the convolution kernel to obtain the convolution result; the central pixel block is set to a fixed value of 0 after adjustment.

[0048] In some embodiments, after the image to be detected is input into the detection model, the image to be detected can be represented by a three-dimensional matrix of n×n×C, where n represents the length and width of the image to be detected, and C represents the number of color channels of the image to be detected. The detection model includes several convolution layers. In the process of the convolution layer performing convolution processing on the n×n×C image to be detected, when the first convolution layer performs convolution calculation, each local pixel block in the pixel area block of the image to be detected covered by the convolution kernel is adjusted according to the influence of the central pixel block of the pixel area block on other local pixel blocks, and then multiplied with the convolution kernel and summed to obtain the convolution result; when other convolution layers perform convolution calculation, each element of the convolution kernel is directly multiplied with each local pixel block in the pixel area block of the image to be detected covered by it, and then summed to obtain the convolution result; the central pixel block is set to a fixed value of 0 after adjustment.

[0049] During the convolution process of each convolution layer, the convolution kernel is slid on the image to be detected according to the preset step size to perform dot product calculation until the entire image to be detected is traversed and calculated to obtain the features after convolution processing; the fully connected layer integrates the features extracted by each convolution layer to form a global feature representation, which helps the model learn the global association of the image to be detected, which is crucial for classification tasks or other tasks that require understanding of the global context. An output layer is connected after the fully connected layer, which is responsible for converting the internal representation of the detection model into a prediction of the recognition category. The loss function used in the model training process can be the softmax function, and the learning rate of the model starts from 0.1 and decreases with the number of calculation iterations.

[0050] In some embodiments, a bias term may be added to the dot product result in the convolution calculation, and the convolution result after adding the bias term may be nonlinearly transformed through an activation function such as ReLU to introduce nonlinear characteristics and obtain the convolution result output by the local pixel block. This enables the feature to perform nonlinear transformation and helps the detection model fit more complex functions, thereby improving the expressiveness and generalization capabilities of the model.

[0051] As an optional implementation, the several convolutional layers of the detection model are divided into multiple convolutional units, and the convolutional kernel size, number of channels and step size of the convolutional layers contained in each convolutional unit are the same.

[0052] In some embodiments, the constructed detection model includes several convolution layers divided into multiple convolution units, and the convolution kernel size, number of channels and step size of the convolution layers contained in each convolution unit are the same. The type of convolution kernel can be determined according to different detection tasks. For example, edge detection can use Sobel, Prewitt or Laplacian filters to highlight the boundaries or edges in the image; blur smoothing can use mean or Gaussian filters to make the image smoother. For the image to be detected represented by the three-dimensional matrix n×n×C, each convolution layer requires C convolution kernels, and each convolution kernel corresponds to a color channel of the input image; the step size of the convolution kernel refers to the number of steps that the convolution kernel moves on the input feature map, which is used to control the distance that the convolution kernel slides when performing convolution calculations on the input image. Choosing a suitable step size is very important for controlling the size of the output feature map; the size of the convolution kernel is mainly based on the size of the receptive field, computational complexity, feature extraction capability and other factors. The size of the convolution kernel determines the size of the pixel area block that it can capture. A larger convolution kernel can capture more context information, while a smaller convolution kernel is sensitive to local features. In the embodiment of the present invention, the size of the convolution kernel of the convolution layer is set to 3×3, and the step size is 2.

[0053] As an optional implementation, the convolution units each include a single convolution layer and a residual unit formed by residual connections of at least two convolution layers, and the residual units are cascaded.

[0054] In some embodiments, the first convolution layer of each convolution unit is a separate convolution layer, and the remaining convolution layers constitute a plurality of cascade-connected residual units, and the cascaded residual units are connected after the separate convolution layer.

[0055] In the embodiment of the present invention, the residual unit can solve the gradient vanishing problem and the degradation problem that occurs during the training of the deep neural network, thereby improving the generalization ability of the detection model and avoiding overfitting of the model.

[0056] As an optional implementation, the number of channels of the convolution kernels of the convolution layers contained in the convolution units connected in sequence increases in sequence.

[0057] Increasing the number of channels means that each layer can learn more types of features. In some embodiments, the number of channels of the convolution kernels of the convolution layers contained in the sequentially connected convolution units is increased sequentially, so that each convolution layer can focus on learning abstract features at different levels to enrich the feature space.

[0058] As an optional implementation, the detection model includes a first convolution unit, a second convolution unit, a third convolution unit, and a fourth convolution unit consisting of 36 convolution layers with a convolution kernel size of 3×3, and the residual unit in each convolution unit includes 2 convolution layers, wherein,

[0059] The first convolution unit includes two residual units, and the number of channels of the convolution kernel of each convolution layer is 64;

[0060] The second convolution unit includes 4 residual units, and the number of channels of the convolution kernel of each convolution layer is 128;

[0061] The third convolution unit includes 8 residual units, and the number of channels of the convolution kernel of each convolution layer is 256;

[0062] The fourth convolution unit includes 2 residual units, and the number of channels of the convolution kernel of each convolution layer is 512.

[0063] In model design, more layers mean that the model has a stronger ability to learn complex feature mappings. However, increased depth also brings about gradient vanishing / explosion problems and the risk of overfitting. After testing and comparison, the embodiment of the present invention designs the detection model with 36 layers, which has a better balance.

[0064] In some embodiments, a schematic diagram of the model structure of the detection module is as follows: Figure 2 As shown, Figure 2 In , Conv1X, Conv2X, Conv3X, and Conv4X represent the first convolution unit, the second convolution unit, the third convolution unit, and the fourth convolution unit, respectively. Each convolution unit uses a convolution kernel of size 3×3, and S (Stride) is 2. The first convolution unit includes the convolution layers of the 1st to 5th layers, and the number of channels of the convolution kernel of each convolution layer is 64. The first layer is a separate convolution layer, expressed as [3×3,64]×1, and the 2nd to 5th layers are two cascaded residual units, expressed as The second convolution unit includes the 6th to 14th layers, and the number of channels of the convolution kernel of each convolution layer is 128. The 6th layer is a separate convolution layer, expressed as [3×3,128]×1, and the 7th to 14th layers are 4 cascaded residual units, expressed as The third convolution unit includes the 15th to 31st layers, and the number of channels of the convolution kernels of each convolution layer is 256. The 15th layer is a separate convolution layer, expressed as [3×3,256]×1, and the 16th to 31st layers are 8 cascaded residual units, expressed as The fourth convolution unit includes the 32nd to 36th convolution layers, and the number of channels of the convolution kernels of each convolution layer is 512. The 32nd layer is a separate convolution layer, expressed as [3×3,512]×1, and the 33rd to 36th layers are two cascaded residual units, expressed as

[0065] In an embodiment of the present invention, a new detection model structure is designed for small target detection in a nighttime environment. The model framework integrates a 36-layer deep learning network and is customized considering noisy and low-light scenes, so that the model can remain robust to various types of noise without generating additional noise images for training. The model has stronger generalization capabilities, does not need to rely on specific noise samples for training, can autonomously be compatible with object detection in different scenarios, enhances light source adaptability, greatly reduces the dependence of detection on light in nighttime environments, improves the ability to detect small targets, and improves the accuracy and timeliness of detection under low-light conditions.

[0066] After the constructed detection model is trained, the image to be detected is input and processed by several convolution layers. The extracted features are integrated by the fully connected layer and the image detection result is output. When the first convolution layer of the detection model performs convolution calculation, each local pixel block in the pixel area block of the image to be detected covered by the convolution kernel is adjusted according to the influence of the central pixel block of the pixel area block on other local pixel blocks, and then multiplied with the convolution kernel to obtain the convolution result. The central pixel block is set to a fixed value of 0 after adjustment.

[0067] According to experimental results, if each convolutional layer of the detection model uses the above-mentioned improved convolution processing, the network will gradually lose detail information. However, if only the first convolutional layer is improved, the needs of small targets in low light and the preservation of detail information can be well taken into account.

[0068] As an optional implementation, each local pixel block in the pixel area block of the image to be detected covered by the convolution kernel is adjusted according to the influence of the central pixel block of the pixel area block and then multiplied with the convolution kernel to obtain a convolution result, including:

[0069] After calculating the difference between each local pixel block and the central fitness factor determined according to the central pixel block, each local pixel block and the central influence coefficient of the central pixel block, the convolution result is obtained by multiplying the convolution kernel. The corresponding convolution formula includes:

[0070] y=∑W i ·(X i -δ)

[0071] Among them, y is the convolution result, W i is the i-th element of the convolution kernel matrix, X iis the ith local pixel block of the pixel area block, and δ is the central fitness factor. It should be noted that the pixel value of the central pixel block after adjustment is a fixed value of 0, and the result after convolution is 0.

[0072] In some embodiments, a schematic diagram of the first convolutional layer performing convolution calculation is shown as follows: Figure 3 As shown, the pixel area block size is 3×3, X1-X4 and X6-X9 are local pixel blocks, X5 is the central pixel block, y is the convolution result, the pixel value of the central pixel area is set to 0, and the central fitness factor is determined according to the central pixel block, each local pixel block and the central influence coefficient of the central pixel block. After the difference between each local pixel block and the central fitness factor is calculated, it is multiplied with the element corresponding to the convolution kernel to obtain the convolution result of the pixel area block.

[0073] In an embodiment of the present invention, since the noise of the pixel area block may confuse the subsequent feature representation process and cause the detection of false features that have no physical association, when processing the feature map, the pixel difference between the local pixel block and the central fitness factor is used to replace the original pixel value at that position for convolution calculation, which can effectively suppress the noise in the image, enhance the recognition ability of the detection model, and improve the detection model's recognition ability for low-light scenes.

[0074] As an optional implementation manner, the central fitness factor determined according to the central pixel block, each of the local pixel blocks and the central influence coefficient of the central pixel block includes:

[0075] The calculation formula of the central fitness factor includes:

[0076]

[0077] The pixel area block size is 3×3, i is the i-th local pixel block of the pixel area block, x5 is the central pixel block, δ is the central fitness factor, ε is the central influence coefficient, ε∈[0,1].

[0078] In some embodiments, the center influence coefficient ε is a constant, ε=0 represents the minimum corresponding center influence, and ε=1 represents the maximum corresponding center influence. Appropriate center influence parameters can filter low light noise and improve detection accuracy; on the contrary, improper center influence parameter selection will reduce model accuracy. According to the results of multiple tests and evaluations of the detection model, ε in the embodiment of the present invention is 0.75, which has a better effect.

[0079] In the embodiment of the present invention, the pixel difference between the local pixel block and the central fitness factor is used to replace the original pixel value at the position for convolution calculation, which can effectively suppress the noise in the image.

[0080] The embodiment of the present invention adjusts the local pixel blocks in the pixel area block by utilizing the influence of the central pixel block on the local pixel blocks, which can effectively suppress the noise in the image to be detected, improve the image recognition ability of the detection model for small targets in low-light scenes at night, and thus improve the accuracy of the detection model.

[0081] To verify the accuracy and effectiveness of the above detection model, a comparative experiment can be conducted, using the same 2000 test data sets to train the model combined with DenseNet and the above detection model. In low-light and dark light scenes, the accuracy of the detection model proposed in this application and the model combined with DenseNet are 80.5% and 75.4% respectively. It can be seen that the detection model proposed in this application can significantly improve the accuracy of small target detection under low-light and dark light conditions.

[0082] Example 2

[0083] Based on the above-mentioned small target detection method in night environment, the embodiment of the present invention further provides a small target detection device in night environment, and its structural schematic diagram is as follows: Figure 4 As shown, the small target detection device 40 in the night environment includes a data acquisition module 41 and a detection module 42;

[0084] The data acquisition module 41 is used to acquire the image to be detected;

[0085] The detection module 42 is used to input the image to be detected into the trained detection model, and after convolution processing of several convolution layers, the extracted features are integrated by the fully connected layer to output the image detection result; wherein, when the first convolution layer of the detection model performs convolution calculation, each local pixel block in the pixel area block of the image to be detected covered by the convolution kernel is adjusted according to the influence of the central pixel block of the pixel area block on other local pixel blocks, and then multiplied with the convolution kernel to obtain the convolution result; the central pixel block is set to a fixed value of 0 after adjustment.

[0086] For other details about the implementation of the above technical solution by each module in the above-mentioned small target detection device 40 in the night environment, reference may be made to the description of the small target detection method in the night environment provided in the above-mentioned invention embodiment, which will not be repeated here.

[0087] Example 3

[0088] Based on the above-mentioned small target detection method in night environment, the embodiment of the present invention further provides a small target detection device in night environment, and its structural schematic diagram is as follows: Figure 5 As shown, the small target detection device 50 in the night environment includes a processor 51 and a memory 52 coupled to the processor 51. The memory 52 stores a computer program, and when the computer program is executed by the processor 51, the processor 51 executes the steps of the small target detection method in the night environment in the above embodiment.

[0089] For other details about the processor 51 in the small target detection device 50 in the night environment implementing the above technical solution, please refer to the description of the small target detection method in the night environment provided in the above invention embodiment, which will not be repeated here.

[0090] Among them, the processor 51 can also be called a CPU (Central Processing Unit), and the processor 51 may be an integrated circuit chip with signal processing capabilities; the processor 51 can also be a general-purpose processor, DSP (Digital Signal Process), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, among which the general-purpose processor can be a microprocessor or the processor 51 can also be any conventional processor, etc.

[0091] Example 4

[0092] The embodiment of the present invention further provides a computer-readable storage medium, a schematic diagram of which is shown in FIG. Figure 6 As shown, a readable computer program 61 is stored on the storage medium 60; wherein, the computer program 61 can be stored in the above storage medium 60 in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium 60 includes: U disk, mobile hard disk, magnetic disk or optical disk, ROM (Read-Only Memory), RAM (Random Access Memory) and other media that can store program codes, or terminal devices such as computers, servers, mobile phones, and tablets.

[0093] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0094] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0095] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium.

[0096] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0097] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or a data center that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0098] The technical solution provided by the present application is introduced in detail above. The principles and implementation methods of the present application are explained by using specific examples in the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0099] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0100] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0101] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0103] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A small target detection method in a night environment, characterized in that: include: Acquire the image to be detected; The image to be detected is input into a trained detection model, and after convolution processing of several convolution layers, the extracted features are integrated by a fully connected layer, and the image detection result is output; wherein, when the first convolution layer of the detection model performs convolution calculation, each local pixel block in the pixel area block of the image to be detected covered by the convolution kernel is adjusted according to the influence of the central pixel block of the pixel area block on other local pixel blocks, and then multiplied with the convolution kernel to obtain the convolution result; the central pixel block is set to a fixed value of 0 after adjustment.

2. The small target detection method in nighttime environment according to claim 1, characterized in that: Each local pixel block in the pixel area block of the image to be detected covered by the convolution kernel is adjusted according to the influence of the central pixel block of the pixel area block and then multiplied with the convolution kernel to obtain a convolution result, including: After calculating the difference between each local pixel block and the central fitness factor determined according to the central pixel block, each local pixel block and the central influence coefficient of the central pixel block, a convolution result is obtained by multiplying the convolution kernel. The corresponding convolution formula includes: y=∑W i ·(X i -δ) Among them, y is the convolution result, W i is the i-th element of the convolution kernel matrix, X i is the i-th local pixel block of the pixel area block, and δ is the central fitness factor.

3. The small target detection method in nighttime environment according to claim 2, characterized in that: The central fitness factor determined according to the central pixel block, each of the local pixel blocks and the central influence coefficient of the central pixel block includes: The calculation formula of the central fitness factor includes: The pixel area block size is 3×3, i is the i-th local pixel block of the pixel area block, x5 is the central pixel block, δ is the central fitness factor, ε is the central influence coefficient, ε∈[0,1].

4. The small target detection method in nighttime environment according to claim 1, characterized in that: The several convolutional layers of the detection model are divided into a plurality of convolutional units, and the convolutional kernel size, number of channels and step size of the convolutional layers contained in each of the convolutional units are the same.

5. The small target detection method in nighttime environment according to claim 4, characterized in that: The convolution units each include a single convolution layer and a residual unit formed by residual connections of at least two convolution layers, and the residual units are cascaded.

6. The method for detecting small targets in nighttime environment according to claim 5, characterized in that: The convolution layers contained in the convolution units connected sequentially have the convolution kernels with increasing numbers of channels.

7. The small target detection method in nighttime environment according to claim 6, characterized in that: The detection model includes a first convolution unit, a second convolution unit, a third convolution unit and a fourth convolution unit consisting of 36 convolution layers with a convolution kernel size of 3×3, and the residual unit in each convolution unit includes 2 convolution layers, wherein, The first convolution unit includes two residual units, and the number of channels of the convolution kernel of each convolution layer is 64; The second convolution unit includes 4 residual units, and the number of channels of the convolution kernel of each convolution layer is 128; The third convolution unit includes 8 residual units, and the number of channels of the convolution kernel of each convolution layer is 256; The fourth convolution unit includes 2 residual units, and the number of channels of the convolution kernel of each convolution layer is 512.

8. A small target detection device in a night environment, characterized in that: It includes a data acquisition module and a detection module; The data acquisition module is used to acquire the image to be detected; The detection module is used to input the image to be detected into a trained detection model, and after convolution processing of several convolution layers, the extracted features are integrated by a fully connected layer to output an image detection result; wherein, when the first convolution layer of the detection model performs convolution calculation, each local pixel block in the pixel area block of the image to be detected covered by the convolution kernel is adjusted according to the influence of the central pixel block of the pixel area block on other local pixel blocks, and then multiplied with the convolution kernel to obtain a convolution result; the central pixel block is set to a fixed value of 0 after adjustment.

9. A small target detection device in a night environment, characterized in that: comprising a memory and a processor, wherein: The memory is used to store computer programs; The processor is used to read the computer program in the memory and execute the steps of the small target detection method in a night environment according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A readable computer program is stored thereon, and when the program is executed by a processor, the steps of the small target detection method in a night environment as described in any one of claims 1 to 7 are implemented.

Images