Object detection method, device, equipment and storage medium

Abstract

The present application provides an object detection method, device, equipment and storage medium, the method comprising: generating a spectral image according to the terahertz beam and spectral information of an object to be detected; marking edge pixel points in the spectral image according to gradient eigenvalues ​​to obtain an edge-marked image; obtaining a structural similarity index, an image quality index and a feature similarity index between pixel blocks of edge pixel points, and obtaining a similarity value between each pixel block and other pixel blocks in combination with a preset weight coefficient; dividing the pixel blocks into a plurality of block sets according to the similarity values, calculating a target pixel value according to the pixel blocks in each block set, correcting the pixel blocks in the block set according to the target pixel value, and generating an edge-corrected image; inputting the edge-corrected image into a pre-trained Mask R-CNN model, and outputting type information and location information of suspected dangerous goods on the object to be detected.

Description

Technical Field

[0001] The present application relates to the technical field of object detection, and in particular to an object detection method, device, equipment and storage medium. Background Art

[0002] Terahertz waves refer to waves with a frequency between 0.1 and 10 THz (1 THz = Hz), or electromagnetic waves with a wavelength within 3mm - 30um, which have many advantages: such as low photon energy (electron volts), which can prevent biological tissue from being damaged by radiation ionization; it has strong penetration ability for many non-metallic and non-polar materials that are opaque in the visible and infrared bands, but is very sensitive to moisture absorption. Therefore, terahertz waves are gradually being used to identify hidden objects, such as security gates and security inspection machines. Affected by the accuracy of hardware such as femtosecond lasers, choppers, and phase locks, traditional terahertz wave imaging generally has problems of low energy and high noise, which directly leads to the blurry image of terahertz spectrum imaging, large noise interference, and affects the resolution and contrast of the image. Reducing the noise problem in terahertz imaging by optimizing system hardware is costly and the effect is also limited. Summary of the invention

[0003] The present application provides an object detection method, device, equipment and storage medium for optimizing a scanning image of a terahertz wave and reducing the cost of removing noise from the terahertz image.

[0004] In a first aspect, an embodiment of the present application provides an object detection method, the method comprising:

[0005] Receiving a terahertz beam of an object to be detected, and generating a spectral image of the object to be detected according to the terahertz beam and preset spectral information;

[0006] Calculating the gradient eigenvalue of each pixel in the spectral image, and marking the edge pixel points in the spectral image according to the gradient eigenvalue to obtain an edge marked image;

[0007] Extracting pixel blocks based on the edge pixel points, calculating the structural similarity index, image quality index and feature similarity index of each pixel block with other pixel blocks, and obtaining similarity values ​​between each pixel block and other pixel blocks according to a preset weight coefficient, the structural similarity index, the image quality index and the feature similarity index;

[0008] Dividing the pixel blocks into a plurality of block sets according to the similarity values, calculating target pixel values ​​according to the pixel blocks in each of the block sets, and correcting the pixel blocks in the block sets according to the target pixel values ​​to generate an edge-corrected image;

[0009] The edge-corrected image is input into a pre-trained Mask R-CNN model, and the type information and location information of the suspected dangerous goods present on the object to be detected are output.

[0010] In a second aspect, an embodiment of the present application provides an object detection device, the device comprising:

[0011] A beam processing module, used for receiving a terahertz beam of an object to be detected, and generating a spectral image of the object to be detected according to the terahertz beam and preset spectral information;

[0012] An edge annotation module is used to calculate the gradient eigenvalue of each pixel in the spectral image, and annotate the edge pixels in the spectral image according to the gradient eigenvalue to obtain an edge annotation image;

[0013] a parameter operation module, configured to extract pixel blocks based on the edge pixel points, calculate a structural similarity index, an image quality index and a feature similarity index between each pixel block and other pixel blocks, and obtain a similarity value between each pixel block and other pixel blocks according to a preset weight coefficient, the structural similarity index, the image quality index and the feature similarity index;

[0014] An edge correction module, used for dividing the pixel blocks into a plurality of block sets according to the similarity values, calculating a target pixel value according to the pixel blocks in each of the block sets, and correcting the pixel blocks in the block sets according to the target pixel values ​​to generate an edge-corrected image;

[0015] The image processing module is used to input the edge-corrected image into a pre-trained Mask R-CNN model and output type information and location information of the suspected dangerous goods on the object to be detected.

[0016] In a third aspect, an embodiment of the present application provides a terminal device, wherein the terminal device includes a memory and a processor;

[0017] The memory is used to store computer programs;

[0018] The processor is used to execute the computer program and implement the object detection method as described in any one of the embodiments of the present application when executing the computer program.

[0019] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor enables the processor to implement an object detection method as described in any one of the embodiments of the present application.

[0020] The embodiment of the present application provides an object detection method, which includes: receiving a terahertz beam of an object to be detected, generating a spectral image of the object to be detected according to the terahertz beam and preset spectral information; calculating the gradient eigenvalue of each pixel point in the spectral image, marking the edge pixel points in the spectral image according to the gradient eigenvalue, and obtaining an edge-marked image; extracting pixel blocks based on the edge pixel points, calculating the structural similarity index, image quality index and feature similarity index of each pixel block with other pixel blocks, and obtaining the similarity value of each pixel block with other pixel blocks according to preset weight coefficients, structural similarity indexes, image quality indexes and feature similarity indexes; dividing the pixel blocks into multiple block sets according to the similarity values, calculating the target pixel values ​​according to the pixel blocks in each block set, correcting the pixel blocks in the block set according to the target pixel values, and generating an edge-corrected image; inputting the edge-corrected image into a pre-trained Mask R-CNN model, and outputting the type information and location information of the suspected dangerous goods on the object to be detected. Through the above method, the gradient eigenvalue of the terahertz spectrum image is calculated, the edge pixel points of the object in the terahertz spectrum image are found according to the gradient eigenvalue, and the edge pixel points are marked, pixel blocks are extracted along the edge pixel points, the similarity values ​​between the pixel blocks are calculated, and the pixel blocks with the same characteristics are divided into multiple block sets according to the similarity values, and the target pixel value of the block set is obtained according to the pixel blocks in the same block set to correct the pixel blocks in the block set to obtain an edge-corrected image with obvious edges, thereby improving the resolution and contrast of the terahertz image, so as to improve the recognition success rate of the object in the object recognition model and reduce the cost of removing noise from the terahertz image. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0022] Figure 1 A schematic flow chart of an object detection method provided in an embodiment of the present application;

[0023] Figure 2 A schematic block diagram of an object detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0026] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0027] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0028] In the embodiment of the present application, the object detection method in the following embodiment is implemented by a set of object detection system. The object detection system includes an optical unit, a collection unit, and a computing unit.

[0029] See also Figure 1 , Figure 1 The embodiment of the present application also provides a schematic flow chart of an object detection method. Figure 1 As shown, the specific steps of the object detection method include: S100-S500.

[0030] S100 , receiving a terahertz beam of an object to be detected, and generating a spectral image of the object to be detected according to the terahertz beam and preset spectral information.

[0031] Exemplarily, the objects to be detected include people, boxes and bags. Therefore, the beam to be detected needs a certain penetration ability, and it is more necessary to ensure that the beam used for detection is harmless to the human body. In an embodiment of the present application, the characteristic of the terahertz beam being harmless to the human body is utilized, and suspicious items are scanned by active terahertz detection, that is, by designing a set of optical units to guide the terahertz beam to irradiate the object to be detected, and collect the reflected terahertz beam. The optical unit includes elements such as lenses, reflectors, and apertures for focusing and controlling the beam path. After receiving the terahertz beam reflected by the object to be detected by the collection unit, the reflected terahertz beam is processed using signal processing techniques such as fast Fourier transform (FFT), and frequency domain information is extracted from the time domain signal of the terahertz beam to obtain a terahertz spectrum. According to the terahertz spectrum, a spectral image is synthesized by comparing it with a preset spectral information library, such as a pseudo-color coded spectral image, in which different colors represent different material components or characteristics, thereby intuitively displaying the obscured items on the object to be detected.

[0032] Spectral recognition technology can identify the characteristics of suspicious items, such as flammables and explosives. These dangerous items do not need to be judged by their shape, and warnings are issued when they are found. However, there are still some dangerous items made of common materials, such as sharp plastic knives, which need to be identified by shape.

[0033] S200 , calculating the gradient eigenvalue of each pixel in the spectral image, and marking the edge pixels in the spectral image according to the gradient eigenvalues ​​to obtain an edge marked image.

[0034] Exemplarily, the first-order derivative of each pixel on the spectral image is calculated in the horizontal and vertical directions to obtain the horizontal gradient value and vertical gradient value of each pixel. The gradient characteristic value of each pixel is obtained according to the square sum of the horizontal gradient value and the vertical gradient value. If the gradient characteristic value is greater than a preset threshold, for example, the preset threshold is 70, the pixel is determined to be an edge pixel. After identifying the edge pixel, the grayscale value of the non-edge pixel is set to 255 (pure white), so that the effect of highlighting the edge pixel can be achieved.

[0035] S300, extracting pixel blocks based on edge pixel points, calculating the structural similarity index, image quality index and feature similarity index between each pixel block and other pixel blocks, and obtaining the similarity value between each pixel block and other pixel blocks according to the preset weight coefficient, structural similarity index, image quality index and feature similarity index.

[0036] Exemplarily, when evaluating the similarity between pixel blocks, the structural similarity index, image quality index and feature similarity index are integrated to perform similarity evaluation. The structural similarity index (SSIM) and image quality index (UQI) are calculated by brightness contrast function, gray value contrast function and structure value contrast function, and the feature similarity index (FSIM) is calculated by phase value contrast function and gradient amplitude contrast function.

[0037] After determining the structural similarity index, image quality index and feature similarity index, the similarity value between each pixel block and other pixel blocks is obtained by combining the preset weight coefficients a, b and c. The specific calculation formula is:

[0038] ;

[0039] Among them, SQF is the similarity value.

[0040] In conventional methods, pixel blocks are weighted by Euclidean distance to determine the correlation between different pixel blocks to obtain the target pixel value. However, this method is too simple. If an object is blocked by another object, it is easy to introduce interference factors. For example, the weighting coefficient of the pixel block at the edge of the blocked object is too high, or the weighting coefficient corresponding to the pixel block at the discontinuous edge is too low, which will eventually lead to a decrease in the accuracy of the evaluation results. In this application, the structural similarity index (SSIM), image quality index (UQI) and feature similarity index (FSIM) are comprehensively considered, which can more accurately discover similar features between different pixel blocks to improve the accuracy of the evaluation results.

[0041] S400, dividing the pixel blocks into a plurality of block sets according to the similarity values, calculating the target pixel values ​​according to the pixel blocks in each block set, correcting the pixel blocks in the block set according to the target pixel values, and generating an edge-corrected image.

[0042] Exemplarily, the similarity values ​​of pixel blocks in the same block set cannot be less than a preset similarity threshold, for example, the preset similarity threshold is 0.85. After determining the target pixel value of each block set, each pixel block in the set can be corrected to make it closer to the target value.

[0043] For each set of blocks, different methods can be used to calculate a representative "target pixel value", including:

[0044] Average value method: Take the average color value of all pixels in the block set as the target pixel value.

[0045] Weighted average method: Calculate the average value by giving different weights based on the area, center location or other factors of the block set.

[0046] Median method: Take the median of each channel color value as the target pixel value to reduce the impact of outliers.

[0047] Pattern recognition: Find the most frequently occurring pixel value in a set of blocks as the target pixel value.

[0048] After determining the target pixel value of each block set, each pixel block in the block set can be corrected to make it closer to the target pixel value. The correction method includes direct replacement and gradual smooth transition. The edge-corrected image obtained through the above process highlights the edge of the object more clearly, making suspicious objects easier to identify.

[0049] S500: Input the edge-corrected image into a pre-trained Mask R-CNN model, and output type information and location information of the suspected dangerous goods on the object to be detected.

[0050] For example, Mask R-CNN is an advanced instance segmentation network that can simultaneously perform object classification, bounding box location, and accurate pixel-level segmentation. Mask R-CNN first generates candidate regions (RoIs) of potential objects, that is, areas that may contain objects of interest. For each RoI, the model extracts deep features and classifies it to determine whether it contains dangerous goods and which category it belongs to (such as explosives, flammables, etc.). For RoIs classified as dangerous goods, the position of their bounding boxes is further refined, and an accurate pixel-level mask is generated to mark the exact outline of the dangerous goods in the image. Ultimately, the model output contains the type information, bounding box location, and refined segmentation mask of each detected dangerous good, providing a detailed visual guide for security inspections or subsequent actions.

[0051] The embodiment of the present application provides an object detection method, which includes: receiving a terahertz beam of an object to be detected, generating a spectral image of the object to be detected according to the terahertz beam and preset spectral information; calculating the gradient eigenvalue of each pixel point in the spectral image, marking the edge pixel points in the spectral image according to the gradient eigenvalue, and obtaining an edge-marked image; extracting pixel blocks based on the edge pixel points, calculating the structural similarity index, image quality index and feature similarity index of each pixel block with other pixel blocks, and obtaining the similarity value of each pixel block with other pixel blocks according to preset weight coefficients, structural similarity indexes, image quality indexes and feature similarity indexes; dividing the pixel blocks into multiple block sets according to the similarity values, calculating the target pixel values ​​according to the pixel blocks in each block set, correcting the pixel blocks in the block set according to the target pixel values, and generating an edge-corrected image; inputting the edge-corrected image into a pre-trained Mask R-CNN model, and outputting the type information and location information of the suspected dangerous goods on the object to be detected. Through the above method, the gradient eigenvalue of the terahertz spectrum image is calculated, the edge pixel points of the object in the terahertz spectrum image are found according to the gradient eigenvalue, and the edge pixel points are marked, pixel blocks are extracted along the edge pixel points, the similarity values ​​between the pixel blocks are calculated, and the pixel blocks with the same characteristics are divided into multiple block sets according to the similarity values, and the target pixel value of the block set is obtained according to the pixel blocks in the same block set to correct the pixel blocks in the block set to obtain an edge-corrected image with obvious edges, thereby improving the resolution and contrast of the terahertz image, so as to improve the recognition success rate of the object in the object recognition model and reduce the cost of removing noise from the terahertz image.

[0052] In order to more clearly introduce the technical solution of the present application, the technical solution of the present application will be introduced through specific embodiments below. It should be noted that the specific embodiments are used to expand the technical solution of the present application, but are not intended to limit the present application.

[0053] In some embodiments, when the object detection system is used to calculate the gradient eigenvalue of each pixel point in the spectral image, mark the edge pixel points in the spectral image according to the gradient eigenvalue, and obtain the edge marked image, it is specifically used to implement the following specific steps: S211-S213.

[0054] S211, calculating the horizontal gradient value and the vertical gradient value of each pixel.

[0055] S212: Calculate the square sum of the horizontal gradient value and the vertical gradient value to obtain a gradient eigenvalue.

[0056] S213: Mark pixel points whose gradient feature values ​​are greater than a preset feature threshold as edge pixel points, and set other pixel points other than the edge pixel points to a preset grayscale value to obtain an edge marked image.

[0057] Exemplarily, the system uses the Sobel algorithm and the Scharr algorithm to calculate the horizontal gradient value of the image. and vertical gradient value The gradient eigenvalue is I, where If the gradient feature value is greater than a preset threshold, for example, the preset threshold is 70, the pixel is determined to be an edge pixel. After identifying the edge pixel, the grayscale value of non-edge pixels other than the edge pixel is set to a preset grayscale value, for example, the preset grayscale value is 255 (pure white), and an edge annotation image is obtained. In this way, the recognition accuracy of the edge of the object can be improved, thereby improving the object recognition accuracy of the system.

[0058] In some embodiments, when the object detection system is used to calculate the structural similarity index, image quality index and feature similarity index between each pixel block and other pixel blocks, it is specifically used to implement the following specific steps: S311-S314.

[0059] S311, obtaining the average brightness value, variance and covariance of each pixel block.

[0060] S312: Obtain a structural similarity index according to a preset first weight array, an average brightness value, a variance, and a covariance.

[0061] Exemplarily, the specific calculation formula of the structural similarity index is:

[0062] ;

[0063] in, and are the average brightness values ​​(means) of the two pixel blocks being compared. and are the variances of the two pixel blocks being compared, reflecting the contrast of the image. It is the covariance of the two pixel blocks being compared, reflecting the similarity of their structural information. and are the weight coefficients in the first weight array, Usually 0.01, Usually 0.03.

[0064] S313: Obtain an image quality index according to a preset second weight array, an average brightness value, a variance, and a covariance.

[0065] Exemplarily, the specific calculation formula of the image quality index is:

[0066] ;

[0067] in, are the weight coefficients in the second weight array.

[0068] S314, obtaining the phase value and gradient amplitude of each pixel block, and determining the feature similarity index according to the preset third weight array, the phase value and the gradient amplitude.

[0069] Exemplarily, the specific calculation formula of the feature similarity index is:

[0070] ;

[0071] in, To compare the phase consistency of two pixel blocks, To compare the similarity of the gradient magnitudes of two pixel blocks, and are the weight coefficients in the third weight array.

[0072] In some embodiments, when the object detection system is used to calculate the target pixel value according to the pixel blocks in each block set, it is specifically used to implement the following specific steps: S411-S412.

[0073] S411, obtaining the pixel value of each pixel in the pixel block.

[0074] S412, sort the pixel values ​​from small to large, and use the median pixel value of the sort as the target pixel value.

[0075] In some embodiments, the pre-trained Mask R-CNN model includes: a Backbone network, a feature fusion layer, an RPN layer, a RolAlign layer, a classification head, and a mask prediction head.

[0076] When the object detection system is used to input the edge-corrected image into a pre-trained Mask R-CNN model and output the type information and location information of the suspicious dangerous goods on the object to be detected, it is specifically used to implement the following specific steps: S511-S516.

[0077] S511 . In the Backbone network, feature extraction is performed on the edge-corrected image by using a preset first mapping scale, a second mapping scale, and a third mapping scale to obtain a first feature image, a second feature image, and a third feature image.

[0078] Exemplarily, the Backbone network performs edge detection and possible edge enhancement on the input image to ensure clear object boundaries, which is particularly important for subsequent accurate segmentation. In the Backbone network, feature extraction of edge-corrected images at different scales is achieved through preset different step sizes (i.e., the step size of the convolution kernel) or pooling layers. The first mapping scale: coarse granularity, capturing the large-scale contextual information of the image to obtain the first feature image. The second mapping scale: medium granularity, balancing details and context to obtain the second feature image. The third mapping scale: fine granularity, focusing on details and edge information to obtain the third feature image.

[0079] S512: At the feature fusion layer, the first feature image, the second feature image, and the third feature image are fused into a feature pyramid.

[0080] Exemplarily, in the feature fusion layer, the first feature image with lower resolution but rich semantic information is upsampled to increase its spatial resolution so that it is aligned with the feature map with higher resolution. The third feature image with higher resolution but relatively sparse semantic information is downsampled to reduce its spatial dimension while retaining more contextual information. The second feature image is in the middle and can be adjusted to match other levels as needed. The low-resolution feature map is upsampled and superimposed on the high-resolution feature map, or vice versa, and feature fusion is performed directly at the element level or at the channel level.

[0081] S513. At the RPN layer, a candidate region frame is added to the third feature image of the feature pyramid and the object existence probability in each candidate region frame is calculated. A target region frame is determined from the candidate region frame according to the object existence probability. The target region frame is a candidate region frame whose object existence probability is greater than a preset probability threshold.

[0082] Exemplarily, the third feature image usually has a high resolution but relatively sparse semantic information, which can further enhance the model's detection capability and overall performance for small-sized objects. In the RPN layer, a candidate region frame is generated directly on the third feature image, and its high-resolution characteristics are used to accurately locate the object. The RPN layer generates a set of dense candidate region frames on the third feature image according to the preset scale and aspect ratio. Since the spatial resolution of the feature map of this layer is high, the distribution of the candidate region frames can be more dense, which is conducive to covering more potential small objects. The features in the candidate region frame are used to predict the probability of the object in the selected region frame. This step is performed directly on the high-resolution features of the third feature image, which improves the accuracy of the prediction. A preset probability threshold, such as 0.85, is pre-set to distinguish between background and foreground areas. For all candidate region frames whose probability of existence of objects is greater than the preset probability threshold, a non-maximum suppression algorithm can also be applied to remove the candidate region frames with high overlap and then set as the target region frame to retain the frame that is most likely to contain the object as the final target region frame.

[0083] S514. In the RolAlign layer, a preset number of sampling pixel points is determined in each target region frame, and a feature vector value of the target region frame is obtained according to a preset bilinear interpolation function and the sampling pixel points.

[0084] Exemplarily, within each RoI, a preset number of sampling points are selected according to a pre-set pattern (such as a uniform grid). These sampling points can be located on any floating-point coordinates, rather than being limited to pixel boundaries, thereby improving the calculation accuracy of features. RoI Align uses a bilinear interpolation algorithm to estimate the eigenvalues ​​of each sampling point. The bilinear interpolation algorithm calculates the eigenvalues ​​of the sampling point in a weighted average manner by focusing on the pixel values ​​of the four nearest pixels around each sampling point, ensuring a high degree of smoothness and accuracy. Finally, the eigenvalues ​​sampled from each RoI are aggregated into a fixed-size feature map as input to the subsequent classification or segmentation network. This aggregation method not only maintains the integrity of the internal information of the RoI, but also meets the requirements of the subsequent network for fixed-size input.

[0085] S515. In the classification head, classification is performed according to a preset dangerous goods database and a feature vector value to obtain type information corresponding to the target area frame. The type information includes: item category information and item category probability.

[0086] Exemplarily, the classification head includes: a fully connected layer, an activation function, a Softmax layer, and a loss function. The classification head is refined through a multi-layer fully connected network, and finally outputs the probability distribution of each RoI corresponding to each category with the help of the Softmax function. The classification head loss function calculates the cross-entropy loss (Cross-Entropy Loss) or other classification losses to guide the model to learn the correct classification decision.

[0087] The classification head first needs to convert the feature vector value output from the RoIAlign layer into a form suitable for the classification task. Based on the converted feature vector value, the classification head predicts which of the preset dangerous goods database the object represented by the RoI belongs to, and identifies the specific category of the object, such as knives, axes, etc. The classification head may give multiple category labels, so it is also necessary to provide a probability distribution for each category label to indicate the model's confidence in each possible category, so that the final classification is determined according to the probability in the subsequent steps to obtain the item category information.

[0088] S516. In the mask prediction head, a two-dimensional feature matrix corresponding to the item category information is output, and position information corresponding to the target area box is output according to the two-dimensional feature matrix.

[0089] Exemplarily, the mask prediction head generates a two-dimensional feature matrix that matches the size of the target candidate box, indicating whether each pixel belongs to the target object. The two-dimensional feature matrix usually includes two values ​​0 and 255. The element values ​​of the two-dimensional feature matrix correspond to the pixel points of the target candidate box one by one. The element value corresponding to the pixel points in the background area is 0, and the element value corresponding to the pixel points in the object area is 255. According to the two-dimensional feature matrix, the mask prediction head outputs the position information corresponding to the target area box, which is the position information of the object in the target area box.

[0090] The embodiment of the present application provides an object detection method, which includes: receiving a terahertz beam of an object to be detected, generating a spectral image of the object to be detected according to the terahertz beam and preset spectral information; calculating the gradient eigenvalue of each pixel point in the spectral image, marking the edge pixel points in the spectral image according to the gradient eigenvalue, and obtaining an edge-marked image; extracting pixel blocks based on the edge pixel points, calculating the structural similarity index, image quality index and feature similarity index of each pixel block with other pixel blocks, and obtaining the similarity value of each pixel block with other pixel blocks according to preset weight coefficients, structural similarity indexes, image quality indexes and feature similarity indexes; dividing the pixel blocks into multiple block sets according to the similarity values, calculating the target pixel values ​​according to the pixel blocks in each block set, correcting the pixel blocks in the block set according to the target pixel values, and generating an edge-corrected image; inputting the edge-corrected image into a pre-trained Mask R-CNN model, and outputting the type information and location information of the suspected dangerous goods on the object to be detected. Through the above method, the gradient eigenvalue of the terahertz spectrum image is calculated, the edge pixel points of the object in the terahertz spectrum image are found according to the gradient eigenvalue, and the edge pixel points are marked, pixel blocks are extracted along the edge pixel points, the similarity values ​​between the pixel blocks are calculated, and the pixel blocks with the same characteristics are divided into multiple block sets according to the similarity values, and the target pixel value of the block set is obtained according to the pixel blocks in the same block set to correct the pixel blocks in the block set to obtain an edge-corrected image with obvious edges, thereby improving the resolution and contrast of the terahertz image, so as to improve the recognition success rate of the object in the object recognition model and reduce the cost of removing noise from the terahertz image.

[0091] See also Figure 2 , Figure 2 The embodiment of the present application further provides a schematic block diagram of an object detection device, wherein the object detection device 300 is used to execute the aforementioned object detection method. The object detection device 300 can be configured in a server.

[0092] Among them, the server can be an independent server or a server cluster, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN) and big data and artificial intelligence platforms.

[0093] like Figure 2 As shown, the object detection device 300 includes: a beam processing module 301 , an edge marking module 302 , a parameter calculation module 303 , an edge correction module 304 and an image processing module 305 .

[0094] The beam processing module 301 is used to receive the terahertz beam of the object to be detected, and generate a spectral image of the object to be detected according to the terahertz beam and preset spectral information.

[0095] The edge labeling module 302 is used to calculate the gradient eigenvalue of each pixel in the spectral image, and label the edge pixels in the spectral image according to the gradient eigenvalue to obtain an edge labeled image.

[0096] In some embodiments, the edge annotation module 302 is used to implement the calculation of the gradient eigenvalue of each pixel point in the spectral image, annotate the edge pixel points in the spectral image according to the gradient eigenvalue, and obtain the edge annotated image, and is specifically used to implement: calculating the horizontal gradient value and the vertical gradient value of each pixel point; calculating the sum of the squares of the horizontal gradient value and the vertical gradient value to obtain the gradient eigenvalue; annotating the pixel points whose gradient eigenvalue is greater than a preset characteristic threshold as the edge pixel points, and setting the other pixel points other than the edge pixel points to a preset grayscale value to obtain the edge annotated image.

[0097] The parameter operation module 303 is used to extract pixel blocks based on edge pixel points, calculate the structural similarity index, image quality index and feature similarity index between each pixel block and other pixel blocks, and obtain the similarity value between each pixel block and other pixel blocks according to the preset weight coefficient, structural similarity index, image quality index and feature similarity index.

[0098] In some embodiments, when the parameter operation module 303 is used to calculate the structural similarity index, image quality index and feature similarity index between each pixel block and other pixel blocks, it is specifically used to implement: obtaining the average brightness value, variance and covariance of each pixel block; obtaining the structural similarity index according to the preset first weight array, average brightness value, variance and covariance; obtaining the image quality index according to the preset second weight array, average brightness value, variance and covariance; obtaining the phase value and gradient amplitude of each pixel block, and determining the feature similarity index according to the preset third weight array, phase value and gradient amplitude.

[0099] The edge correction module 304 is used to divide the pixel blocks into multiple block sets according to the similarity values, calculate the target pixel value according to the pixel blocks in each block set, correct the pixel blocks in the block set according to the target pixel value, and generate an edge-corrected image.

[0100] In some embodiments, when the edge correction module 304 is used to calculate the target pixel value based on the pixel blocks in each block set, it is specifically used to implement: obtaining the pixel value of each pixel point in the pixel block; sorting the pixel values ​​from small to large, and taking the median pixel value of the sorting as the target pixel value.

[0101] The image processing module 305 is used to input the edge-corrected image into a pre-trained Mask R-CNN model, and output type information and location information of the suspected dangerous goods on the object to be detected.

[0102] In some embodiments, the pre-trained Mask R-CNN model includes: a Backbone network, a feature fusion layer, an RPN layer, a RolAlign layer, a classification head and a mask prediction head. When the image processing module 305 is used to input the edge-corrected image into the pre-trained Mask R-CNN model and output the type information and location information of the suspicious dangerous goods on the object to be detected, it is specifically used to implement: in the Backbone network, the edge-corrected image is respectively extracted by a preset first mapping scale, a second mapping scale, and a third mapping scale to obtain a first feature image, a second feature image, and a third feature image; in the feature fusion layer, the first feature image, the second feature image, and the third feature image are fused into a feature pyramid; in the RPN layer, a candidate region frame is added to the third feature image of the feature pyramid and the probability of existence of an object in each candidate region frame is calculated, and the probability of existence of the object is obtained from the candidate region frame according to the probability of existence of the object. The target area box is determined by the frame, and the target area box is a candidate area box in which the probability of the object existing is greater than the preset probability threshold; in the RolAlign layer, a preset number of sampling pixel points are determined in each target area box, and the feature vector value of the target area box is obtained according to the preset bilinear interpolation function and the sampling pixel points; in the classification head, classification is performed according to the preset dangerous goods database and the feature vector value to obtain the type information corresponding to the target area box, and the type information includes: item category information and item category probability; in the mask prediction head, a two-dimensional feature matrix corresponding to the item category information is output, and the position information corresponding to the target area box is output according to the two-dimensional feature matrix.

[0103] An embodiment of the present application provides a terminal device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement an object detection method such as any one of the embodiments of the present application when executing the computer program.

[0104] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements any one of the object detection methods in the embodiments of the present application.

[0105] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for detecting an object, characterized in that: The method comprises: Receiving a terahertz beam of an object to be detected, and generating a spectral image of the object to be detected according to the terahertz beam and preset spectral information; Calculating the gradient eigenvalue of each pixel in the spectral image, and marking the edge pixel points in the spectral image according to the gradient eigenvalue to obtain an edge marked image; Extract pixel blocks based on the edge pixel points, obtain the average brightness value, variance and covariance of each pixel block; obtain a structural similarity index according to a preset first weight array, the average brightness value, the variance and the covariance; obtain an image quality index according to a preset second weight array, the average brightness value, the variance and the covariance; obtain a phase value and a gradient amplitude of each pixel block, determine a feature similarity index according to a preset third weight array, the phase value and the gradient amplitude, and obtain a similarity value between each pixel block and other pixel blocks according to a preset weight coefficient, the structural similarity index, the image quality index and the feature similarity index; Dividing the pixel blocks into a plurality of block sets according to the similarity values, calculating target pixel values ​​according to the pixel blocks in each of the block sets, and correcting the pixel blocks in the block sets according to the target pixel values ​​to generate an edge-corrected image; The edge-corrected image is input into a pre-trained Mask R-CNN model, and the type information and location information of the suspected dangerous goods present on the object to be detected are output.

2. The object detection method according to claim 1, characterized in that: The step of calculating the target pixel value according to the pixel blocks in each of the block sets includes: Obtaining a pixel value of each pixel in the pixel block; The pixel values ​​are sorted from small to large, and the median pixel value of the sorting is used as the target pixel value.

3. The object detection method according to claim 1, characterized in that: The pre-trained Mask R-CNN model includes: a Backbone network, a feature fusion layer, an RPN layer, a RolAlign layer, a classification head and a mask prediction head. The edge-corrected image is input into the pre-trained Mask R-CNN model, and the type information and location information of the suspicious dangerous goods on the object to be detected are output, including: In the Backbone network, feature extraction is performed on the edge-corrected image by using a preset first mapping scale, a second mapping scale, and a third mapping scale to obtain a first feature image, a second feature image, and a third feature image; At the feature fusion layer, the first feature image, the second feature image and the third feature image are fused into a feature pyramid; At the RPN layer, adding candidate area boxes to the third feature image of the feature pyramid and calculating the object existence probability in each candidate area box, and determining a target area box from the candidate area boxes according to the object existence probability, the target area box being a candidate area box whose object existence probability is greater than a preset probability threshold; In the RolAlign layer, a preset number of sampling pixel points are determined in each of the target area frames, and a feature vector value of the target area frame is obtained according to a preset bilinear interpolation function and the sampling pixel points; In the classification head, classification is performed according to a preset dangerous goods database and the feature vector value to obtain the type information corresponding to the target area frame, wherein the type information includes: item category information and item category probability; In the mask prediction head, a two-dimensional feature matrix corresponding to the item category information is output, and position information corresponding to the target area box is output according to the two-dimensional feature matrix.

4. The object detection method according to claim 1, characterized in that: The step of calculating the gradient eigenvalue of each pixel in the spectral image and marking edge pixels in the spectral image according to the gradient eigenvalue to obtain an edge marked image includes: Calculate the horizontal gradient value and the vertical gradient value of each pixel; Calculating the square sum of the horizontal gradient value and the vertical gradient value to obtain the gradient characteristic value; The pixel points whose gradient feature values ​​are greater than a preset feature threshold are marked as edge pixel points, and other pixel points other than the edge pixel points are set to preset grayscale values ​​to obtain the edge marked image.

5. An object detection device, characterized in that: The device comprises: A beam processing module, used for receiving a terahertz beam of an object to be detected, and generating a spectral image of the object to be detected according to the terahertz beam and preset spectral information; An edge annotation module is used to calculate the gradient eigenvalue of each pixel in the spectral image, and annotate the edge pixels in the spectral image according to the gradient eigenvalue to obtain an edge annotation image; A parameter operation module is used to extract pixel blocks based on the edge pixel points, obtain the average brightness value, variance and covariance of each pixel block; obtain a structural similarity index according to a preset first weight array, the average brightness value, the variance and the covariance; obtain an image quality index according to a preset second weight array, the average brightness value, the variance and the covariance; obtain a phase value and a gradient amplitude of each pixel block, determine a feature similarity index according to a preset third weight array, the phase value and the gradient amplitude, and obtain a similarity value between each pixel block and other pixel blocks according to a preset weight coefficient, the structural similarity index, the image quality index and the feature similarity index; An edge correction module, used for dividing the pixel blocks into a plurality of block sets according to the similarity values, calculating a target pixel value according to the pixel blocks in each of the block sets, correcting the pixel blocks in the block sets according to the target pixel values, and generating an edge-corrected image; The image processing module is used to input the edge-corrected image into a pre-trained Mask R-CNN model and output type information and location information of the suspected dangerous goods on the object to be detected.

6. A terminal device, characterized in that: The terminal device includes a memory and a processor; The memory is used to store computer programs; The processor is used to execute the computer program and implement the object detection method according to any one of claims 1 to 4 when executing the computer program.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the object detection method according to any one of claims 1 to 4.

Images