Improved millimeter wave image target detection and recognition method for YOLO v8

Abstract

The application relates to an improved millimeter wave image target detection and recognition method of YOLOv8, and belongs to the technical field of image processing. The system comprises the following steps: inputting an acquired millimeter wave image into a pre-trained improved YOLOv8 neural network; obtaining target position and category information at an output layer, and performing target detection and recognition; generating corresponding detection results according to target information, including target categories, coordinates and confidence, so that efficient detection and accurate recognition of the target are realized. The system has the advantages of fast detection speed and high precision.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing, and relates to a millimeter wave image target detection and recognition method for improving YOLOv8. BACKGROUND

[0002] Current common millimeter wave target detection methods are mainly based on traditional feature extraction and classification algorithms, and their precision is limited and requires a large amount of computing resources. At the same time, target detection methods based on deep learning have been widely used in the field of visible light images, but there are great differences in millimeter wave images, and network structure design and pre-training need to be specifically designed. SUMMARY

[0003] Therefore, the purpose of the application is to provide a millimeter wave image target detection and recognition method for improving YOLOv8.

[0004] To achieve the above purpose, the application provides the following technical scheme:

[0005] A millimeter wave image target detection and recognition method for improving YOLOv8, comprising the following steps:

[0006] S1: obtaining a millimeter wave image

[0007] S11: using a radar chip to actively obtain the spatial position, speed, reflection intensity and other information of a target by transmitting and receiving low-power frequency-modulated continuous wave (FMCW) millimeter waves.

[0008] Set the corresponding millimeter wave radar parameters, such as working frequency f, distance resolution ΔR and direction angle θ, etc. According to the radar equation, the minimum detectable power P min :

[0009]

[0010] where P t is the transmit power; G t and G r are the transmit and receive antenna gains, respectively; λ is the wavelength; σ is the target radar cross section; R is the distance; and L is the system loss.

[0011] S12: using the RMA imaging algorithm to obtain the millimeter wave radar three-dimensional imaging process as follows:

[0012] S121: obtaining four-dimensional echo data s(x m ,x n ,z,k);

[0013] S122: processing the four-dimensional echo data along the scanning direction (x m ,x n,k) perform a three-dimensional Fourier transform to obtain a four-dimensional wave number domain;

[0014] S123: performing interpolation and dimension conversion on the wavenumber domain echo data to obtain three-dimensional wavenumber domain image reconstruction data;

[0015] S124: Perform a three-dimensional inverse Fourier transform on the three-dimensional wavenumber domain echo data to obtain final three-dimensional image reconstruction data.

[0016] The raw data collected by the radar board is obtained through Ethernet. Assuming that the radar sampling position is (x, y, 0), the spatial coordinates of the target to be imaged are (x', y', z0), and the target reflectivity is p(x', y'), the backscattered data received by the radar is:

[0017]

[0018] R represents the distance from the center of the transceiver antenna to the target, and the average distance from the target to the radar is z0. -2 You can use (z0R) -1 Instead, for a stationary target, z0 is a constant and can be ignored. The final received data can be approximately expressed as:

[0019]

[0020] Spherical waves can be considered as the superposition of plane waves, then:

[0021]

[0022] in:

[0023]

[0024] so,

[0025]

[0026] Changing the order of integration reveals The corresponding two-dimensional Fourier transform of the reflectivity function, FT 2D [p(x,y)]=p(k x ,k y ),and It also represents the two-dimensional inverse Fourier transform. So

[0027]

[0028] Then the two-dimensional target reflectivity can be reconstructed:

[0029]

[0030] S13: Convert the preprocessed data into an image format (such as PNG or JPEG) to obtain a millimeter wave image. First, the amplitude and phase of the data are calculated, and then the amplitude data is mapped to a grayscale value range (0-255) to obtain the final millimeter wave image.

[0031] S2: Improved YOLO v8 neural network structure

[0032] S21: Backbone Network Replacement: Darknet53 is replaced with the lightweight ShuffleNet backbone network. The ShuffleNet network introduces the Shuffle module, which reduces computational complexity and parameter requirements, improving model efficiency and accuracy. Specifically, the Shuffle module reduces computational complexity and improves feature expression through grouped convolution and channel shuffling operations. In grouped convolution, input channels are divided into multiple groups, and independent convolution operations are performed within each group, reducing computational complexity. Channel shuffling adds channel shuffling after the convolution operation to improve the model's feature expression capabilities.

[0033] S22: Post-processing Algorithm Optimization: We improved YOLOv8's post-processing algorithm by using the non-maximum suppression (NMS) algorithm to filter out overlapping objects. We also introduced a pixel correction method to improve the model's accuracy and robustness. Specifically, this method adjusts the target box coordinates based on the local image gradient information, bringing them closer to the ideal boundary.

[0034] S2: Image preprocessing and feature extraction

[0035] S21: Preprocess the acquired millimeter wave image, such as scaling, cropping, and color space conversion, to make it meet the input requirements of the improved YOLO v8. For example, scale the image to the fixed size required by the network (such as 416×416, 640×640).

[0036] S22: The preprocessed image is fed into the improved YOLO v8 neural network, which uses the ShuffleNet backbone to extract feature information. The network output is calculated layer by layer using operations such as convolution, activation, and pooling.

[0037] S23: Utilize the multi-scale feature fusion mechanism of YOLO v8, through upsampling and skip connection methods, to fuse feature maps of different depths, thereby enhancing detection performance. Specifically, the following formula can be used:

[0038]

[0039] Among them, F(x) is the fused feature map; U is the upsampling operation; α is the weight parameter; F l(x) and F s(x)Represent deep and shallow feature maps respectively; Represents a feature fusion operation (such as addition or concatenation).

[0040] S3: Object Detection and Recognition

[0041] S31: Decode the network output according to the predefined anchor box shape to obtain the candidate target box and confidence. This step usually includes calculating the center coordinates, width and height of the predicted bounding box and the category confidence. The specific calculation formula is as follows:

[0042] b x =σ(t x )+c x

[0043] b y =σ(t y )+c y

[0044] b w =p w exp(t w )

[0045] b h =p h exp(t h )

[0046] Pr(Class i |Object)=σ(t ci )

[0047] Among them, b x 、b y 、b w 、b h are the center coordinates and width and height of the predicted bounding box respectively; t x , t y , t w , t h is the network output value; c x 、c y is the center coordinate of the anchor box; p w 、p h is the width and height of the anchor box; σ is the Sigmoid function; t ci Represents the confidence level of the i-th category; Pr(Class i |Object) is the probability that the target belongs to the i-th category.

[0048] S32: Screen the candidate target frames and remove target frames with confidence levels lower than a threshold. For example, if the confidence threshold is set to 0.5, only target frames with confidence levels greater than or equal to 0.5 are retained.

[0049] S33: Apply the non-maximum suppression (NMS) algorithm to further optimize the filtered target boxes. Calculate the IoU (intersection over union) between the target boxes. If the IoU between a target box and other high-confidence target boxes is greater than a predetermined threshold (e.g., 0.5), the target box is considered redundant and is removed.

[0050] S34: Applying a pixel correction method to the optimized target frame, adjusting the target frame coordinates based on the local gradient information of the image to bring it closer to the ideal boundary. Specifically, the target frame edge coordinates are adjusted by calculating the direction of the maximum gradient in the neighborhood.

[0051] S4: Output the final detection results, including the target box position, size, and category information. At this point, the target in the millimeter wave image has been successfully detected and recognized.

[0052] This paper improves the accuracy and efficiency of target detection and recognition in millimeter-wave imagery by introducing the lightweight ShuffleNet backbone network and optimizing the YOLO v8 neural network structure. Furthermore, non-maximum suppression (NMS) and pixel correction methods are used to optimize the detection results, further improving the accuracy and robustness of the model. This paper is suitable for applications in fields such as unmanned driving and security monitoring, and has broad potential.

[0053] The beneficial effects of the present invention are:

[0054] 1. Fast: Compared to other object detection algorithms, YOLO has a fast recognition speed and can perform real-time object detection on a GPU. YOLO-based real-time object detection has high accuracy and can detect objects at speeds of up to xx frames per second.

[0055] 2. High Accuracy: Utilizing a fully convolutional neural network, a single forward pass through the entire image directly determines the probability and location of a target's presence. Compared to R-CNN methods, this method combines multiple detection and regression steps, and combines it with millimeter-wave radar detection to utilize a wide range of electromagnetic frequency bands, scanning electromagnetic wave reflections from the exterior of a person or object to detect the material and structure within. Compared to traditional metal detectors, it more accurately detects non-metallic and low-density materials hidden within a person or object, achieving similar or even better detection accuracy.

[0056] 3. Generalization: YOLO can handle images of various sizes, targets of various sizes and scales, and can detect objects in complex environments.

[0057] 4. Adaptation: YOLO can enhance its detection capabilities through self-online learning, such as detecting in new scenes and on new objects.

[0058] 5. Does not rely on traditional image segmentation, target detection and other technologies: The YOLO algorithm does not require pre-segmentation of objects in the image. It takes the entire image as input and directly outputs the category and location information of each object in the image.

[0059] Other advantages, objects, and features of the present invention will be described in part in the following description and, in part, will be apparent to those skilled in the art upon examination of the following description or may be learned from practice of the present invention. The objects and other advantages of the present invention may be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below with reference to the accompanying drawings, in which:

[0061] Figure 1 The figure is a flowchart of an improved YOLO v8 millimeter wave image target detection and recognition method provided in an embodiment of the present application.

[0062] Figure 2 This is a test result diagram of the training model of the millimeter wave image target detection and recognition method based on the improved YOLO v8 of the present invention, with the input being a kitchen knife.

[0063] Figure 3 This is a test result diagram of the training model of the millimeter wave image target detection and recognition method based on the improved YOLO v8 of the present invention, with the input being a spatula.

[0064] Figure 4 This is a test result diagram of the training model of the millimeter wave image target detection and recognition method based on the improved YOLO v8 of the present invention for the input of scissors.

[0065] Figure 5 This is a test result diagram of the training model of the millimeter wave image target detection and recognition method based on the improved YOLO v8 of the present invention for the input of a long sharp knife.

[0066] Figure 6 This is a test result diagram of the training model of the millimeter wave image target detection and recognition method based on the improved YOLO v8 of the present invention, with the input being a fruit knife. DETAILED DESCRIPTION

[0067] The following describes the embodiments of the present invention by means of specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0068] Among them, the accompanying drawings are only for illustrative purposes and represent only schematic diagrams rather than actual pictures, and should not be understood as limiting the present invention. In order to better illustrate the embodiments of the present invention, some parts of the accompanying drawings may be omitted, enlarged or reduced, and do not represent the dimensions of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions may be omitted in the accompanying drawings.

[0069] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "back", etc. indicating directions or positional relationships, they are based on the directions or positional relationships shown in the drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0070] See also Figure 1 This content is a flowchart of the millimeter wave image target detection and recognition method using YOLO v8. The specific method is as follows:

[0071] a. Acquire millimeter wave images.

[0072] 1. Set the millimeter-wave radar to operate at 60 GHz, with a range resolution of 5 cm and a directional angle range of 360 degrees.

[0073] 2. Use millimeter-wave radar to scan the target area, obtain a set of intensity value data in distance and direction, and use the holographic reconstruction algorithm to obtain the millimeter-wave image of the target after projection in the distance direction.

[0074] 3. Apply Gaussian filter to smooth the raw data to remove noise, and use enhancement algorithm to enhance the target signal.

[0075] 4. The preprocessed data is converted into a PNG format image for subsequent analysis and display.

[0076] b. The image is input into the trained improved YOLO v8 neural network.

[0077] 1. The feature representation of the millimeter wave image is extracted through the convolution and pooling layers of the Shuffle ghost network.

[0078] 2. The feature maps of different scales extracted by the backbone network are fused with the feature fusion layer of YOLO v8 to obtain more rich feature information.

[0079] 3. The fused feature maps are input into the convolution layer, activation function and pooling layer of YOLO v8, and the network output is calculated layer by layer.

[0080] c. The target position and category information are obtained at the output layer, and target detection and recognition are performed.

[0081] 1. According to the feature map output by the network and the pre-defined anchor box shape, the position and confidence of the candidate target box are decoded.

[0082] 2. Set a threshold, such as 0.5, to filter out candidate target boxes with confidence higher than the threshold.

[0083] 3. Apply the NMS algorithm to eliminate candidate target boxes with high overlap and only keep the most representative target boxes.

[0084] 4. Use the pixel point correction method to fine-tune the position of the target box to improve the accuracy and precision of the target box.

[0085] d. According to the target information, generate the corresponding detection results, including target category, coordinates, and confidence.

[0086] 1. Extract the class label and confidence of each target from the target box after NMS processing.

[0087] 2. According to the feature map output by the network and the position information of the target box, convert the coordinates of the target box to the coordinates in the original image space.

[0088] 3. Represent the target category, coordinates and confidence in the JSON or XML format data structure as the final detection result.

[0089] For this example, the improved YOLO v8 millimeter wave image target detection and recognition method model is trained, and there are 5 input categories, including kitchen knife, spatula, scissors, long sharp knife and fruit knife. For different inputs and their corresponding output results, please refer to Figures 2 to 6 .

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should all be included in the scope of the claims of the present invention.

Claims

1. Improve YOLO v8's millimeter wave image target detection and recognition method, characterized by: The method comprises the following steps: S1: Acquire millimeter wave images; S11: Uses a radar chip to actively obtain the target's spatial position, velocity, and reflection intensity by transmitting and receiving low-power frequency-modulated continuous (FMCW) millimeter waves. Set the corresponding millimeter wave radar parameters, including operating frequency f, range resolution ΔR and direction angle θ; calculate the required minimum detectable power P according to the radar equation min : Among them, P t is the transmission power; G t and G r are the transmitting and receiving antenna gains respectively; λ is the wavelength; σ is the target radar cross-section; R is the distance; L is the system loss; S12: Use RMA imaging algorithm to obtain millimeter wave radar 3D imaging; The raw data collected by the radar board is obtained through Ethernet. Assuming that the radar sampling position is (x, y, 0), the spatial coordinates of the target to be imaged are (x', y', z0), and the target reflectivity is p(x', y'), the backscattered data received by the radar is: R represents the distance from the center of the transmitting and receiving antenna to the target, and the average distance from the target to the radar is z0; in the above formula, R -2 Use (z0R) -1 Instead, for a stationary target, z0 is a constant and can be ignored; the final received data is expressed as: Spherical waves are considered to be the superposition of plane waves, then: in: Then we have: Changing the order of integration reveals The corresponding two-dimensional Fourier transform of the reflectivity function, FT 2D [p(x,y)]=p(k x ,k y ),and represents the two-dimensional inverse Fourier transform; then Then reconstruct the two-dimensional target reflectivity: S13: converting the preprocessed data into an image format to obtain a millimeter wave image; S2: Input the image into the trained improved YOLO v8 neural network; The lightweight network ShuffleNet is used to replace Darknet53. The post-processing algorithm of YOLOv8 is improved to use the non-maximum suppression (NMS) algorithm to filter overlapping objects. At the same time, a pixel correction method is introduced to improve the accuracy and robustness of the model. S21: Preprocess the acquired millimeter wave image, including scaling, cropping, and color space conversion, to make it meet the input requirements of the improved YOLO v8; S22: Input the preprocessed image into the ShuffleNet backbone network to extract feature information; based on convolution, activation and pooling operations, calculate the network output layer by layer; S23: Using the multi-scale feature fusion mechanism of YOLO v8, feature maps of different depths are fused through upsampling and skip connection methods, using the following formula: F(x)=U(αF l(x) )⊕(1-α)F s(x) Among them, F(x) is the fused feature map; U is the upsampling operation; α is the weight parameter; F l(x) and F s(x) Represent deep and shallow feature maps respectively; ⊕ represents feature fusion operation, i.e. addition or splicing; S3: Obtain target location and category information in the output layer, and perform target detection and recognition; S31: Decode the network output according to the predefined anchor box shape to obtain the candidate target box and confidence, including calculating the center coordinates, width and height of the predicted bounding box and the category confidence. The specific calculation formula is as follows: b x =σ(t x )+c x b y =σ(t y )+c y b w =p w exp(t w ) b h =p h exp(t h ) P r (Class i |Object)=σ(t ci ) Among them, b x 、b y 、b w 、b h are the center coordinates and width and height of the predicted bounding box respectively; t x , t y , t w , t h is the network output value; c x 、c y is the center coordinate of the anchor box; p w 、p h is the width and height of the anchor box; σ is the Sigmoid function; t ci Represents the confidence level of the i-th category; Pr(Class i |Object) is the probability that the target belongs to the i-th category; S32: Screen the candidate target frames and remove target frames with confidence levels lower than a threshold. If the confidence threshold is set to 0.5, only target frames with confidence levels greater than or equal to 0.5 are retained. S33: Apply the non-maximum suppression (NMS) algorithm to optimize the filtered target boxes; calculate the intersection over union (IoU) between the target boxes. If the IoU between a target box and other high-confidence target boxes is greater than a predetermined threshold, the target box is considered redundant and is removed. S34: Applying a pixel correction method to the optimized target frame, adjusting the target frame coordinates according to the local gradient information of the image so that it is closer to the ideal boundary; specifically, adjusting the target frame edge coordinates by calculating the direction of the maximum gradient in the neighborhood; S4: Generate corresponding detection results based on target information, including target category, coordinates and confidence level; S41: Extract the target category and confidence level in the target frame after NMS processing. S42: Convert the coordinates of the target box from the network output space back to the original image space; S43: Generate detection results based on target category, coordinates and confidence level.

2. The improved YOLO v8 millimeter wave image target detection and recognition method according to claim 1, characterized in that: The S12 specifically includes the following steps: S121: Acquire four-dimensional echo data s(x m ,x n ,z,l); S122: Calculate the four-dimensional echo data along the scanning direction (x m ,x n ,k) perform a three-dimensional Fourier transform to obtain a four-dimensional wave number domain; S123: performing interpolation and dimension conversion on the wavenumber domain echo data to obtain three-dimensional wavenumber domain image reconstruction data; S124: Perform a three-dimensional inverse Fourier transform on the three-dimensional wavenumber domain echo data to obtain final three-dimensional image reconstruction data.

3. The improved YOLO v8 millimeter wave image target detection and recognition method according to claim 2, characterized in that: In S43, the detection result is expressed in JSON or XML format.

Images