Image detection method

Abstract

The invention relates to an image detection method, belongs to the technical field of ray image detection, and solves the problem of low detection accuracy of a target object in a complex scene in the prior art. The image detection method comprises the following steps: carrying out edge extraction on an image to be detected, and segmenting the obtained edge image to obtain a binary image; extracting a plurality of contours from the binarized image, and calculating a rotation angle of each contour according to a plurality of pixel points of each contour; the rotation angle of each contour is compared with a preset angle threshold value, and whether the rotation angle of at least one contour is larger than the preset angle threshold value exists or not is judged; if yes, inputting the to-be-detected image into the rotating image detection model to obtain a prediction category and a prediction rotating bounding box of each target; and if not, inputting the to-be-detected image into the horizontal image detection model to obtain a prediction category and a prediction horizontal bounding box of each target. And the detection precision of the target object in a complex scene is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of radiographic image detection, and in particular to an image detection method. Background Art

[0002] With the development of society and economic prosperity, the frequency of people's travel and the scale of express logistics are constantly expanding. Security inspection plays a vital role in maintaining public safety. At present, X-ray scanning is a commonly used technical means in security inspection. It can penetrate the surface of objects and help security inspectors identify potential prohibited items. However, this process not only places high demands on the attention and judgment of security inspectors, but also makes it difficult to maintain long-term concentration. Frequent personnel rotation also increases the burden on human resources.

[0003] In recent years, with the rise of deep learning technology, contraband detection has also made breakthrough progress. The deep learning model can automatically extract image features and learn complex patterns, significantly improving the accuracy and robustness of detection. This not only brings new solutions to security inspections, but also provides a more solid guarantee for public safety.

[0004] However, current X-ray target detection methods mainly use horizontal rectangular frames to constrain objects, which perform well in detecting objects in normal postures. However, when detecting irregularly placed objects and objects with severe overlapping interference, the irregular placement of objects is ignored, resulting in large errors in target detection results, making it difficult to meet the needs of complex scenes.

[0005] Therefore, there is an urgent need for a technical solution that can accurately detect objects in complex scenes. Summary of the invention

[0006] In view of the above analysis, an embodiment of the present invention aims to provide an image detection method to solve the problem of low detection accuracy of target objects in complex scenes in the prior art. An embodiment of the present invention provides an image detection method, the image detection method comprising:

[0007] Perform edge extraction on the image to be detected to obtain an edge image, and segment the edge image according to a preset segmentation pixel threshold to obtain a binary image;

[0008] Extract multiple contours from the binary image, each contour includes multiple pixel points; calculate the rotation angle of each contour according to the multiple pixel points of each contour;

[0009] The rotation angle of each contour is compared with the preset angle threshold to determine whether there is at least one contour whose rotation angle is greater than the preset angle threshold; if yes, the image to be detected is input into the rotation image detection model to obtain the predicted category and predicted rotation bounding box of each target in the image to be detected; if no, the image to be detected is input into the horizontal image detection model to obtain the predicted category and predicted horizontal bounding box of each target.

[0010] Based on the further improvement of the above image detection method, the edge extraction of the image to be detected is performed to obtain an edge image, and the edge image is segmented according to a preset segmentation pixel threshold to obtain a binary image, including:

[0011] Use the preset horizontal Sobel convolution kernel and the preset vertical Sobel convolution kernel to perform convolution operations on the image to be detected respectively to obtain a horizontal edge image and a vertical edge image; merge the horizontal edge image and the vertical edge image to obtain an intermediate edge image;

[0012] Normalizing the middle edge image, and then restoring the normalized middle edge image to a normal pixel value to obtain an edge image;

[0013] The pixel value of each pixel in the edge image is compared with the preset segmentation pixel threshold, and the pixels greater than the preset segmentation pixel threshold are regarded as the first category, and the pixels less than or equal to the preset segmentation pixel threshold are regarded as the second category, so as to obtain a binary image.

[0014] Based on the further improvement of the above image detection method, the extraction of multiple contours from the binary image, each contour including multiple pixel points, includes:

[0015] According to the preset connected regions, pixels in the binary image that are adjacent and whose pixel values ​​all belong to the first category are taken as a connected region to obtain a plurality of connected regions; wherein the preset connected regions include eight-connected or four-connected;

[0016] In each connected region, starting from any pixel of the first category, all adjacent pixels of the first category are marked sequentially along the boundary of each connected region to obtain the contour corresponding to each connected region;

[0017] Compare the sizes of several contours, delete the largest contour, and use the remaining contours as the final multiple contours.

[0018] Based on the further improvement of the above image detection method, the rotation angle of each contour is calculated according to the multiple pixel points of each contour to obtain the rotation angle of multiple contours, including:

[0019] Calculate the centroid of each contour according to the multiple pixel points of each contour, and calculate the covariance matrix of each contour according to the centroid of each contour and the multiple pixel points of each contour;

[0020] Calculate the maximum eigenvalue corresponding to each contour according to the covariance matrix of each contour, and calculate the main direction vector according to the covariance matrix and the maximum eigenvalue of each contour;

[0021] Calculates the rotation angle based on the principal direction vector.

[0022] Based on the further improvement of the above image detection method, the image to be detected is input into the rotation image detection model to obtain the predicted category and predicted rotation bounding box of each target in the image to be detected, including:

[0023] The image to be detected is input into the rotation image detection model to obtain the predicted category and predicted rotation bounding box information of each target in the image to be detected; the predicted rotation bounding box information includes the predicted center point coordinates, predicted covariance matrix and predicted rotation angle of the rotation bounding box; wherein the rotation image detection model is obtained by training based on the improved YOLOv8 neural network structure, and the improved YOLOv8 neural network structure includes a dual-stream attention module and a terminal feature enhancement module, the dual-stream attention module is used to enhance the channel information and feature information when the original backbone network generates a feature map, and the terminal feature enhancement module is used to enhance the feature of the feature map output by the original neck network;

[0024] The coordinates of the four corner points of each target are calculated according to the predicted rotation bounding box information of each target, and the coordinates of the four corner points of each target are connected as the predicted rotation bounding box of each target.

[0025] Based on the further improvement of the above image detection method, the rotation image detection model includes a backbone network, a neck network and a detection head;

[0026] The backbone network includes an original backbone network and a dual-stream attention module; the original backbone network is used to extract features from the input image to be detected, obtain the third feature map, the fourth feature map and the fifth feature map and output them to the neck network; the dual-stream attention module enhances channel information and feature information when the original backbone network extracts features to generate the third feature map to the fifth feature map;

[0027] The neck network includes an original neck network and a terminal feature enhancement module; the original neck network is used to perform feature fusion on the third feature map, the fourth feature map and the fifth feature map to obtain the second fused feature map, the third fused feature map and the fourth fused feature map and output them to the terminal feature enhancement module; the terminal feature enhancement module is used to perform feature enhancement on the second fused feature map, the third fused feature map and the fourth fused feature map respectively to obtain the enhanced fifth fused feature map, the sixth fused feature map and the seventh fused feature map and output them to the detection head;

[0028] The detection head is used to predict the predicted category and predicted rotation bounding box information of each target in the image to be detected according to the fifth fused feature map, the sixth fused feature map and the seventh fused feature map.

[0029] Based on the further improvement of the above image detection method, the backbone network includes a first convolution module, a first feature extraction layer, a second feature extraction layer, a third feature extraction layer and a fourth feature extraction layer connected in sequence, and outputs a first feature map, a second feature map, a third feature map, a fourth feature map and a fifth feature map respectively;

[0030] The first feature extraction layer, the second feature extraction layer and the third feature extraction layer have the same structure, and all include a second convolution module and a first residual convolution module connected in sequence; the fourth feature extraction layer includes a second convolution module, a first residual convolution module, a dual-stream attention module and a spatial pyramid pooling fast module SPPF connected in sequence.

[0031] Based on the further improvement of the above image detection method, the coordinates of the four corner points of each target are calculated by the following formula:

[0032]

[0033] in, Represents the predicted center point coordinates of each target, represents the prediction covariance matrix of each target, θ pred represents the predicted rotation angle of each object.

[0034] Based on the further improvement of the above image detection method, the rotation image detection model is trained through the following steps:

[0035] Acquire a number of radiographic images, preprocess the number of radiographic images, and mark the category labels of each target in each preprocessed radiographic image and the coordinates of the four corner points of the rotation bounding box;

[0036] Calculate the center point coordinate label and covariance matrix label of the rotation bounding box of each target according to the coordinates of the four corner points of the rotation bounding box of each target; take a ray image and the category label, center point coordinate label and covariance matrix label corresponding to each target marked in the image as a training sample to obtain several training samples; divide the several training samples into a training set and a validation set;

[0037] The preset neural network model is trained for a preset number of iterations using the training set, and a trained rotated image detection model is determined based on the evaluation results of the validation set at the end of each iteration.

[0038] Based on the further improvement of the above image detection method, the preset neural network model is trained for a preset number of iterations using the training set, and the trained rotation image detection model is determined according to the evaluation result of the validation set at the end of each iteration process, including:

[0039] In each iteration, multiple training samples are randomly selected from the training set, and the multiple training samples are divided into multiple groups. All the training samples in each group are used to train the preset neural network model in turn to obtain the predicted categories, predicted center point coordinates and predicted covariance matrix of all targets in all the training samples in each group;

[0040] Calculate the loss function values ​​of all training samples in the current group, update the weight parameters of the preset neural network model in combination with the preset gradient descent training method, and use the updated preset neural network model to start training the next group of training samples until all groups of training samples have completed training;

[0041] After all groups of training samples have completed training, the preset neural network model obtained in this iteration process is evaluated using the samples in the validation set to obtain the evaluation results of this iteration;

[0042] Determine whether the number of iterations meets the preset number of iterations. If so, use the preset neural network model corresponding to the optimal evaluation result as the trained rotation image detection model; if not, continue training for the next iteration process.

[0043] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0044] 1. By comparing the rotation angles of each contour in the image to be detected, different detection models are set for the image to be detected according to the comparison results: if there is at least one contour in the image to be detected with a rotation angle greater than the preset angle threshold, the rotation angle detection model is used to detect the image to be detected with high detection accuracy; otherwise, the horizontal image detection model is used to detect the image to be detected, which improves the detection accuracy of target objects in complex scenes as a whole;

[0045] 2. The image to be detected is predicted by the rotation image detection model to obtain the predicted category and predicted rotation bounding box of each target in the image to be detected. The rotation bounding box can effectively constrain each target in the image, reduce the background part of each target, make the bounding box of each target clearer, and improve the recognition accuracy of each target in the image;

[0046] 3. By adding a dual-stream attention module to the YOLOv8 neural network structure, the rotation image detection model enhances the channel information and feature information when the original backbone network generates the feature map, and adds a terminal feature enhancement module to enhance the feature map output by the original neck network, thereby achieving accurate recognition of rotated targets in the image to be detected.

[0047] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be achieved and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0049] Figure 1 A schematic diagram of a flow chart of an image detection method provided by an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of a binary image obtained by segmenting an edge image provided by an embodiment of the present invention;

[0051] Figure 3 A schematic diagram of the structure of a rotation image detection model provided by an embodiment of the present invention;

[0052] Figure 4 A schematic diagram of the structure of a dual-stream attention module provided in an embodiment of the present invention;

[0053] Figure 5 A schematic diagram of the structure of a first terminal feature enhancement module, a second terminal feature enhancement module or a third terminal feature enhancement module provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0055] A specific embodiment of the present invention discloses an image detection method, such as Figure 1 As shown, the image detection method includes:

[0056] Step S1: extracting edges of the image to be detected to obtain an edge image, and segmenting the edge image according to a preset segmentation pixel threshold to obtain a binary image;

[0057] Step S2: extracting multiple contours from the binary image, each contour including multiple pixel points; calculating the rotation angle of each contour according to the multiple pixel points of each contour;

[0058] Step S3: Compare the rotation angle of each contour with the preset angle threshold to determine whether there is at least one contour whose rotation angle is greater than the preset angle threshold; if yes, input the image to be detected into the rotation image detection model to obtain the predicted category and predicted rotation bounding box of each target in the image to be detected; if no, input the image to be detected into the horizontal image detection model to obtain the predicted category and predicted horizontal bounding box of each target.

[0059] Specifically, Figure 1 As shown, in step S1, edge extraction is performed on the image to be detected to obtain an edge image. The edge image is the position where the grayscale changes most dramatically in the image to be detected, that is, the place where the spatial gradient value is the largest. If the image to be detected is a color image, it is necessary to convert the color image into a grayscale image before edge extraction. Specifically, the preset segmentation pixel threshold is reasonably set according to the actual situation, and the edge image is segmented to obtain a binary image, such as Figure 2 shown.

[0060] Preferably, the step of extracting edges from the image to be detected to obtain an edge image, and segmenting the edge image according to a preset segmentation pixel threshold to obtain a binary image includes:

[0061] Use the preset horizontal Sobel convolution kernel and the preset vertical Sobel convolution kernel to perform convolution operations on the image to be detected respectively to obtain a horizontal edge image and a vertical edge image; merge the horizontal edge image and the vertical edge image to obtain an intermediate edge image;

[0062] Normalizing the middle edge image, and then restoring the normalized middle edge image to a normal pixel value to obtain an edge image;

[0063] The pixel value of each pixel in the edge image is compared with the preset segmentation pixel threshold, and the pixels greater than the preset segmentation pixel threshold are regarded as the first category, and the pixels less than or equal to the preset segmentation pixel threshold are regarded as the second category, so as to obtain a binary image.

[0064] Specifically, the preset horizontal Sobel convolution kernel can be: The preset vertical Sobel convolution kernel can be: A horizontal edge image is obtained by convolving the image to be detected with a preset horizontal Sobel convolution kernel, and a vertical edge image is obtained by convolving the image to be detected with a preset vertical Sobel convolution kernel.

[0065] Specifically, the horizontal edge image and the vertical edge image may be combined to obtain the intermediate edge image in the following manner:

[0066]

[0067] Among them, E represents the pixel value of any pixel in the middle edge image, Gx represents the pixel value of the corresponding pixel in the horizontal edge image, and Gy represents the pixel value of the corresponding pixel in the vertical edge image.

[0068] At this time, the pixel values ​​in the intermediate edge image may be normalized to a range of 0 to 1, and then the normalized pixel values ​​may be multiplied by 255 to restore to normal pixel values, thereby obtaining an edge image.

[0069] Exemplarily, the preset segmentation pixel threshold is set to 60, pixels greater than the preset segmentation pixel threshold are set to 255, 255 is used as the first type of pixels, and pixels less than or equal to the preset segmentation pixel threshold are set to 0, 0 is used as the second type of pixels, such as Figure 2 The binary image is shown.

[0070] Specifically, Figure 1 As shown, in step S2, multiple contours in the binary image are extracted, and the rotation angle of each contour is calculated.

[0071] Preferably, the extracting of multiple contours from the binary image, each contour comprising multiple pixel points, comprises:

[0072] According to the preset connected regions, pixels in the binary image that are adjacent and whose pixel values ​​all belong to the first category are taken as a connected region to obtain a plurality of connected regions; wherein the preset connected regions include eight-connected or four-connected;

[0073] In each connected region, starting from any pixel of the first category, all adjacent pixels of the first category are marked sequentially along the boundary of each connected region to obtain the contour corresponding to each connected region;

[0074] Compare the sizes of several contours, delete the largest contour, and use the remaining contours as the final multiple contours.

[0075] Specifically, the preset connected region can be set to 4-connected or 8-connected, and the pixels with a pixel value of 255 and coordinates satisfying the preset connected region are taken as a connected region to obtain a plurality of connected regions, each of which represents a different object or parts of an object. Specifically, in each connected region, starting from any pixel, all adjacent pixels of the first category are sequentially marked along the boundary of each connected region to obtain the contour corresponding to each connected region.

[0076] It is worth noting that, among the several contours obtained, the largest contour is the contour of the image to be detected. The largest contour is deleted, and the remaining contours are the contours of each target object included in the image to be detected.

[0077] Specifically, in step S2, the rotation angle of the remaining contour is calculated.

[0078] Preferably, the step of calculating the rotation angle of each contour according to the plurality of pixel points of each contour to obtain the rotation angles of the plurality of contours comprises:

[0079] Calculate the centroid of each contour according to the multiple pixel points of each contour, and calculate the covariance matrix of each contour according to the centroid of each contour and the multiple pixel points of each contour;

[0080] Calculate the maximum eigenvalue corresponding to each contour according to the covariance matrix of each contour, and calculate the main direction vector according to the covariance matrix and the maximum eigenvalue of each contour;

[0081] Calculates the rotation angle based on the principal direction vector.

[0082] Specifically, the n pixels of each contour are set to (x1, y1), (x2, y2), ..., (x n ,y n );

[0083] The centroid of each contour is for:

[0084] The covariance matrix Σ of each profile is:

[0085]

[0086] The maximum eigenvalue λ1 of each contour is: And there exists: Σv1=λ1v1, where v1 represents the main direction vector.

[0087] After calculating the main direction vector v1, the rotation angle is calculated using the following formula:

[0088]

[0089] Among them, θ represents the rotation angle of the contour, v 1y 、v 1x Represent the x and y components of v1 respectively.

[0090] Specifically, Figure 1 As shown, in step S3, the preset angle threshold can be set to 30°, and the rotation angle of each contour obtained in step S2 is compared with the preset angle threshold. According to the comparison result, the image to be detected is input into different image detection models.

[0091] Specifically, the rotational image detection model can predict the category and rotational bounding box of each target in the image based on the input image; the horizontal image detection model can predict the category and horizontal bounding box of each target in the image based on the input image.

[0092] Specifically, if the rotation angle of each contour is less than or equal to the preset angle threshold, the image to be detected is input into the horizontal image detection model, and the predicted category and predicted horizontal bounding box of each target in the image to be detected are predicted. It can be understood that the horizontal image detection model is a model for detecting images in the prior art. Exemplarily, the horizontal image detection model can be a YOLO model, an R-CNN model, or a Fast R-CNN model.

[0093] Specifically, if at least one of the rotation angles of each contour is greater than a preset angle threshold, the image to be detected is input into the rotation image detection model to predict the predicted category and predicted rotation bounding box of each target in the detection image.

[0094] Preferably, the step of inputting the image to be detected into a rotation image detection model to obtain a predicted category and a predicted rotation bounding box of each target in the image to be detected includes:

[0095] The image to be detected is input into the rotation image detection model to obtain the predicted category and predicted rotation bounding box information of each target in the image to be detected; the predicted rotation bounding box information includes the predicted center point coordinates, predicted covariance matrix and predicted rotation angle of the rotation bounding box; wherein the rotation image detection model is obtained by training based on the improved YOLOv8 neural network structure, and the improved YOLOv8 neural network structure includes a dual-stream attention module and a terminal feature enhancement module, the dual-stream attention module is used to enhance the channel information and feature information when the original backbone network generates a feature map, and the terminal feature enhancement module is used to enhance the feature of the feature map output by the original neck network;

[0096] The coordinates of the four corner points of each target are calculated according to the predicted rotation bounding box information of each target, and the coordinates of the four corner points of each target are connected as the predicted rotation bounding box of each target.

[0097] Specifically, the rotated image detection model is trained based on the improved YOLOv8 neural network structure. The improved YOLOv8 neural network structure adds a dual-stream attention module and a terminal feature enhancement module on the basis of the YOLOv8 neural network structure.

[0098] Specifically, the dual-stream attention module acts on the original backbone network. When the original backbone network extracts the features of the image to be detected, the channel information and feature information of the feature map generated in the middle are enhanced, the feature map is focused, and the recognition accuracy of the image to be detected is improved.

[0099] Specifically, the terminal feature enhancement module acts on the original neck network to further enhance the feature map output by the original neck network to the detection head, thereby improving the recognition accuracy of the image to be detected.

[0100] It is worth noting that by predicting the image to be detected by the rotation image detection model to obtain the rotation bounding box of each target in the image to be detected, the interference of each target in the image to be detected can be effectively reduced, and the detection accuracy of each target can be improved.

[0101] Specifically, Figure 3 As shown, the rotation image detection model includes a backbone network, a neck network and a detection head;

[0102] The backbone network includes an original backbone network and a dual-stream attention module; the original backbone network is used to extract features from the input image to be detected, obtain the third feature map, the fourth feature map and the fifth feature map and output them to the neck network; the dual-stream attention module enhances channel information and feature information when the original backbone network extracts features to generate the third feature map to the fifth feature map;

[0103] The neck network includes an original neck network and a terminal feature enhancement module; the original neck network is used to perform feature fusion on the third feature map, the fourth feature map and the fifth feature map to obtain the second fused feature map, the third fused feature map and the fourth fused feature map and output them to the terminal feature enhancement module; the terminal feature enhancement module is used to perform feature enhancement on the second fused feature map, the third fused feature map and the fourth fused feature map respectively to obtain the enhanced fifth fused feature map, the sixth fused feature map and the seventh fused feature map and output them to the detection head;

[0104] The detection head is used to predict the predicted category and predicted rotation bounding box information of each target in the image to be detected according to the fifth fused feature map, the sixth fused feature map and the seventh fused feature map.

[0105] Specifically, Figure 3 As shown, the backbone network extracts features from the image to be detected input to the rotation image detection model to obtain the first feature map P1, the second feature map P2, the third feature map P3, the fourth feature map P4 and the fifth feature map P5. The original backbone network extracts features from the input image to be detected, and the dual-stream attention module enhances the channel information and feature information when the original backbone network extracts features to generate the third to fifth feature maps.

[0106] Preferably, if Figure 3 As shown, the backbone network includes a first convolution module, a first feature extraction layer, a second feature extraction layer, a third feature extraction layer and a fourth feature extraction layer connected in sequence, and outputs a first feature map, a second feature map, a third feature map, a fourth feature map and a fifth feature map respectively;

[0107] The first feature extraction layer, the second feature extraction layer and the third feature extraction layer have the same structure, and all include a second convolution module and a first residual convolution module connected in sequence; the fourth feature extraction layer includes a second convolution module, a first residual convolution module, a dual-stream attention module and a spatial pyramid pooling fast module SPPF connected in sequence.

[0108] Specifically, Figure 3 As shown, the structures of the first feature extraction layer, the second feature extraction layer and the third feature extraction layer are the same, and all include a second convolution module and a first residual convolution module connected in sequence; the fourth feature extraction layer includes a second convolution module, a first residual convolution module, a dual-stream attention module and a spatial pyramid pooling fast module SPPF connected in sequence.

[0109] It is worth mentioning that the dual-stream attention module acts on the fourth feature extraction layer to enhance the deep feature map generated by the backbone network.

[0110] Specifically, Figure 3 As shown, the image to be detected passes through the first convolution module to generate a first feature map P1, the first feature map P1 passes through the first feature extraction layer to generate a second feature map P2, the second feature map passes through the second feature extraction layer to generate a third feature map P3, and the third feature map P3 passes through the third feature extraction layer to generate a fourth feature map P4.

[0111] Specifically, when the fourth feature map P4 passes through the fourth feature extraction layer, it passes through the second convolution module and the first residual convolution module in sequence to obtain the sixth feature map P6. The sixth feature map P6 is enhanced with the channel information and feature information by the dual-stream attention module to obtain the seventh feature map P7. The seventh feature map P7 is passed through the spatial pyramid pooling fast module SPPF to obtain the fifth feature map P5 and output it to the neck network.

[0112] For example, Figure 3 As shown in the figure, the size of the image to be detected input to the rotation image detection model is 640*640*3. After the convolution operation, batch normalization operation and Swish activation function operation of the first convolution module, a feature map of 320*320*64 is obtained as the first feature map P1; the first feature map P1 is subjected to convolution operation, batch normalization operation and Swish activation function operation by the second convolution module of the first feature extraction layer, and the residual convolution operation is performed by the first residual convolution module to obtain a feature map of 160*160*128 as the second feature map P2; the second feature map P2 is subjected to convolution operation, batch normalization operation and Swish activation function operation by the second convolution module of the second feature extraction layer, and the residual convolution operation is performed by the first residual convolution module to obtain a feature map of 80*80*256 as the second feature map P 3; The third feature map P3 is subjected to convolution operation, batch normalization operation and Swish activation function operation by the second convolution module of the third feature extraction layer, and is subjected to residual convolution operation by the first residual convolution module to obtain a 40*40*512 feature map as the fourth feature map P4; The fourth feature map P4 is subjected to convolution operation, batch normalization operation and Swish activation function operation by the second convolution module of the fourth feature extraction layer, and is subjected to residual convolution operation by the first residual convolution module to obtain a 20*20*512 feature map as the sixth feature map P6; The sixth feature map P6 is subjected to channel information and feature information enhancement by the dual-stream attention module to obtain a 20*20*512 feature map as the seventh feature map P7; The seventh feature map P7 is subjected to SPPF to obtain a 20*20*512 feature map as the fifth feature map.

[0113] Preferably, if Figure 4 As shown, the dual-stream attention module includes a left branch, a right branch and a first matrix addition layer;

[0114] The left branch is used to enhance the channel information of the input sixth feature map, obtain the eighth feature map and input it to the first matrix addition layer;

[0115] The right branch is used to enhance the feature information of the input sixth feature map, obtain the ninth feature map and input it into the first matrix addition layer;

[0116] The eighth feature map and the ninth feature map are added in the first matrix addition layer to obtain the seventh feature map.

[0117] Specifically, in the dual-stream attention module, the left branch enhances the channel information of the sixth feature map, the right branch enhances the feature information of the sixth feature map, and the first matrix addition layer adds the eighth feature map and the ninth feature map to obtain the seventh feature map.

[0118] Preferably, if Figure 4 As shown, the left branch includes a maximum pooling layer, an average pooling layer, a first fully connected layer, a second fully connected layer, a second matrix addition layer, an activation function layer and a multiplication module;

[0119] The sixth feature map is subjected to the maximum pooling layer and the first fully connected layer to obtain the tenth feature map, and the sixth feature map is subjected to the average pooling layer and the second fully connected layer to obtain the eleventh feature map;

[0120] The tenth feature map and the eleventh feature map are added in the second matrix addition layer, and the added feature maps are passed through the activation function layer to obtain the twelfth feature map;

[0121] The twelfth feature map and the sixth feature map are multiplied in the multiplication module to obtain the eighth feature map.

[0122] Specifically, the maximum pooling layer and the average pooling layer in the left branch are pooling operations performed on the channel axis of the sixth feature map. Exemplarily, the sixth feature map P6 input to the left branch is width*height*channel, expressed as H1*W1*C1, then the output dimension after passing through the maximum pooling layer and the average pooling layer is 1*1*C1, and the subsequent first fully connected layer, second fully connected layer, second matrix addition layer, and activation function layer do not change the feature dimension. Finally, the input sixth feature map P6, which is H1*W1*C1, is multiplied by the multiplication module with the twelfth feature map output by the activation function layer. Figure 1 *1*C1, and multiply them to obtain the enhanced eighth feature map P8, which is H1*W1*C1.

[0123] Preferably, if Figure 4 As shown, the right branch includes a maximum pooling layer, an average pooling layer, a matrix concatenation layer, a convolution layer, an activation function layer and a multiplication module;

[0124] The sixth feature map is passed through the maximum pooling layer to obtain the thirteenth feature map, and the sixth feature map is passed through the average pooling layer to obtain the fourteenth feature map;

[0125] The thirteenth feature map and the fourteenth feature map are spliced ​​in the matrix splicing layer, and the spliced ​​feature maps are sequentially passed through the convolution layer and the activation function layer to obtain the fifteenth feature map;

[0126] The fifteenth feature map and the sixth feature map are multiplied in the multiplication module to obtain a ninth feature map.

[0127] Specifically, the maximum pooling layer and the average pooling layer in the right branch are pooling operations performed on the feature axis of the sixth feature map. Exemplarily, the sixth feature map P6 input to the right branch is width*height*channel, expressed as H1*W1*C1, then the output dimension after maximum pooling and average pooling is H1*W1*1, the subsequent matrix splicing layer splices the dimension to H1*W1*2, and the convolution layer reduces the dimension to H1*W1*1, the activation function layer does not change the dimension, and finally the multiplication module is used to multiply the input sixth feature map P6, which is H1*W1*C1, with the fifteenth feature map P15, which is H1*W1*1, output by the activation function layer, to obtain the enhanced ninth feature map, which is H1*W1*C1.

[0128] Specifically, the activation function layers in the left branch and the right branch are Sigmod activation function layers.

[0129] It is worth mentioning that the left branch and the right side of the dual-stream attention module respectively enhance the channel information and feature information of the input sixth feature map P6, and finally obtain the final enhanced seventh feature map P7 through the first matrix addition layer, thereby improving the feature extraction capability of the rotated image detection model and further improving the recognition accuracy of the image to be detected.

[0130] Specifically, Figure 4 As shown, the backbone network inputs the obtained third feature map P3, fourth feature map P4 and fifth feature map P5 into the neck network, the fifth feature map P5 is spliced ​​and fused with the fourth feature map through the first sampling fusion layer to obtain the first fused feature map S1; the first fused feature map S1 is spliced ​​and fused with the third feature map P3 through the second sampling fusion layer to obtain the second fused feature map S2; the second fused feature map S2 is spliced ​​and fused with the first fused feature map S1 through the first convolution fusion layer to obtain the third fused feature map S3; the third fused feature map S3 is spliced ​​and fused with the fifth feature map P5 through the second convolution fusion layer to obtain the fourth fused feature map S4.

[0131] Specifically, the first sampling fusion layer and the second sampling fusion layer have the same structure, both including an upsampling layer, a splicing layer and a second residual convolution module connected in sequence; the first convolution fusion layer and the second convolution fusion layer have the same structure, both including a third convolution module, a splicing layer and a third residual convolution module connected in sequence.

[0132] For example, Figure 3As shown, the third feature map P3 is 80*80*256, the fourth feature map P4 is 40*40*512, and the fifth feature map P5 is 20*20*512; the fifth feature map P5 is subjected to the upsampling layer of the first sampling fusion layer to obtain a feature map 40*40*512, which is input into the splicing layer, and is spliced ​​with the fourth feature map P4 in the splicing layer to obtain a feature map 40*40*512, and is subjected to the second residual convolution module to obtain a feature map 40*40*512 as the first fused feature map S1; similarly, the first fused feature map S1 is subjected to the upsampling layer of the second sampling fusion layer to obtain a feature map 80*80*512, which is input into the splicing layer, and is spliced ​​with the third feature map P3 in the splicing layer to obtain a feature map 80*80*768, and is subjected to the second residual convolution module to obtain a feature map 80*80*256 as the second fused feature map S2. Specifically, the second fused feature map S2 passes through the third convolution module of the first convolution fusion layer to obtain a feature map 40*40*256, which is input into the splicing layer, and is spliced ​​with the first fused feature map S1 in the splicing layer to obtain a feature map 40*40*768, and passes through the third residual convolution module to obtain a feature map 40*40*512, which is used as the third fused feature map S3; similarly, the third fused feature map S3 passes through the third convolution module of the second convolution fusion layer to obtain a feature map 20*20*512, which is input into the splicing layer, and is spliced ​​with the fifth feature map P5 in the splicing layer to obtain a feature map 20*20*1024, and passes through the third residual convolution module to obtain a feature map 20*20*512, which is used as the fourth fused feature map S4.

[0133] Preferably, if Figure 3 As shown, the terminal feature enhancement module includes three terminal feature enhancement modules with the same structure: a first terminal feature enhancement module, a second terminal feature enhancement module and a third terminal feature enhancement module;

[0134] The first terminal feature enhancement module, the second terminal feature enhancement module and the third terminal feature enhancement module respectively enhance the features of the input second fusion feature map, the third fusion feature map and the fourth fusion feature map, and output the fifth fusion feature map, the sixth fusion feature map and the seventh fusion feature map.

[0135] Specifically, the first end feature enhancement module, the second end feature enhancement module, and the third end feature enhancement module do not change the size of the input feature map. Figure 3 As shown, the second fused feature map S2, the third fused feature map S3 and the fourth fused feature map S4 are respectively subjected to the first terminal feature enhancement module, the second terminal feature enhancement module and the third terminal feature enhancement module to obtain the fifth fused feature map S5, the sixth fused feature map S6 and the seventh fused feature map S7.

[0136] Preferably, if Figure 5As shown, the first terminal feature enhancement module, the second terminal feature enhancement module or the third terminal feature enhancement module all include a maximum pooling layer, a third fully connected layer, a first activation function layer, a fourth fully connected layer, a second activation function layer and a multiplication module;

[0137] The input feature map is sequentially passed through the maximum pooling layer, the third fully connected layer, the first activation function layer, the fourth fully connected layer, and the second activation function layer to obtain the intermediate feature map;

[0138] The intermediate feature map and the input feature map are multiplied in the multiplication module to obtain the output feature map.

[0139] Specifically, the maximum pooling layer of the first terminal feature enhancement module, the second terminal feature enhancement module, or the third terminal feature enhancement module is a pooling operation performed on the channel axis of the input feature map. For example, the input feature map is expressed as H2*W2*C2, then the output dimension after maximum pooling is 1*1*C2, and the subsequent third fully connected layer, the first activation function layer, the fourth fully connected layer, and the second activation function layer do not change the size of the feature map to obtain the intermediate feature map. Figure 1 *1*C2, the feature map H2*W2*C2 input in the multiplication module and the intermediate feature Figure 1 *1*C2 are multiplied to obtain the enhanced feature map H2*W2*C2, that is, the output feature map is H2*W2*C2.

[0140] It is worth noting that the first terminal feature enhancement module, the second terminal feature enhancement module or the third terminal feature enhancement module enhances the feature map input to the detection head, thereby improving the recognition accuracy of the rotated image detection model for the image to be detected.

[0141] Specifically, in the first terminal feature enhancement module, the second terminal feature enhancement module or the third terminal feature enhancement module, the first activation function layer is a Swish activation function layer, and the second activation function layer is a Sigmod activation function layer.

[0142] Specifically, Figure 3 As shown, the fifth fused feature map S5, the sixth fused feature map S6 and the seventh fused feature map S7 are input to the detection head, and are predicted by the first prediction layer, the second prediction layer and the third prediction layer respectively, and all the prediction results are analyzed in the post-processing layer to obtain the predicted category and predicted rotated bounding box information of each target in the image to be detected.

[0143] Specifically, the coordinates of the four corner points of each target are calculated according to the predicted rotated bounding box information of each target, and the coordinates of the four corner points of each target are connected to serve as the predicted rotated bounding box of each target.

[0144] Preferably, the coordinates of the four corner points of each target are calculated by the following formula:

[0145]

[0146] in, Represents the predicted center point coordinates of each target, represents the prediction covariance matrix of each target, θ pred represents the predicted rotation angle of each target, (x top-final ,y top-final )、(x right-final ,y right-final )、(x bottom-final ,y bottom-final ) and (x left-final ,y left-final ) represents the coordinates of the four corner points.

[0147] Preferably, the rotation image detection model is trained by the following steps:

[0148] Acquire a number of radiographic images, preprocess the number of radiographic images, and mark the category labels of each target in each preprocessed radiographic image and the coordinates of the four corner points of the rotation bounding box;

[0149] Calculate the center point coordinate label and covariance matrix label of the rotation bounding box of each target according to the coordinates of the four corner points of the rotation bounding box of each target; take a ray image and the category label, center point coordinate label and covariance matrix label corresponding to each target marked in the image as a training sample to obtain several training samples; divide the several training samples into a training set and a validation set;

[0150] The preset neural network model is trained for a preset number of iterations using the training set, and a trained rotated image detection model is determined based on the evaluation results of the validation set at the end of each iteration.

[0151] Specifically, when training the rotation image detection model, the category of each target in the preprocessed radiographic image is annotated to determine the category label of each target in a one-hot label format, such as [0,1,0,0,0,0], indicating that all targets include a total of 6 categories, and this target belongs to the second category.

[0152] Specifically, the coordinates of the four corner points of the rotated bounding box refer to the coordinates of the four corner points of the rotated bounding box. After marking each target, the coordinates of the four corner points and the category label of each target can be determined.

[0153] Specifically, according to the coordinates of the four corner points of each target, the center point coordinates and the covariance matrix of the rotation bounding box of each target are calculated as the center point coordinate label and the covariance matrix label of each target.

[0154] Specifically, the center point coordinates and covariance matrix of the rotation bounding box of each target are calculated by the following formula:

[0155]

[0156] Among them, (x0, y0) represents the coordinates of the center point, (x1, y1), (x2, y2), (x3, y3) and (x4, y4) represent the coordinates of the four corner points, ∑ represents the covariance matrix, k = 1, 2, 3, 4.

[0157] Specifically, the category label, center point coordinate label and covariance matrix label of each target in each radiation image are obtained, and a radiation image and the category labels, center point coordinate labels and covariance matrix labels of each target in the image are used as a training sample to obtain several training samples, and the several training samples are divided into a training set and a validation set.

[0158] Specifically, the training set is used to train the rotation image detection model for a preset number of iterations to obtain a neural network model for each iteration. At the same time, the validation set is used to evaluate the neural network model for each iteration to obtain an evaluation result for each iteration. The evaluation results of the neural network models of all iterations are compared, and the neural network model corresponding to the optimal evaluation result is used as the trained rotation image detection model.

[0159] Exemplarily, the preset number of iterations is set to 100 in advance, 100 neural network models are obtained, and corresponding evaluation results are obtained at the same time. The 100 evaluation results are compared, and the neural network model with the best evaluation result is selected as the trained rotated image detection model.

[0160] Specifically, at the beginning of each iteration process, multiple training samples are randomly selected from the training set, and the multiple training samples are divided into multiple groups. For example, 10,000 training samples are selected and divided into 10 groups, with 1,000 training samples in each group. The training order of the 10 groups is determined, and the rotation image detection model is trained using each group of training samples in turn.

[0161] Specifically, after each group of training samples is input into the rotation image detection model, the predicted categories, predicted center point coordinates and predicted covariance matrix of all targets in each group of training samples are obtained, the loss function value of each group of training samples is calculated, and the weight parameters of the rotation image detection model are updated according to the loss function value. The updated rotation image detection model is used as the rotation image detection model for the next group of training samples, and the training process of the next group of training samples is started until all groups of training samples are trained.

[0162] Specifically, determine the category labels and predicted categories of all targets in each group of training samples, and calculate the category loss value using the following formula:

[0163]

[0164] Among them, L BCE represents the category loss value, M represents the number of all targets in all training samples of the current group, N represents the number of categories, and y i，j represents the predicted category value of the i-th category of the j-th target, p i，j Represents the class label value of the i-th class of the j-th target.

[0165] Specifically, determine the center point coordinate labels and covariance matrix labels of all targets in each group of training samples, as well as the predicted center point coordinates and predicted covariance matrix, and calculate the rotation bounding box loss value using the following formula:

[0166]

[0167] Among them, L ProbIoU represents the rotation bounding box loss value, BD represents the distance function, represents the predicted center point coordinates of the jth target, represents the predicted covariance matrix of the jth target, The center point coordinate label of the jth target, Denotes the covariance matrix label of the j-th target.

[0168] After obtaining the category loss value and rotation bounding box loss value of each group of training samples, the loss function value is calculated by the following formula:

[0169] L=ω1L BCE +ω2L probIoU ;

[0170] 1 = ω1 + ω2;

[0171] Among them, L represents the loss function value, ω1 represents the preset category weight value, and ω2 represents the preset rotation bounding box weight value.

[0172] Specifically, the loss function value corresponding to each group of training samples is calculated, and the weight parameters of the rotation image detection model are updated according to the preset gradient descent training method to obtain the updated rotation image detection model.

[0173] After all groups of training samples have completed training, the rotation image detection model obtained by the last group of training samples is evaluated using the validation set to obtain the evaluation results of this iteration.

[0174] Preferably, the preset gradient descent training method is any one of the following:

[0175] Stochastic gradient descent;

[0176] Adam gradient descent algorithm;

[0177] Batch Gradient Descent.

[0178] Specifically, the stochastic gradient descent (SGD) method has high computational efficiency, strong adaptability, and can jump out of the local minimum, which helps the rotation image detection model jump out of the local minimum and find the optimal solution.

[0179] Specifically, the Adam gradient descent algorithm combines the characteristics of the momentum gradient descent method and the RMSProp algorithm. It updates the model parameters through adaptive learning rate and second-order moment estimation, which can accelerate the convergence speed of the rotation image detection model in the gradient descent process and avoid falling into the local optimal solution.

[0180] Specifically, batch gradient descent has stable convergence and can obtain a stable rotation image detection model more quickly.

[0181] Specifically, after obtaining the evaluation result of this iteration, the rotation image detection model obtained in this iteration process is used to start the training process of the next iteration until the number of iterations meets the preset number of iterations. The evaluation results of the preset number of iterations are compared, and the rotation image detection model corresponding to the best evaluation result is selected as the trained rotation image detection model.

[0182] Specifically, Figure 1 As shown, in step S3, a suitable image detection model is selected according to the rotation angle of each target object in the image to be detected, and the rotation image detection model and the horizontal image detection model are used to predict the image to be detected to obtain the category and bounding box of each target, thereby improving the detection accuracy of each target.

[0183] Compared with the prior art, an image detection method provided by an embodiment of the present invention compares the rotation angles of each contour in the image to be detected, and sets different detection models for the image to be detected according to the comparison result: if there is at least one contour in the image to be detected with a rotation angle greater than a preset angle threshold, the detection accuracy of the image to be detected by using the rotation angle detection model is high; otherwise, the image to be detected is detected by using the horizontal image detection model, which improves the detection accuracy of target objects in complex scenes as a whole; at the same time, the image to be detected is predicted by the rotation image detection model to obtain the predicted category and predicted rotation bounding box of each target in the image to be detected, and the rotation bounding box can effectively constrain each target in the image, reduce the background part in each target, make the bounding box of each target clearer, and improve the recognition accuracy of each target in the image; and, by adding a dual-stream attention module to the rotation image detection model on the basis of the YOLOv8 neural network structure, the channel information and feature information are enhanced when the original backbone network generates a feature map, and the terminal feature enhancement module is added to enhance the feature map output by the original neck network, thereby realizing accurate recognition of rotated targets in the image to be detected.

[0184] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0185] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. An image detection method, characterized in that: The image detection method comprises: Perform edge extraction on the image to be detected to obtain an edge image, and segment the edge image according to a preset segmentation pixel threshold to obtain a binary image; Extract multiple contours from the binary image, each contour includes multiple pixel points; calculate the rotation angle of each contour according to the multiple pixel points of each contour; The rotation angle of each contour is compared with the preset angle threshold to determine whether there is at least one contour whose rotation angle is greater than the preset angle threshold; if yes, the image to be detected is input into the rotation image detection model to obtain the predicted category and predicted rotation bounding box of each target in the image to be detected; if no, the image to be detected is input into the horizontal image detection model to obtain the predicted category and predicted horizontal bounding box of each target.

2. The image detection method according to claim 1, characterized in that: The step of extracting edges of the image to be detected to obtain an edge image, and segmenting the edge image according to a preset segmentation pixel threshold to obtain a binary image comprises: Use the preset horizontal Sobel convolution kernel and the preset vertical Sobel convolution kernel to perform convolution operations on the image to be detected respectively to obtain a horizontal edge image and a vertical edge image; merge the horizontal edge image and the vertical edge image to obtain an intermediate edge image; Normalizing the middle edge image, and then restoring the normalized middle edge image to a normal pixel value to obtain an edge image; The pixel value of each pixel in the edge image is compared with the preset segmentation pixel threshold, and the pixels greater than the preset segmentation pixel threshold are regarded as the first category, and the pixels less than or equal to the preset segmentation pixel threshold are regarded as the second category, so as to obtain a binary image.

3. The image detection method according to claim 1, characterized in that: The step of extracting multiple contours from the binary image, each contour comprising multiple pixel points, includes: According to the preset connected regions, pixels in the binary image that are adjacent and whose pixel values ​​all belong to the first category are taken as a connected region to obtain a plurality of connected regions; wherein the preset connected regions include eight-connected or four-connected; In each connected region, starting from any pixel of the first category, all adjacent pixels of the first category are marked sequentially along the boundary of each connected region to obtain the contour corresponding to each connected region; Compare the sizes of several contours, delete the largest contour, and use the remaining contours as the final multiple contours.

4. The image detection method according to claim 3, characterized in that: The step of calculating the rotation angle of each contour according to the plurality of pixel points of each contour to obtain the rotation angles of the plurality of contours comprises: Calculate the centroid of each contour according to the multiple pixel points of each contour, and calculate the covariance matrix of each contour according to the centroid of each contour and the multiple pixel points of each contour; Calculate the maximum eigenvalue corresponding to each contour according to the covariance matrix of each contour, and calculate the main direction vector according to the covariance matrix and the maximum eigenvalue of each contour; Calculates the rotation angle based on the principal direction vector.

5. The image detection method according to claim 1, characterized in that: The step of inputting the image to be detected into the rotation image detection model to obtain the predicted category and predicted rotation bounding box of each target in the image to be detected includes: The image to be detected is input into the rotation image detection model to obtain the predicted category and predicted rotation bounding box information of each target in the image to be detected; the predicted rotation bounding box information includes the predicted center point coordinates, predicted covariance matrix and predicted rotation angle of the rotation bounding box; wherein the rotation image detection model is obtained by training based on the improved YOLOv8 neural network structure, and the improved YOLOv8 neural network structure includes a dual-stream attention module and a terminal feature enhancement module, the dual-stream attention module is used to enhance the channel information and feature information when the original backbone network generates a feature map, and the terminal feature enhancement module is used to enhance the feature of the feature map output by the original neck network; The coordinates of the four corner points of each target are calculated according to the predicted rotation bounding box information of each target, and the coordinates of the four corner points of each target are connected as the predicted rotation bounding box of each target.

6. The image detection method according to claim 5, characterized in that: The rotation image detection model includes a backbone network, a neck network and a detection head; The backbone network includes an original backbone network and a dual-stream attention module; the original backbone network is used to extract features from the input image to be detected, obtain the third feature map, the fourth feature map and the fifth feature map and output them to the neck network; The dual-stream attention module enhances the channel information and feature information when the original backbone network performs feature extraction to generate the third to fifth feature maps; The neck network includes the original neck network and the terminal feature enhancement module; The original neck network is used to perform feature fusion on the third feature map, the fourth feature map and the fifth feature map to obtain the second fused feature map, the third fused feature map and the fourth fused feature map and output them to the terminal feature enhancement module; The terminal feature enhancement module is used to enhance the features of the second fusion feature map, the third fusion feature map and the fourth fusion feature map respectively, obtain the enhanced fifth fusion feature map, the sixth fusion feature map and the seventh fusion feature map and output them to the detection head; The detection head is used to predict the predicted category and predicted rotation bounding box information of each target in the image to be detected according to the fifth fused feature map, the sixth fused feature map and the seventh fused feature map.

7. The image detection method according to claim 6, characterized in that: The backbone network includes a first convolution module, a first feature extraction layer, a second feature extraction layer, a third feature extraction layer and a fourth feature extraction layer connected in sequence, and outputs a first feature map, a second feature map, a third feature map, a fourth feature map and a fifth feature map respectively; The first feature extraction layer, the second feature extraction layer and the third feature extraction layer have the same structure, and all include a second convolution module and a first residual convolution module connected in sequence; the fourth feature extraction layer includes a second convolution module, a first residual convolution module, a dual-stream attention module and a spatial pyramid pooling fast module SPPF connected in sequence.

8. The image detection method according to claim 5, characterized in that: The coordinates of the four corner points of each target are calculated using the following formula: in, Represents the predicted center point coordinates of each target, represents the prediction covariance matrix of each target, θ pred represents the predicted rotation angle of each object.

9. The image detection method according to claim 5, characterized in that: Train the rotated image detection model by following these steps: Acquire a number of radiographic images, preprocess the number of radiographic images, and mark the category labels of each target in each preprocessed radiographic image and the coordinates of the four corner points of the rotation bounding box; Calculate the center point coordinate label and covariance matrix label of the rotation bounding box of each target according to the coordinates of the four corner points of the rotation bounding box of each target; take a ray image and the category label, center point coordinate label and covariance matrix label corresponding to each target marked in the image as a training sample to obtain a plurality of training samples; Divide several training samples into training sets and validation sets; The preset neural network model is trained for a preset number of iterations using the training set, and a trained rotated image detection model is determined based on the evaluation results of the validation set at the end of each iteration.

10. The image detection method according to claim 9, characterized in that: The method of using the training set to train the preset neural network model for a preset number of iterations, and determining a trained rotation image detection model according to the evaluation result of the validation set at the end of each iteration process, includes: In each iteration, multiple training samples are randomly selected from the training set, and the multiple training samples are divided into multiple groups. All the training samples in each group are used to train the preset neural network model in turn to obtain the predicted categories, predicted center point coordinates and predicted covariance matrix of all targets in all the training samples in each group; Calculate the loss function values ​​of all training samples in the current group, update the weight parameters of the preset neural network model in combination with the preset gradient descent training method, and use the updated preset neural network model to start training the next group of training samples until all groups of training samples have completed training; After all groups of training samples have completed training, the preset neural network model obtained in this iteration process is evaluated using the samples in the validation set to obtain the evaluation results of this iteration; Determine whether the number of iterations meets the preset number of iterations. If so, use the preset neural network model corresponding to the optimal evaluation result as the trained rotation image detection model; if not, continue training for the next iteration process.

Images