Real-time automatic identification method and system for strong echo region of nigra based on YOLOV5, storage medium and electronic equipment

Abstract

The invention provides a real-time automatic identification method and system for a nigra strong echo region based on YOLOV5, a storage medium and electronic equipment. The method comprises the following steps: acquiring a craniocerebral ultrasonic scanning image of an examined patient; based on a nigra strong echo region recognition model, performing nigra strong echo region recognition on the craniocerebral ultrasonic scanning image to obtain a recognition result; wherein the nigra strong echo region recognition model is a model which is trained in advance based on a YOLOV5 algorithm; and outputting an identification result. The YOLOv5 algorithm has higher recognition precision and stronger generalization ability, and can better adapt to nigra strong echo regions of different sizes and shapes; in the identification of the nigra strong echo region, the target region, namely the nigra strong echo region, can be quickly and accurately captured from the medical image, the accuracy and efficiency of diagnosis are greatly improved, and misdiagnosis and missed diagnosis risks caused by human factors are also reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of computer data processing, and particularly relates to a real-time automatic recognition method, system, storage medium and electronic device for substantia nigra hyperechogenic area based on YOLOV5. Background Art

[0002] At present, in the field of medical image analysis, the recognition of the substantia nigra hyperechogenic area is of great significance for the early diagnosis of neurodegenerative diseases such as Parkinson's disease. However, existing recognition methods, such as those based on manual feature extraction and traditional machine learning techniques, are often limited by the complexity and accuracy of feature extraction; for example, some studies rely on the texture analysis of ultrasonic images, but this method will significantly reduce the generalization ability and accuracy of the method when facing different devices and individual differences of patients. In addition, traditional methods are computationally intensive and inefficient when dealing with high-dimensional data, which makes it difficult to meet the requirements of real-time recognition. Summary of the Invention

[0003] One of the purposes of the present invention is to provide a real-time automatic recognition method for the substantia nigra hyperechogenic area based on YOLOV5. The YOLOv5 algorithm has higher recognition accuracy and stronger generalization ability, and can better adapt to substantia nigra hyperechogenic areas of different sizes and shapes; in the recognition of the substantia nigra hyperechogenic area, the present application can quickly and accurately capture the target area, that is, the substantia nigra hyperechogenic area, from medical images, greatly improving the accuracy and efficiency of diagnosis, and also reducing the risk of misdiagnosis and missed diagnosis caused by human factors.

[0004] A real-time automatic recognition method for the substantia nigra hyperechogenic area based on YOLOV5 provided by an embodiment of the present invention includes:

[0005] Obtain the cranial ultrasound scan image of the patient under examination;

[0006] Based on the substantia nigra hyperechogenic area recognition model, perform the recognition of the substantia nigra hyperechogenic area on the cranial ultrasound scan image to obtain the recognition result; wherein, the substantia nigra hyperechogenic area recognition model is a model pre-trained based on the YOLOV5 algorithm;

[0007] Output the recognition result.

[0008] Optionally, the pre-training steps of the substantia nigra hyperechogenic area recognition model are as follows:

[0009] Improve the YOLOV5 algorithm to obtain the improved YOLOV5 algorithm;

[0010] Obtain training samples;

[0011] Preprocess the training samples to obtain the preprocessed training samples;

[0012] Based on the improved YOLOV5 algorithm, train the substantia nigra hyperechoic area recognition model according to the preprocessed training samples.

[0013] Optionally, improving the YOLOV5 algorithm to obtain the improved YOLOV5 algorithm includes:

[0014] Replacing the ordinary convolution Conv in the YOLOV5 algorithm with the Gsconv method;

[0015] And replacing the FPN structure in the YOLOV5 algorithm with the BiFPN structure;

[0016] And replacing the GioU loss function in the YOLOV5 algorithm with the CIoU loss function.

[0017] Optionally, obtaining the training samples includes:

[0018] Using an ultrasonic device to perform multi-angle scans on a large number of patients to obtain scan images; among them, the scan images include normal images and hyperechoic area images; the scan images contain echo areas with different depths, sizes, and shapes;

[0019] Taking the scan images as training samples.

[0020] Optionally, preprocessing the training samples to obtain the preprocessed training samples includes:

[0021] Performing grayscale processing on the training samples;

[0022] And performing denoising processing on the training samples using a Gaussian filter;

[0023] And performing normalization processing on the training samples.

[0024] Optionally, based on the improved YOLOV5 algorithm, training the substantia nigra hyperechoic area recognition model according to the preprocessed training samples includes:

[0025] Annotating the preprocessed training samples; among them, the annotated categories include: normal substantia nigra area and substantia nigra area with hyperecho;

[0026] Dividing the preprocessed training samples after annotation into a training set and a test set according to a ratio of 8:2;

[0027] Based on the improved YOLOV5 algorithm, training an initial recognition model according to the training set;

[0028] Testing the initial recognition model based on the test set;

[0029] Taking the initial recognition model that passes the test as the substantia nigra hyperechoic area recognition model.

[0030] Optionally, the CIoU loss function includes:

[0031]

[0032] where b, b gt are the center points of the predicted bounding box and the ground truth bounding box respectively, v is the similarity of the aspect ratio, a is the weight coefficient, IoU is the intersection over union, ω gt and h gt are the width and height of the ground truth bounding box respectively, ω and h are the width and height of the predicted bounding box respectively, L CIoU is the CIoU loss, ρ is the Euclidean distance function, and e is the diagonal length of the minimum rectangle of the predicted bounding box and the ground truth bounding box.

[0033] A real-time automatic identification system for substantia nigra hyperechoic area based on YOLOV5 provided by an embodiment of the present invention includes:

[0034] An image acquisition module, configured to acquire a cranial ultrasound scan image of an examined patient;

[0035] A substantia nigra hyperechoic area identification module, configured to identify the substantia nigra hyperechoic area in the cranial ultrasound scan image based on a substantia nigra hyperechoic area identification model, and obtain an identification result; wherein, the substantia nigra hyperechoic area identification model is a model pre-trained based on the YOLOV5 algorithm;

[0036] A result output module, configured to output the identification result.

[0037] A computer-readable storage medium provided by an embodiment of the present invention, on which a computer program is stored, and a processor executes the computer program to implement the method described in any one of the above.

[0038] An electronic device provided by an embodiment of the present invention, the electronic device includes a memory and a processor, a computer program is stored in the memory, and the processor executes the computer program to implement the method described in any one of the above.

[0039] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained by the structures specifically pointed out in the written specification and the drawings.

[0040] The technical solutions of the present invention will be further described in detail below with reference to the drawings and embodiments. Description of the Drawings

[0041] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the accompanying drawings:

[0042] Figure 1 It is a flowchart of a real-time automatic recognition method for the substantia nigra hyperechoic area based on YOLOV5 in an embodiment of the present invention;

[0043] Figure 2 It is a structural diagram of the Gsconv method in an embodiment of the present invention;

[0044] Figure 3 It is a schematic diagram of the BiFPN structure in an embodiment of the present invention;

[0045] Figure 4 It is a structural diagram of the improved YOLOV5 algorithm in an embodiment of the present invention;

[0046] Figure 5 It is an implementation flowchart of a real-time automatic recognition method for the substantia nigra hyperechoic area based on YOLOV5 in an embodiment of the present invention;

[0047] Figure 6 It is a schematic diagram of a real-time automatic recognition system for the substantia nigra hyperechoic area based on YOLOV5 in an embodiment of the present invention. Detailed implementation manners

[0048] The following describes the preferred embodiments of the present invention with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0049] An embodiment of the present invention provides a real-time automatic recognition method for the substantia nigra hyperechoic area based on YOLOV5, as Figure 1 shown, including:

[0050] S1. Obtain the cranial ultrasound scan image of the patient under examination;

[0051] S2. Based on the substantia nigra hyperechoic area recognition model, perform recognition on the cranial ultrasound scan image to obtain a recognition result; wherein, the substantia nigra hyperechoic area recognition model is a model pre-trained based on the YOLOV5 algorithm;

[0052] S3. Output the recognition result;

[0053] Among them, the pre-training steps of the substantia nigra hyperechoic area recognition model are as follows:

[0054] Improve the YOLOV5 algorithm to obtain an improved YOLOV5 algorithm;

[0055] Obtain training samples;

[0056] Preprocess the training samples to obtain the preprocessed training samples;

[0057] Based on the improved YOLOV5 algorithm, train the substantia nigra hyperechogenic area recognition model according to the preprocessed training samples.

[0058] In the above technical solution, the cranial ultrasound scan images of the patient include ultrasound pictures and videos, both of which are acquired on the ultrasound device for examining the patient; the substantia nigra hyperechogenic area recognition model performs substantia nigra hyperechogenic area recognition on the cranial ultrasound scan images. If the substantia nigra hyperechogenic area is recognized, the recognition result includes the substantia nigra hyperechogenic area, and the recognition result is output to remind the doctor to pay attention to this result in time, which is convenient for the doctor to conduct subsequent analysis and research, so as to provide a more accurate treatment plan for the patient.

[0059] When training the substantia nigra hyperechogenic area recognition model, first improve the basic YOLOV5 algorithm, preprocess the training samples for training, and train the substantia nigra hyperechogenic area recognition model based on the improved YOLOV5 algorithm according to the preprocessed training samples.

[0060] Compared with traditional image recognition methods, the YOLOv5 algorithm has higher recognition accuracy and stronger generalization ability, and can better adapt to substantia nigra hyperechogenic areas of different sizes and shapes; in the recognition of the substantia nigra hyperechogenic area, this application can quickly and accurately capture the target area, that is, the substantia nigra hyperechogenic area, from medical images, greatly improving the accuracy and efficiency of diagnosis, and also reducing the risk of misdiagnosis and missed diagnosis caused by human factors.

[0061] In one embodiment, the improvement of the YOLOV5 algorithm to obtain the improved YOLOV5 algorithm includes:

[0062] Replace the ordinary convolution Conv in the YOLOV5 algorithm with the Gsconv method;

[0063] And replace the FPN structure in the YOLOV5 algorithm with the BiFPN structure;

[0064] And replace the GioU loss function in the YOLOV5 algorithm with the CIoU loss function.

[0065] In the above technical solution, YOLOv5 uses a splicing method to fuse features. After three different splicing operations, three feature maps of different sizes are obtained, and then the CSP structure and convolution operations are added respectively, and predictions are made on the final three feature maps. The YOLOv5 network mainly includes four modules: the input end, the backbone network, the Neck network, and the Head output layer (Prediction). The backbone network of YOLOv5 is divided into three layers, and each layer is mainly composed of the CBH and CSP structures. YOLOv5 proposes the Focus structure in the first layer of the backbone network and adds the SPP layer in the third layer. The CSP structure with residual components is used in the backbone network, and the residual components are replaced by convolution operations in the neck; the CSP structure divides the feature map into two parts, one part continues to perform convolution operations to obtain more profound feature information, and the other part is spliced with the feature map after the previous part of the convolution operation.

[0066] As Figure 2 shown, to meet the requirement of ensuring accuracy while achieving fast detection on edge devices, the method of Gsconv is used. Gsconv not only has high-efficient feature extraction ability but also can effectively process multi-scale information, thus improving the accuracy while maintaining the detection speed. The core idea of Gsconv is to extract feature information in the image through a special convolution operation and simultaneously process targets of different scales. This structure enables Gsconv to capture the detailed information in the image more comprehensively, thereby improving the accuracy of object detection.

[0067] As Figure 3 shown, in order to reduce the computational cost and the number of parameters while ensuring accuracy, an optimization method of cross-scale connection (BiFPN) is used: (1) Nodes with only unilateral input are deleted. Since these nodes do not perform feature fusion and contribute less to the network, a simplified bidirectional network can be obtained by deleting these nodes. (2) A skip connection is added between the input node and the output node at the same scale. In this way, within the same layer, more feature information can be fused without adding too much computational cost. This skip connection helps to enhance the transmission and fusion of features and improve the representation ability of the model. (3) Each top-down and bottom-up path is regarded as a feature layer, and feature fusion is repeated multiple times. In this way, higher-level feature fusion can be achieved, further improving the accuracy of the model.

[0068] Although GIoU can well reflect the distance and overlap degree between the predicted bounding box and the ground truth bounding box, when IoU and GIoU are equal, it is impossible to determine whether the predicted bounding box is horizontal or vertical, which leads to a slow convergence speed of the calculation. Therefore, the CIoU loss function is introduced. It considers three issues: the coverage area, the distance between the centers of the two bounding boxes, and their aspect ratios. Using this loss function can improve the regression localization accuracy and also ensure an accelerated convergence speed of the predicted bounding box during training.

[0069] As Figure 4 shown, it is the structural diagram of the improved YOLOV5 algorithm.

[0070] In one embodiment, the obtaining of the training samples includes:

[0071] Using an ultrasonic device to perform multi-angle scans on a large number of patients to obtain scan images; wherein, the scan images include normal images and hyperechoic region images; the scan images contain echo regions with different depths, sizes, and shapes;

[0072] Taking the scan images as training samples.

[0073] In the above technical solution, an ultrasonic device is used to scan the brains of multiple patients at multiple angles. Since the collection of ultrasonic image data of different substantia nigra regions needs to cover various possible manifestations of hyperechoic regions in the substantia nigra, including echo regions with different depths, sizes, and shapes. In this way, using a large number of training images to train and test the YOLOv5 model, a trained YOLOv5 model can be obtained, which can improve the recognition accuracy of hyperechoic regions in the substantia nigra based on YOLOv5.

[0074] In one embodiment, the preprocessing of the training samples to obtain the preprocessed training samples includes:

[0075] Performing grayscale processing on the training samples;

[0076] And, using a Gaussian filter to perform denoising processing on the training samples;

[0077] And, performing normalization processing on the training samples.

[0078] In the above technical solution, in the data preprocessing stage, a series of processes need to be performed on the original images to reduce data redundancy and noise interference and improve the efficiency and accuracy of model training. For example, grayscale processing can simplify image information and reduce computational complexity; using a Gaussian filter for denoising processing helps to eliminate random noise in the image and highlight the characteristics of hyperechoic regions in the substantia nigra; normalization processing can ensure consistent feature scales between different images and lay a foundation for the subsequent training of the YOLOv5 model.

[0079] In one embodiment, based on the improved YOLOV5 algorithm, a substantia nigra hyperechoic area recognition model is trained according to the preprocessed training samples, including:

[0080] Annotate the preprocessed training samples; among them, the annotated categories include: normal substantia nigra area and substantia nigra area with hyperecho.

[0081] Divide the preprocessed and annotated training samples into a training set and a test set according to a ratio of 8:2.

[0082] Based on the improved YOLOV5 algorithm, train an initial recognition model according to the training set.

[0083] Based on the test set, test the initial recognition model.

[0084] Take the initial recognition model that passes the test as the substantia nigra hyperechoic area recognition model.

[0085] In the above technical solution, two situations of the normal substantia nigra area and the substantia nigra area with hyperecho in the relevant images are annotated to construct corresponding training sets and test sets (including image and annotated output data files). The following steps are included: Use the Make Sense annotation tool to annotate the hyperechoic area in the multiple ultrasound images as the target, mark the normal substantia nigra area as "N", and mark the hyperecho in the substantia nigra area as "S"; associate the obtained txt file (which records the position of the target) with the picture, place a part of the corresponding txt files and pictures in the training folder to form a training set; place the other part of the corresponding txt files and pictures in the test folder to form a test set. In this way, by annotating the hyperechoic area as the target and using the annotated training set and test set to train the YOLOv5 model, a trained YOLOv5 model can be obtained, which can improve the recognition accuracy of the substantia nigra hyperechoic area based on YOLOv5.

[0086] Use the above training set and test set to train and test the YOLOv5 model to obtain a trained YOLOv5 model. The following steps are included: Test the YOLOv5 model with the obtained relevant image test set to further determine whether the YOLOv5 model has been trained. If it is determined that the YOLOv5 model has been trained, the trained YOLOv5 model can be used to detect the image to be detected; if it is determined that the YOLOv5 model has not been trained, continue to train the YOLOv5 model, thereby improving the recognition accuracy of the substantia nigra hyperechoic area based on YOLOv5.

[0087] Among them, the test set includes a process test set and a final test set, and the test folder includes a process test folder and a final test folder; a part of the corresponding txt files and images are placed in the process test folder to form a process test set; the other part of the corresponding txt files and images are placed in the final test folder to form a final test set.

[0088] The YOLOv5 model has been trained with a large number of professionally labeled images of the substantia nigra with strong echogenicity areas and already has a high recognition ability. It can accurately separate the substantia nigra with strong echogenicity areas from the complex image background and precisely depict its contour. Therefore, the recognition results output by the system include not only basic information such as the position, size, and shape of the substantia nigra with strong echogenicity areas, but also detailed data on its intensity and distribution.

[0089] In one embodiment,

[0090] The CIoU loss function includes:

[0091]

[0092]

[0093] Among them, b, b gt are the center points of the predicted bounding box and the ground truth bounding box respectively, v is the similarity of the aspect ratio, a is the weight coefficient, IoU is the intersection over union, ω gt and h gt are the width and height of the ground truth bounding box respectively, ω and h are the width and height of the predicted bounding box respectively, L CIoU is the CIoU loss, ρ is the Euclidean distance function, and e is the diagonal length of the minimum rectangle of the predicted bounding box and the ground truth bounding box.

[0094] In one embodiment, as Figure 5 shown, when this application is used, it can also obtain the image to be detected (including ultrasound pictures and videos, all of which are images collected on an ultrasound device) as the image to be detected. Use the trained YOLOv5 model to detect the image to be detected and test according to whether there is strong echogenicity in the substantia nigra area. The YOLOv5 model can detect the ultrasound images of the substantia nigra area collected in real time, and then obtain the detection results. After the system's judgment, a voice broadcast is made for the results with strong echogenicity to remind the doctor to pay attention to the results in time. Evaluate the above test results to obtain a final conclusion, and then give a corresponding treatment plan for the problematic images.

[0095] An embodiment of the present invention provides a real-time automatic recognition system for the substantia nigra with strong echogenicity areas based on YOLOV5, as Figure 6 shown, including:

[0096] An image acquisition module 1 for acquiring cranial ultrasound scan images of an examined patient;

[0097] A substantia nigra hyperechoic area recognition module 2 for recognizing the substantia nigra hyperechoic area in the cranial ultrasound scan image based on a substantia nigra hyperechoic area recognition model to obtain a recognition result; wherein, the substantia nigra hyperechoic area recognition model is a model pre-trained based on the YOLOV5 algorithm;

[0098] A result output module 3 for outputting the recognition result.

[0099] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and a processor executes the computer program to implement the method as described in any one of the above.

[0100] An embodiment of the present invention provides an electronic device, which includes a memory and a processor. A computer program is stored in the memory, and the processor executes the computer program to implement the method as described in any one of the above.

[0101] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. A real-time automatic recognition method for the substantia nigra hyperechoic area based on YOLOV5, characterized in that, Including: Obtain the cranial ultrasound scan image of the patient to be examined; Based on the substantia nigra hyperechoic area recognition model, perform substantia nigra hyperechoic area recognition on the cranial ultrasound scan image to obtain the recognition result; among them, the substantia nigra hyperechoic area recognition model is a model pre-trained based on the YOLOV5 algorithm; Output the recognition result.

2. The real-time automatic recognition method for substantia nigra hyperechoic area based on YOLOV5 according to claim 1, wherein, The pre-training steps of the substantia nigra hyperechoic area recognition model are as follows: Improve the YOLOV5 algorithm to obtain the improved YOLOV5 algorithm; Obtain training samples; Preprocess the training samples to obtain the preprocessed training samples; Based on the improved YOLOV5 algorithm, train the substantia nigra hyperechoic area recognition model according to the preprocessed training samples.

3. The real-time automatic identification method of the substantia nigra hyperechoic area based on YOLOV5 according to claim 2, characterized in that The improvement of the YOLOV5 algorithm to obtain the improved YOLOV5 algorithm includes: Replace the ordinary convolution Conv in the YOLOV5 algorithm with the Gsconv method; And replace the FPN structure in the YOLOV5 algorithm with the BiFPN structure; And replace the GioU loss function in the YOLOV5 algorithm with the CIoU loss function.

4. The real-time automatic identification method of substantia nigra hyperechoic area based on YOLOV5 according to claim 2, characterized in that, The obtaining of the training samples includes: Use an ultrasound device to perform multi-angle scans on a large number of patients to obtain scan images; among them, the scan images include normal images and hyperechoic area images; the scan images contain hyperechoic areas with different depths, sizes, and shapes; Use the scan images as training samples.

5. The real-time automatic identification method of substantia nigra hyperechoic area based on YOLOV5 according to claim 2, characterized in that The preprocessing of the training samples to obtain the preprocessed training samples includes: Perform grayscale processing on the training samples; And use a Gaussian filter to perform denoising processing on the training samples; And perform normalization processing on the training samples.

6. The real-time automatic recognition method of the substantia nigra hyperechoic area based on YOLOV5 according to claim 2, characterized in that, The training of the substantia nigra hyperechoic area recognition model based on the improved YOLOV5 algorithm according to the preprocessed training samples includes: Annotate the preprocessed training samples; among them, the annotated categories include: normal substantia nigra area and substantia nigra area with hyperecho; Divide the annotated preprocessed training samples into a training set and a test set according to a ratio of 8:2; Based on the improved YOLOV5 algorithm, train an initial recognition model according to the training set; Based on the test set, test the initial recognition model; Use the initial recognition model that passes the test as the substantia nigra hyperechoic area recognition model.

7. The real-time automatic recognition method for substantia nigra hyperechoic area based on YOLOV5 according to claim 3, wherein The CIoU loss function includes: Among them, b and b gt are the center points of the predicted box and the ground truth box respectively, v is the similarity of the aspect ratio, a is the weight coefficient, IoU is the intersection over union, ω gt and h gt are the width and height of the ground truth box respectively, ω and h are the width and height of the predicted box respectively, L CIoU is the CIoU loss, ρ is the Euclidean distance function, and e is the diagonal length of the minimum rectangle of the predicted box and the ground truth box.

8. A real-time automatic identification system for substantia nigra hyperechoic area based on YOLOV5, characterized in that, Including: An image acquisition module for obtaining the cranial ultrasound scan image of the patient to be examined; A substantia nigra hyperechoic area recognition module for performing substantia nigra hyperechoic area recognition on the cranial ultrasound scan image based on the substantia nigra hyperechoic area recognition model to obtain the recognition result; among them, the substantia nigra hyperechoic area recognition model is a model pre-trained based on the YOLOV5 algorithm; A result output module for outputting the recognition result.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and the processor executes the computer program to implement the method according to any one of claims 1-7.

10. An electronic device, characterized in that, The electronic device includes a memory and a processor, a computer program is stored in the memory, and the processor executes the computer program to implement the method according to any one of claims 1-7.

Images