Image detection method, network model training method and related device

Abstract

The present application discloses an image detection method, a network model training method and a related device, which obtains an image to be detected in a target domain, and then inputs the image to be detected into a first network model for removing the background to determine at least one region of interest; and inputs at least one region of interest into a second network model to obtain a discriminant feature corresponding to at least one region of interest in source domain data, the source domain data is used to indicate the labeled data, and then the detection result is determined according to the discriminant feature. Thus, image detection in the target domain is realized, so that the unlabeled data in the target domain can be labeled and determined by the labeled data in the source domain, reducing the detection error caused by the domain change and improving the accuracy of image detection.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image detection method, a network model training method, and related devices. Background Art

[0002] With the improvement of computer performance, the excellent performance of deep neural networks has been remarkable. Image detection is an application of neural networks. Its task is to detect the location and category of objects of interest in pictures. Compared with image classification, it requires more fine-grained annotation, which increases the cost and workload of obtaining annotation data.

[0003] Generally, a training set, i.e., source domain data, is used to train the network model to obtain the correspondence between image features and annotations, and then the network model is used to process the target domain data to obtain relevant features.

[0004] However, since the source domain and target domain data are often collected in different environments, there are usually differences in their distribution, such as brightness, angle, clarity, etc. At this time, the performance of the neural network model trained by the training set in the test set will usually decrease, and it will cause errors in the image detection results, affecting the accuracy of image detection. Summary of the invention

[0005] In view of this, the present application provides an image detection method, which can effectively reduce the detection error caused by changes in the image field and improve the accuracy of image detection.

[0006] In a first aspect, the present application provides an image detection method, which can be applied to a system or program for medical image detection, and specifically comprises: obtaining an image to be detected, wherein the image to be detected is target domain data, and the target domain data is used to indicate unlabeled data;

[0007] Inputting the image to be detected into a first network model to determine at least one region of interest, wherein the first network model is used to remove the background of the image to be detected;

[0008] Inputting at least one of the regions of interest into a second network model to obtain a discriminant feature corresponding to the at least one region of interest in source domain data, wherein the source domain data is used to indicate annotation data, and the second network model associates the source domain data with a plurality of target domain data through the region of interest, wherein the discriminant feature is a key region in the region of interest for determining a detection result;

[0009] A detection result is determined according to the discriminant feature, and the detection result is used to indicate the position of the region of interest in the image to be detected and the corresponding annotation data.

[0010] Optionally, in some possible implementations of the present application, the image to be detected is a medical detection image, the position of the region of interest is the position of a lesion, and the annotation data is the type of the lesion.

[0011] Optionally, in some possible implementations of the present application, inputting the image to be detected into a first network model to determine at least one region of interest includes:

[0012] Generating at least one anchor frame on the image to be detected;

[0013] Inputting the anchor frame into a first network model to obtain an anchor frame including a foreground, wherein the foreground is used to indicate valid data in the image to be detected;

[0014] At least one region of interest is determined according to the anchor frame containing the foreground.

[0015] Optionally, in some possible implementations of the present application, determining the detection result according to the discriminant feature includes:

[0016] Obtaining the area corresponding to the discriminant feature;

[0017] If the area corresponding to the discriminant feature meets a preset condition, a detection result is determined according to the discriminant feature, and the preset condition is set based on the proportion of the background in the area corresponding to the discriminant feature.

[0018] Optionally, in some possible implementations of the present application, after acquiring the image to be detected, the method further includes:

[0019] Obtaining parameters of target area data corresponding to the image to be detected;

[0020] Adjusting the parameters of the target domain data corresponding to the image to be detected to the corresponding parameters in the source domain to obtain the image to be detected after image conversion;

[0021] The step of inputting the image to be detected into a first network model to determine at least one region of interest includes:

[0022] The converted image to be detected is input into the first network model to determine at least one region of interest.

[0023] A second aspect of the present application provides a training method for a network model, specifically comprising: obtaining a set of images to be trained, the set of images to be trained comprising a source domain data set and a target domain data set, the source domain data set comprising at least one source domain data, the target domain data set comprising at least one target domain data, the source domain data being used to indicate labeled data, and the target domain data being used to indicate unlabeled data;

[0024] Using the source domain data set and the target domain data set to train a first network model;

[0025] Determine at least one region of interest according to the trained first network model;

[0026] The second network model is trained using the region of interest, the source domain data set and the target domain data set, and the trained second network model is used to indicate the annotation data corresponding to the region of interest in the target domain data.

[0027] Optionally, in some possible implementations of the present application, the method further includes:

[0028] The second network model is trained using a first loss function, the source domain data set, and the target domain data set to obtain a first gradient of the second network model, wherein the first loss function is used to align features of the source domain data set with features of the target domain data set, the first loss function belongs to an adversarial learning loss function, and the first gradient of the second network model is used to update parameters of the second network model.

[0029] Optionally, in some possible implementations of the present application, the adopting the first loss function, the source domain data set, and the target domain data set to train the second network model to obtain a first gradient of the second network model includes:

[0030] Inputting the first loss function, the source domain data set and the target domain data set into a gradient reversal layer to invert the objective function;

[0031] The second network model is trained according to the inverted objective function to obtain a first gradient of the second network model.

[0032] Optionally, in some possible implementations of the present application, the method further includes:

[0033] Determine the region of interest in the source domain data set and the region of interest in the target domain data set respectively according to the trained first network model;

[0034] The second network model is trained using a second loss function, a region of interest in the source domain data set, and a region of interest in the target domain data set to obtain a second gradient of the second network model, wherein the second loss function is used to align features of the region of interest in the source domain data set with features of the region of interest in the target domain data set, the second loss function belongs to an adversarial learning loss function, and the second gradient of the second network model is used to update parameters of the second network model.

[0035] Optionally, in some possible implementations of the present application, the method further includes:

[0036] Determining a plurality of anchor boxes in the source domain data set;

[0037] Setting weights for the source domain data set and the target domain data set according to the proportion of the foreground in the anchor frame;

[0038] The adopting the first loss function, the source domain data set and the target domain data set to train the second network model to obtain a first gradient of the second network model includes:

[0039] The second network model is trained using a first loss function, a weighted source domain data set, and a weighted target domain data set to obtain a first gradient of the second network model.

[0040] A third aspect of the present application provides an image detection device, comprising: an acquisition unit, configured to acquire an image to be detected, wherein the image to be detected is target domain data, and the target domain data is used to indicate unlabeled data;

[0041] A determination unit, configured to input the image to be detected into a first network model to determine at least one region of interest, wherein the first network model is used to remove the background of the image to be detected;

[0042] An input unit, used to input at least one of the regions of interest into a second network model to obtain a discriminant feature corresponding to the at least one region of interest in source domain data, wherein the source domain data is used to indicate annotation data, and the second network model associates the source domain data with a plurality of target domain data through the region of interest, and the discriminant feature is a key area in the region of interest for determining a detection result;

[0043] A detection unit is used to determine a detection result according to the discriminant feature, and the detection result is used to indicate the position of the region of interest in the image to be detected and the corresponding annotation data.

[0044] Optionally, in some possible implementations of the present application, the determining unit is specifically configured to generate at least one anchor frame on the image to be detected;

[0045] The determining unit is specifically configured to input the anchor frame into a first network model to obtain an anchor frame including a foreground, wherein the foreground is used to indicate valid data in the image to be detected;

[0046] The determining unit is specifically configured to determine at least one region of interest according to the anchor frame containing the foreground.

[0047] Optionally, in some possible implementations of the present application, the detection unit is specifically used to obtain the area corresponding to the discriminant feature;

[0048] The detection unit is specifically used to determine the detection result according to the discriminant feature if the area corresponding to the discriminant feature meets a preset condition, and the preset condition is set based on the proportion of the background in the area corresponding to the discriminant feature.

[0049] Optionally, in some possible implementations of the present application, the acquisition unit is further used to acquire parameters of the target field data corresponding to the image to be detected;

[0050] The acquisition unit is further used to adjust the parameters of the target domain data corresponding to the image to be detected to the corresponding parameters in the source domain, so as to obtain the image to be detected after image conversion;

[0051] The determining unit is specifically configured to input the converted image to be detected into the first network model to determine at least one region of interest.

[0052] A fourth aspect of the present application provides a training device for a neural network model, comprising:

[0053] an acquisition unit, configured to acquire a set of images to be trained, wherein the set of images to be trained includes a source domain data set and a target domain data set, wherein the source domain data set includes at least one source domain data, and the target domain data set includes at least one target domain data, wherein the source domain data is used to indicate labeled data, and the target domain data is used to indicate unlabeled data;

[0054] A training unit, configured to train a first network model using the source domain data set and the target domain data set;

[0055] A determining unit, configured to determine at least one region of interest according to the trained first network model;

[0056] The training unit is further used to train a second network model using the region of interest, the source domain data set and the target domain data set, and the trained second network model is used to indicate the labeled data corresponding to the region of interest in the target domain data.

[0057] Optionally, in some possible implementations of the present application, the training unit is also used to train the second network model using a first loss function, the source domain data set, and the target domain data set to obtain a first gradient of the second network model, wherein the first loss function is used to align features of the source domain data set with features of the target domain data set, the first loss function belongs to an adversarial learning loss function, and the first gradient of the second network model is used to update parameters of the second network model.

[0058] Optionally, in some possible implementations of the present application, the training unit is specifically configured to input the first loss function, the source domain data set, and the target domain data set into a gradient reversal layer to invert the objective function;

[0059] The training unit is specifically used to train the second network model according to the inverted objective function to obtain the first gradient of the second network model.

[0060] Optionally, in some possible implementations of the present application, the training unit is further used to determine the region of interest in the source domain data set and the region of interest in the target domain data set respectively according to the trained first network model;

[0061] The training unit is also used to train the second network model using a second loss function, the region of interest in the source domain data set, and the region of interest in the target domain data set to obtain a second gradient of the second network model, wherein the second loss function is used to align features of the region of interest in the source domain data set with features of the region of interest in the target domain data set, the second loss function belongs to an adversarial learning loss function, and the second gradient of the second network model is used to update parameters of the second network model.

[0062] Optionally, in some possible implementations of the present application, the training unit is further used to determine a plurality of anchor frames in the source domain data set;

[0063] The training unit is further used to set weights for the source domain data set and the target domain data set according to the proportion of the foreground in the anchor frame;

[0064] The training unit is specifically used to train the second network model using a first loss function, a weighted source domain data set, and a weighted target domain data set to obtain a first gradient of the second network model.

[0065] The fifth aspect of the present application provides a medical detection device, including: a memory, a processor and a bus system; the memory is used to store program code; the processor is used to execute the image detection method described in the first aspect or any one of the first aspects according to the instructions in the program code.

[0066] The fifth aspect of the present application provides a server, including: a memory, a processor and a bus system; the memory is used to store program code; the processor is used to execute the network model training method described in the second aspect or any one of the second aspects according to the instructions in the program code.

[0067] In a sixth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, the computer executes the image detection method described in the first aspect or any one of the first aspects, or the network model training method described in the second aspect or any one of the second aspects.

[0068] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0069] By acquiring an image to be detected in the target domain, and then inputting the image to be detected into the first network model to determine at least one region of interest; and inputting at least one region of interest into the second network model to obtain the discriminant feature corresponding to at least one region of interest in the source domain data, the source domain data is used to indicate the annotation data, and then determine the detection result according to the discriminant feature, wherein the detection result is used to indicate the position of the region of interest in the image to be detected and the corresponding annotation data. Thus, image detection in the target domain is realized. Since the target domain and the source domain are associated through the network model, the unlabeled data in the target domain can be labeled and determined through the labeled data in the source domain, reducing the detection error caused by the domain change and improving the accuracy of image detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0071] Figure 1A schematic diagram of the architecture of a medical image detection system in an embodiment of the present application;

[0072] Figure 2 A flow chart of an image detection process provided by an embodiment of the present application;

[0073] Figure 3 A flowchart of an image detection method provided in an embodiment of the present application;

[0074] Figure 4 A schematic diagram of an image detection scenario provided in an embodiment of the present application;

[0075] Figure 5 A schematic diagram of an image conversion process provided in an embodiment of the present application;

[0076] Figure 6 A flowchart of a network model training method provided in an embodiment of the present application;

[0077] Figure 7 An architecture diagram of a network model in image detection provided in an embodiment of the present application;

[0078] Figure 8 A schematic diagram of the structure of an image detection device provided in an embodiment of the present application;

[0079] Fig. 9 A schematic diagram of the structure of a network model training device provided in an embodiment of the present application;

[0080] Fig.10 A schematic diagram of the structure of a medical detection device provided in an embodiment of the present application;

[0081] Fig.11 A schematic diagram of the structure of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0082] The embodiment of the present application provides a method and related device for image detection, which can be applied to a system or program for medical image detection, by obtaining an image to be detected in a target domain, and then inputting the image to be detected into a first network model to determine at least one region of interest; and inputting at least one region of interest into a second network model to obtain a discriminant feature corresponding to at least one region of interest in source domain data, the source domain data is used to indicate the annotation data, and then determine the detection result according to the discriminant feature, wherein the detection result is used to indicate the position of the region of interest in the image to be detected and the corresponding annotation data. Thus, image detection in the target domain is realized. Since the target domain and the source domain are associated through the network model, the unlabeled data in the target domain can be labeled and determined through the labeled data in the source domain, which reduces the detection error caused by the domain change and improves the accuracy of image detection.

[0083] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0084] First, some terms that may appear in the embodiments of the present application are explained.

[0085] Deep detection model: An object detection model based on a deep neural network that inputs an image and identifies the location and category of the target object in the image.

[0086] Region of interest (ROI): In image processing, the area to be processed is outlined from the image being processed in the form of boxes, circles, ellipses, irregular polygons, etc. through various operators and functions.

[0087] Source domain data: includes manually annotated data sets. For example, during the model development phase, the training data provided by the partner hospital includes the characteristics of various lesions and their corresponding location information.

[0088] Target domain data: There is a lack of manually annotated datasets, and the data distribution is different from the source domain.

[0089] Domain shift: There is a difference in the data distribution between the two domains.

[0090] Domain adaptation: A type of task in machine learning where a model trained on labeled data from a source domain generalizes well to a target domain.

[0091] Feature alignment: Aligning the feature distributions of two fields is a method to solve domain adaptation.

[0092] It should be understood that the present application can be applied to medical diagnosis scenarios based on artificial intelligence (AI), specifically for detecting and analyzing input medical images, that is, using a medical image detection model to output the analysis results of the medical images, so that medical staff or researchers can obtain more accurate diagnosis results. Figure 1 , Figure 1 This is a schematic diagram of the architecture of the medical image detection system in the embodiment of the present application. As shown in the figure, a large number of medical images can be obtained through medical detection equipment. It should be noted that medical images include but are not limited to computed tomography (CT) images, magnetic resonance imaging (MRI) images, ultrasound (US) images and molybdenum target images.

[0093] Among them, AI-based medical fields include computer vision technology (CV). Computer vision is a science that studies how to make machines "see". To put it more specifically, it refers to machine vision such as using cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphic processing to make computer processing into images that are more suitable for human eye observation or transmission to instruments for detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multidimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, three-dimensional object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

[0094] Among them, CT images are composed of a certain number of pixels with different grayscales from black to white arranged in a matrix. These pixels reflect the X-ray absorption coefficient of the corresponding voxels. CT images are represented by different grayscales, reflecting the degree of absorption of X-rays by organs and tissues. Therefore, like the black and white images shown in X-ray images, black shadows represent low absorption areas, that is, low-density areas, such as lungs with a lot of gas; white shadows represent high absorption areas, that is, high-density areas, such as bones. However, compared with X-ray images, CT images have high density resolution, that is, high density resolution. Therefore, CT images can better display organs composed of soft tissues, such as the brain, spinal cord, mediastinum, lungs, liver, gallbladder, pancreas, and pelvic organs, and display images of lesions on a good anatomical image background.

[0095] MRI images have been applied to imaging diagnosis of all systems in the body. The best results are achieved in the brain, spinal cord, heart and large blood vessels, joint bones, soft tissues and pelvic cavity. For cardiovascular diseases, not only can the anatomical changes of each chamber, large blood vessels and valves be observed, but also ventricular analysis can be performed, qualitative and semi-quantitative diagnosis can be made, multiple sections can be made, the spatial resolution is high, and the whole picture of the heart and lesions, as well as their relationship with surrounding structures, are displayed, which is superior to other X-ray imaging, two-dimensional ultrasound, radionuclide and CT examinations. When diagnosing brain and spinal cord lesions, coronary, sagittal and cross-sectional images can be made.

[0096] US images reflect the difference in acoustic parameters in the medium, and can obtain information different from optics, X-rays, and Y-rays. Ultrasound has good resolution for human soft tissues, which is helpful for identifying tiny lesions in biological tissues. When ultrasound images display living tissues, they can obtain the required images without dyeing.

[0097] Mammography is a new digital imaging technology that combines traditional radiological technology with modern computer technology. It ultimately converts the analog image of ordinary X-ray photography into a digital image that can be quantified and processed, making a qualitative leap in traditional X-ray film technology and image quality, making it easier for radiologists to find suspicious malignant lesions in breast radiographs, and is considered to be a method that helps to improve the early detection rate of breast cancer. This application mainly introduces the mammography of the breast as an example.

[0098] The medical detection device sends the medical images to the server, and the medical image detection model trained in the server can detect these medical images. For example, if a calcified area is detected in the medical image, the area is extracted to further determine whether the calcification is malignant or benign. If it is a malignant calcification, a malignant calcification positioning result will be generated, and the server will send the malignant calcification positioning result to the terminal device. The terminal device can generate a report based on the malignant calcification positioning result and print it out, or it can directly display the malignant calcification positioning result on the display screen.

[0099] It should be noted that terminal devices include but are not limited to PDAs, mobile phones, printers, personal computers, laptops and tablet computers.

[0100] It is understandable that the medical detection device, server and terminal device included in the medical image detection system can be three independent devices or integrated into the same system, which is not limited here.

[0101] With the improvement of computer performance, the excellent performance of deep neural networks has been remarkable. Image detection is an application of neural networks. Its task is to detect the location and category of objects of interest in pictures. Compared with image classification, it requires more fine-grained annotation, which increases the cost and workload of obtaining annotation data.

[0102] Generally, a training set, i.e., source domain data, is used to train the network model to obtain the correspondence between image features and annotations, and then the network model is used to process the target domain data to obtain relevant features.

[0103] However, since the source domain and target domain data are often collected in different environments, there are usually differences in their distribution, such as brightness, angle, clarity, etc. At this time, the performance of the neural network model trained by the training set in the test set will usually decrease, and it will cause errors in the image detection results, affecting the accuracy of image detection.

[0104] In order to solve the above problems, this application proposes an image detection method, which is applied to Figure 2 In the process framework of image detection shown in Figure 2 As shown, it is a process architecture diagram of an image detection provided in an embodiment of the present application. First, the medical detection equipment collects images of relevant parts of the user, and then sends the collected images to the network model to obtain the disease information contained in the image, and the location of the disease, wherein the above-mentioned network model is trained with active domain data and target domain data.

[0105] It is understandable that the method provided by the present application can be written as a program as a processing logic in a hardware system, or as an image detection device, which implements the above processing logic in an integrated or external manner. As an implementation method, the image detection device obtains an image to be detected in a target domain, and then inputs the image to be detected into a first network model to determine at least one region of interest; and inputs at least one region of interest into a second network model to obtain a discriminant feature corresponding to at least one region of interest in the source domain data, and the source domain data is used to indicate the annotation data, and then the detection result is determined according to the discriminant feature, wherein the detection result is used to indicate the position of the region of interest in the image to be detected and the corresponding annotation data. Thus, image detection in the target domain is realized. Since the target domain and the source domain are associated through the network model, the unlabeled data in the target domain can be labeled and determined by the labeled data in the source domain, which reduces the detection error caused by the domain change and improves the accuracy of image detection.

[0106] Combined with the above process architecture, the image detection method in this application is introduced below. Figure 3 , Figure 3 A flowchart of an image detection method provided in an embodiment of the present application, the embodiment of the present application at least includes the following steps:

[0107] 301. Obtain an image to be detected.

[0108] In this embodiment, the image to be detected is target domain data, and the target domain data is used to indicate unlabeled data.

[0109] It can be understood that the image to be detected can be a medical detection image, and the annotation data is the type of the lesion. That is, the present application can obtain the location and type of the disease that may exist in the image to be detected by inputting the image to be detected. The specific disease type can be referred to Figure 1 The examples in the chapter are explained in detail and will not be repeated here.

[0110] In addition, due to the differences between the target domain and the source domain, when the image to be detected does not contain the target domain data, deviations may occur in the feature recognition process. For hospital scenarios, different hospitals may use different parameters for the same instrument, such as detection angle, brightness, clarity, etc., which can cause deviations in feature detection. Therefore, the target domain data can be associated with the source domain data before feature detection.

[0111] 302. Input the image to be detected into a first network model to determine at least one region of interest;

[0112] In this embodiment, the first network model can be a region proposal network model (RPN); the function of the first network model is to capture the region of interest in the image to be detected, where the region of interest can be pre-set, for example: setting the capture area to be the image area containing the lump; the region of interest can also be automatically generated, for example: capturing the area with uneven pigment distribution in the image; the region of interest can also be determined by combining the above methods, and the specific method depends on the actual scenario.

[0113] It should be noted that while the first network model captures the region of interest in the image to be detected, it is also equivalent to removing the background of the image to be detected. The background of the image to be detected may have different presentation forms in different scenes. For example, in a medical image scene, the background can be the unilluminated black part of the image; in a landscape photo scene, the background can be a large area of ​​single pigment such as the sky or the ocean.

[0114] It can be understood that since the image to be detected contains the foreground and the background, the foreground is the area containing valid data, and the background is the area not containing valid data, such as Figure 4 As shown, it is a scene schematic diagram of an image detection provided by an embodiment of the present application, in which a foreground A1, a background A2 and an area of ​​interest A3 are shown, wherein the foreground A1 contains an image that may indicate disease information, the background A2 generally has no specific content, and the area of ​​interest A3 contains a portion of the foreground A1. It should be noted that the frame of the area of ​​interest in the figure is square, which can be circular in the actual scene. Diamond or irregular graphics, the specific shape depends on the actual scene, and is not limited here. In addition, a region of interest is shown in the figure, and in the actual scene, there may be multiple regions of interest, wherein the multiple regions of interest may be independent of each other or may have a certain overlap area; it should also be noted that the positions indicated by the annotations in the figure are examples, and the areas with similar image features to the annotations have similar properties to the annotated areas.

[0115] Optionally, considering that the region of interest may overlap with the background, and the background is useless data, which will occupy additional resources in the feature judgment process, the selection of the region of interest can be optimized. First, at least one anchor frame is generated on the image to be detected; then the anchor frame is input into the first network model to obtain an anchor frame containing the foreground; and then at least one region of interest is determined according to the anchor frame containing the foreground. Among them, the size and aspect ratio of the anchor frame can be preset, the sizes of multiple anchor frames can be different, and the sizes can be adjusted according to the content in the anchor frame. By continuously adjusting the content in the anchor frame, a relatively accurate region of interest can be obtained.

[0116] Optionally, since the detection error is caused by the difference in domains, after obtaining the image to be detected, the image to be detected can also be input into an image converter to convert the image to be detected from the target domain to the source domain; specifically, first obtain the parameters of the target domain data corresponding to the image to be detected; then adjust the parameters of the target domain data corresponding to the image to be detected to the corresponding parameters in the source domain to obtain the image to be detected after image conversion; and then input the converted image to be detected into the first network model to determine at least one of the regions of interest.

[0117] For the conversion process of the above image converter, you can refer to Figure 5 Conduct, such as Figure 5 As shown, it is a schematic diagram of the process of image conversion provided by an embodiment of the present application; in the figure, X and Y represent pictures in two fields, G is a converter that converts pictures in X into Y style, and F is the reverse converter. Our goal is to train G and F given X and Y. Dx and Dy are discriminators in two fields, which are used to determine the quality of generated pictures. In order to ensure that the converted image can retain the core content, additional consistency constraints are introduced in the cyclic adversarial process, so that F(G(x))≈x, G(F(y))≈y, further optimizing the accuracy of domain conversion, thereby realizing the conversion of target domain data to source domain data, and performing the subsequent detection process, which can improve the recall rate of features.

[0118] 303. Input at least one of the regions of interest into a second network model to obtain a discriminant feature corresponding to the at least one region of interest in the source domain data.

[0119] In this embodiment, the source domain data is used to indicate the labeled data, and the second network model associates the source domain data with the multiple target domain data through the region of interest; wherein, the second network model can be a regional convolutional neural network (RCNN) or other network models used for deep learning, which is not limited here.

[0120] It can be understood that the discriminant feature is a feature in the source domain data, which has a corresponding area in the target domain. For example, if the discriminant feature is a mass detected at a brightness of 50% and an angle of 45° in the source domain data, then there is an area corresponding to the mass in the image at a brightness of 60% and an angle of 40° in the target domain. At this time, the annotated data of the discriminant feature will be annotated to the target domain data accordingly, thereby eliminating the influence of domain changes on feature detection; in addition, the discriminant feature can be understood as a key area in the region of interest for determining the detection result. The setting of the key area has different interpretations in different scenarios. In medical images, the key area is the image area indicating the characteristics of the lesion, for example: the texture image area of ​​the lesion as a mass; in face recognition, the key area is the image area indicating the characteristics of the face, for example: the image area of ​​the pupil in the human eye; the specific form depends on the actual scene and is not limited here.

[0121] In a possible scenario for the above process, after RCNN obtains the ROI generated by RPN, it performs pooling feature ROI-pooling on the feature map generated by the backbone feature extractor backbone according to the position of the ROI to obtain the feature corresponding to the ROI, that is, the discriminant feature.

[0122] 304. Determine a detection result according to the discriminant feature.

[0123] In this embodiment, the detection result is used to indicate the position of the region of interest in the image to be detected and the corresponding annotation data, wherein the position of the region of interest is the position of the lesion in the image to be detected, and the annotation data is the type of the lesion, such as a mass, erosion or damage.

[0124] Optionally, since the detection result may introduce some background areas due to the conversion of the field, the feature elements containing the background can be further optimized. Specifically, the area corresponding to the discriminant feature is first obtained; when the area corresponding to the discriminant feature meets the preset conditions, the detection result is determined according to the discriminant feature, wherein the preset conditions are set based on the proportion of the background in the area corresponding to the discriminant feature; for example: the discriminant feature with a background proportion of less than 10% is specified as the detection result. It can be understood that for the discriminant features that do not meet the preset conditions, the ROI corresponding to the discriminant features can be further adjusted to make the proportion of the background in the corresponding area meet the preset conditions.

[0125] It can be seen from the above embodiments that, by obtaining an image to be detected in the target domain, and then inputting the image to be detected into the first network model to determine at least one region of interest; and inputting at least one region of interest into the second network model to obtain at least one discriminant feature corresponding to the region of interest in the source domain data, the source domain data is used to indicate the annotation data, and then determine the detection result according to the discriminant feature, wherein the detection result is used to indicate the position of the region of interest in the image to be detected and the corresponding annotation data. Thus, image detection in the target domain is realized. On the one hand, since the target domain and the source domain are associated through the network model, the unlabeled data in the target domain can be labeled and determined through the labeled data of the source domain, which reduces the detection error caused by the domain change and improves the accuracy of image detection; on the other hand, it reduces the workload caused by the domain change and improves the detection efficiency.

[0126] The above embodiment introduces the processing method after image input, which involves the application of the first network model and the second network model. It can be understood that the first network model and the second network model provided in this embodiment are not conventional source models, but are obtained by training relevant features through a preset data set; the method for training the network model in this application will be introduced below, please refer to Figure 6 , Figure 6 A flowchart of a network model training method provided in an embodiment of the present application, the embodiment of the present application at least includes the following steps:

[0127] 601. Obtain a set of images to be trained.

[0128] In this embodiment, the image set to be trained includes a source domain data set and a target domain data set, the source domain data set includes at least one source domain data, the target domain data set includes at least one target domain data, the source domain data is used to indicate labeled data, and the target domain data is used to indicate unlabeled data.

[0129] It can be understood that the source domain data set may include different feature detection methods for the same disease, and may also include detection methods for the same feature corresponding to different diseases. For example, the source domain data set includes a data set of different pathological features of tuberculosis; the source domain data set may also include a feature data set for masses in different diseases. The specific form of the data set depends on the actual scenario and should include the corresponding target domain data, which is not limited here.

[0130] 602. Use the source domain data set and the target domain data set to train a first network model.

[0131] In this embodiment, the first network model may be an RPN. The training process of the RPN is to collect a preset region of interest containing disease features through the RPN; correspondingly, the feature distribution of different diseases in the source domain data and the size information corresponding to the relevant features may be used in the training process of the RPN.

[0132] 603. Determine at least one region of interest according to the trained first network model.

[0133] In this embodiment, the original domain data is associated with the target domain area through the region of interest.

[0134] 604. Train a second network model using the region of interest, the source domain data set, and the target domain data set.

[0135] In this embodiment, the trained second network model is used to indicate the labeling data corresponding to the region of interest in the target domain data.

[0136] Optionally, in order to further improve the correspondence between the source domain data and the target domain data, feature alignment may be performed on the source domain data and the target domain data. The feature alignment process may be performed in the following manner.

[0137] 1. Perform feature alignment on image features.

[0138] Specifically, the second network model can be trained using a first loss function, the source domain data set, and the target domain data set to obtain a first gradient of the second network model, wherein the first loss function is used to align features of the source domain data set with features of the target domain data set, the first loss function belongs to an adversarial learning loss function, and the first gradient of the second network model is used to update parameters of the second network model.

[0139] In a possible implementation, the source domain data X S The feature map extracted by backbone is F S ∈R H ×W×C , where H and W are the height and width of the feature map, respectively, and C is the number of channels; correspondingly, the target data X T The feature map is F T ∈R H ×W×CSince it is assumed that the domain difference mainly comes from the low-level features of the image, if the two feature maps can be aligned, the domain difference caused by the image style difference can be eliminated. It can be understood that the feature alignment process aligns the features of the patch rather than the features of the entire image. That is, each output on the feature map represents a patch on the original image. The values ​​of each channel at the same position are taken and concatenated to obtain the patch feature f of the source domain. S ∈R C and the patch features f in the target domain T ∈R C .

[0140] Then, a domain discriminator DD is trained, and the first loss function defining the image-level feature alignment can be:

[0141] L img =y DD log DD(f;θ DD )+(1-y DD )log(1-DD(f;θ DD ))

[0142] where y DD represents the domain label of the input image (source domain is 1, target domain is 0), DD(f;θ DD ) is the probability that the input image belongs to the source domain calculated by the domain discriminator, where θ DD is the training parameter of the domain discriminator, and the training objective is L img , where θ B is the training parameter of backbone. The first gradient is obtained by reaching the training target, and then the parameters of the second network model are updated.

[0143] Optionally, to improve the efficiency of the above training, the objective function can be inverted, that is, minimized. Specifically, a gradient reversal layer (GRL) can be introduced to align features adversarially. The function of GRL is to flip the sign of the input gradient, so that in the minimization operation, the parameters after GRL are to minimize the objective function, and the parameters before GRL are to maximize the objective function, thereby improving the efficiency of adversarial learning.

[0144] 2. Align features of ROI.

[0145] Specifically, first, the regions of interest in the source domain data set and the regions of interest in the target domain data set are determined respectively according to the trained first network model; then the second network model is trained using a second loss function, the regions of interest in the source domain data set, and the regions of interest in the target domain data set to obtain a second gradient of the second network model, wherein the second loss function is used to align features of the regions of interest in the source domain data set with features of the regions of interest in the target domain data set, the second loss function belongs to an adversarial learning loss function, and the second gradient of the second network model is used to update parameters of the second network model.

[0146] In a possible implementation, for each ROI, the features of the ROI are obtained after ROI-pooling, and the source domain data and the target domain data are recorded as and The ROI features are also aligned once, using the second loss function:

[0147]

[0148] Where L ins The objective function of instance-level feature alignment, p r Represents the instance-level domain discriminator for the ROI feature f r The calculated probability of belonging to the source domain, the training target is θ DD2 The training parameters of the domain discriminator for feature alignment of ROI are obtained by achieving the training target, and then the parameters of the second network model are updated.

[0149] Optionally, by combining the above two feature alignment methods, the overall loss function L can be obtained as:

[0150] L=L img +L ins +L det

[0151] Among them, L det is the supervision loss of the detection model.

[0152] It is understandable that one or more network models in the above embodiments can be applied to Figure 3 In the image detection scenario of the embodiment.

[0153] It can be seen from the above embodiments that by aligning the source domain data with the target domain data, the correlation between the source domain data and the target domain data in the network model is improved, and the accuracy of the image detection process is further improved.

[0154] In another possible scenario, since a large number of anchors generated a priori by the model may contain a large amount of background, and in medical image detection, more attention is paid to the foreground area that needs to be detected, and the style changes of the background do not need to be paid too much attention, if the foreground anchor and the background anchor are treated equally, the feature alignment loss generated by the vast majority of background anchors will mask the feature alignment loss of the foreground anchor, affecting the correlation between the source domain data and the target domain data.

[0155] To solve the above problems, we can Figure 6 Based on the embodiment, a weight is set for each anchor. Specifically, the weight w can be:

[0156] w=y(1-p) γ +(1-y)p γ

[0157] Where y is the label assigned to the anchor (if it is foreground, y=1, if it is background, it is 0), and p is the confidence calculated by the current model that the anchor is the foreground.

[0158] Then corresponding to the feature alignment process, the first loss function can be adjusted as:

[0159]

[0160] For the related parameter description and subsequent operations, please refer to the above Figure 6 The first loss function in the corresponding embodiment is not described in detail here.

[0161] By setting weights, the problem of anchor box category imbalance is solved, and the influence of anchor boxes containing background in feature alignment is reduced; and the problem of incomplete coverage of anchor boxes containing foreground during training is alleviated.

[0162] The above embodiments introduce various training processes for network models. The image detection method provided by the present application is described below in conjunction with a specific scenario. Figure 7 The figure shows an architecture diagram of a network model in image detection provided in an embodiment of the present application.

[0163] It should be noted that this embodiment involves two network models, which are RPN and RCNN in this scenario. Optionally, the two network models can also be merged into one network model, and the two corresponding functions can be merged, for example, into a fast regional convolutional neural network (Faster-RCNN). The specific combination form depends on the actual scenario and is not limited here.

[0164] First, the image to be detected is subjected to backbone feature extraction to obtain a backbone feature set, wherein the source domain data and the target domain data in the backbone feature set are feature aligned by an image domain discriminator, and are also passed through a gradient flipping layer before being input into the image domain discriminator to minimize the objective function.

[0165] For the above backbone feature set, on the one hand, the backbone feature set is input into RPN to select relevant candidate boxes to determine the anchor box, and further determine the ROI; on the other hand, the backbone feature set is input into the pooling layer in RCNN, and combined with the ROI determined on the one hand to obtain the features corresponding to the ROI, and the features corresponding to the ROI are aligned after the gradient flip layer.

[0166] Finally, according to the features corresponding to the obtained ROI, the type of the disease and the location where the disease occurs are determined.

[0167] Understandable, yes. Figure 3 or Figure 6 The relevant features in the embodiments can all be referred to in the process of the architecture diagram and will not be elaborated here.

[0168] In order to better implement the above solution of the embodiment of the present application, the following also provides related devices for implementing the above solution. Figure 8 , Figure 8 The present invention provides a schematic diagram of the structure of an image detection device according to an embodiment of the present invention. The image detection device 800 includes:

[0169] An acquisition unit 801 is used to acquire an image to be detected, where the image to be detected is target domain data, and the target domain data is used to indicate unlabeled data;

[0170] A determination unit 802 is used to input the image to be detected into a first network model to determine at least one region of interest, wherein the first network model is used to remove the background of the image to be detected;

[0171] An input unit 803 is used to input at least one of the regions of interest into a second network model to obtain a discriminant feature corresponding to at least one of the regions of interest in source domain data, where the source domain data is used to indicate the annotation data, and the second network model associates the source domain data with a plurality of the target domain data through the region of interest, where the discriminant feature is a key region in the region of interest for determining a detection result;

[0172] The detection unit 804 is used to determine a detection result according to the discriminant feature, and the detection result is used to indicate the position of the region of interest in the image to be detected and the corresponding annotation data.

[0173] Optionally, in some possible implementations of the present application, the determining unit is specifically configured to generate at least one anchor frame on the image to be detected;

[0174] The determining unit 802 is specifically configured to input the anchor frame into the first network model to obtain an anchor frame including a foreground, where the foreground is used to indicate valid data in the image to be detected;

[0175] The determining unit 802 is specifically configured to determine at least one region of interest according to the anchor frame containing the foreground.

[0176] Optionally, in some possible implementations of the present application, the detection unit 804 is specifically used to obtain the area corresponding to the discriminant feature;

[0177] The detection unit 804 is specifically configured to determine a detection result according to the discriminant feature if the area corresponding to the discriminant feature meets a preset condition, wherein the preset condition is set based on a proportion of the background in the area corresponding to the discriminant feature.

[0178] Optionally, in some possible implementations of the present application, the acquisition unit 801 is further used to acquire parameters of the target area data corresponding to the image to be detected;

[0179] The acquisition unit 801 is further used to adjust the parameters of the target domain data corresponding to the image to be detected to the corresponding parameters in the source domain, so as to obtain the image to be detected after image conversion;

[0180] The determining unit 802 is specifically configured to input the converted image to be detected into the first network model to determine at least one region of interest.

[0181] By acquiring an image to be detected in the target domain, and then inputting the image to be detected into the first network model to determine at least one region of interest; and inputting at least one region of interest into the second network model to obtain the discriminant feature corresponding to at least one region of interest in the source domain data, the source domain data is used to indicate the annotation data, and then determine the detection result according to the discriminant feature, wherein the detection result is used to indicate the position of the region of interest in the image to be detected and the corresponding annotation data. Thus, image detection in the target domain is realized. Since the target domain and the source domain are associated through the network model, the unlabeled data in the target domain can be labeled and determined through the labeled data in the source domain, reducing the detection error caused by the domain change and improving the accuracy of image detection.

[0182] This embodiment also provides a network model training device 900, such as Fig. 9 As shown, including:

[0183] An acquisition unit 901 is used to acquire a set of images to be trained, the set of images to be trained includes a source domain data set and a target domain data set, the source domain data set includes at least one source domain data, the target domain data set includes at least one target domain data, the source domain data is used to indicate labeled data, and the target domain data is used to indicate unlabeled data;

[0184] A training unit 902, configured to train a first network model using the source domain data set and the target domain data set;

[0185] A determining unit 903 is used to determine at least one region of interest according to the trained first network model;

[0186] The training unit 902 is also used to train a second network model using the region of interest, the source domain data set and the target domain data set, and the trained second network model is used to indicate the annotation data corresponding to the region of interest in the target domain data.

[0187] Optionally, in some possible implementations of the present application, the training unit 902 is also used to train the second network model using a first loss function, the source domain data set, and the target domain data set to obtain a first gradient of the second network model, wherein the first loss function is used to align features of the source domain data set with features of the target domain data set, the first loss function belongs to an adversarial learning loss function, and the first gradient of the second network model is used to update parameters of the second network model.

[0188] Optionally, in some possible implementations of the present application, the training unit 902 is specifically configured to input the first loss function, the source domain data set, and the target domain data set into a gradient reversal layer to invert the objective function;

[0189] The training unit 902 is specifically used to train the second network model according to the inverted objective function to obtain the first gradient of the second network model.

[0190] Optionally, in some possible implementations of the present application, the training unit 902 is further used to determine the region of interest in the source domain data set and the region of interest in the target domain data set according to the trained first network model;

[0191] The training unit 902 is also used to train the second network model using a second loss function, the region of interest in the source domain data set, and the region of interest in the target domain data set to obtain a second gradient of the second network model, wherein the second loss function is used to align features of the region of interest in the source domain data set with features of the region of interest in the target domain data set, the second loss function belongs to an adversarial learning loss function, and the second gradient of the second network model is used to update parameters of the second network model.

[0192] Optionally, in some possible implementations of the present application, the training unit 902 is further configured to determine a plurality of anchor frames in the source domain data set;

[0193] The training unit 902 is further configured to set weights for the source domain data set and the target domain data set according to the proportion of the foreground in the anchor frame;

[0194] The training unit 902 is specifically used to train the second network model using a first loss function, a weighted source domain data set, and a weighted target domain data set to obtain a first gradient of the second network model.

[0195] The present application also provides a medical detection device, such as Fig.10 Shown is a structural schematic diagram of a medical detection device provided in an embodiment of the present application. For the sake of convenience of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application.

[0196] Fig.10 FIG. 1 is a block diagram of a partial structure of a medical detection device provided in an embodiment of the present application. Fig.10 The medical detection device includes: a radio frequency (RF) circuit 1010, a memory 1020, an input unit 1030, a display unit 1040, a sensor 1050, an audio circuit 1060, a wireless fidelity (WiFi) module 1070, a processor 1080, and a power supply 1090. Those skilled in the art will appreciate that Fig.10 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0197] Combine the following Fig.10 A detailed introduction to the various components of medical testing equipment:

[0198] The RF circuit 1010 can be used for receiving and sending signals during the process of sending and receiving information or making calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 1080 for processing; in addition, the designed uplink data is sent to the base station. Generally, the RF circuit 1010 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF circuit 1010 can also communicate with the network and other devices through wireless communication. The above wireless communication can use any communication standard or protocol, including but not limited to the global system of mobile communication (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short messaging service (SMS), etc.

[0199] The memory 1020 can be used to store software programs and modules. The processor 1080 executes various functional applications and data processing of the medical detection device by running the software programs and modules stored in the memory 1020. The memory 1020 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the medical detection device (such as audio data, a phone book, etc.), etc. In addition, the memory 1020 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0200] The input unit 1030 can be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the medical detection device. Specifically, the input unit 1030 may include a touch panel 1031 and other input devices 1032. The touch panel 1031, also known as a touch screen, can collect user touch operations on or near it (such as operations performed by users using fingers, styluses, or any other suitable objects or accessories on or near the touch panel 1031, and air touch operations within a certain range on the touch panel 1031), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 1031 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact point coordinates, and then sends it to the processor 1080, and can receive and execute commands sent by the processor 1080. In addition, the touch panel 1031 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch panel 1031, the input unit 1030 can also include other input devices 1032. Specifically, the other input devices 1032 can include but are not limited to one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, etc.

[0201] The display unit 1040 may be used to display information input by the user or information provided to the user and various menus of the medical detection device. The display unit 1040 may include a display panel 1041. Optionally, the display panel 1041 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like. Further, the touch panel 1031 may cover the display panel 1041. When the touch panel 1031 detects a touch operation on or near it, it is transmitted to the processor 1080 to determine the type of touch event. Subsequently, the processor 1080 provides a corresponding visual output on the display panel 1041 according to the type of touch event. Although in Fig.10 In the embodiment, the touch panel 1031 and the display panel 1041 are used as two independent components to realize the input and output functions of the medical detection device, but in some embodiments, the touch panel 1031 and the display panel 1041 can be integrated to realize the input and output functions of the medical detection device.

[0202] The medical detection device may also include at least one sensor 1050, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 1041 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 1041 and / or the backlight when the medical detection device is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary, which can be used for applications that identify the posture of the medical detection device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that can also be configured in the medical detection device, they will not be repeated here.

[0203] The audio circuit 1060, the speaker 1061, and the microphone 1062 can provide an audio interface between the user and the medical detection device. The audio circuit 1060 can transmit the received audio data to the speaker 1061 after converting the electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 1062 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1060 and converted into audio data, and then the audio data is output to the processor 1080 for processing, and then sent to another medical detection device through the RF circuit 1010, or the audio data is output to the memory 1020 for further processing.

[0204] WiFi is a short-range wireless transmission technology. Medical testing equipment can help users send and receive emails, browse web pages, and access streaming media through WiFi module 1070. It provides users with wireless broadband Internet access. Fig.10 A WiFi module 1070 is shown, but it is understandable that it is not a necessary component of the medical detection device and can be omitted as needed without changing the essence of the invention.

[0205] The processor 1080 is the control center of the medical detection device. It uses various interfaces and lines to connect various parts of the entire medical detection device. It executes various functions of the medical detection device and processes data by running or executing software programs and / or modules stored in the memory 1020, and calling data stored in the memory 1020. Optionally, the processor 1080 may include one or more processing units; optionally, the processor 1080 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1080.

[0206] The medical detection equipment also includes a power supply 1090 (such as a battery) for supplying power to each component. Optionally, the power supply can be logically connected to the processor 1080 through a power management system, thereby managing functions such as charging, discharging, and power consumption through the power management system.

[0207] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0208] In the embodiment of the present application, the processor 1080 included in the terminal also has the function of executing each step of the above-mentioned image detection method.

[0209] The present application also provides a server. Fig.11 , Fig.11 1 is a structural diagram of a server provided in an embodiment of the present invention. The server 1100 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1122 (for example, one or more processors) and memory 1132, and one or more storage media 1130 (for example, one or more mass storage devices) storing application programs 1142 or data 1144. Among them, the memory 1132 and the storage medium 1130 may be short-term storage or permanent storage. The program stored in the storage medium 1130 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 1122 may be configured to communicate with the storage medium 1130 to execute a series of instruction operations in the storage medium 1130 on the server 1100.

[0210] The server 1100 may also include one or more power supplies 1126, one or more wired or wireless network interfaces 1150, one or more input and output interfaces 1158, and / or one or more operating systems 1141, such as Windows Server 2003. TM , Mac OS X TM , Unix TM ,LinuxTM, FreeBSD TM etc.

[0211] The steps performed by the server in the above embodiment can be based on the Fig.11 The server structure shown.

[0212] The present application also provides a computer-readable storage medium in which an image detection instruction is stored. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned Figures 2 to 7 The illustrated embodiment describes the steps performed by the image detection device in the method.

[0213] The present application also provides a computer program product including an image detection instruction, which, when executed on a computer, enables the computer to execute the above-mentioned Figures 2 to 7 The illustrated embodiment describes the steps performed by the image detection device in the method.

[0214] The present application also provides an image detection system, which may include: Figure 8 The image detection device in the described embodiment, or Fig. 9 The described network model training device.

[0215] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0216] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0217] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0218] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0219] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, an image detection device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0220] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image detection method, characterized in that: include: Acquire an image to be detected, wherein the image to be detected is target domain data, the target domain data is used to indicate unlabeled data, and the image to be detected is a medical detection image; Obtaining parameters of target area data corresponding to the image to be detected; Adjusting the parameters of the target domain data corresponding to the image to be detected to the corresponding parameters in the source domain to obtain the image to be detected after image conversion; Inputting the converted image to be detected into a first network model to determine at least one region of interest, wherein the first network model is used to remove the background of the converted image to be detected; Inputting at least one of the regions of interest into a second network model to obtain a discriminant feature corresponding to the at least one region of interest in source domain data, wherein the source domain data is used to indicate the labeled data, and the second network model associates the source domain data with the target domain data through the region of interest, and the discriminant feature is a key area in the region of interest for determining a detection result; Obtaining the area corresponding to the discriminant feature; If the area corresponding to the discriminant feature meets the preset condition, determining the detection result according to the discriminant feature; For the region of interest corresponding to the discriminant feature that does not meet the preset condition, regional adjustment is performed so that the proportion of the background in the corresponding region meets the preset condition, and the detection result is determined according to the discriminant feature, the preset condition is set based on the proportion of the background in the region corresponding to the discriminant feature, and the detection result is used to indicate the position of the region of interest in the converted image to be detected and the corresponding annotation data, the position of the region of interest is the position of the lesion in the converted image to be detected, and the annotation data is the type of the lesion.

2. The method according to claim 1, characterized in that The step of inputting the converted image to be detected into a first network model to determine at least one region of interest comprises: Generating at least one anchor frame on the converted image to be detected; Inputting the anchor frame into a first network model to obtain an anchor frame containing a foreground, wherein the foreground is used to indicate valid data in the converted image to be detected; At least one region of interest is determined according to the anchor frame containing the foreground.

3. The method according to any one of claims 1 to 2, characterized in that: The step of inputting the converted image to be detected into a first network model to determine at least one region of interest comprises: The converted image to be detected is input into the first network model to determine at least one region of interest.

4. A network model training method, characterized in that: include: Acquire a set of images to be trained, the set of images to be trained comprising a source domain data set and a target domain data set, the source domain data set comprising at least one source domain data, the target domain data set comprising a feature data set for masses in different diseases and at least one target domain data, the source domain data being used to indicate labeled data, and the target domain data being used to indicate unlabeled data; Using the source domain data set and the target domain data set to train a first network model; Determine at least one region of interest according to the trained first network model; The second network model is trained using the region of interest, the source domain data set and the target domain data set, so that according to the trained second network model, a discriminant feature corresponding to at least one region of interest in the source domain data is obtained, and a detection result is determined according to the discriminant feature, wherein the detection result is used to indicate the position of the region of interest in the converted image to be detected and the corresponding annotation data, the converted image to be detected is obtained by adjusting the parameters of the target domain data corresponding to the image to be detected to the corresponding parameters in the source domain, the position of the region of interest is the position of the lesion in the converted image to be detected, and the annotation data is the type of the lesion; Wherein, determining the detection result according to the discriminant feature includes: Obtaining the area corresponding to the discriminant feature; If the area corresponding to the discriminant feature meets the preset condition, determining the detection result according to the discriminant feature; For the region of interest corresponding to the discriminant feature that does not meet the preset conditions, regional adjustment is performed so that the proportion of the background in the corresponding region meets the preset conditions, and the detection result is determined according to the discriminant feature, and the preset conditions are set based on the proportion of the background in the region corresponding to the discriminant feature.

5. The method according to claim 4, characterized in that The method further comprises: The second network model is trained using a first loss function, the source domain data set, and the target domain data set to obtain a first gradient of the second network model, wherein the first loss function is used to align features of the source domain data set with features of the target domain data set, the first loss function belongs to an adversarial learning loss function, and the first gradient of the second network model is used to update parameters of the second network model.

6. The method according to claim 5, characterized in that The adopting the first loss function, the source domain data set and the target domain data set to train the second network model to obtain a first gradient of the second network model includes: Inputting the first loss function, the source domain data set and the target domain data set into a gradient reversal layer to invert the objective function; The second network model is trained according to the inverted objective function to obtain a first gradient of the second network model.

7. The method according to claim 5, characterized in that The method further comprises: Determine the region of interest in the source domain data set and the region of interest in the target domain data set respectively according to the trained first network model; The second network model is trained using a second loss function, a region of interest in the source domain data set, and a region of interest in the target domain data set to obtain a second gradient of the second network model, wherein the second loss function is used to align features of the region of interest in the source domain data set with features of the region of interest in the target domain data set, the second loss function belongs to an adversarial learning loss function, and the second gradient of the second network model is used to update parameters of the second network model.

8. The method according to any one of claims 5 to 7, characterized in that: The method further comprises: Determining a plurality of anchor boxes in the source domain data set; Setting weights for the source domain data set and the target domain data set according to the proportion of the foreground in the anchor frame; The adopting the first loss function, the source domain data set and the target domain data set to train the second network model to obtain a first gradient of the second network model includes: The second network model is trained using a first loss function, a weighted source domain data set, and a weighted target domain data set to obtain a first gradient of the second network model.

9. An image detection device, characterized in that: include: An acquisition unit, used for acquiring an image to be detected, wherein the image to be detected is target domain data, the target domain data is used to indicate unlabeled data, and the image to be detected is a medical detection image; The acquisition unit is further used to acquire parameters of the target area data corresponding to the image to be detected; The acquisition unit is further used to adjust the parameters of the target domain data corresponding to the image to be detected to the corresponding parameters in the source domain, so as to obtain the image to be detected after image conversion; A determination unit, configured to input the converted image to be detected into a first network model to determine at least one region of interest, wherein the first network model is configured to remove a background of the converted image to be detected; An input unit, used to input at least one of the regions of interest into a second network model to obtain a discriminant feature corresponding to the at least one region of interest in source domain data, wherein the source domain data is used to indicate annotation data, and the second network model associates the source domain data with a plurality of target domain data through the region of interest, and the discriminant feature is a key area in the region of interest for determining a detection result; A detection unit, used to obtain the area corresponding to the discriminant feature; If the area corresponding to the discriminant feature meets the preset conditions, the detection result is determined according to the discriminant feature; for the area of ​​interest corresponding to the discriminant feature that does not meet the preset conditions, the area is adjusted so that the proportion of the background in the corresponding area meets the preset conditions, and the detection result is determined according to the discriminant feature, the preset condition is set based on the proportion of the background in the area corresponding to the discriminant feature, and the detection result is used to indicate the position of the area of ​​interest in the converted image to be detected and the corresponding annotation data, the position of the area of ​​interest is the position of the lesion in the converted image to be detected, and the annotation data is the type of the lesion.

10. The device according to claim 9, characterized in that The determining unit is specifically configured to generate at least one anchor frame on the converted image to be detected; The determining unit is specifically configured to input the anchor frame into a first network model to obtain an anchor frame including a foreground, wherein the foreground is used to indicate valid data in the converted image to be detected; The determining unit is specifically configured to determine at least one region of interest according to the anchor frame containing the foreground.

11. The device according to any one of claims 9 to 10, characterized in that: Also includes: The determining unit is specifically configured to input the converted image to be detected into the first network model to determine at least one region of interest.

12. A network model training device, characterized in that: include: an acquisition unit, configured to acquire a set of images to be trained, wherein the set of images to be trained includes a source domain data set and a target domain data set, wherein the source domain data set includes a feature data set for masses in different diseases and at least one source domain data, and the target domain data set includes at least one target domain data, wherein the source domain data is used to indicate labeled data, and the target domain data is used to indicate unlabeled data; A training unit, configured to train a first network model using the source domain data set and the target domain data set; A determining unit, configured to determine at least one region of interest according to the trained first network model; The training unit is further used to train a second network model using the region of interest, the source domain data set and the target domain data set, so that according to the trained second network model, a discriminant feature corresponding to at least one region of interest in the source domain data is obtained, and a detection result is determined according to the discriminant feature, wherein the detection result is used to indicate the position of the region of interest in the converted image to be detected and the corresponding annotation data, the converted image to be detected is obtained by adjusting the parameters of the target domain data corresponding to the image to be detected to the corresponding parameters in the source domain, the position of the region of interest is the position of the lesion in the converted image to be detected, and the annotation data is the type of the lesion; Wherein, determining the detection result according to the discriminant feature includes: Obtaining the area corresponding to the discriminant feature; If the area corresponding to the discriminant feature meets the preset condition, determining the detection result according to the discriminant feature; For the region of interest corresponding to the discriminant feature that does not meet the preset conditions, regional adjustment is performed so that the proportion of the background in the corresponding region meets the preset conditions, and the detection result is determined according to the discriminant feature, and the preset conditions are set based on the proportion of the background in the region corresponding to the discriminant feature.

13. The device according to claim 12, characterized in that The training unit is also used to train the second network model using a first loss function, the source domain data set and the target domain data set to obtain a first gradient of the second network model, wherein the first loss function is used to align features of the source domain data set and features of the target domain data set, the first loss function belongs to an adversarial learning loss function, and the first gradient of the second network model is used to update parameters of the second network model.

14. The device according to claim 13, characterized in that The training unit is specifically configured to input the first loss function, the source domain data set and the target domain data set into a gradient reversal layer so as to reverse the objective function; The training unit is specifically used to train the second network model according to the inverted objective function to obtain the first gradient of the second network model.

15. The device according to claim 13, characterized in that The training unit is further used to determine the region of interest in the source domain data set and the region of interest in the target domain data set respectively according to the trained first network model; The training unit is also used to train the second network model using a second loss function, the region of interest in the source domain data set, and the region of interest in the target domain data set to obtain a second gradient of the second network model, wherein the second loss function is used to align features of the region of interest in the source domain data set with features of the region of interest in the target domain data set, the second loss function belongs to an adversarial learning loss function, and the second gradient of the second network model is used to update parameters of the second network model.

16. The device according to any one of claims 12 to 15, characterized in that: The training unit is further used to determine a plurality of anchor frames in the source domain data set; The training unit is further used to set weights for the source domain data set and the target domain data set according to the proportion of the foreground in the anchor frame; The training unit is specifically used to train the second network model using a first loss function, a weighted source domain data set, and a weighted target domain data set to obtain a first gradient of the second network model.

17. A medical testing device, characterized in that: include: memories, transceivers, processors, and bus systems; Wherein, the memory is used to store programs; The processor is used to execute the program in the memory, comprising the following steps: Acquire an image to be detected, wherein the image to be detected is target domain data, the target domain data is used to indicate unlabeled data, and the image to be detected is a medical detection image; Obtaining parameters of target area data corresponding to the image to be detected; Adjusting the parameters of the target domain data corresponding to the image to be detected to the corresponding parameters in the source domain to obtain the image to be detected after image conversion; Inputting the converted image to be detected into a first network model to determine at least one region of interest, wherein the first network model is used to remove the background of the converted image to be detected; Inputting at least one of the regions of interest into a second network model to obtain a discriminant feature corresponding to the at least one region of interest in source domain data, wherein the source domain data is used to indicate annotation data, and the second network model associates the source domain data with a plurality of target domain data through the region of interest, wherein the discriminant feature is a key region in the region of interest for determining a detection result; Obtaining the area corresponding to the discriminant feature; If the area corresponding to the discriminant feature meets the preset condition, determining the detection result according to the discriminant feature; For the region of interest corresponding to the discriminant feature that does not meet the preset condition, the region is adjusted so that the proportion of the background in the corresponding region meets the preset condition, and the detection result is determined according to the discriminant feature, the preset condition is set based on the proportion of the background in the region corresponding to the discriminant feature, and the detection result is used to indicate the position of the region of interest in the converted image to be detected and the corresponding annotation data, the position of the region of interest is the position of the lesion in the converted image to be detected, and the annotation data is the type of the lesion; The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

18. A server, characterized in that: include: memories, transceivers, processors, and bus systems; Wherein, the memory is used to store programs; The processor is used to execute the program in the memory, comprising the following steps: Acquire a set of images to be trained, the set of images to be trained comprising a source domain data set and a target domain data set, the source domain data set comprising at least one source domain data, the target domain data set comprising a feature data set for masses in different diseases and at least one target domain data, the source domain data being used to indicate labeled data, and the target domain data being used to indicate unlabeled data; Using the source domain data set and the target domain data set to train a first network model; Determine at least one region of interest according to the trained first network model; The second network model is trained using the region of interest, the source domain data set and the target domain data set, so that according to the trained second network model, a discriminant feature corresponding to at least one region of interest in the source domain data is obtained, and a detection result is determined according to the discriminant feature, wherein the detection result is used to indicate the position of the region of interest in the converted image to be detected and the corresponding annotation data, the converted image to be detected is obtained by adjusting the parameters of the target domain data corresponding to the image to be detected to the corresponding parameters in the source domain, the position of the region of interest is the position of the lesion in the converted image to be detected, and the annotation data is the type of the lesion; Wherein, determining the detection result according to the discriminant feature includes: Obtaining the area corresponding to the discriminant feature; If the area corresponding to the discriminant feature meets the preset condition, determining the detection result according to the discriminant feature; Performing regional adjustment on the region of interest corresponding to the discriminant feature that does not meet the preset condition so that the proportion of the background in the corresponding region meets the preset condition, and determining the detection result according to the discriminant feature, wherein the preset condition is set based on the proportion of the background in the region corresponding to the discriminant feature; The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

19. A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, which, when executed on a computer, enables the computer to execute the image detection method described in any one of claims 1 to 3, or the network model training method described in any one of claims 4 to 8.

Images