An image processing method, apparatus, device, and storage medium

Abstract

Embodiments of the application disclose an image processing method, device and equipment and a storage medium, wherein the method comprises: obtaining a target image to be processed; performing multi-level feature extraction on the target image and performing scale normalization processing on a feature map extracted at each level to obtain a plurality of feature maps; performing feature fusion on the plurality of feature maps to obtain fused features, determining respective weights of the plurality of levels according to the fused features, and weighting the plurality of feature maps according to the respective weights of the plurality of levels to obtain a target feature map; and determining a detection result corresponding to the target image according to the target feature map. The method can realize global cross-level information flow between high and low levels and improve image detection performance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to an image processing method and device, equipment and a storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, deep learning technology is widely used in the field of image processing. Nowadays, a model is trained through deep learning technology, and image detection is performed based on the model to detect a specific region in an image. For example, a model is used to detect a lesion in a pathological image to provide reference data for doctors, thereby facilitating medical staff to make decisions.

[0003] At present, a model trained through deep learning technology generally only performs attention activation or multi-scale feature extraction at a specific position of a network, which leads to poor information flow between high-level and low-level features, thereby greatly limiting the network performance of the model.

[0004] Therefore, it is necessary to provide a new model capable of realizing global cross-level information flow between high-level and low-level features, improving the network performance of the model, and thus improving the image processing performance. SUMMARY

[0005] The embodiments of the present application provide an image processing method, device and storage medium, which can realize global cross-level information flow between high-level and low-level features, improve the network performance of the model, and thus improve the image processing performance.

[0006] Therefore, the first aspect of the present application provides an image processing method, which comprises:

[0007] obtaining a target image to be processed;

[0008] performing multi-level feature extraction on the target image and performing scale normalization processing on a feature map extracted at each level to obtain a plurality of feature maps;

[0009] performing feature fusion on the plurality of feature maps to obtain fused features, determining a weight of each level according to the fused features, and weighting the plurality of feature maps according to the weight of each level to obtain a target feature map;

[0010] determining a detection result corresponding to the target image according to the target feature map.

[0011] The second aspect of the present application provides an image processing device, which comprises:

[0012] a target image acquisition module configured to obtain a target image to be processed;

[0013] The prediction module is used to perform multi-level feature extraction on the target image and scale normalize the feature maps extracted at each level to obtain multiple feature maps;

[0014] The prediction module is further configured to perform feature fusion on the multiple feature maps to obtain fused features, determine the weights of multiple layers according to the fused features, and weight the multiple feature maps according to the weights of the multiple layers to obtain a target feature map;

[0015] The prediction module is further configured to determine the detection result corresponding to the target image based on the target feature map.

[0016] A third aspect of this application provides an image processing apparatus, the apparatus comprising a processor and a memory:

[0017] The memory is used to store program code and transmit the program code to the processor;

[0018] The processor is configured to execute the steps of the image processing method as described in the first aspect above, according to the instructions in the program code.

[0019] A fourth aspect of this application provides a computer-readable storage medium for storing program code for performing the steps of the image processing method described in the first aspect.

[0020] The fifth aspect of this application provides a computer program product including instructions that, when run on a computer, cause the computer to perform the steps of the image processing method described in the first aspect.

[0021] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0022] This application provides an image processing method. First, a target image to be processed is acquired. Then, multi-level feature extraction is performed on the target image, and the feature maps extracted at each level are scale-normalized to obtain multiple feature maps. Next, the multiple feature maps are fused to obtain fused features. Weights for each level are determined based on the fused features. Finally, the multiple feature maps are weighted according to their respective weights to obtain a target feature map. Because the selection of target feature maps involves a global selection process for feature maps extracted at each level, comprehensively considering all different information dimensions of image features extracted from all levels, the selected target feature map contains the most valuable information for the detection task. Based on this, the detection result corresponding to the target image is determined according to the target feature map, thereby improving the accuracy of the detection result. Attached Figure Description

[0023] Figure 1 A schematic diagram of the structure of existing deep learning networks;

[0024] Figure 2 This is a schematic diagram of the structure of a server for image processing provided in an embodiment of this application;

[0025] Figure 3 This is a schematic diagram of the structure of a terminal device for image processing provided in an embodiment of this application;

[0026] Figure 4 This is a schematic diagram illustrating an application scenario of an image processing method provided in an embodiment of this application;

[0027] Figure 5 A schematic flowchart of the image processing method provided in the embodiments of this application;

[0028] Figure 6 This is a schematic diagram of the processing architecture of the image detection model provided in the embodiments of this application;

[0029] Figure 7 A schematic flowchart illustrating the training method for the image detection model provided in this application embodiment;

[0030] Figure 8 A schematic diagram illustrating the processing procedure of the feature extraction module provided in this embodiment of the application;

[0031] Figure 9 A schematic diagram of the processing flow of the feature map selection module provided in the embodiments of this application;

[0032] Figure 10 This is a schematic diagram of the network structure of the detection result prediction module provided in the embodiments of this application;

[0033] Figure 11 This is a schematic diagram illustrating an application scenario of another image processing method provided in an embodiment of this application.

[0034] Figure 12 A schematic diagram of the structure of an image processing device corresponding to the image processing method provided in the embodiments of this application;

[0035] Figure 13 This is a schematic diagram of another image quality control device provided in an embodiment of this application;

[0036] Figure 14 This is a schematic diagram of another image quality control device provided in an embodiment of this application;

[0037] Figure 15 This is a schematic diagram of another image quality control device provided in an embodiment of this application. Detailed Implementation

[0038] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0039] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0040] In existing deep learning network models, attention mechanism modules or multi-scale feature extraction modules are usually added to the middle or back end of the network. The inventors have found that this design is not conducive to the network capturing the correlation information between feature maps at different levels, which will have a certain impact on the network's processing performance.

[0041] To better understand why existing deep learning network models struggle to capture the relationships between feature maps at different levels, we will first combine... Figure 1 This section provides a brief introduction to existing deep learning networks. (See also...) Figure 1 , Figure 1 This is a schematic diagram of an exemplary existing deep learning network structure. Figure 1 As shown, the left half 101 of the network represents the feature extraction layer, and the right half 102 represents the non-feature extraction layer. Existing deep learning networks typically perform attention activation or multi-scale feature extraction processing on any one of positions 1, 2, and 3.

[0042] like Figure 1As shown, existing technologies only perform attention activation or multi-scale feature extraction at a specific location in the network (such as location 1, location 2, or location 3), and the output of one feature extraction layer becomes the input of the next. Information flow is only possible between two directly connected feature extraction layers; information cannot flow between layers if two feature extraction layers are not directly connected. In this case, adding attention mechanisms or multi-scale feature extraction modules only to specific locations in the network will prevent the reference to features extracted from various layers in the network during the determination of detection results. This may affect the accuracy of the determined detection results and limit the network's processing performance.

[0043] To achieve global cross-level information flow between high and low levels in the model and improve the model's processing performance, this application provides an image processing method. This method extracts multi-level and multi-scale features from the target image, then fuses features from different levels and scales, selects the most valuable feature map combination for detecting the target image, and then fuses the selected feature map combination to obtain the target feature map. Finally, the detection result of the target image is determined based on the target feature map.

[0044] In the above image processing method, the image detection model can spontaneously select the most valuable feature map combination for image detection tasks from the feature maps of different levels and scales extracted by the feature extraction module based on the feature map selection module, so as to fuse them to obtain the target feature map, thereby realizing the information flow between different levels in the image detection model and improving the processing performance of the image detection model.

[0045] It should be noted that the embodiments of this application also provide devices for processing images, such as servers and terminal devices. The server can be an application server or a web server; in actual deployment, it can be a standalone server or a cluster server. The terminal device can specifically be a computer, smartphone, personal digital assistant (PDA), tablet computer, etc.

[0046] The following section describes the server and terminal device for image processing provided in the embodiments of this application from the perspective of hardware implementation.

[0047] See Figure 2 , Figure 2This is a schematic diagram of a server for image processing provided in an embodiment of this application. The server 200 includes a central processing unit 222, a graphics processor 231, a memory 232, and a storage medium 230. The central processing unit 222 or the graphics processor 231, or both, in the server 200 cooperate to call computer programs stored in the storage medium 230, thereby executing the image processing method provided in this embodiment of the application.

[0048] Storage medium 230 can store application programs 242, data 244, and operating system 241 (such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.); memory 232 and storage medium 230 can be temporary or persistent storage. In addition, server 200 may also include power supply 226, wired or wireless network interface 250, and input / output interface 258.

[0049] See Figure 3 , Figure 3 This is a schematic diagram of a terminal device for image processing provided in an embodiment of this application. For ease of explanation, Figure 3 Only the parts relevant to the embodiments of this application are shown. The following description uses a computer as an example of a terminal device.

[0050] like Figure 3 As shown, the computer includes: a radio frequency (RF) circuit 310, a memory 320, an input unit 330, a display unit 340, a sensor 350, an audio circuit 360, a wireless fidelity (WiFi) module 370, a processor 380, and a power supply 390. The memory 320 can be used to store software programs and modules; the processor 380 is the control center of the computer, which can connect to various parts of the computer via interfaces and lines. Specifically, the processor 380 includes a central processing unit 381 and a graphics processing unit 382. The central processing unit 381 or the graphics processing unit 382, ​​or both, cooperate to execute the image processing method provided in this application embodiment by running the software programs and modules stored in the memory 320.

[0051] It should be understood that Figure 2 The server structure shown and Figure 3 The terminal device structures shown are examples. In practical applications, the image processing method provided in this application embodiment can also be applied to servers or terminal devices with other structures.

[0052] To facilitate understanding of the technical solutions provided in the embodiments of this application, the following example of applying the image processing method provided in the embodiments of this application to a server will be used to exemplarily introduce the application scenarios to which the image processing method provided in the embodiments of this application is applicable.

[0053] See Figure 4 , Figure 4 This is a schematic diagram illustrating an application scenario of the image processing method provided in the embodiments of this application. For example... Figure 4 As shown, the application scenario includes: an image acquisition device 401 and an image processing server 402; wherein, the image acquisition device 401 can acquire the image to be processed as the target image under the operation of the operator, and send the target image to the image processing server 402; the image processing server 402 is equipped with an image detection model, and the image processing server 402 is used to execute the image processing method provided in the embodiments of this application to obtain the detection result of the target image.

[0054] In practical applications, the image acquisition device 401 can acquire images in response to user operations. The user can specify the target image to be processed according to their needs and send the target image to the image processing server 402. After receiving the target image, the image processing server 402 can use a pre-deployed image detection model to detect the target image and obtain the detection result.

[0055] It should be noted that the image detection model deployed in the image processing server 402 includes a feature extraction module, a feature map selection module, and a detection result prediction module. Specifically, the feature extraction module performs multi-level feature extraction on the input target image and performs scale normalization on the feature maps extracted at each level to obtain multiple feature maps of different scales. Then, the feature map selection module fuses the multiple feature maps to obtain fused features, determines the weights of each level based on the fused features, and weights the multiple feature maps according to the weights of each level to obtain the target feature map. Finally, the detection result prediction module determines the detection result corresponding to the target image based on the target feature map selected by the feature map selection module.

[0056] Thus, based on an image detection model that includes a feature extraction module, a feature map selection module, and a detection result prediction module, feature maps of different levels and scales are learned from the target image to be processed. The most valuable feature map combination for image detection is selected from these different levels and scales and fused to obtain the target feature map. The detection result corresponding to the target image is then determined based on this target feature map. This method realizes the information flow between high and low levels in the model, thereby improving the detection accuracy of the target image.

[0057] It should be noted that the image processing method provided in this application embodiment can be applied to any scenario and can process any type of image to achieve target detection, recognition, segmentation, etc. in the image. For example, it can be used for face recognition, target region segmentation in pathological images in the medical field, etc. No limitations are made here on the application scenarios to which the image processing method provided in this application embodiment is applicable.

[0058] The image processing method provided in this application will be described below through embodiments.

[0059] See Figure 5 , Figure 5 This is a schematic flowchart illustrating an image processing method provided in an embodiment of this application. For ease of description, the following embodiment uses a server as the execution entity to introduce the image processing method. Figure 5 As shown, the image processing method includes the following steps:

[0060] Step 501: Obtain the target image to be processed.

[0061] In practical applications, images can be acquired using terminal devices with camera capabilities. The image to be processed can then be selected as the target image and uploaded to a server. The server then receives the target image and processes it using its deployed image detection model to obtain the corresponding detection results. In practice, different types of images can be acquired as target images for processing in different application scenarios.

[0062] Taking the method provided in this application embodiment as an example of applying it to a face recognition scenario, the target image to be processed can be an image captured in real time by a terminal device, or an image stored in a terminal device; the terminal device here can specifically be a mobile phone, camera, or other devices; taking the target image as an image captured in real time by a terminal device as an example, after the terminal device captures the image, it can directly upload the captured image as the target image to the server, so that the server can subsequently use the image detection model deployed therein to detect the target image and identify the face region in the target image.

[0063] In practical applications, the terminal device can transmit only one target image to the server each time, or it can transmit multiple target images to the server at once. There is no limit to the number of target images that the server can acquire at one time.

[0064] It should be understood that when the image processing method provided in this application is executed by a terminal device, the terminal device can directly use the image it has acquired or the image it has stored as the target image to be processed, and then directly perform subsequent processing on the target image.

[0065] Furthermore, the image processing method provided in this application embodiment can also be applied to the field of medical image detection to assist doctors in obtaining relevant reference information based on pathological images. In the scenario of detecting pathological images, the pathological images to be processed can be provided by medical personnel or patients.

[0066] In one possible implementation, when the device used to detect pathological images is a server, the server can receive pathological images sent by a first user from a first user terminal and use those pathological images as target images to be processed.

[0067] The aforementioned first user terminal can be a client for relevant medical staff or a client for patients. When the first user terminal is a client for relevant medical staff, the device supporting the operation of the first user terminal can be a terminal device within the hospital. Through this first user terminal, pathological images acquired by relevant image acquisition devices can be used as target images, or specific pathological images can be selected from pathological images stored in a pathological image database as target images, and then the target images can be uploaded to a server used for pathological image detection. When the first user terminal is a client for patients, the device supporting the operation of the first user terminal can be an image uploading device within the hospital or a terminal device used by the patient personally. Through this first user terminal, the patient can specify one or more pathological images as target images, and then upload the target images to a server used for pathological image detection.

[0068] In another possible implementation, when the device used to detect pathological images is a terminal device, the terminal device can directly use the pathological image specified by the user as the target image. Specifically, the user can specify the pathological image acquired in real time by the terminal device as the target image, or the user can select one or more images from the pathological images stored in the terminal device as the target image. After the terminal device determines that the user has determined the target image, it performs relevant detection processing on the target image determined by the user.

[0069] It should be understood that in practical applications, different types of images can be acquired using different image acquisition devices as target images to be processed, depending on different needs. This application does not limit the device used to acquire images, nor does it limit the type of target image acquired.

[0070] Step 502: Perform multi-level feature extraction on the target image and scale normalize the feature maps extracted at each level to obtain multiple feature maps;

[0071] Feature extraction of a target image is performed through multiple layers of multi-level, multi-scale feature extraction. Each layer extracts features of different dimensions. For example, some feature extraction layers are used to extract edge features in the target image, while others are used to extract core features, and so on. Different layers correspond to different feature extraction targets. At the same time, the feature maps extracted by different layers are at different scales. Multi-level feature extraction can also be understood as multi-level, multi-size feature extraction. The feature maps extracted by each layer are of different sizes. In order to facilitate subsequent processing, it is also necessary to perform scale standardization on the feature maps extracted by each layer, so that the feature maps extracted by each layer are uniformly standardized to feature maps of the same scale.

[0072] Step 503: Perform feature fusion on the multiple feature maps to obtain fused features, determine the weights of each of the multiple layers based on the fused features, and weight the multiple feature maps according to the weights of each of the multiple layers to obtain the target feature map;

[0073] After the above multi-level feature extraction and scale unification processes, in order to select the feature maps that are most valuable for subsequent detection, it is also necessary to determine the weights of each level through an attention mechanism. Based on the weights, the feature maps that are valuable for this detection are determined, and these feature maps are then fused into the target feature map.

[0074] Step 504: Determine the detection result corresponding to the target image based on the target feature map.

[0075] After selecting the target feature map, image detection is performed based on the target feature map to determine the corresponding detection result, that is, to detect a specific region in the target image.

[0076] In practical implementation, after the server obtains the target image, it can input the target image into the image detection model deployed on the server, and use the image detection model to perform the relevant processing of the target image in steps 502 to 504 above to obtain the detection result corresponding to the target image.

[0077] It should be noted that the above image detection model includes a feature extraction module, a feature map selection module, and a detection result prediction module. The feature extraction module is used to perform multi-level feature extraction on the input target image and to perform scale normalization on the feature maps extracted at each level to obtain multiple feature maps of the same scale. The feature map selection module is used to select features from the multiple feature maps obtained by the feature extraction module to obtain the target feature map. The detection result prediction module is used to determine the detection result corresponding to the target image based on the target feature map.

[0078] Specifically, the feature map selection module is used to perform feature fusion on the multiple feature maps to obtain fused features, determine the weights of multiple levels according to the fused features, and weight the multiple feature maps according to the weights of the multiple levels to obtain the target feature map.

[0079] In its implementation, the feature map selection module selects the target feature map in the following way:

[0080] First, the multiple feature maps are input into the feature map data fusion layer. The feature map data fusion layer sums the multiple feature maps to obtain a feature fusion map, which is then output to the global pooling layer.

[0081] Then, the features are compressed into a 1x1 vector by the global pooling layer and output to the fully connected layer;

[0082] Next, the 1x1 vector is mapped to the feature space through the fully connected layer, and the weight values ​​of each level feature at each position are obtained by activating the feature space through the activation layer.

[0083] Finally, attention activation (i.e. weighted activation) is performed on the multiple feature maps of each level according to the weight values ​​of the features at each position, and the activated feature maps are fused through the fusion layer to obtain the target feature map.

[0084] To facilitate understanding the working principle of the above image detection model, the following will combine... Figure 6 The processing architecture of an exemplary image detection model is described below. Figure 6 As shown. The image detection model includes: a feature extraction module 601, a feature map selection module 602, and a detection result prediction module 603.

[0085] After the target image to be processed is input into the image detection model, multiple feature extraction layers in the feature extraction module 601 extract features from the target image layer by layer, obtaining feature maps of different scales extracted by each feature extraction layer. Then, scale standardization processing is performed on the feature maps of different scales to unify all feature maps to the same scale. Next, the standardized feature maps are input into the feature map selection module 602, which performs fusion processing based on the input feature maps to obtain the most valuable target feature map. Finally, this target feature map is input into the detection result prediction module 603, which performs detection on the input target feature map and determines the corresponding detection result.

[0086] It is understandable that the above detection results are determined based on actual image processing needs. In different application scenarios, the processing needs for the target image may be different, and correspondingly, the detection results determined by the image detection model for different processing needs will also be different.

[0087] Taking the method provided in this application embodiment as an example of applying it to a face recognition scenario, after the server obtains the target image uploaded by the terminal device, it identifies the face region in the target image based on the image detection model, and then uses the identified face region in the target image as the detection result. Taking the method provided in this application embodiment as an example of applying it to a pathological image detection scenario, after the server obtains the target image uploaded by the doctor or patient through relevant equipment, the target image can be a medical imaging image, which refers to an image generated by medical imaging equipment. Based on the image detection model, a specific region in the target image is identified, which can specifically be a region with abnormal cell morphology. Then, the specific region identified in the target image is used as the detection result to provide relevant reference information for the doctor.

[0088] It should be noted that the image detection model in this application embodiment can also be applied to various other application scenarios. For different application scenarios, the image detection model in this application embodiment can obtain different detection results accordingly. This application does not specifically limit the detection results determined by the image detection model.

[0089] It should be noted that when the method provided in this application is applied to a face recognition scenario, if the execution subject of the method is a server, the server, after determining the face region in the target image based on the image detection model, can return the detection result to the terminal device, so that the terminal device can mark the face region in the target image based on the detection result. If the execution subject of the method is a terminal device, the terminal device can directly mark the face region in the target image based on the face region determined by the image detection model.

[0090] It should be noted that when the method provided in this application embodiment is applied to a pathological image detection scenario, if the executing entity of the method is a server, after the server determines the detection result corresponding to the target image based on the image detection model, it can return the detection result corresponding to the target image to the first user terminal, that is, return the detection result corresponding to the target image to the first user terminal that sent the target image, so that the first user terminal can identify a specific region in the target image based on the detection result it received; and / or, after the server determines the detection result corresponding to the target image based on the image detection model, it can send the detection result corresponding to the target image to the second user terminal of the second user corresponding to the first user, so that the second user terminal can identify a specific region in the target image based on the detection result it received.

[0091] It should be understood that when the first user terminal is a medical staff member, the second user terminal can be the patient corresponding to the target image; when the first user terminal is a patient, the second user terminal can be the medical staff member of that patient.

[0092] When the method provided in this application is applied to a pathological image detection scenario, if the executing entity of the method is a terminal device, after the terminal device determines the detection result corresponding to the target image based on the image detection model, it can directly highlight a specific area on the target image based on the detection result; that is, the terminal device can directly mark a specific area in the target image based on the detection result it determines, such as circling the detected specific area with a highlight line.

[0093] It should be noted that when the method provided in this application embodiment is applied to other application scenarios, other methods can be used to perform subsequent processing on the detection results generated by the image detection model. This application does not limit the subsequent processing methods used for the detection results.

[0094] In the above image processing method, after multi-level feature extraction of the target image, a global screening is performed on the feature maps extracted at each level. The image features of all different information dimensions extracted at all levels are comprehensively considered to select the target feature map that is most valuable to the detection task. The amount of information contained in the target feature map is considered from each feature extraction dimension and is obtained through global screening. Therefore, its information content is the most effective, which can improve the accuracy of the final detection result.

[0095] Furthermore, in practical implementation, a pre-trained image detection model can be used. This model can spontaneously select the most valuable feature map combinations for processing the target task from feature maps of different levels and scales it has learned, and fuse them to obtain the target feature map. This enables information flow between different levels in the image detection model and improves its processing performance. Specifically, the feature map selection module in the image detection model can select feature map combinations from feature maps of different levels and scales extracted by the feature extraction module, and fuse them to obtain the target feature map, thereby realizing information flow between high and low levels in the model. Correspondingly, the detection result prediction module determines the detection result based on the target feature map obtained by fusing the feature maps extracted from multiple levels and scales, which can effectively ensure the accuracy of the determined detection result.

[0096] It should be understood that, in practical applications, whether the image processing method provided in this application can accurately detect the target image to be processed mainly depends on the performance of the image detection model, and the performance of the image detection model is closely related to the training process of the image detection model. The training method of the image quality detection model provided in this application will be introduced below through embodiments.

[0097] See Figure 7 , Figure 7 This is a flowchart illustrating the training method for the image detection model provided in this application embodiment. For ease of description, the following embodiment uses a server as the execution entity to introduce the training method for the image detection model. Figure 7 As shown, the training method for this image detection model includes the following steps:

[0098] Step 701: Obtain the training sample set, wherein each training sample in the training sample set includes a sample image and the corresponding label of the sample image.

[0099] Before training an image detection model, it is usually necessary to obtain a large number of image training samples to form an image training sample set. Each image training sample includes a sample image and a corresponding label. The label is used to identify the standard detection result of the sample image in the detection task to be processed by the image detection model. For example, if the detection task to be processed by the image detection model is to identify the target region in the image, then the label corresponding to the sample image is the location of the target region in the sample image.

[0100] In practical applications, corresponding image training sample sets can be obtained for different needs. For example, in face recognition applications, the server can collect a sufficient number of sample images for face recognition from a database and generate corresponding training samples based on the collected sample images. These training samples include the sample images and corresponding face annotations, which can identify the location of face regions in the sample images. In applications involving pathological image detection, the server can obtain pathological images from a hospital's pathological image database as sample images and generate corresponding training samples based on the collected pathological images. These training samples include the pathological images and corresponding annotations, which can identify the location of specific regions and / or the abnormality types corresponding to specific regions in the pathological images. These annotations can be provided by doctors after annotation.

[0101] In applications involving pathological image detection, the server can perform detection on specific types of pathological images. Taking cervical endoscopy images as an example, the server can retrieve cervical endoscopy images from a hospital's pathological image database, then determine the corresponding annotation labels for each image. These labels can be used to annotate the lesion type corresponding to the lesion area in the cervical endoscopy image. Specifically, lesion types can include low-grade lesions, high-grade lesions, and cervical cancer. Furthermore, the cervical endoscopy images and their corresponding annotation labels are used to form training samples, and multiple training samples obtained in this way form a training sample set. It should be understood that the image detection model trained based on this training sample set can perform lesion segmentation on cervical endoscopy images, that is, identify the lesion area and the corresponding lesion type in the cervical endoscopy image.

[0102] It is understood that the image detection model training method provided in this application embodiment can not only be applied to the above application scenarios, but also to other scenarios according to actual needs; when the image detection model training method provided in this application embodiment is applied to other scenarios, the corresponding training sample set is obtained according to different needs to train the image detection model.

[0103] Step 702: Iteratively update the parameters of the pre-built image detection model using the gradient descent algorithm based on the training sample set until the image detection model is in a convergent state.

[0104] The server pre-builds an image detection model. After obtaining the image training sample set, it inputs the training samples in the training sample set into the pre-built image detection model and iteratively updates the parameters in the model using the gradient descent algorithm until the image detection model is in a convergent state.

[0105] During training, the server inputs sample images from the training image set into a pre-built image detection model. The feature extraction module within the model performs multi-layer feature extraction on the input sample images to obtain multiple feature maps at different scales. These feature maps are then scale-normalized to obtain multiple feature maps at the same scale. The feature map selection model then fuses these multiple feature maps to obtain fused features. Based on these fused features, weights are determined for each level, and the multiple feature maps are weighted according to these weights to obtain a target feature map. The detection result prediction model within the image detection model determines the detection result corresponding to the sample image based on this target feature map. Furthermore, the server calculates the loss function value based on the deviation between the detection result output by the image detection model and the corresponding label in the training samples, and then updates the parameters in the model using a gradient descent algorithm. This process is repeated iteratively to continuously update the model parameters in the image detection model until the model reaches convergence.

[0106] The following section uses the trained image detection model for face recognition of human images as an example to introduce the specific implementation process of step 702.

[0107] The server pre-builds an image detection model. After obtaining the corresponding training sample set (each training sample in the training sample set includes a sample person image and its corresponding label, with the label used to mark the location of the face region in the sample person image), the sample person images in the training sample are input into the image detection model. After processing by each module in the image detection model, the prediction result corresponding to the sample pathological image is obtained. In each iteration, the prediction error between the sample pathological image output by the image detection model and the label corresponding to the sample person image in the training sample is determined. Then, the prediction error is backpropagated to the image detection model, the gradient is calculated based on the gradient descent algorithm, and the parameters of the image detection model are updated accordingly. This iterative update is repeated until the image detection model is in a convergent state, that is, until the detection accuracy of the trained image detection model meets the preset conditions.

[0108] It is understandable that in practical applications, the server can use different types of training samples to train the pre-built image detection model to obtain an image detection model that can handle different detection tasks. This application does not impose any limitations on the detection tasks handled by the image detection model.

[0109] It should be noted that the above image detection model includes a feature extraction module, a feature map selection module, and a detection result prediction module. The feature extraction module may include a feature extraction layer for extracting feature maps of different scales and a size normalization processing layer; the feature map selection module may include a feature map data integration layer, a global pooling layer, a fully connected layer, an activation layer, and a fusion layer; the detection result prediction module may include a convolutional layer, an upsampling layer, and a classification layer.

[0110] The feature extraction module, feature map selection module, and detection result prediction module in the image detection model provided in this application embodiment are described below.

[0111] First, the feature extraction module in the image detection model is introduced. The feature extraction module processes the input target image through multiple feature extraction layers to extract feature images of different scales, and then standardizes these feature images to a uniform scale. In practical applications, the network structure of the feature extraction module (also known as a feature library) can be set according to actual needs. For example, ResCNN-101 networks or VGG networks can be used as feature extraction modules.

[0112] Specifically, taking the ResNet-101 network as an example, this section explains the feature extraction module in an image detection model. (See also...) Figure 8 , Figure 8 This is a network structure diagram of a ResCNN-101 network as the feature extraction module provided in an embodiment of this application. The feature extraction module 800 includes a feature extraction layer 801 and a scale normalization layer 802; wherein the feature extraction layer 801 uses ResNet-101 as the network base.

[0113] The ResNet-101 model comprises five convolutional layers: Conv1, Conv2_x, Conv3_x, Conv4_x, and Conv5_x. Each convolutional layer is followed by an activation layer (e.g., ReLU) and a batch normalization (BN) layer. Conv1 serves as the input layer for the feature extraction module, while Conv2_x, Conv3_x, Conv4_x, and Conv5_x are four cascaded feature extraction layers. These four different feature extraction layers extract features from the target image layer by layer, resulting in feature maps at different scales. Each of the four feature extraction layers is followed by a scale normalization layer to standardize the feature maps output by each feature extraction layer at different scales, resulting in multiple feature maps of the same scale. The specific structural parameters of the ResNet-101 network model are shown in Table 1.

[0114] Table 1 ResNet-101 structural parameters

[0115]

[0116]

[0117] Based on the parameters shown in Table 1 above, the parameters of each layer in the ResNet-101 network model include the kernel size, the number of kernels, and the stride. Taking the input layer Conv1 as an example, the kernel size of this layer is 7x7, the number of kernels is 64, and the stride is 2. The feature extraction layer Conv2_x includes one max pooling layer and three block convolutions. The kernel size in the max pooling layer is 3x3, and the stride is 2. One block convolution includes three convolutional layers with kernel sizes of 1x1,64, 3x3,64, and 1x1,64, respectively. Similarly, the specific structural parameters of the feature extraction layers Conv3_x, Conv4_x, and Conv5_x are shown in the table below, and will not be repeated here.

[0118] Understandably, the target image to be processed is input from Conv1 of the feature extraction module. After passing through convolutional layers, ReLU layers, and BN layers, the output of Conv1 is used as the input to the feature extraction layer Conv2_x. Then, after passing through a max pooling layer and three block convolutions, it is input to the next feature extraction layer Conv3_x and its connected scale normalization layer. The feature extraction layer Conv3_x takes the output of Conv2_x, passes it through four block convolutions, and then inputs it to the next feature extraction layer Conv4_x and its connected scale normalization layer. Similarly, The feature extraction layer Conv4_x takes the output of Conv3_x and passes it through 23 block convolutions before feeding it into the next feature extraction layer Conv5_x and the scale normalization layer connected to it. The feature extraction layer Conv5_x takes the output of Conv4_x and passes it through 3 block convolutions before feeding it into the scale normalization layer connected to it. The scale normalization layer connected to each feature extraction layer performs scale normalization on the input feature map, unifying each feature map to the same scale and number of channels (e.g., 2048 channels), and uses it as input to the feature map selection module.

[0119] It should be noted that in practical applications, the specific network structure of the feature extraction module is not limited to the ResCNN-101 network mentioned above. Other networks that can perform multi-level feature extraction on target images can be used as the basic model for the feature extraction module.

[0120] Next, we will introduce the feature map selection module in the image detection model. (See also...) Figure 9 , Figure 9A schematic diagram of the processing flow for the feature map selection module. Figure 9 Taking the input of two feature maps from the feature map extraction module to the feature map selection module as an example, the processing flow of the feature map selection model is introduced. The feature map selection module shown includes: a feature map data integration layer, a global pooling layer, a fully connected layer, an activation layer, and a fusion layer.

[0121] The feature extraction module processes the two feature maps it obtains. and Input to the feature map selection module; where the feature map and These correspond to the two feature extraction layers in the feature extraction module, and the two feature maps. and It contains feature information at different scales. The feature map selection module obtains the feature map. and Then, the feature map data fusion layer first combines the two input feature maps. and The features are added along the channel direction to obtain a feature fusion tensor U that integrates features from different levels. Then, the global pooling layer and the fully connected layer are connected through a global pooling layer F. gp Optimization and Fully Connected F fc The operation will compress the data into a one-dimensional vector z; where global pooling and fully connected operations can be expressed as equations (1) and (2), respectively:

[0122]

[0123] z = F fc (s)=δ(B(W s (2)

[0124] In equation (1), s is Figure 9 In this context, 's' represents the feature obtained after processing by the global pooling layer; F gp U represents the global pooling layer function; c for Figure 9 U in the figure is a feature map. and The feature map is obtained by adding the features along the channel direction; H and W are the length and width of the feature map, respectively, C is the number of channels of the feature tensor (e.g., C = 2048), and (i,j) represents the pixel coordinates in the feature map.

[0125] In equation (2), z is Figure 9 In the diagram, z represents a feature obtained after processing by a fully connected layer; F fc The fully connected layer function is represented by s; s is the same as s in equation (1); δ and B represent the processing functions of the ReLU layer and BN layer, respectively; W sIn this equation, W represents a nonlinear transformation, and s is the same as s in equation (1). s This means that the nonlinear transformation W compresses s from a length of 2048 to an eigenvector of a specified length.

[0126] After obtaining the one-dimensional vector z, perform two convolutions on the one-dimensional vector z with a kernel size of 1x1 to generate two one-dimensional vectors of length C. Then, activate the one-dimensional vectors generated by the convolution using a classifier softmax and calculate the weights a and b of each one-dimensional vector corresponding to different level features. The weights a and b are defined by equations (3) and (4), respectively:

[0127]

[0128]

[0129] Where a and b represent the weights of the two feature levels, respectively; A c and B c Both are non-linear transformation operations, specifically two 1x1 convolution operations, used to transform a one-dimensional vector z into a one-dimensional vector of length C. A c z represents the result of nonlinear transformation A c Convert z (which is the z in equation (2)) into a vector of length 2048, B c z represents the result of nonlinear transformation B c Convert z (which is the z in equation (2)) into a vector of length 2048; convert A c z and B c z, as the exponent of the Euler number e, participates in the weight calculation in equations (3) and (4).

[0130] Finally, the activation layer and fusion layer adjust the feature maps of different levels based on the calculated weight information a and b. and Activation and fusion are performed to obtain a target feature map V that integrates features from different levels, and this map is then input into the detection result prediction module. The activation process of the weights on the feature map is similar to the activation process of the SE-Net network model, and the fusion process can specifically involve summing the activated feature maps.

[0131] It is understandable that when there are multiple levels of feature maps, the feature map selection module can adopt a similar operation process to the above-described operation on two levels of feature maps. For example, when the feature extraction module obtains three feature maps, the feature map selection module can select the target feature map by using the above-described processing method for these three feature maps. This application does not impose any limitation on the number of feature maps processed by the feature map selection module.

[0132] Next, the detection result prediction module in the image detection model will be introduced. See [link / reference] Figure 10 , Figure 10 This is a schematic diagram of the detection result prediction module. Figure 10 The detection result prediction module shown includes: convolutional layer 1001, upsampling layer 1002, and classification prediction layer 1003.

[0133] After the feature map selection module inputs the target feature map V into the detection result prediction module, the convolutional layer 1001 and the upsampling layer 1002 perform convolution and upsampling processing on the target feature map V to obtain an image with the same scale as the target image. Then, the classification prediction layer 1003 uses the classifier softmax to detect the image and obtain the detection result corresponding to the target image. The detection result can be the location of the target region contained in the target image, such as the location of the face region contained in the target image, or the location of a specific region contained in the pathological image, or the location of other regions, which is not limited here.

[0134] In the training method of the aforementioned image detection model, the server first acquires a training sample set, in which each sample includes a sample image and its corresponding label. Then, the parameters of the pre-built image detection model are iteratively updated using the gradient descent algorithm based on the training sample set until the image detection model meets the convergence condition. The image detection model trained based on this method can learn feature maps at different levels and scales based on its feature extraction module, and select the most valuable feature map combination for image detection tasks from the feature maps at different levels and scales based on the feature map selection module, so as to fuse them to obtain the target feature map. This realizes the flow of information between different levels in the model, which helps to improve the processing performance of the model. The detection result prediction module in the model further determines the corresponding detection result based on the above target feature map, which can effectively ensure the accuracy of the determined detection result.

[0135] To facilitate a further understanding of the image processing method provided in the embodiments of this application, the following example of using the image processing method provided in the embodiments of this application to detect a specific region in a cervical endoscopy image will be used to provide an overall introduction to the image processing method provided in the embodiments of this application.

[0136] refer to Figure 11 , Figure 11 This is a schematic diagram illustrating the implementation architecture of the image processing method provided in this application embodiment. In the scenario of detecting lesion areas in cervical endoscopic images, it includes a pathological image database 1101, a terminal 1102, and a server 1103.

[0137] The pathological image database 1101 stores cervical endoscopy images of different patients. When training an image detection model for detecting cervical endoscopy images, the server 1103 can obtain a large number of cervical endoscopy images from the pathological image database 1101. These images are then labeled with corresponding tags by relevant experts. These tags identify specific regions in the cervical endoscopy images and the corresponding abnormality types. Training samples are formed using the cervical endoscopy images and their corresponding tags. Multiple training samples in this way are then used to form a training sample set. The server 1103 uses this training sample set to iteratively train the model parameters in the pre-built image detection model until the image detection model converges. Once the training of the image detection model is complete, it can be applied to actual detection tasks.

[0138] When using an image processing model to detect cervical endoscopy images, the user can send the cervical endoscopy image to be detected to the server 1103 via terminal 1102. After receiving the cervical endoscopy image, the server 1103 uses it as the target image to be processed and inputs it into a pre-trained image detection model. Multiple feature extraction layers in the feature extraction module of the image detection model extract features from the target image layer by layer, thereby obtaining the feature map extracted by each feature extraction layer. The feature maps are then scaled to obtain multiple feature maps with uniform scale, and these multiple feature maps are input into the feature map selection module. The feature map selection module selects features from these multiple feature maps to obtain the target feature map, and inputs the target feature map into the detection result prediction module. The detection result prediction module determines the detection result corresponding to the target image based on the target feature map, that is, it determines the specific region corresponding to the input cervical endoscopy image and the type of abnormality present in the specific region.

[0139] After determining the detection result corresponding to the cervical endoscopy image based on the image detection model, the server 1103 returns the detection result to the terminal 1102 so that the terminal 1102 can highlight specific areas based on the cervical endoscopy image it sends.

[0140] In conjunction with the image processing method described above, this application also provides a corresponding image processing apparatus to enable the practical application and implementation of the above-described image processing method.

[0141] See Figure 12 , Figure 12 For the above text Figure 5 The image processing method shown is illustrated in the schematic diagram of an image processing apparatus 1200, which includes:

[0142] Target image acquisition module 1201 is used to acquire the target image to be processed;

[0143] Prediction module 1202 is used to perform multi-level feature extraction on the target image and scale normalization on the feature map extracted at each level to obtain multiple feature maps;

[0144] The prediction module 1202 is further configured to perform feature fusion on the plurality of feature maps to obtain fused features, determine the weights of each of the plurality of levels according to the fused features, and weight the plurality of feature maps according to the weights of each of the plurality of levels to obtain a target feature map;

[0145] The prediction module 1202 is further configured to determine the detection result corresponding to the target image based on the target feature map.

[0146] Optionally, the prediction module is specifically used to perform multi-level feature extraction on the target image through the feature extraction module in the pre-trained image detection model and to perform scale normalization on the feature maps extracted at each level to obtain multiple feature maps; and to perform feature fusion on the multiple feature maps through the feature map selection module in the image detection model to obtain fused features, determine the weights of each of the multiple levels according to the fused features, weight the multiple feature maps according to the weights of each of the multiple levels to obtain a target feature map, and determine the detection result corresponding to the target image through the detection result prediction module in the image detection model based on the target feature map.

[0147] The prediction module, when selecting a target feature map based on the feature map selection module, specifically performs the following steps: inputting the multiple feature maps into the feature map data fusion layer; summing the multiple feature maps through the feature map data fusion layer to obtain a feature fusion map; outputting the feature fusion map to the global pooling layer; compressing the feature fusion map into a 1x1 vector through the global pooling layer; outputting the 1x1 vector to the fully connected layer; mapping the 1x1 vector to the feature space through the fully connected layer; activating the features at each level at each position through the activation layer in the feature space to obtain the weight values ​​of each level feature at each position; performing attention activation (i.e., weighted activation) on the multiple feature maps at each level according to the weight values ​​of each level feature at each position; and performing feature fusion on the activated feature maps through the fusion layer to obtain the target feature map.

[0148] Optional, in Figure 12 Based on the image processing apparatus shown, see Figure 13 , Figure 13 This is a schematic diagram of another image quality control device 1300 provided in an embodiment of this application. In device 1300, the target image acquisition module 1201 is specifically used to: receive an image of a first user sent by a first user terminal as the target image to be processed;

[0149] The device 1300 also includes:

[0150] Feedback module 1301 is used to return the detection result corresponding to the target image to the first user terminal; and / or send the detection result corresponding to the target image to the second user terminal of the second user corresponding to the first user.

[0151] Optional, in Figure 12 Based on the image processing apparatus shown, see Figure 14 , Figure 14 This is a schematic diagram of another image quality control device 1400 provided in an embodiment of this application. In the device 1400, the target image acquisition module 1201 is specifically used to: determine the image specified by the user as the target image to be processed;

[0152] The device 1400 also includes:

[0153] Display module 1401 is used to highlight the detection results corresponding to the target image based on the target image.

[0154] Optional, in Figure 12 Based on the image processing apparatus shown, see Figure 15 , Figure 15 This is a schematic diagram of another image quality control device 1500 provided in an embodiment of this application. The device 1500 further includes:

[0155] The training data acquisition module 1501 is used to acquire a training sample set, wherein each training sample in the training sample set includes a sample image and a corresponding label for the sample image.

[0156] The model training module 1502 is used to iteratively update the parameters of the pre-built image detection model using a gradient descent algorithm based on the training sample set until the image detection model is in a convergent state. The image detection model includes a feature extraction module, a feature map selection module, and a detection result prediction module. The feature extraction module includes convolutional layers and size normalization layers for extracting feature maps of different scales. The feature map selection module includes a feature map data fusion layer, a global pooling layer, a fully connected layer, an activation layer, and a fusion layer. The detection result prediction module includes convolutional layers, upsampling layers, and a classification layer.

[0157] Optional, in Figure 15 Based on the image processing device shown, each training sample in the training sample set includes a sample human image and a corresponding label, wherein the label is used to mark the position corresponding to the face region in the human image; then the model training module 1502 is specifically used for:

[0158] The sample person image is input into a pre-constructed image detection model to obtain the prediction result corresponding to the sample person image output by the image detection model.

[0159] In each iteration, the prediction error is determined based on the prediction result and label corresponding to the sample person image. The prediction error is then backpropagated to the image detection model to calculate the gradient and update the parameters of the image detection model. This iterative update continues until the image detection model is in a convergent state.

[0160] Optional, in Figure 12 Based on the image processing device shown, the feature extraction module includes four cascaded feature extraction layers and a scale normalization layer connected to each feature extraction layer. Different feature extraction layers correspond to different convolution scales. The feature extraction layers input the extracted feature maps into the connected scale normalization layer, which is used to perform scale normalization processing on the received feature maps.

[0161] In the aforementioned image processing apparatus, the image detection model can spontaneously select the most valuable feature map combinations for the target task from feature maps of different levels and scales it has learned, and fuse them to obtain the target feature map. This enables information flow between different levels in the image detection model and improves its processing performance. Specifically, the feature map selection module in the image detection model can select feature map combinations from feature maps of different levels and scales extracted by the feature extraction module, and fuse them to obtain the target feature map, thereby realizing information flow between high and low levels in the model. Correspondingly, the detection result prediction module determines the detection result based on the target feature map obtained by fusing feature maps of different levels and scales, which can effectively ensure the accuracy of the determined detection result.

[0162] This application also provides a computer-readable storage medium for storing program code that executes any one of the implementation methods of the image processing method described in the foregoing embodiments.

[0163] This application also provides a computer program product including instructions that, when run on a computer, cause the computer to execute any one of the implementation methods of the image processing method described in the foregoing embodiments.

[0164] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0165] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0166] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0167] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0168] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0169] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An image processing method, characterized in that, The method, applicable to global cross-level information flow between high and low levels, includes: Acquire the target image to be processed, wherein the target image is a medical image; The feature extraction module in the pre-trained image detection model performs multi-level feature extraction on the target image from different feature extraction layers, and performs scale normalization on the feature maps extracted at each layer to obtain multiple feature maps. The feature extraction dimensions, feature extraction targets, and feature map scales are different in different feature extraction layers. The feature extraction module is a ResCNN-101 network, and the scale normalization process unifies feature maps of different scales into feature maps of the same scale. The multiple feature maps are input into the feature map data fusion layer, and the multiple feature maps are summed in the channel direction through the feature map data fusion layer to obtain a feature fusion map, which is then output to the global pooling layer. The features are compressed into a 1x1 vector by the global pooling layer using the following formula (1), and the 1x1 vector is output to the fully connected layer. Formula (1); s represents the feature obtained after processing by the global pooling layer. This represents the global pooling layer function. Here, H and W represent the length and width of the feature map, respectively; C represents the number of channels in the feature tensor; and (i,j) represents the pixel coordinates in the feature map. The 1x1 vector is mapped to the feature space by the fully connected layer using the following formula (2), and the weight values ​​of each layer feature at each position are obtained by activation by the activation layer in the feature space. Formula (2); Z represents the feature obtained after processing by the fully connected layer; Represents the fully connected layer function; and These represent the processing functions of the ReLU layer and the BN layer, respectively. W in the equation represents a nonlinear transformation. Attention activation is performed on the multiple feature maps of each level according to the weight values ​​of each feature at each position, and the activated feature maps are fused through a fusion layer to obtain the target feature map. The amount of information included in the target feature map is obtained by considering each feature extraction dimension and globally filtering. The detection result corresponding to the target image is determined based on the target feature map; Based on the detection results corresponding to the target image, the target region is highlighted on the target image.

2. The image processing method according to claim 1, characterized in that, The target image is subjected to multi-level feature extraction by the feature extraction module in the pre-trained image detection model, and the feature map extracted at each level is scaled to obtain multiple feature maps; as well as, The feature map selection module in the image detection model performs feature fusion on the multiple feature maps to obtain fused features. The weights of multiple layers are determined based on the fused features. The multiple feature maps are then weighted according to the weights of the multiple layers to obtain the target feature map. The detection result prediction module in the image detection model determines the detection result corresponding to the target image based on the target feature map.

3. The image processing method according to claim 2, characterized in that, The method further includes: Obtain a training sample set, wherein each training sample in the training sample set includes a sample image and a corresponding label for the sample image; The parameters of the pre-built image detection model are iteratively updated using the gradient descent algorithm based on the training sample set until the image detection model is in a convergent state. The image detection model includes a feature extraction module, a feature map selection module, and a detection result prediction module. The feature extraction module includes a feature extraction layer and a size normalization layer for extracting feature maps at different scales. The feature map selection module includes a cascaded feature map data fusion layer, a global pooling layer, a fully connected layer, an activation layer, and a fusion layer, used to fuse the multiple feature maps to obtain fused features. Based on the fused features, the weights of each layer are determined, and the multiple feature maps are weighted according to their respective weights to obtain a target feature map. The detection result prediction module includes a convolutional layer, an upsampling layer, and a classification layer.

4. The image processing method according to claim 3, characterized in that, Each training sample in the training sample set includes a sample person image and a corresponding label, wherein the label is used to mark the position of the face region in the person image; The step of iteratively updating the parameters of the pre-built image detection model using the gradient descent algorithm based on the training sample set until the image detection model is in a convergent state includes: The sample person image is input into a pre-constructed image detection model to obtain the prediction result corresponding to the sample person image output by the image detection model. In each iteration, the prediction error is determined based on the prediction result and label corresponding to the sample person image. The prediction error is then backpropagated to the image detection model to calculate the gradient and update the parameters of the image detection model. This iterative update continues until the image detection model is in a convergent state.

5. The image processing method according to any one of claims 2 to 4, characterized in that, The feature extraction module includes four cascaded feature extraction layers and a scale normalization layer connected to each feature extraction layer. Different feature extraction layers correspond to different convolution scales. The feature extraction layers input the extracted feature maps into the connected scale normalization layer, which is used to perform scale normalization processing on the received feature maps.

6. The image processing method according to claim 1, characterized in that, The acquisition of the target image to be processed includes: Receive the image sent by the first user terminal as the target image to be processed; The method further includes: Return the detection result corresponding to the target image to the first user terminal; and / or, The detection result corresponding to the target image is sent to the second user terminal of the second user corresponding to the first user.

7. The image processing method according to claim 1, characterized in that, The acquisition of the target image to be processed includes: The user-specified image is selected as the target image to be processed.

8. An image processing apparatus, characterized in that, The device, used for global cross-level information flow between high and low levels, includes: The target image acquisition module is used to acquire the target image to be processed, wherein the target image is a medical image. The prediction module is used to perform multi-level feature extraction on the target image from different feature extraction layers through the feature extraction module in the pre-trained image detection model, and to perform scale normalization on the feature maps extracted at each layer to obtain multiple feature maps. The feature extraction layers have different feature dimensions, feature extraction targets, and feature map scales. The feature extraction module is a ResCNN-101 network, and the scale normalization process unifies feature maps of different scales into feature maps of the same scale. The prediction module is further configured to input the plurality of feature maps into the feature map data fusion layer, sum the plurality of feature maps in the channel direction through the feature map data fusion layer to obtain a feature fusion map, and output the feature fusion map to the global pooling layer; The features are compressed into a 1x1 vector by the global pooling layer using the following formula (1), and the 1x1 vector is output to the fully connected layer. Formula (1); s represents the feature obtained after processing by the global pooling layer. This represents the global pooling layer function. Here, H and W represent the length and width of the feature map, respectively; C represents the number of channels in the feature tensor; and (i,j) represents the pixel coordinates in the feature map. The 1x1 vector is mapped to the feature space by the fully connected layer using the following formula (2), and the weight values ​​of each layer feature at each position are obtained by activation by the activation layer in the feature space. Formula (2); Z represents the feature obtained after processing by the fully connected layer; Represents the fully connected layer function; and These represent the processing functions of the ReLU layer and the BN layer, respectively. W in the equation represents a nonlinear transformation. Attention activation is performed on the multiple feature maps of each level according to the weight values ​​of each feature at each position, and the activated feature maps are fused through a fusion layer to obtain the target feature map. The amount of information included in the target feature map is obtained by considering each feature extraction dimension and globally filtering. The prediction module is further configured to determine the detection result corresponding to the target image based on the target feature map; The display module is used to highlight the target area based on the detection results corresponding to the target image.

9. The image processing apparatus according to claim 8, characterized in that, The prediction module is specifically used to perform multi-level feature extraction on the target image through the feature extraction module in the pre-trained image detection model, and to perform scale standardization on the feature maps extracted at each level to obtain multiple feature maps; and to perform feature fusion on the multiple feature maps through the feature map selection module in the image detection model to obtain fused features, determine the weights of each of the multiple levels according to the fused features, and weight the multiple feature maps according to the weights of each of the multiple levels to obtain a target feature map, and to determine the detection result corresponding to the target image through the detection result prediction module in the image detection model based on the target feature map.

10. The image processing apparatus according to claim 8, characterized in that, The device further includes: The training data acquisition module is used to acquire a training sample set, wherein each training sample in the training sample set includes a sample image and a corresponding label for the sample image; The model training module is used to iteratively update the parameters of a pre-built image detection model using a gradient descent algorithm based on the training sample set until the image detection model reaches a convergent state. The image detection model includes a feature extraction module, a feature map selection module, and a detection result prediction module. The feature extraction module includes convolutional layers and size normalization layers for extracting feature maps at different scales. The feature map selection module includes a feature map data fusion layer, a global pooling layer, a fully connected layer, an activation layer, and a fusion layer. The detection result prediction module includes convolutional layers, upsampling layers, and a classification layer.

11. A device, characterized in that, include: Processor and memory, among which, The memory is used to store computer programs; The processor is configured to execute the image processing method according to any one of claims 1 to 7 according to the computer program.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program for performing the method according to any one of claims 1 to 7.

Images