Image detection method and device, electronic equipment and computer readable storage medium

By performing feature extraction and diffusion processing on the image to generate noisy images, filtering and processing candidate boxes, the problem of inaccurate detection of small objects in the prior art is solved, and higher detection accuracy is achieved.

CN118918382APending Publication Date: 2024-11-08PING AN TECH (SHENZHEN) CO LTD

Patent Information

Application Number
CN202411064727.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing example segmentation technology detects small objects due to the limitation of input image resolution, especially the problem that the size and shape of the small objects do not match the anchor points of fixed size and proportion.

Method used

Noise images are generated by feature extraction and diffusion processing of the image to be detected, initial candidate boxes are generated and selected candidate boxes are filtered, prediction and reverse diffusion processing are performed to obtain a clear mask, and finally classification processing is performed to improve the detection accuracy of small objects.

Benefits of technology

The recognition process is converted into a diffusion process from noise to mask, reducing the dependence on the resolution of the input image, improving the detection and segmentation capabilities of small objects, making the detection and recognition of small objects in the image more accurate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918382A_ABST
    Figure CN118918382A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of medical science and technology, and provides an image detection method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: obtaining a to-be-detected image; performing feature extraction on the to-be-detected image to obtain a feature map; performing diffusion processing on the feature map to obtain a noise image; generating a plurality of initial candidate frames on the noise image, and screening out a plurality of selected candidate frames from the plurality of initial candidate frames according to attribute information of each initial candidate frame; performing prediction processing on the mask of each selected candidate frame on the noise image to obtain a noise mask; performing reverse diffusion processing on each noise mask to obtain a clear mask corresponding to the selected candidate box; and performing classification processing on each clear mask to obtain a classification identification result corresponding to the selected candidate box. According to the technical scheme, the small object in the image can be detected and recognized more accurately.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present application relate to but are not limited to the field of medical technology, and in particular to an image detection method, device, electronic device, and computer-readable storage medium. Background Art

[0002] With the continuous improvement of medical technology, instance segmentation technology has been widely used in the medical field, such as the detection of cells, tissues and lesions. However, existing instance segmentation techniques are usually limited by the resolution of the input image, resulting in some small objects that cannot be accurately detected and segmented. For example, current instance segmentation methods generally use a region generation network to generate candidate boxes, then use the region of interest alignment layer to extract the features of each candidate box, and finally use classification, regression and mask branches to output categories, bounding boxes and masks. However, because the region generation network uses anchors of fixed size and scale to generate candidate boxes, the size and shape of small objects may not match the anchors, resulting in inaccurate detection of small objects. Summary of the invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] In order to solve the problems mentioned in the above background technology, the embodiments of the present application provide an image detection method, device, electronic device and computer-readable storage medium, so that the detection and identification of small objects in images can be more accurate.

[0005] In a first aspect, an embodiment of the present application provides an image detection method, comprising:

[0006] Acquire the image to be detected;

[0007] Extracting features from the image to be detected to obtain a feature map;

[0008] Performing diffusion processing on the feature map to obtain a noise image;

[0009] Generating a plurality of initial candidate frames on the noise image, and selecting a plurality of selected candidate frames from the plurality of initial candidate frames according to attribute information of each of the initial candidate frames;

[0010] Performing prediction processing on the mask of each of the selected candidate boxes on the noise image to obtain a noise mask;

[0011] Performing a reverse diffusion process on each of the noise masks to obtain a clear mask corresponding to the selected candidate box;

[0012] Classification processing is performed on each of the clear masks to obtain a classification recognition result corresponding to the selected candidate box.

[0013] According to some embodiments of the present application, the step of performing diffusion processing on the feature map to obtain a noise image includes:

[0014] Determine a preset proportion of Gaussian noise from a preset diffusion model;

[0015] The Gaussian noise is added to each pixel in the feature image to obtain the noise image.

[0016] According to some embodiments of the present application, the attribute information includes candidate box confidence and candidate box score information, and the generating of multiple initial candidate boxes on the noisy image and screening out a plurality of selected candidate boxes from the multiple initial candidate boxes according to the attribute information of each of the initial candidate boxes includes:

[0017] Generate a plurality of initial candidate boxes on the noise image using a pre-trained region generation network;

[0018] Analyze and process each of the initial candidate boxes to obtain the candidate box confidence and the candidate box score information;

[0019] Comparing the candidate box confidence with a preset confidence threshold to obtain confidence comparison information; and comparing the candidate box score information with a preset candidate box score threshold to obtain score comparison information;

[0020] According to the confidence comparison information and the score comparison information, a plurality of the selected candidate frames are selected from the plurality of the initial candidate frames.

[0021] According to some embodiments of the present application, performing prediction processing on the mask of each selected candidate box on the noise image to obtain a noise mask includes:

[0022] Performing feature detection on each of the selected candidate frames to obtain candidate frame feature information;

[0023] Generate an initial mask according to the feature information of the candidate box, wherein the initial mask carries a precision value;

[0024] Comparing the precision value of the initial mask with a preset precision threshold;

[0025] When the precision value of the initial mask is greater than or equal to the precision threshold, the initial mask is determined as the noise mask.

[0026] According to some embodiments of the present application, performing inverse diffusion processing on each of the noise masks to obtain a clear mask corresponding to the selected candidate box includes:

[0027] determining a mapping function from a recurrent neural network in the diffusion model;

[0028] The noise mask is converted into the clear mask corresponding to the selected candidate box according to the mapping function.

[0029] According to some embodiments of the present application, the classifying process of each clear mask to obtain a classification recognition result corresponding to the selected candidate box includes:

[0030] Analyzing and processing the features of each clear mask to obtain mask feature information;

[0031] According to the pre-trained classifier and the mask feature information, the clear mask is divided and processed to obtain the classification recognition result corresponding to the selected candidate box.

[0032] According to some embodiments of the present application, after classifying each of the clear masks to obtain a classification recognition result corresponding to the selected candidate box, the method further includes:

[0033] Comparing the classification recognition result with the description text of the image to be detected to obtain a comparison loss;

[0034] The network parameters of the diffusion model are adjusted according to the contrast loss.

[0035] In a second aspect, the embodiment of the present application further provides an image detection device, including:

[0036] A first processing module, used for acquiring an image to be detected;

[0037] A second processing module is used to extract features from the image to be detected to obtain a feature map;

[0038] A third processing module, used for performing diffusion processing on the feature map to obtain a noise image;

[0039] A fourth processing module, configured to generate a plurality of initial candidate frames on the noise image, and select a plurality of selected candidate frames from the plurality of initial candidate frames according to attribute information of each of the initial candidate frames;

[0040] A fifth processing module, configured to perform prediction processing on the mask of each of the selected candidate boxes on the noise image to obtain a noise mask;

[0041] A sixth processing module, configured to perform inverse diffusion processing on each of the noise masks to obtain a clear mask corresponding to the selected candidate box;

[0042] The seventh processing module is used to classify each of the clear masks to obtain a classification recognition result corresponding to the selected candidate box.

[0043] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image detection method as described in the first aspect above is implemented.

[0044] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the image detection method described in the first aspect above.

[0045] The image detection method according to the embodiment provided in the present application has at least the following beneficial effects: in the process of image detection processing, firstly, the image to be detected is acquired, then the feature extraction of the image to be detected is performed to obtain a feature map, and then the feature map is diffused to obtain a noise image; then, multiple initial candidate frames are generated on the noise image, and a number of selected candidate frames are screened out from the multiple initial candidate frames according to the attribute information of each initial candidate frame; then, the mask of each selected candidate frame on the noise image is predicted to obtain a noise mask; then, each noise mask is reversely diffused to obtain a cleaning mask corresponding to the selected candidate frame; finally, each clear mask is classified to obtain a classification recognition result corresponding to the selected candidate frame; through the above technical scheme, the recognition process is converted into a diffusion process from noise to mask, which does not need to rely on complex feature extraction and mask prediction, thereby reducing the dependence on the resolution of the input image, improving the detection and segmentation capabilities of small objects, and making the detection and recognition of small objects in the image more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0047] Figure 1 is a flow chart of an image detection method provided by an embodiment of the present application;

[0048] Figure 2 is a specific flow chart of step S300 provided in one embodiment of the present application;

[0049] Figure 3 is a specific flow chart of step S600 provided in one embodiment of the present application;

[0050] Figure 4 is a specific flow chart of step S200 provided in one embodiment of the present application;

[0051] Figure 5 is a specific flow chart of step S210 provided in one embodiment of the present application;

[0052] Figure 6 is a specific flow chart of step S240 provided in one embodiment of the present application;

[0053] Figure 7 is a flow chart of an image detection method provided by another embodiment of the present application;

[0054] Figure 8 is a schematic diagram of a speech recognition device provided by an embodiment of the present application;

[0055] Fig. 9 It is a schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] It should be noted that although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification and claims and the above drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0058] It should be noted that, unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0059] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0060] AI is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. Artificial intelligence is a branch of computer science. Artificial intelligence attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is also a theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0061] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0062] Artificial intelligence is AI. AI is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0063] The servers involved in artificial intelligence technology can be independent servers or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0064] The present application provides an image detection method, device, electronic device and computer-readable storage medium. In the process of image detection processing, first, an image to be detected is obtained, then feature extraction is performed on the image to be detected to obtain a feature map, and then diffusion processing is performed on the feature map to obtain a noise image; then multiple initial candidate frames are generated on the noise image, and a number of selected candidate frames are screened out from the multiple initial candidate frames according to the attribute information of each initial candidate frame; then the mask of each selected candidate frame on the noise image is predicted to obtain a noise mask; then each noise mask is reversely diffused to obtain a cleaning mask corresponding to the selected candidate frame; finally, each clear mask is classified to obtain a classification recognition result corresponding to the selected candidate frame; through the above technical solution, the recognition process is converted into a diffusion process from noise to mask, which does not need to rely on complex feature extraction and mask prediction, thereby reducing the dependence on the resolution of the input image, improving the detection and segmentation capabilities of small objects, and making the detection and recognition of small objects in the image more accurate.

[0065] The image detection method provided in the embodiment of the present application relates to the field of medical technology. The image detection method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or can be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the image detection method, etc., but is not limited to the above forms.

[0066] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0067] It should be noted that in each specific implementation of the present application, when it comes to the need to perform relevant processing based on data related to user identity or characteristics such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0068] The embodiments of the present application are further described below in conjunction with the accompanying drawings.

[0069] like Figure 1 As shown, Figure 1 It is a flowchart of an image detection method provided by an embodiment of the present application. The image detection method includes but is not limited to steps S100 to S700.

[0070] Step S100, obtaining an image to be detected;

[0071] Step S200, extracting features from the image to be detected to obtain a feature map;

[0072] Step S300, performing diffusion processing on the feature map to obtain a noise image;

[0073] Step S400, generating a plurality of initial candidate frames on the noise image, and selecting a plurality of selected candidate frames from the plurality of initial candidate frames according to attribute information of each initial candidate frame;

[0074] Step S500, predicting the mask of each selected candidate box on the noise image to obtain a noise mask;

[0075] Step S600, performing reverse diffusion processing on each noise mask to obtain a clear mask corresponding to the selected candidate box;

[0076] Step S700 , classifying each clear mask to obtain a classification recognition result corresponding to the selected candidate box.

[0077] It should be noted that in the process of image detection and processing, the image to be detected is first obtained, then the feature extraction of the image to be detected is performed to obtain a feature map, and then the feature map is diffused to obtain a noise image; then multiple initial candidate frames are generated on the noise image, and a number of selected candidate frames are screened out from the multiple initial candidate frames according to the attribute information of each initial candidate frame; then the mask of each selected candidate frame on the noise image is predicted to obtain a noise mask; then each noise mask is reversely diffused to obtain a cleaning mask corresponding to the selected candidate frame; finally, each clear mask is classified to obtain a classification recognition result corresponding to the selected candidate frame; through the above technical scheme, the recognition process is converted into a diffusion process from noise to mask, which does not need to rely on complex feature extraction and mask prediction, thereby reducing the dependence on the resolution of the input image, improving the detection and segmentation capabilities of small objects, and making the detection and recognition of small objects in the image more accurate.

[0078] It is worth noting that the image to be detected may be a medical image, and in the process of obtaining the medical image, the user's permission is required. The user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. Only after the user's separate permission or consent is clearly obtained can the medical image be subjected to subsequent image detection and processing. The user's relevant privacy information will also be kept confidential, and relevant laws, regulations and standards will be respected.

[0079] It is worth noting that in the process of detecting and processing the image to be detected, the feature map can be obtained by first extracting the features of the image to be detected, and then the noise image can be obtained by diffusing the feature map, converting the recognition process into a diffusion process from noise to mask, which does not need to rely on complex feature extraction and mask prediction, thereby reducing the dependence on the resolution of the input image, improving the detection and segmentation capabilities of small objects, and making the detection and recognition of small objects in the image more accurate. Exemplarily, a diffusion model is used to add Gaussian noise to each pixel in the image to obtain a noise image. The diffusion model is a probability-based generation model that can diffuse a low-dimensional Gaussian noise into the image space, thereby realizing the generation process from noise to data. Its structure is: the input end is the input image, the output end is the output noise image, and the middle is the various steps of the diffusion model. The diffusion model consists of multiple diffusion steps, each of which adds a certain proportion of Gaussian noise to each pixel in the input image, so that the input image gradually becomes blurred and noisy. Finally, the input image will be completely diffused into a low-dimensional Gaussian noise. Applying the diffusion model to the instance segmentation task in this way allows instance segmentation to recover clear object shapes from Gaussian noise instead of relying on complex feature extraction and mask prediction.

[0080] It is worth noting that multiple initial candidate boxes are generated on the noisy image, and then a number of selected candidate boxes are screened out from the multiple initial candidate boxes according to the attribute information of each initial candidate box, and then the mask of each selected candidate box on the noisy image is predicted to obtain the noise mask; then each noise mask is subjected to reverse diffusion processing to obtain a clear mask corresponding to the selected candidate box, and then each clear mask is classified to obtain the classification recognition result corresponding to the selected candidate box, thereby improving the recognition accuracy of small objects in the image.

[0081] It is worth noting that multiple initial candidate boxes are generated on the noisy image, and then several selected candidate boxes can be screened out from the multiple initial candidate boxes according to the attribute information of each initial candidate box, and then the mask of each selected candidate box on the noisy image is predicted and processed to obtain the noise mask. The candidate box generation region proposal network is used to generate candidate boxes on the noisy image, and high-quality candidate boxes are screened out using non-maximum suppression. The candidate box generator is a method for detecting areas where objects may exist in an image. It can generate some candidate boxes based on information such as image features or edges, and screen out high-quality candidate boxes based on indicators such as the confidence or score of the candidate boxes. This application uses a candidate box generator to assist the diffusion model in instance segmentation, thereby simplifying the instance segmentation process and improving the efficiency of instance segmentation. The mask generator is used to predict the mask of each candidate box on the noisy image to obtain a set of noise masks. The mask generator is a method for predicting the category and instance label of each pixel inside each candidate box. It can generate some masks based on information such as the features or positions of the candidate boxes, and optimize the prediction of the mask based on indicators such as the accuracy or loss of the mask. This application uses a mask generator to assist the diffusion model in instance segmentation, thereby simplifying the instance segmentation process and improving the efficiency of instance segmentation. Applying the diffusion model to the instance segmentation task allows instance segmentation to recover clear object shapes from Gaussian noise, rather than relying on complex feature extraction and mask prediction. The diffusion model is used to transform the instance segmentation task into a diffusion process from noise to mask, thereby avoiding dependence on the input image resolution and improving the detection and segmentation capabilities of small objects.

[0082] It is worth noting that the clear mask corresponding to the selected candidate box can be obtained by reverse diffusion of each noise mask; the noise mask of each candidate box is reversely diffused using the diffusion model to obtain the clear mask of each candidate box. The diffusion model can not only diffuse noise to data, but also reversely diffuse data to noise, thereby realizing the reverse generation process from data to noise. Applying the diffusion model to the instance segmentation task allows instance segmentation to recover clear object shapes from noise masks instead of relying on complex feature extraction and mask prediction. The diffusion model uses a recurrent neural network to learn the mapping function from noise mask to clear mask, thereby realizing the diffusion process from Gaussian noise to instance mask. The diffusion model can complete the diffusion in one or more steps, thereby improving the segmentation accuracy of small objects.

[0083] It is worth noting that each clear mask is classified to obtain a classification recognition result corresponding to the selected candidate box; the classifier uses a fully connected layer to classify the clear mask of each candidate box and outputs the category of each candidate box. A classifier is a method for predicting the category to which each candidate box belongs. It can generate some categories based on information such as the features or shape of the candidate box or mask, and optimize the prediction of the category based on indicators such as the accuracy or loss of the category. The present invention uses a classifier to assist the diffusion model in instance segmentation, thereby simplifying the process of instance segmentation and improving the efficiency of instance segmentation.

[0084] In some embodiments, Figure 2 As shown, the above step S300 may include but is not limited to step S310 to step S320.

[0085] Step S310, determining a preset proportion of Gaussian noise from a preset diffusion model;

[0086] Step S320, adding Gaussian noise to each pixel in the feature map to obtain a noisy image.

[0087] It should be noted that in the process of diffusion processing of the feature map, a preset proportion of Gaussian noise is first determined from a preset diffusion model; then Gaussian noise is added to each pixel in the feature map to obtain the corresponding noise image. Among them, Gaussian noise is added to each pixel in the image using a diffusion model to obtain a noise image. The diffusion model is a probability-based generation model that can diffuse a low-dimensional Gaussian noise into the image space, thereby realizing the generation process from noise to data. The diffusion model consists of multiple diffusion steps, each of which adds a certain proportion of Gaussian noise to each pixel in the input image, so that the input image gradually becomes blurred and noisy. In the end, the input image will be completely diffused into a low-dimensional Gaussian noise. In this way, the diffusion model is applied to the instance segmentation task, so that the instance segmentation can recover a clear object shape from Gaussian noise, instead of relying on complex feature extraction and mask prediction.

[0088] It is worth noting that the preset diffusion model is a pre-trained diffusion model. A preset proportion of Gaussian noise can be determined from the preset diffusion model. By adding Gaussian noise to each pixel in the feature map, a noise image can be obtained, so that instance segmentation can restore a clear object shape from the Gaussian noise without relying on complex feature extraction and mask prediction, making image detection easier and faster.

[0089] For example, in the field of medical technology, by adding Gaussian noise to each pixel in a medical feature map, a medical noise image can be obtained to prepare for subsequent small object recognition.

[0090] In some embodiments, Figure 3 As shown, the attribute information includes the candidate box confidence and the candidate box score information, and the above step S400 may include but is not limited to steps S410 to S440.

[0091] Step S410, generating a plurality of initial candidate boxes on the noisy image using a pre-trained region generation network;

[0092] Step S420, analyzing and processing each initial candidate box to obtain candidate box confidence and candidate box score information;

[0093] Step S430, comparing the candidate box confidence with a preset confidence threshold to obtain confidence comparison information; and comparing the candidate box score information with a preset candidate box score threshold to obtain score comparison information;

[0094] Step S440 , selecting a number of selected candidate frames from the multiple initial candidate frames according to the confidence comparison information and the score comparison information.

[0095] It should be noted that, in the process of generating multiple initial candidate boxes on a noisy image and selecting a number of selected candidate boxes from the multiple initial candidate boxes according to the attribute information of each initial candidate box, a pre-trained region generation network can be used to generate multiple initial candidate boxes on the noisy image first; then, each initial candidate box is analyzed and processed to obtain the candidate box confidence and candidate box score information; then, the candidate box confidence is compared with a preset confidence threshold to obtain confidence comparison information; and the candidate box score information is compared with a preset candidate box score threshold to obtain score comparison information; based on the confidence comparison information and the score comparison information, a number of selected candidate boxes can be selected from the multiple initial candidate boxes; through the above technical solution, the instance segmentation process can be well simplified and the efficiency of instance segmentation can be well improved.

[0096] It is worth noting that a candidate box generation region proposal network is used to generate candidate boxes on a noisy image, and non-maximum suppression is used to screen out high-quality candidate boxes. A candidate box generator is a method for detecting areas in an image where objects may exist. It can generate some candidate boxes based on information such as image features or edges, and screen out high-quality candidate boxes based on indicators such as the confidence or score of the candidate boxes, i.e., the selected candidate boxes in the embodiments of the present application. The present invention uses a candidate box generator to assist the diffusion model in instance segmentation, thereby simplifying the process of instance segmentation and improving the efficiency of instance segmentation.

[0097] In some embodiments, Figure 4 As shown, the above step S500 may include but is not limited to step S510 to step S540.

[0098] Step S510, performing feature detection on each selected candidate box to obtain candidate box feature information;

[0099] Step S520, generating an initial mask according to the feature information of the candidate box, wherein the initial mask carries a precision value;

[0100] Step S530, comparing the accuracy value of the initial mask with a preset accuracy threshold;

[0101] Step S540: When the precision value of the initial mask is greater than or equal to the precision threshold, the initial mask is determined as a noise mask.

[0102] It should be noted that, in the process of predicting the mask of each selected candidate box on the noise image to obtain the noise mask, first, feature detection is performed on each selected candidate box to obtain the candidate box feature information; then, an initial mask is generated according to the candidate box feature information, and then the accuracy value of the initial mask is compared with the preset accuracy threshold; when the accuracy value of the initial mask is greater than or equal to the accuracy threshold, the initial mask can be determined as a noise mask; through the above technical scheme, the noise mask is used to assist the diffusion model in instance segmentation, thereby simplifying the instance segmentation process and improving the efficiency of instance segmentation.

[0103] In some embodiments, Figure 5 As shown, the above step S600 may include but is not limited to step S610 to step S620.

[0104] Step S610, determining a mapping function from a recursive neural network in a diffusion model;

[0105] Step S620: convert the noise mask into a clear mask corresponding to the selected candidate box according to the mapping function.

[0106] It should be noted that, in the process of performing reverse diffusion processing on each noise mask to obtain a clear mask corresponding to the selected candidate box, the mapping function is first determined from the recursive neural network in the diffusion model, and then the noise mask is converted into a clear mask corresponding to the selected candidate box according to the mapping function; the diffusion model uses a recursive neural network to learn the mapping function from the noise mask to the clear mask, thereby realizing the diffusion process from Gaussian noise to the instance mask; the diffusion process can complete the diffusion in one step or multiple steps, thereby improving the segmentation accuracy of small objects while reducing the computational complexity.

[0107] In some embodiments, Figure 6 As shown, the above step S700 may include but is not limited to step S710 to step S720.

[0108] Step S710, analyzing and processing the features of each clear mask to obtain mask feature information;

[0109] Step S720 , dividing the clear mask according to the pre-trained classifier and the mask feature information to obtain a classification recognition result corresponding to the selected candidate box.

[0110] It should be noted that, in the process of classifying each clear mask and obtaining the classification and recognition results corresponding to the selected candidate boxes, the features of each clear mask are first analyzed and processed to obtain the mask feature information; then, according to the pre-trained classifier and the mask feature information, the clear mask is divided and processed to obtain the classification and recognition results corresponding to the selected candidate boxes; the present application uses a classifier to assist the diffusion model in instance segmentation, thereby simplifying the instance segmentation process and improving the efficiency of instance segmentation.

[0111] In some embodiments, Figure 7 As shown, after executing the above step S700, it may include but not be limited to step S810 and step S820.

[0112] Step S810, comparing the classification recognition result with the description text of the image to be detected to obtain a comparison loss;

[0113] Step S820: adjusting the network parameters of the diffusion model according to the contrast loss.

[0114] It should be noted that, in the process of classifying each clear mask, after obtaining the classification recognition result corresponding to the selected candidate box, the classification recognition result can be compared with the description text of the image to be detected to obtain the contrast loss; then the network parameters of the diffusion model are adjusted according to the contrast loss to update the diffusion model, so that small objects in the image to be detected can be recognized more quickly and accurately in the future.

[0115] In addition, if Figure 8 As shown, an embodiment of the present application further provides an image detection device 10, comprising:

[0116] The first processing module 100 is used to obtain an image to be detected;

[0117] The second processing module 200 is used to extract features from the image to be detected to obtain a feature map;

[0118] The third processing module 300 is used to perform diffusion processing on the feature map to obtain a noise image;

[0119] The fourth processing module 400 is used to generate a plurality of initial candidate frames on the noise image, and select a plurality of selected candidate frames from the plurality of initial candidate frames according to the attribute information of each initial candidate frame;

[0120] A fifth processing module 500 is used to perform prediction processing on the mask of each selected candidate box on the noise image to obtain a noise mask;

[0121] The sixth processing module 600 is used to perform inverse diffusion processing on each noise mask to obtain a clear mask corresponding to the selected candidate box;

[0122] The seventh processing module 700 is used to classify each clear mask to obtain a classification recognition result corresponding to the selected candidate box.

[0123] It should be noted that in the process of image detection and processing, the image to be detected is first obtained, then the feature extraction of the image to be detected is performed to obtain a feature map, and then the feature map is diffused to obtain a noise image; then multiple initial candidate frames are generated on the noise image, and a number of selected candidate frames are screened out from the multiple initial candidate frames according to the attribute information of each initial candidate frame; then the mask of each selected candidate frame on the noise image is predicted to obtain a noise mask; then each noise mask is reversely diffused to obtain a cleaning mask corresponding to the selected candidate frame; finally, each clear mask is classified to obtain a classification recognition result corresponding to the selected candidate frame; through the above technical scheme, the recognition process is converted into a diffusion process from noise to mask, which does not need to rely on complex feature extraction and mask prediction, thereby reducing the dependence on the resolution of the input image, improving the detection and segmentation capabilities of small objects, and making the detection and recognition of small objects in the image more accurate.

[0124] The specific implementation of the image detection device 10 is substantially the same as the specific implementation of the above-mentioned image detection method, and will not be described in detail herein.

[0125] In addition, if Fig. 9 As shown, an embodiment of the present application further provides an electronic device 800 , which includes: a memory 820 , a processor 810 , and a computer program stored in the memory 820 and executable on the processor 810 .

[0126] The processor 810 and the memory 820 may be connected via a bus or other means.

[0127] The non-transient software programs and instructions required to implement the image detection methods of the above embodiments are stored in the memory 820. When executed by the processor 810, the image detection methods of the above embodiments are executed, for example, the image detection methods described above are executed. Figure 1 The method steps S100 to S700, Figure 2 Steps S310 to S320 of the method, Figure 3 Steps S410 to S440 of the method, Figure 4 Steps S510 to S540 of the method, Figure 5 Steps S610 to S620 of the method, Figure 6 Method steps S710 to S720 and Figure 7Method steps S810 to S820 in.

[0128] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0129] In addition, an embodiment of the present application further provides a computer-readable storage medium, which stores computer-executable instructions. The computer-executable instructions are executed by a processor 810 or a controller, for example, by a processor 810 in the above-mentioned device embodiment, so that the above-mentioned processor 810 can execute the image detection method in the above-mentioned embodiment, for example, execute the above-mentioned Figure 1 The method steps S100 to S700, Figure 2 Steps S310 to S320 of the method, Figure 3 Steps S410 to S440 of the method, Figure 4 Steps S510 to S540 of the method, Figure 5 Steps S610 to S620 of the method, Figure 6 Method steps S710 to S720 and Figure 7 Method steps S810 to S820 in.

[0130] The above embodiments may be used in combination, and modules with the same name in different embodiments may be the same or different.

[0131] The above describes specific embodiments of the present application, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0132] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and computer-readable storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0133] The apparatus, device, computer-readable storage medium and method provided in the embodiments of the present application correspond to each other. Therefore, the apparatus, device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the corresponding apparatus, device and computer storage medium will not be repeated here.

[0134] In the 1990s, it was very clear whether the improvement of a technology was hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented by hardware entity modules. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs. The original code before compiling must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Basic Expression Language), AHDL (Altera Hardware Description Language), Conference, CUPL (Cornell University Programming Language), HDCall, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also be aware that it is easy to obtain a hardware circuit that implements the logical method flow by simply performing some logic programming on the method flow using the above-mentioned hardware description languages ​​and programming it into an integrated circuit.

[0135] The controller can be implemented in any appropriate manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the means for realizing various functions may be regarded as both a software module for realizing the method and a structure within a hardware component.

[0136] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0137] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing the embodiments of the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0138] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the embodiments of the present application may adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0139] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0140] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0141] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0142] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0143] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash memory (Flash RAM). The memory is an example of a computer-readable medium.

[0144] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (Transitory Med ia), such as modulated data signals and carrier waves.

[0145] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0146] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0147] Embodiments of the present application may be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. Embodiments of the present application may also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0148] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0149] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. An image detection method, characterized in that: The method comprises: Acquire the image to be detected; Extracting features from the image to be detected to obtain a feature map; Performing diffusion processing on the feature map to obtain a noise image; Generating a plurality of initial candidate frames on the noise image, and selecting a plurality of selected candidate frames from the plurality of initial candidate frames according to attribute information of each of the initial candidate frames; Performing prediction processing on the mask of each of the selected candidate boxes on the noise image to obtain a noise mask; Performing a reverse diffusion process on each of the noise masks to obtain a clear mask corresponding to the selected candidate box; Classification processing is performed on each of the clear masks to obtain a classification recognition result corresponding to the selected candidate box.

2. The image detection method according to claim 1, characterized in that: The step of performing diffusion processing on the feature map to obtain a noise image includes: Determine a preset proportion of Gaussian noise from a preset diffusion model; The Gaussian noise is added to each pixel in the feature image to obtain the noise image.

3. The image detection method according to claim 1, characterized in that: The attribute information includes candidate box confidence and candidate box score information, and the generating of multiple initial candidate boxes on the noise image and selecting a plurality of selected candidate boxes from the multiple initial candidate boxes according to the attribute information of each of the initial candidate boxes includes: Generate a plurality of initial candidate boxes on the noise image using a pre-trained region generation network; Analyze and process each of the initial candidate boxes to obtain the candidate box confidence and the candidate box score information; Comparing the candidate box confidence with a preset confidence threshold to obtain confidence comparison information; and comparing the candidate box score information with a preset candidate box score threshold to obtain score comparison information; According to the confidence comparison information and the score comparison information, a plurality of the selected candidate frames are selected from the plurality of the initial candidate frames.

4. The image detection method according to claim 1, characterized in that: The predicting process of the mask of each selected candidate box on the noise image to obtain a noise mask includes: Performing feature detection on each of the selected candidate frames to obtain candidate frame feature information; Generate an initial mask according to the feature information of the candidate box, wherein the initial mask carries a precision value; Comparing the precision value of the initial mask with a preset precision threshold; When the precision value of the initial mask is greater than or equal to the precision threshold, the initial mask is determined as the noise mask.

5. The image detection method according to claim 2, characterized in that: The performing reverse diffusion processing on each of the noise masks to obtain a clear mask corresponding to the selected candidate frame includes: determining a mapping function from a recurrent neural network in the diffusion model; The noise mask is converted into the clear mask corresponding to the selected candidate box according to the mapping function.

6. The image detection method according to claim 1, characterized in that: The classifying process is performed on each of the clear masks to obtain a classification recognition result corresponding to the selected candidate frame, including: Analyzing and processing the features of each clear mask to obtain mask feature information; According to the pre-trained classifier and the mask feature information, the clear mask is divided and processed to obtain the classification recognition result corresponding to the selected candidate box.

7. The image detection method according to claim 2, characterized in that: After classifying each of the clear masks to obtain a classification recognition result corresponding to the selected candidate frame, the method further includes: Comparing the classification recognition result with the description text of the image to be detected to obtain a comparison loss; The network parameters of the diffusion model are adjusted according to the contrast loss.

8. An image detection device, characterized in that: include: A first processing module, used for acquiring an image to be detected; A second processing module is used to extract features from the image to be detected to obtain a feature map; A third processing module, used for performing diffusion processing on the feature map to obtain a noise image; A fourth processing module, configured to generate a plurality of initial candidate frames on the noise image, and select a plurality of selected candidate frames from the plurality of initial candidate frames according to attribute information of each of the initial candidate frames; A fifth processing module, configured to perform prediction processing on the mask of each of the selected candidate boxes on the noise image to obtain a noise mask; A sixth processing module, configured to perform inverse diffusion processing on each of the noise masks to obtain a clear mask corresponding to the selected candidate box; The seventh processing module is used to classify each of the clear masks to obtain a classification recognition result corresponding to the selected candidate box.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the image detection method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that: The computer executable instructions are used to execute the image detection method described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Target detection method and device

    CN111723860A

  • Diffusion model-based three-dimensional point cloud target detection method and device

    CN116863426A

  • Target detection method applying diffusion detection model

    CN117893838A

Cited By

  • Image data protection method and device based on distributed nodes

    CN119442331A

  • Counterfeit face detection method, system and device and storage medium

    CN119580332A