Image detection method, device, storage medium, electronic equipment and product

Abstract

The present application discloses an image detection method, device, storage medium, electronic device and product, which relates to the field of artificial intelligence technology. The present application can be applied to technical fields such as blockchain, map vehicle networking, etc. The method is applied to a terminal, and the method includes: obtaining an image to be detected; inputting the image to be detected into a detection model for flower screen detection processing to obtain flower screen detection information, wherein the detection model is trained based on valid images in a sample set, wherein the valid images are images other than noise images in the sample set, and the noise images are images detected from the sample set and calibrated with flower screen label information having noise; and determining the flower screen status of the image to be detected based on the flower screen detection information. The present application can effectively improve the flower screen detection performance of the detection model and improve the accuracy of image flower screen detection.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and specifically to an image detection method, device, storage medium, electronic equipment and product. Background Art

[0002] Image detection is the work of detecting and analyzing images. Image detection usually requires the detection and analysis of image distorted screen conditions. Currently, there are related solutions for detecting image distorted screen conditions through trained detection models. In current solutions, detection models are usually trained by calibrating images with distorted screen label information.

[0003] Since the calibration of flower screen label information usually inevitably has subjective errors, the flower screen label information of the image is noisy, and the detection performance of the trained detection model is poor, the flower screen detection error is high, and the accuracy of image flower screen detection is low. Summary of the invention

[0004] The embodiments of the present application provide an image detection method, device, storage medium, electronic device and product, which can effectively improve the flower screen detection performance of the detection model and improve the accuracy of image flower screen detection.

[0005] To solve the above technical problems, the present application provides the following technical solutions:

[0006] According to one embodiment of the present application, an image detection method includes: acquiring an image to be detected; inputting the image to be detected into a detection model for flower screen detection processing to obtain flower screen detection information, wherein the detection model is trained based on valid images in a sample set, the valid images are images other than noise images in the sample set, and the noise images are images detected from the sample set and calibrated with flower screen label information having noise; determining the flower screen state of the image to be detected based on the flower screen detection information.

[0007] According to one embodiment of the present application, an image detection device includes: an acquisition module, used to acquire an image to be detected; a detection module, used to input the image to be detected into a detection model for flower screen detection processing to obtain flower screen detection information, the detection model is trained based on valid images in a sample set, the valid images are images in the sample set excluding noise images, and the noise images are images detected from the sample set and calibrated with flower screen label information with noise; a determination module, used to determine the flower screen state of the image to be detected based on the flower screen detection information.

[0008] In some embodiments of the present application, the image detection device also includes: a sample set acquisition module, used to acquire the sample set, the sample set includes at least one image, and the corresponding flower screen label information of the image calibration; a noise image detection module, used to detect the image corresponding to the flower screen label information with noise from the at least one image, and obtain the noise image; a valid image determination module, used to determine the image other than the noise image in the at least one image as the valid image; a model training module, used to train the preset detection model according to the valid image to obtain the trained detection model.

[0009] In some embodiments of the present application, the model training module is used to: jointly train the preset detection model according to the valid image and the noise image to obtain the trained detection model.

[0010] In some embodiments of the present application, the model training module includes: a flower screen detection unit, which is used to use the preset detection model to perform flower screen detection processing on at least two enhanced images corresponding to the noise image and the valid image, respectively, to obtain flower screen detection information corresponding to the at least two enhanced images and the valid image; a first detection error analysis unit, which is used to determine the first detection error based on the flower screen detection information of the at least two enhanced images; a second detection error analysis unit, which is used to determine the second detection error based on the flower screen detection information corresponding to the valid image; and a joint adjustment unit, which is used to jointly adjust the preset detection model based on the first detection error and the second detection error to obtain the trained detection model.

[0011] In some embodiments of the present application, the first detection error analysis unit includes: a difference information analysis subunit, used to determine the difference information of the flower screen detection information between each two enhanced images for the at least two enhanced images, and obtain at least one difference information; a first detection error calculation subunit, used to obtain the first detection error based on the at least one difference information.

[0012] In some embodiments of the present application, there is at least one valid image; the second detection error analysis unit includes: a detection error calculation subunit, used to determine the detection error corresponding to each of the valid images based on the difference information between the flower screen detection information corresponding to each of the valid images and the flower screen label information; a second detection error calculation subunit, used to perform calculations on the detection error corresponding to at least one of the valid images to obtain the second detection error.

[0013] In some embodiments of the present application, the joint adjustment unit includes: a fusion operation subunit, used to perform a fusion operation on the first detection error and the second detection error to obtain a target detection error corresponding to the preset detection model; a parameter adjustment subunit, used to adjust the parameters of the preset detection model according to the target detection error until the preset detection model meets the training stop condition, thereby obtaining the trained detection model.

[0014] In some embodiments of the present application, the noise image detection module includes: a label analysis unit, which is used to analyze and process the flower screen label information corresponding to each of the images to obtain calibration error information of the flower screen label information corresponding to each of the images; a noise flower screen label information determination unit, which is used to detect the flower screen label information corresponding to the calibration error information higher than a predetermined error condition to obtain the noise flower screen label information; and a noise image determination unit, which is used to determine the image corresponding to the noise flower screen label information in the at least one image as the noise image.

[0015] In some embodiments of the present application, the label analysis unit is used to: use the preset detection model to perform flower screen detection processing on each of the images to obtain flower screen detection information corresponding to each of the images; fuse the flower screen detection information corresponding to each of the images with the flower screen label information to obtain fused flower screen information corresponding to each of the images; analyze the first difference between the fused flower screen information and the flower screen detection information corresponding to each of the images to obtain the first difference corresponding to each of the images; analyze the second difference between the fused flower screen information and the flower screen label information corresponding to each of the images to obtain the second difference corresponding to each of the images; and obtain the calibration error information of the flower screen label information corresponding to each of the images based on the first difference and the second difference corresponding to each of the images.

[0016] According to another embodiment of the present application, an image detection method includes: obtaining the sample set, the sample set includes at least one image, and the image is calibrated with corresponding flower screen label information; from the at least one image, detecting the image corresponding to the flower screen label information with noise to obtain the noise image; determining the image other than the noise image in the at least one image as the valid image; training the preset detection model according to the valid image to obtain the trained detection model, and the trained detection model is used to perform flower screen detection processing on the image to be detected.

[0017] According to another embodiment of the present application, an image detection device includes: a sample set acquisition module, used to acquire the sample set, wherein the sample set includes at least one image, and the image is calibrated with corresponding flower screen label information; a noise image detection module, used to detect an image corresponding to the flower screen label information with noise from the at least one image, and obtain the noise image; a valid image determination module, used to determine the image other than the noise image in the at least one image as the valid image; a model training module, used to train the preset detection model according to the valid image to obtain the trained detection model, and the trained detection model is used to perform flower screen detection processing on the image to be detected.

[0018] According to another embodiment of the present application, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of a computer, the computer executes the method described in the embodiment of the present application.

[0019] According to another embodiment of the present application, an electronic device includes: a memory storing a computer program; and a processor reading the computer program stored in the memory to execute the method described in the embodiment of the present application.

[0020] According to another embodiment of the present application, a computer program product or a computer program includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations described in the embodiments of the present application.

[0021] In an embodiment of the present application, an image to be detected is obtained; the image to be detected is input into a detection model for flower screen detection processing to obtain flower screen detection information, the detection model is trained based on valid images in a sample set, the valid images are images in the sample set excluding noise images, and the noise images are images detected from the sample set and calibrated with flower screen label information having noise; the flower screen state of the image to be detected is determined based on the flower screen detection information.

[0022] In this way, the images in the sample set are divided into noise images and valid images. The noise image is the image that has been calibrated with the noisy flower screen label information, and the valid image is the image other than the noise image. In turn, the detection model is trained based on the valid images in the sample set, which can improve the detection performance of the detection model, reduce the error of flower screen detection, and thus improve the accuracy of image flower screen detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A schematic diagram of a system to which an embodiment of the present application can be applied is shown.

[0025] Figure 2 A flow chart of an image detection method according to an embodiment of the present application is shown.

[0026] Figure 3 A flow chart of an image detection method according to another embodiment of the present application is shown.

[0027] Figure 4 A flow chart of performing image detection in a sample set under a scenario is shown.

[0028] Figure 5 A flowchart for training a detection model in a scenario is shown.

[0029] Figure 6 A block diagram of an image detection device according to another embodiment of the present application is shown.

[0030] Figure 7 A block diagram of an image detection device according to another embodiment of the present application is shown.

[0031] Figure 8 A block diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0033] It is understandable that in the specific implementation of this application, related data such as content information and interactive behavior information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0034] Figure 1 FIG. 1 is a schematic diagram showing a system 100 to which an embodiment of the present application can be applied. Figure 1 As shown, the system 100 may include a server 101 and a terminal 102 .

[0035] Server 101 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0036] The terminal 102 may be any device, including but not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, VR / AR devices, smart watches, and computers. In some implementations, the terminal 102 and the server 101 may be nodes in a blockchain network. In some implementations, the image to be detected may be an image in a map vehicle networking platform.

[0037] In one implementation of this example, the server 101 or the terminal 102 can: obtain an image to be detected; input the image to be detected into a detection model for flower screen detection processing to obtain flower screen detection information, wherein the detection model is trained based on valid images in a sample set, the valid images are images in the sample set excluding noise images, and the noise images are images detected from the sample set and calibrated with flower screen label information having noise; determine the flower screen status of the image to be detected based on the flower screen detection information.

[0038] Figure 2 The flowchart of the image detection method according to an embodiment of the present application is schematically shown. The execution subject of the image detection method can be any device, such as Figure 1 The server 101 or the terminal 102 is shown.

[0039] like Figure 2 As shown, the image detection method may include steps S210 to S230.

[0040] Step S210, obtaining an image to be detected; step S220, inputting the image to be detected into a detection model for flower screen detection processing to obtain flower screen detection information, wherein the detection model is trained based on valid images in a sample set, and valid images are images in the sample set excluding noise images, and noise images are images detected from the sample set and calibrated with flower screen label information having noise; step S230, determining the flower screen status of the image to be detected based on the flower screen detection information.

[0041] The image to be detected is an image of a screen distortion state to be detected. The image to be detected can be acquired by a specified device according to actual needs. The detection model is a pre-trained model based on machine learning or deep learning, such as a model based on a neural network.

[0042] The sample set includes at least one image, and each image is calibrated with corresponding flower screen label information. The flower screen label information can indicate the flower screen state of the image, such as the confidence that the image belongs to a flower screen or the confidence that the image belongs to different flower screen types.

[0043] The noise image is an image detected from the sample set and calibrated with the noisy screen label information, and the noisy screen label information is the screen label information with an error higher than a predetermined standard. The valid image is the image other than the noise image in the sample set.

[0044] The detection model is trained based on the valid images in the sample set. The noise of the flower screen label information marked by the valid images is low. The detection performance of the trained detection model is high. The image to be detected is input into the detection model to perform flower screen detection processing and obtain flower screen detection information corresponding to the image to be detected. Flower screen detection information is detection information that can indicate the flower screen state of the image, such as the confidence that the image belongs to a flower screen or the confidence that the image belongs to different flower screen types.

[0045] After obtaining the flower screen detection information corresponding to the image to be detected, the flower screen state of the image to be detected can be determined according to the flower screen detection information. For example, the flower screen detection information is the confidence that the image belongs to a flower screen. If the confidence is higher than a predetermined threshold, it can be determined that the flower screen state of the image to be detected is a flower screen image. For another example, the flower screen detection information is the confidence that the image belongs to different flower screen types. If the confidence of a certain flower screen type is the highest, it can be determined that the flower screen state of the image to be detected is the flower screen type with the highest confidence.

[0046] In this way, based on steps S210 to S230, the images in the sample set are divided into noise images and valid images. The noise image is an image that has been calibrated with noisy flower screen label information, and the valid image is an image other than the noise image. In turn, a detection model is obtained by training the valid images in the sample set, which can improve the detection performance of the detection model, reduce the error of flower screen detection, and thereby improve the accuracy of image flower screen detection.

[0047] The following further describes specific embodiments of the steps performed during image detection.

[0048] See also Figure 3 In one embodiment, the detection model is trained according to the following steps:

[0049] Step S310, obtaining a sample set, the sample set includes at least one image, and the flower screen label information corresponding to the image calibration; step S320, detecting the image corresponding to the flower screen label information with noise from at least one image, and obtaining a noisy image; step S330, determining the image other than the noise image in at least one image as a valid image; step S340, training a preset detection model according to the valid image, and obtaining a trained detection model.

[0050] A sample set for training a detection model can be obtained from a predetermined position. By detecting the flower screen label information of each image in the sample set, the flower screen label information with noise can be detected, and then the image corresponding to the flower screen label information with noise can be determined as a noise image. And the image other than the noise image in at least one image is determined as a valid image. The preset detection model is trained according to the valid image. The flower screen label information calibrated by the valid image has low noise. Through training, a trained detection model with excellent detection performance can be obtained.

[0051] In one embodiment, step S320, detecting an image corresponding to the noisy screen label information from at least one image to obtain a noisy image, includes:

[0052] Analyze and process the flower screen label information corresponding to each image to obtain the calibration error information of the flower screen label information corresponding to each image; detect the flower screen label information corresponding to the calibration error information higher than the predetermined error condition to obtain the noise flower screen label information; determine the image corresponding to the noise flower screen label information in at least one image as a noise image.

[0053] The calibration error information is information describing the calibration error of the flower screen label information (such as error level or weight, etc.). The flower screen label information corresponding to each image is analyzed and processed to obtain the calibration error information of the flower screen label information corresponding to each image, and the flower screen label information corresponding to the calibration error information higher than the predetermined error condition (such as a predetermined level threshold or a predetermined weight threshold, etc.) is determined as noise flower screen label information, and the image corresponding to the noise flower screen label information is the noise image. Furthermore, the noise image can be detected from the sample set by detecting the calibration error information.

[0054] It can be understood that in another embodiment, step S320, detecting an image corresponding to the noisy flower screen label information from at least one image to obtain a noisy image, includes: detecting an image corresponding to the noisy flower screen label information from at least one image based on an expert review system (such as a manual review system) to obtain a noisy image.

[0055] In one embodiment, the distorted screen label information corresponding to each image is analyzed and processed to obtain calibration error information of the distorted screen label information corresponding to each image, including:

[0056] A preset detection model is used to perform flower screen detection processing on each image to obtain flower screen detection information corresponding to each image; the flower screen detection information corresponding to each image is fused with the flower screen label information to obtain fused flower screen information corresponding to each image; a first difference between the fused flower screen information and the flower screen detection information corresponding to each image is analyzed to obtain a first difference corresponding to each image; a second difference between the fused flower screen information and the flower screen label information corresponding to each image is analyzed to obtain a second difference corresponding to each image; based on the first difference and the second difference corresponding to each image, calibration error information of the flower screen label information corresponding to each image is obtained.

[0057] A method of fusing the flower screen detection information corresponding to each image with the flower screen label information. For example, the flower screen detection information corresponding to image A is the first confidence p, and the flower screen label information is the second confidence y. The fusion method may include fusing the first confidence p with the second confidence y by multiplication or addition, and the fusion result is the fused flower screen information q corresponding to image A.

[0058] In one implementation of this example, the average value of the first confidence level p and the second confidence level y is calculated, that is, the fused screen distortion information q corresponding to the image A=(p+y) / 2.

[0059] A method for analyzing the first difference between the fused flower screen information and the flower screen detection information corresponding to each image. For example, in one method, the difference between the fused flower screen information and the flower screen detection information corresponding to image A is analyzed as the first difference corresponding to image A.

[0060] In one implementation of this example, the method for analyzing the first difference is to calculate the KL divergence of the fused flower screen information and the flower screen detection information. For example, the fused flower screen information corresponding to image A is q, and the flower screen detection information corresponding to image A is the first confidence level p, then the first difference is KL(p||q)=p*log(p)-p*log(q).

[0061] A method for analyzing the second difference between the fused flower screen information and the flower screen label information corresponding to each image, for example, in one method, the difference between the fused flower screen information and the flower screen label information corresponding to image A is analyzed as the second difference corresponding to image A.

[0062] In one implementation of this example, the method for analyzing the second difference is to calculate the KL divergence of the fused flower screen information and the flower screen label information. For example, the fused flower screen information corresponding to image A is q, and the flower screen detection information corresponding to image A is the second confidence y, then the second difference is KL(y||q)=y*log(y)-y*log(q).

[0063] According to the first difference and the second difference corresponding to each image, the calibration error information of the flower screen label information corresponding to each image is obtained. For example, the first difference and the second difference corresponding to image A are summed or multiplied, and the result is used as the calibration error information of the flower screen label information corresponding to image A.

[0064] In one implementation of this example, the calibration error information of the flower screen label information corresponding to each image is obtained, and the calibration error information w=(KL(p||q)) / 2+(KL(y||q)) / 2.

[0065] In one embodiment, step S340, training a preset detection model according to a valid image to obtain a trained detection model, includes: training the preset detection model only according to the valid image to obtain a trained detection model.

[0066] For example, only the valid image is input into the preset detection model for detection processing to obtain the flower screen detection information corresponding to the valid image, and then the flower screen detection information and the flower screen label information corresponding to the valid image are compared to obtain the detection loss. The parameters of the preset detection model are adjusted according to the detection loss function until the detection accuracy or number of training times of the preset detection model meet the requirements, thereby obtaining the trained detection model.

[0067] In one embodiment, step S340, training a preset detection model according to a valid image to obtain a trained detection model, includes: jointly training the preset detection model according to the valid image and the noise image to obtain the trained detection model.

[0068] In this embodiment, the preset detection model is jointly trained by combining the valid image and the noise image. By combining the valid image and the noise image, the order of the training sample images is high, and the detection model can be trained for robust learning, which can further improve the detection reliability and stability of the detection model.

[0069] In one embodiment, a preset detection model is jointly trained according to a valid image and a noise image to obtain a trained detection model, including:

[0070] Using a preset detection model, at least two enhanced images and a valid image corresponding to the noise image are respectively processed for flower screen detection to obtain flower screen detection information corresponding to the at least two enhanced images and the valid image; a first detection error is determined based on the flower screen detection information of the at least two enhanced images; a second detection error is determined based on the flower screen detection information corresponding to the valid image; and the preset detection model is adjusted jointly based on the first detection error and the second detection error to obtain a trained detection model.

[0071] Randomly enhancing the noisy image can obtain at least two enhanced images corresponding to the noisy image. The enhancement means of randomly enhancing the noisy image is not limited to enhancement of illumination, contrast, saturation, etc., and can also be enhanced through more complex data enhancement means.

[0072] A first detection error is determined based on the flower screen detection information of at least two enhanced images, and a second detection error is determined based on the flower screen detection information corresponding to the valid image. The preset detection model can be adjusted based on the first detection error and the second detection error, for example, by adjusting the parameters in the preset detection model until the requirements are met, thereby obtaining a trained detection model.

[0073] In this way, by enhancing the noisy image and training the detection model with at least two enhanced images of the noisy image and the valid image, robust learning of the detection model can be further achieved, and the detection reliability and stability of the detection model can be further improved.

[0074] In one embodiment, there is at least one noise image; and determining the first detection error according to the noise detection information of at least two enhanced images includes:

[0075] For at least two enhanced images, difference information of the screen noise detection information between every two enhanced images is determined to obtain at least one difference information; and a first detection error is obtained according to the at least one difference information.

[0076] Randomly enhance at least one noise image to obtain N enhanced images, which may include x1 to x N , the flower screen detection information of N enhanced images can be N confidence levels p1 to p N , the difference information of the flower screen detection information between the i-th enhanced image and the j-th enhanced image is (p i -p j ), i and j belong to 1 to N.

[0077] According to the method of obtaining the first detection error based on at least one difference information, in one example, the at least one difference information can be averaged as the first detection error. In one implementation of this example, according to the formula For at least one difference information (p i -p j ) is calculated to obtain the first detection error L e , the first detection error L e That is, N confidence levels p1 to p N loss of consistency.

[0078] In one embodiment, there is at least one valid image; and determining the second detection error according to the flower screen detection information corresponding to the valid image includes:

[0079] According to the difference information between the flower screen detection information corresponding to each valid image and the flower screen label information, the detection error corresponding to each valid image is determined; the detection error corresponding to at least one valid image is processed by operation to obtain a second detection error.

[0080] The screen flower detection information p corresponding to the i-th valid image i With the flower screen label information i The difference information is p i -y i , that is, the detection error corresponding to the i-th valid image is p i -y i .

[0081] The detection errors corresponding to the N valid images are processed to obtain the second detection error. In one example, the detection errors corresponding to the N valid images can be multiplied by a preset coefficient to obtain the second detection error. In one implementation of this example, according to the formula The detection error (p i -y i ) is processed to obtain the second detection error L c .

[0082] In one embodiment, adjusting a preset detection model according to the first detection error and the second detection error to obtain a trained detection model includes:

[0083] The first detection error and the second detection error are fused and processed to obtain a target detection error corresponding to a preset detection model; parameters of the preset detection model are adjusted according to the target detection error until the preset detection model meets the training stop condition, thereby obtaining a trained detection model.

[0084] The first detection error and the second detection error are fused and processed to obtain the target detection error. In one example, the first detection error and the second detection error can be fused and processed by multiplying to obtain the product as the target detection error. In one implementation of this example, according to the loss function L=L c +αL e The first detection error L e The second detection error L c The target detection error L is obtained by performing fusion operation, where α is the weight used to control the corresponding two losses.

[0085] The above embodiment is further described below in conjunction with a process of performing image detection in an application scenario. Figure 4 and Figure 5 , Figure 4A flowchart of performing image detection in a sample set in this scenario is shown. Figure 5 The flowchart of the detection model training in this scenario is shown.

[0086] See also Figure 4 The image detection process in this scenario may include steps S410 to S440.

[0087] Step S410, obtaining a sample set, the sample set including at least one image and the corresponding flower screen label information of the image calibration.

[0088] Step S420 includes steps S421 to S424.

[0089] In step S421, an image corresponding to the noisy screen label information is detected from at least one image to obtain a "noise image".

[0090] Among them, step S421, from at least one image, detects the image corresponding to the flower screen label information with noise to obtain a noisy image, which specifically includes: analyzing and processing the flower screen label information corresponding to each image to obtain the calibration error information of the flower screen label information corresponding to each image; detecting the flower screen label information corresponding to the calibration error information higher than the predetermined error condition to obtain the noise flower screen label information; determining the image corresponding to the noise flower screen label information in at least one image as a noise image.

[0091] Furthermore, the flower screen label information corresponding to each image is analyzed and processed to obtain calibration error information of the flower screen label information corresponding to each image, including: using a preset detection model to perform flower screen detection processing on each image to obtain flower screen detection information corresponding to each image; fusing the flower screen detection information corresponding to each image with the flower screen label information to obtain fused flower screen information corresponding to each image; analyzing a first difference between the fused flower screen information and the flower screen detection information corresponding to each image to obtain a first difference corresponding to each image; analyzing a second difference between the fused flower screen information and the flower screen label information corresponding to each image to obtain a second difference corresponding to each image; and obtaining calibration error information of the flower screen label information corresponding to each image based on the first difference and the second difference corresponding to each image.

[0092] In this scenario, the flower screen detection information corresponding to each image is fused with the flower screen label information. For example, the flower screen detection information corresponding to image A is the first confidence p, and the flower screen label information is the second confidence y. The fusion method is to calculate the average value of the first confidence p and the second confidence y, that is, the fused flower screen information q corresponding to image A is q = (p + y) / 2.

[0093] A method for analyzing the first difference between the fused flower screen information and the flower screen detection information corresponding to each image. In this scenario, the KL divergence of the fused flower screen information and the flower screen detection information is calculated. For example, the fused flower screen information corresponding to image A is q, and the flower screen detection information corresponding to image A is the first confidence level p, then the first difference is KL(p||q)=p*log(p)-p*log(q).

[0094] A method for analyzing the second difference between the fused flower screen information and the flower screen label information corresponding to each image. In this scenario, the method for analyzing the second difference is to calculate the KL divergence of the fused flower screen information and the flower screen label information. For example, the fused flower screen information corresponding to image A is q, and the flower screen detection information corresponding to image A is the second confidence y, then the second difference is KL(y||q)=y*log(y)-y*log(q).

[0095] In this scenario, the calibration error information of the flower screen label information corresponding to each image is obtained according to the first difference and the second difference corresponding to each image, specifically the calibration error information w=(KL(p||q)) / 2+(KL(y||q)) / 2.

[0096] In step S422, the distorted screen label information with noise is removed, that is, the distorted screen label information corresponding to the "noise image" is removed.

[0097] In step S423, images other than the noise image in at least one image are determined as "valid images".

[0098] In step S424, the distorted screen label information is retained, that is, the distorted screen label information corresponding to the "valid image" is retained.

[0099] Step S430, robust training, specifically, training the preset detection model according to the effective image to obtain the trained detection model. Wherein, training the preset detection model according to the effective image to obtain the trained detection model specifically includes: jointly training the preset detection model according to the effective image and the noise image to obtain the trained detection model.

[0100] Among them, see Figure 5 , jointly training the preset detection model according to the effective image and the noise image to obtain the trained detection model, including: step S510 to step S550, and after step S550, jointly adjusting the preset detection model according to the first detection error and the second detection error to obtain the trained detection model. Among them, the preset detection model "shares weights" during the joint training, that is, "jointly adjusts model parameters" when jointly training the preset detection model.

[0101] In step S510, the noise image is "randomly enhanced" to obtain at least two enhanced images corresponding to the noise image. The enhancement method of randomly enhancing the noise image is not limited to enhancement of illumination, contrast, saturation, etc., and can also be enhanced by more complex data enhancement methods.

[0102] In step S520, a preset detection model is used to perform a flower screen detection process on at least two enhanced images corresponding to the noise image, so as to obtain flower screen detection information corresponding to the at least two enhanced images.

[0103] In step S530, a first detection error (ie, consistency loss) is determined based on the screen noise detection information of at least two enhanced images.

[0104] Determining a first detection error based on the flower screen detection information of at least two enhanced images includes: for at least two enhanced images, determining difference information of the flower screen detection information between every two enhanced images to obtain at least one difference information; and obtaining the first detection error based on the at least one difference information.

[0105] Specifically, randomly enhancing at least one noise image to obtain a total of N enhanced images may include x1 to x N , the flower screen detection information of N enhanced images can be N confidence levels p1 to p N , the difference information of the flower screen detection information between the i-th enhanced image and the j-th enhanced image is (p i -p j ), i and j belong to 1 to N.

[0106] In this scenario, according to the formula For at least one difference information (p i -p j ) is calculated to obtain the first detection error L e , the first detection error L e That is, N confidence levels p1 to p N loss of consistency.

[0107] In step S540, a preset detection model is used to perform a distorted screen detection process on the valid image to obtain distorted screen detection information corresponding to the valid image.

[0108] In step S550, a second detection error (ie, classification loss) is determined according to the flower screen detection information corresponding to the valid image.

[0109] Determine the second detection error according to the flower screen detection information corresponding to the valid image, including: determine the detection error corresponding to each valid image according to the difference information between the flower screen detection information corresponding to each valid image and the flower screen label information; and perform calculation processing on the detection error corresponding to at least one valid image to obtain the second detection error.

[0110] The screen flower detection information p corresponding to the i-th valid image i With the flower screen label information i The difference information is p i -y i , that is, the detection error corresponding to the i-th valid image is p i -y i The detection errors corresponding to N valid images are processed by calculation to obtain the second detection error. In this scenario, according to the formula The detection error (p i -y i ) is processed to obtain the second detection error L c (i.e. classification loss).

[0111] Furthermore, the step of jointly adjusting the preset detection model according to the first detection error and the second detection error to obtain the trained detection model includes: fusing the first detection error and the second detection error to obtain the target detection error corresponding to the preset detection model; adjusting the parameters of the preset detection model according to the target detection error until the preset detection model meets the conditions for stopping training, thereby obtaining the trained detection model.

[0112] The first detection error and the second detection error are fused and processed to obtain the target detection error. In this scenario, according to the loss function L = L c +αL e The first detection error L e The second detection error L c The target detection error L is obtained by performing fusion operation, where α is the weight used to control the corresponding two losses.

[0113] Step S440, outputting the result. Specifically, obtaining the image to be detected; inputting the image to be detected into the detection model for flower screen detection processing to obtain flower screen detection information; and determining the flower screen state of the image to be detected according to the flower screen detection information.

[0114] After obtaining the flower screen detection information corresponding to the image to be detected, the flower screen state of the image to be detected can be determined according to the flower screen detection information. For example, the flower screen detection information is the confidence that the image belongs to a flower screen. If the confidence is higher than a predetermined threshold, it can be determined that the flower screen state of the image to be detected is a flower screen image. For another example, the flower screen detection information is the confidence that the image belongs to different flower screen types. If the confidence of a certain flower screen type is the highest, it can be determined that the flower screen state of the image to be detected is the flower screen type with the highest confidence.

[0115] In this way, by applying the embodiments of the present application in this scenario, at least the following beneficial effects are achieved: the images in the sample set are divided into noise images and valid images, the noise images are images that have been calibrated with noisy flower screen label information, and the valid images are images other than the noise images. In turn, a detection model is obtained by training the valid images in the sample set, which can improve the detection performance of the detection model, reduce the error of flower screen detection, and thereby improve the accuracy of image flower screen detection.

[0116] In order to better implement the image detection method provided in the embodiment of the present application, the embodiment of the present application also provides an image detection device based on the above image detection method. The meanings of the terms are the same as those in the above image detection method, and the specific implementation details can refer to the description in the method embodiment. Figure 6 A block diagram of an image detection device according to an embodiment of the present application is shown. Figure 7 A block diagram of an image detection device according to another embodiment of the present application is shown.

[0117] like Figure 6 As shown, the image detection device 600 may include an acquisition module 610 , a detection module 620 and a determination module 630 .

[0118] The acquisition module 610 can be used to acquire the image to be detected; the detection module 620 can be used to input the image to be detected into the detection model for flower screen detection processing to obtain flower screen detection information, the detection model is trained based on valid images in the sample set, the valid images are images in the sample set excluding noise images, and the noise images are images detected from the sample set and calibrated with flower screen label information with noise; the determination module 630 can be used to determine the flower screen status of the image to be detected based on the flower screen detection information.

[0119] In some embodiments of the present application, the image detection device also includes: a sample set acquisition module, which can be used to acquire the sample set, the sample set includes at least one image, and the image is calibrated with corresponding flower screen label information; a noise image detection module, which can be used to detect the image corresponding to the flower screen label information with noise from the at least one image to obtain the noise image; a valid image determination module, which can be used to determine the image other than the noise image in the at least one image as the valid image; a model training module, which can be used to train the preset detection model according to the valid image to obtain the trained detection model.

[0120] In some embodiments of the present application, the model training module is used to: jointly train the preset detection model according to the valid image and the noise image to obtain the trained detection model.

[0121] In some embodiments of the present application, the model training module includes: a flower screen detection unit, which is used to use the preset detection model to perform flower screen detection processing on at least two enhanced images corresponding to the noise image and the valid image, respectively, to obtain flower screen detection information corresponding to the at least two enhanced images and the valid image; a first detection error analysis unit, which is used to determine the first detection error based on the flower screen detection information of the at least two enhanced images; a second detection error analysis unit, which is used to determine the second detection error based on the flower screen detection information corresponding to the valid image; and a joint adjustment unit, which is used to jointly adjust the preset detection model based on the first detection error and the second detection error to obtain the trained detection model.

[0122] In some embodiments of the present application, the first detection error analysis unit includes: a difference information analysis subunit, used to determine the difference information of the flower screen detection information between each two enhanced images for the at least two enhanced images, and obtain at least one difference information; a first detection error calculation subunit, used to obtain the first detection error based on the at least one difference information.

[0123] In some embodiments of the present application, there is at least one valid image; the second detection error analysis unit includes: a detection error calculation subunit, used to determine the detection error corresponding to each of the valid images based on the difference information between the flower screen detection information corresponding to each of the valid images and the flower screen label information; a second detection error calculation subunit, used to perform calculations on the detection error corresponding to at least one of the valid images to obtain the second detection error.

[0124] In some embodiments of the present application, the joint adjustment unit includes: a fusion operation subunit, used to perform a fusion operation on the first detection error and the second detection error to obtain a target detection error corresponding to the preset detection model; a parameter adjustment subunit, used to adjust the parameters of the preset detection model according to the target detection error until the preset detection model meets the training stop condition, thereby obtaining the trained detection model.

[0125] In some embodiments of the present application, the noise image detection module includes: a label analysis unit, which is used to analyze and process the flower screen label information corresponding to each of the images to obtain calibration error information of the flower screen label information corresponding to each of the images; a noise flower screen label information determination unit, which is used to detect the flower screen label information corresponding to the calibration error information higher than a predetermined error condition to obtain the noise flower screen label information; and a noise image determination unit, which is used to determine the image corresponding to the noise flower screen label information in the at least one image as the noise image.

[0126] In some embodiments of the present application, the label analysis unit is used to: use the preset detection model to perform flower screen detection processing on each of the images to obtain flower screen detection information corresponding to each of the images; fuse the flower screen detection information corresponding to each of the images with the flower screen label information to obtain fused flower screen information corresponding to each of the images; analyze the first difference between the fused flower screen information and the flower screen detection information corresponding to each of the images to obtain the first difference corresponding to each of the images; analyze the second difference between the fused flower screen information and the flower screen label information corresponding to each of the images to obtain the second difference corresponding to each of the images; and obtain the calibration error information of the flower screen label information corresponding to each of the images based on the first difference and the second difference corresponding to each of the images.

[0127] According to another embodiment of the present application, refer to Figure 7 , an image detection device 700, the device includes: a sample set acquisition module 710, which can be used to obtain the sample set, the sample set includes at least one image, and the image is calibrated with corresponding flower screen label information; a noise image detection module 720, which can be used to detect the image corresponding to the flower screen label information with noise from the at least one image to obtain the noise image; a valid image determination module 730, which can be used to determine the image other than the noise image in the at least one image as the valid image; a model training module 740, which can be used to train the preset detection model according to the valid image to obtain the trained detection model, and the trained detection model is used to perform flower screen detection processing on the image to be detected.

[0128] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0129] In addition, an embodiment of the present application further provides an electronic device, which may be a terminal or a server. Figure 8 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0130] The electronic device may include components such as a processor 801 with one or more processing cores, a memory 802 with one or more computer-readable storage media, a power supply 803, and an input unit 804. Those skilled in the art will appreciate that Figure 8 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0131] The processor 801 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the memory 802, and calling data stored in the memory 802, it executes various functions of the computer device and processes data, thereby detecting the electronic device. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user pages and application programs, etc., and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 801.

[0132] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 802 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0133] The electronic device also includes a power supply 803 for supplying power to each component. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management system, so as to manage charging, discharging, power consumption and other functions through the power management system. The power supply 803 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators and other arbitrary components.

[0134] The electronic device may further include an input unit 804, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0135] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 801 in the electronic device will load the executable files corresponding to the processes of one or more computer programs into the memory 802 according to the following instructions, and the processor 801 will run the computer programs stored in the memory 802, thereby realizing various functions of the aforementioned embodiments of the present application.

[0136] For example, in one embodiment, the processor 801 may execute: detecting content information corresponding to local browsing content, and detecting interactive behavior information corresponding to local content browsing interactive behavior; obtaining candidate recommended content sent by a server, wherein the candidate recommended content is obtained by analyzing and processing the server in response to a local content browsing request; performing analysis and processing based on the content information and the interactive behavior information to obtain target browsing information; adjusting and processing the candidate recommended content according to the target browsing information to obtain target recommended content, wherein the target recommended content is used for image detection.

[0137] For example, in one embodiment, the processor 801 can execute: receiving a content browsing request sent by a terminal; performing analysis and processing in response to the content browsing request to obtain candidate recommended content; and sending the candidate recommended content to the terminal so that the terminal adjusts the candidate recommended content according to target browsing information to obtain target recommended content, wherein the target recommended content is used for image detection, and the target browsing information is obtained by analyzing and processing the interaction behavior information corresponding to the detected content browsing interaction behavior and the content information corresponding to the browsed content by the terminal.

[0138] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0139] To this end, an embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. The computer program can be loaded by a processor to execute the steps in any method provided in the embodiment of the present application.

[0140] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0141] Since the computer program stored in the computer-readable storage medium can execute the steps in any method provided in the embodiments of the present application, the beneficial effects that can be achieved by the method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0142] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method provided in various optional implementations in the above embodiments of the present application.

[0143] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0144] It should be understood that the present application is not limited to the embodiments that have been described above and shown in the accompanying drawings, but various modifications and changes may be made without departing from the scope thereof.

Claims

1. An image detection method, characterized in that: include: Acquire the image to be detected; Input the image to be detected into a detection model for flower screen detection processing to obtain flower screen detection information, wherein the detection model is trained based on valid images in a sample set, the valid images are images in the sample set excluding noise images, and the noise images are images detected from the sample set and calibrated with flower screen label information having noise; The training process of the detection model includes: obtaining the sample set, wherein the sample set includes at least one image, and the image is calibrated with corresponding flower screen label information; from the at least one image, detecting the image corresponding to the flower screen label information with noise to obtain the noise image; determining the image other than the noise image in the at least one image as the valid image; and jointly training the preset detection model according to the valid image and the noise image to obtain the trained detection model; Among them, the detection model preset according to the effective image and the noise image is jointly trained to obtain the trained detection model, including: obtaining at least two enhanced images corresponding to the noise image, and for the at least two enhanced images, determining the difference information of the flower screen detection information between each two enhanced images to obtain at least one difference information; obtaining a first detection error according to the at least one difference information; determining the detection error corresponding to each effective image according to the difference information between the flower screen detection information and the flower screen label information corresponding to each effective image; performing calculation processing on the detection error corresponding to at least one effective image to obtain a second detection error; adjusting the preset detection model according to the first detection error and the second detection error to obtain the trained detection model; The distorted screen state of the image to be detected is determined according to the distorted screen detection information.

2. The method according to claim 1, characterized in that The step of jointly training the preset detection model according to the valid image and the noise image to obtain the trained detection model includes: Using the preset detection model, respectively performing flower screen detection processing on at least two enhanced images corresponding to the noise image and the effective image, to obtain flower screen detection information corresponding to the at least two enhanced images and the effective image; Determining the first detection error according to the screen distortion detection information of the at least two enhanced images; determining the second detection error according to the flower screen detection information corresponding to the valid image; The preset detection model is adjusted jointly according to the first detection error and the second detection error to obtain the trained detection model.

3. The method according to claim 2, characterized in that The step of adjusting the preset detection model according to the first detection error and the second detection error to obtain the trained detection model includes: The first detection error and the second detection error are fused and processed to obtain a target detection error corresponding to the preset detection model; The parameters of the preset detection model are adjusted according to the target detection error until the preset detection model meets the training stop condition, thereby obtaining the trained detection model.

4. The method according to any one of claims 1 to 3, characterized in that: The step of detecting an image corresponding to the noisy screen label information from the at least one image to obtain the noisy image includes: Analyze and process the flower screen label information corresponding to each of the images to obtain calibration error information of the flower screen label information corresponding to each of the images; Detecting the noise screen label information corresponding to the calibration error information higher than the predetermined error condition to obtain the noise screen label information; An image corresponding to the noise and aliasing label information in the at least one image is determined as the noise image.

5. The method according to claim 4, characterized in that The analyzing and processing the flower screen label information corresponding to each of the images to obtain the calibration error information of the flower screen label information corresponding to each of the images includes: Using the preset detection model to perform a flower screen detection process on each of the images, and obtaining flower screen detection information corresponding to each of the images; The flower screen detection information corresponding to each of the images is fused with the flower screen label information to obtain fused flower screen information corresponding to each of the images; Analyze the first difference between the fused flower screen information and the flower screen detection information corresponding to each of the images to obtain the first difference corresponding to each of the images; Analyze the second difference between the fused flower screen information and the flower screen label information corresponding to each of the images to obtain the second difference corresponding to each of the images; According to the first difference degree and the second difference degree corresponding to each of the images, calibration error information of the flower screen label information corresponding to each of the images is obtained.

6. An image detection device, characterized in that: include: An acquisition module, used for acquiring an image to be detected; A detection module is used to input the image to be detected into a detection model for flower screen detection processing to obtain flower screen detection information, wherein the detection model is trained based on valid images in a sample set, wherein the valid images are images other than noise images in the sample set, and the noise images are images detected from the sample set and calibrated with flower screen label information having noise; the device is also used to: obtain the sample set, wherein the sample set includes at least one image, and the flower screen label information corresponding to the image calibration; and from the at least one image, detect the image corresponding to the flower screen label information having noise to obtain the noise image; Determine an image other than the noise image in the at least one image as the valid image; Jointly training the preset detection model according to the valid image and the noise image to obtain the trained detection model; Among them, the detection model preset according to the effective image and the noise image is jointly trained to obtain the trained detection model, including: obtaining at least two enhanced images corresponding to the noise image, and for the at least two enhanced images, determining the difference information of the flower screen detection information between each two enhanced images to obtain at least one difference information; obtaining a first detection error according to the at least one difference information; determining the detection error corresponding to each effective image according to the difference information between the flower screen detection information and the flower screen label information corresponding to each effective image; performing calculation processing on the detection error corresponding to at least one effective image to obtain a second detection error; adjusting the preset detection model according to the first detection error and the second detection error to obtain the trained detection model; A determination module is used to determine the distorted screen state of the image to be detected according to the distorted screen detection information.

7. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor of a computer, the computer is caused to execute the method according to any one of claims 1 to 5.

8. An electronic device, characterized in that: include: a memory storing a computer program; A processor reads a computer program stored in a memory to execute the method according to any one of claims 1 to 5.

9. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images