Image foreign matter recognition result verification method and device, equipment and medium

Abstract

The invention discloses a construction method of a foreign matter recognition result verification model, an image foreign matter recognition result verification method and device, equipment and a medium. The image foreign matter recognition result verification method comprises the steps that a target foreign matter image output by a foreign matter recognition model is acquired, and a foreign matter background image is extracted from the target foreign matter image; performing channel fusion processing on the target foreign matter image and the foreign matter background image based on a preset image channel fusion model to obtain a target fusion image; inputting the target fusion image into a foreign matter recognition result verification model to obtain a foreign matter probability table of the target foreign matter image; and obtaining a verification result of a foreign matter recognition result of the target foreign matter image according to the foreign matter probability table. Through the technical scheme of the invention, verification of the image foreign matter recognition result can be realized, the accuracy of the foreign matter recognition result is improved, and meanwhile, the verification efficiency of the foreign matter recognition result is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method for constructing a foreign body recognition result verification model, and a method device, equipment and medium for verifying image foreign body recognition results. Background Art

[0002] In today's era of rapid development of computer technology, computer image processing technology based on computer vision principles is increasingly used in real-time target detection. Target detection aims to accurately find all interesting targets in an image and determine their position and size. This is one of the key issues in the field of computer vision. However, due to the significant differences in appearance, shape, and posture of various objects, coupled with the interference of various factors such as illumination changes and object occlusion during imaging, target detection has become a very challenging task in the field of vision. At present, the YOLO series of models has dominated the field of real-time target detection, especially the release of YOLO-v10, which has reached an advanced level in terms of performance and efficiency trade-offs, with an average precision AP of 54.4% on the COCO dataset. But even so, in complex situations such as low light, object rotation, and dense scenes, the model still has false detections.

[0003] In order to solve the problem of false detection in target detection, there are many methods for filtering out false alarms. For example, the structural similarity measure SSIM is calculated between the target image and the background image of the corresponding area to filter out false alarms. However, this method has many defects. First, the amount of calculation is huge, resulting in slow calculation speed; second, the calculation result of SSIM is easily affected by the image size and is highly sensitive to noise; third, if the image changes greatly, it is necessary to manually set the threshold according to the scene, and the accuracy of the threshold is directly related to the effect; fourth, SSIM often focuses too much on simple patterns and ignores global differences, resulting in unreliable measurement results, and thus the filtering effect of false alarms is not satisfactory. In addition, there are also means of using cosine similarity measurement to filter out false alarms. However, cosine similarity only considers the direction of the vector, but does not attach importance to the importance of the vector, and may miss some key visual features of the image. In addition, when processing sparse vectors, the calculation speed is slow because the product of all non-zero elements needs to be calculated. Especially when the angle between vectors is large, cosine similarity may not accurately reflect the similarity between them.

[0004] In summary, in image processing, the existing technology has the problem that the accuracy of foreign body recognition results is generally low and the verification efficiency of foreign body recognition results is low. Summary of the invention

[0005] The present invention provides a method for constructing a foreign body recognition result verification model, an image foreign body recognition result verification method device, equipment and medium, which can solve the problems of generally low accuracy of foreign body recognition results and low efficiency of foreign body recognition result verification in the prior art.

[0006] In a first aspect, an embodiment of the present invention provides a method for constructing a foreign body recognition result verification model, the method comprising:

[0007] Acquire an original sample set, wherein the original sample includes at least one target image, a background image matching the target image, and an image label, wherein the target image is an image containing a foreign object identified by a foreign object recognition model, and the image label indicates whether the recognition result of the foreign object recognition model is a false alarm;

[0008] Based on a preset image channel fusion model, channel fusion processing is performed on the target image and the background image in each original sample to obtain a fused image of each sample that matches each original sample;

[0009] A training sample set is formed according to each sample fusion image and image label matched with each original sample, and the image label is used as the annotation data in the training sample;

[0010] The consistent classification network model is trained using the training sample set to obtain a foreign object recognition result verification model.

[0011] In a second aspect, an embodiment of the present invention provides a method for verifying an image foreign body recognition result, the method comprising:

[0012] Obtaining a target foreign object image output by a foreign object recognition model, and extracting a foreign object background image from the target foreign object image;

[0013] Based on a preset image channel fusion model, channel fusion processing is performed on the target foreign body image and the foreign body background image to obtain a target fused image;

[0014] Inputting the target fused image into a foreign body recognition result verification model trained by the method according to any one of claims 1 to 4 to obtain a foreign body probability table of the target foreign body image;

[0015] According to the foreign object probability table, a verification result of the foreign object recognition result of the target foreign object image is obtained.

[0016] In a third aspect, an embodiment of the present invention provides a device for constructing a foreign body recognition result verification model, the device comprising:

[0017] The original sample acquisition module is used to acquire an original sample set, wherein the original sample includes at least one target image, a background image matching the target image, and an image label, the target image is an image containing foreign matter identified by the foreign matter recognition model, and the image label marks whether the recognition result of the foreign matter recognition model is a false alarm;

[0018] An image fusion module is used to perform channel fusion processing on the target image and the background image in each original sample based on a preset image channel fusion model to obtain a fused image of each sample that matches each original sample;

[0019] A training sample generation module, used to form a training sample set according to each sample fusion image and image label matching each original sample, and the image label is used as the annotation data in the training sample;

[0020] The model training module is used to train the consistency classification network model using the training sample set to obtain a foreign body recognition result verification model.

[0021] In a fourth aspect, an embodiment of the present invention provides an image foreign body recognition result verification device, the device comprising:

[0022] A foreign body recognition module is used to obtain a target foreign body image output by a foreign body recognition model and extract a foreign body background image from the target foreign body image;

[0023] A channel fusion module is used to perform channel fusion processing on the target foreign body image and the foreign body background image based on a preset image channel fusion model to obtain a target fused image;

[0024] A model processing module, used for inputting the target fusion image into a foreign body recognition result verification model trained by the method according to any one of claims 1 to 4, to obtain a foreign body probability table of the target foreign body image;

[0025] The result acquisition module is used to obtain the verification result of the foreign body recognition result of the target foreign body image according to the foreign body probability table.

[0026] In a fifth aspect, an embodiment of the present invention provides an electronic device, the electronic device comprising:

[0027] at least one processor; and

[0028] a memory communicatively connected to the at least one processor; wherein,

[0029] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a method for constructing a foreign body recognition result verification model and an image foreign body recognition result verification method described in any embodiment of the present invention.

[0030] In a sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a method for constructing a foreign body recognition result verification model and an image foreign body recognition result verification method described in any embodiment of the present invention when executed.

[0031] The technical solution of the embodiment of the present invention first obtains an original sample set, and performs channel fusion processing on the target image and the background image in each original sample based on a preset image channel fusion model to obtain each sample fusion image matching each original sample, then forms a training sample set according to each sample fusion image and image label matching each original sample, and uses the training sample set to train a consistency classification network model to obtain a foreign body recognition result verification model, then obtains the target foreign body image output by the foreign body recognition model, and extracts the foreign body background image from the target foreign body image, then performs channel fusion processing on the target foreign body image and the foreign body background image based on a preset image channel fusion model to obtain a target fusion image, and inputs the target fusion image into the trained foreign body recognition result verification model to obtain a foreign body probability table of the target foreign body image, and finally obtains the verification result of the foreign body recognition result of the target foreign body image according to the foreign body probability table, thereby solving the problems of generally low accuracy of foreign body recognition results and low efficiency of foreign body recognition result verification in the prior art, realizing the verification of image foreign body recognition results, improving the accuracy of foreign body recognition results, and improving the efficiency of foreign body recognition result verification.

[0032] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0034] Figure 1is a flow chart of a method for constructing a foreign body recognition result verification model provided in accordance with the first embodiment of the present invention;

[0035] Figure 2 is a flow chart of a method for verifying image foreign body recognition results provided according to Embodiment 2 of the present invention;

[0036] Figure 3 2 is a schematic diagram of a structure of a device for constructing a foreign body recognition result verification model provided by Embodiment 3 of the present invention;

[0037] Figure 4 is a structural schematic diagram of an image foreign body recognition result verification device provided according to a fourth embodiment of the present invention;

[0038] Figure 5 The present invention is a schematic diagram of the structure of an electronic device for implementing a method for constructing a foreign body recognition result verification model and an image foreign body recognition result verification method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0040] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, any variation of the terms "including" and "having" is intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0041] Embodiment 1

[0042] Figure 1A flowchart of a method for constructing a foreign object recognition result verification model provided in Example 1 of the present invention. This embodiment is applicable to the situation where a foreign object recognition result verification model is to be constructed. The method can be executed by a construction device for the foreign object recognition result verification model. The construction device for the foreign object recognition result verification model can be implemented in the form of hardware and / or software. The construction device for the foreign object recognition result verification model can be configured in a terminal or server with the function of constructing a foreign object recognition result verification model.

[0043] like Figure 1 As shown, the method includes:

[0044] S110: Obtain an original sample set.

[0045] Among them, the original sample includes at least one target image, a background image matching the target image, and an image label. The target image is an image containing foreign matter identified by the foreign matter recognition model, and the image label indicates whether the recognition result of the foreign matter recognition model is a false alarm.

[0046] Wherein, the original sample set includes at least one target image. Exemplarily, in a monitoring image on a certain railway line, the part where foreign objects are detected can be used as a target image. At the same time, background images matching these target images are also included. For example, a surrounding background image on a certain railway line where no foreign objects are detected. In addition, there is an image label. The role of the image label is to clearly mark whether the recognition result of the foreign object recognition model for each target image is a false alarm. For example, if the model mistakenly identifies an object that is not actually a foreign object as a foreign object, the corresponding image label will be marked as a false alarm; conversely, if it is indeed a foreign object and the identification is correct, the image label will be marked as a non-false alarm.

[0047] Optionally, after obtaining the original sample set, it also includes: based on a preset data enhancement rule, performing data enhancement processing on the target image and background image in each original sample to obtain each target image and background image after data enhancement; and / or based on a preset image scaling rule, resizing the target image and background image in each original sample to obtain each target image and background image that meets the set size.

[0048] Specifically, after obtaining the original sample set, firstly, based on the preset data enhancement rules, the target image and the background image in each original sample are comprehensively enhanced; wherein, the data enhancement rules may include but are not limited to flipping, rotating, color changing, adding noise and other operations; exemplarily, for a target image in which a foreign object is detected in a factory workshop, operations such as horizontal flipping or 90-degree clockwise rotation may be performed, and for the background image, the color may be fine-tuned or some random slight noise may be added. Through these processes, the target images and the background images after data enhancement are obtained to enrich the diversity and quantity of the samples and improve the generalization ability of the model; further, the target image and the background image in each original sample may be resized based on the preset image scaling rules; wherein, the image scaling rules may be set according to actual needs and the input requirements of the model; exemplarily, it may be scaled according to a fixed ratio, or adaptively scaled according to the restrictions of the long side and the short side. For example, for a target image with too high a resolution, it may be reduced to a size of 224×224 pixels that meets the model input requirements; for the background image, corresponding adjustments are also made to obtain target images and background images that meet the set sizes to meet the needs of subsequent model processing and calculations.

[0049] S120 , performing channel fusion processing on the target image and the background image in each original sample based on a preset image channel fusion model to obtain fused images of each sample that match each original sample.

[0050] Among them, the channel fusion model can be a model with channel fusion function of images, such as a convolutional neural network, an attention mechanism model, a generative adversarial network, and a multi-scale fusion model.

[0051] Relevant personnel in this field should understand that the image channels of common images may include red, green, and blue channels, or in some special image types, there may also be transparency channels, etc. Through the role of this image channel fusion model, the features of the target image and the background image in each channel can be deeply fused and optimized; for example, for a target image containing a specific foreign body, it may have unique texture or color features in a certain channel, and the corresponding background image may have complementary information in other channels. Through the precise calculation and processing of the image channel fusion model, the valuable information in these different channels can be combined and complemented. Finally, the fused images of each sample matching the original samples are obtained, providing a more comprehensive, richer, and more accurate information basis for subsequent image processing and analysis.

[0052] S130 , forming a training sample set according to the sample fusion images and image labels that match the original samples, and the image labels are used as annotation data in the training samples.

[0053] The image labels can be used as a clear annotation and description of each sample fusion image. For example, the image label may indicate that the foreign matter in a sample fusion image is real and not a false alarm; or on the contrary, it may indicate that it is a false alarm; further, the image labels serve as the annotation data in the training samples, providing clear guidance and reference standards for subsequent model training. Through these annotation data, the model can learn how to accurately identify and judge the foreign matter in the image, thereby continuously optimizing its own performance and accuracy.

[0054] S140: Use the training sample set to train the consistency classification network model to obtain a foreign body recognition result verification model.

[0055] The consistency classification network model includes a feature extraction layer, a convolution layer, a linear classification layer and a parameter adjustment layer connected in sequence.

[0056] Specifically, the consistency classification network model is trained using the training sample set to obtain a foreign body recognition result verification model, including: obtaining a target training sample in the training sample set, and inputting the target training sample into the consistency classification network model; performing a feature extraction operation on the sample fusion image in the target training sample through the feature extraction layer in the consistency classification network model to obtain a target fusion feature; processing the target fusion feature through the convolution layer in the consistency classification network model to obtain a target consistency difference matching the target fusion feature; processing the target consistency difference through the linear classification layer in the consistency classification network model to obtain a detection classification result; adjusting the parameters of the consistency classification network model according to the detection classification result and the image label in the target training sample through the parameter adjustment layer in the consistency classification network model; returning to execute the operation of obtaining the target training sample in the training sample set until the training end condition is met, and determining the trained consistency classification network model as the foreign body recognition result verification model.

[0057] Specifically, based on the above steps, the target training samples can be some samples randomly selected from the training sample set or selected according to specific rules; further, the feature extraction layer in the consistency classification network model is used to perform feature extraction operations on the sample fusion image in the target training samples, so as to obtain target fusion features, and the feature extraction layer can be used to extract representative and distinguishing features from the input image, such as image edges, textures, color distribution and other features; further, the above target fusion features are processed by the convolution layer in the consistency classification network model to obtain target consistency differences that match the target fusion features, and the convolution layer can be used to capture local patterns and correlations in the features through convolution operations; further, the linear classification layer in the consistency classification network model is used to process the target consistency differences to obtain the detection classification results, wherein the linear classification layer is based on the learned The classification boundary of the input is used to make a classification decision to determine whether the foreign body in the image is correctly identified; further, the parameter adjustment layer in the consistency classification network model is used to adjust the parameters of the consistency classification network model according to the detected classification results and the image labels in the target training samples. The parameter adjustment layer is used to update the model's weights, biases and other parameters according to the difference between the model's output and the true label to optimize the model performance. Finally, the operation of obtaining the target training sample in the training sample set is performed again, and this continues until the end of training conditions are met. The end of training conditions can be that the accuracy of the model on the verification set reaches a certain level, or the number of training rounds reaches a preset value, etc., which can be adjusted by relevant personnel according to the actual implementation scenario, and this embodiment does not limit this; finally, the trained consistency classification network model is identified as a foreign body recognition result verification model, which is used to verify the accuracy of new foreign body recognition results.

[0058] The technical solution of the embodiment of the present invention first obtains the original sample set, and performs channel fusion processing on the target image and the background image in each original sample based on a preset image channel fusion model to obtain each sample fusion image matching each original sample, then forms a training sample set according to each sample fusion image matching each original sample and the image label, and finally uses the training sample set to train the consistency classification network model to obtain a foreign body recognition result verification model, thereby realizing the verification of the image foreign body recognition result, improving the accuracy of the foreign body recognition result, and at the same time improving the efficiency of the foreign body recognition result verification.

[0059] Embodiment 2

[0060] Figure 2A flowchart of a method for verifying image foreign body recognition results is provided for Embodiment 2 of the present invention. This embodiment is applicable to situations where image foreign body recognition results are to be verified. The method can be performed by an image foreign body recognition result verification device. The image foreign body recognition result verification device can be implemented in the form of hardware and / or software. The image foreign body recognition result verification device can be configured in a terminal or server having an image foreign body recognition result verification function.

[0061] like Figure 2 As shown, the method includes:

[0062] S210, obtaining a target foreign object image output by a foreign object recognition model, and extracting a foreign object background image from the target foreign object image.

[0063] Among them, the foreign object recognition model is a model that can be used to detect and mark the presence of foreign objects in an image; exemplarily, in the daily monitoring of railways, a deep learning model is used to determine whether there are foreign objects on railway tracks; further, after obtaining the target foreign object image, a foreign object background image is extracted from it; wherein, the foreign object background image refers to the image area surrounding the foreign object in the target foreign object image except for the part marked as a foreign object. For example, if the target foreign object image is a photo of a mobile phone screen, and the marked foreign object is a tiny scratch, then the screen image outside the scratch constitutes the foreign object background image.

[0064] S220 , performing channel fusion processing on the target foreign body image and the foreign body background image based on a preset image channel fusion model to obtain a target fused image.

[0065] S230: Input the target fused image into a foreign body recognition result verification model to obtain a foreign body probability table of the target foreign body image.

[0066] S240. Obtain a verification result of a foreign object recognition result of the target foreign object image according to the foreign object probability table.

[0067] Specifically, according to the foreign object probability table, a verification result of the foreign object recognition result of the target foreign object image is obtained, including: obtaining the probability of a foreign object matching the target foreign object image according to the foreign object probability table; when the foreign object probability is not less than a preset probability threshold, determining that the foreign object recognition result of the target foreign object image is correct, and giving a foreign object prompt according to the target foreign object image; when the foreign object probability is less than the preset probability threshold, determining that the foreign object recognition result of the target foreign object image is wrong, and giving up giving a foreign object prompt according to the target foreign object image.

[0068] In a specific implementation scenario of this embodiment, the process of obtaining the verification result of the foreign body recognition result of the target foreign body image according to the foreign body probability table is as follows: First, the foreign body probability corresponding to the target foreign body image is obtained according to the foreign body probability table. The foreign body probability here refers to the probability value of the target foreign body image being judged to contain foreign bodies. For example, in the analysis of an electronic component detection image, the foreign body probability obtained may be 0.8, indicating that there is an 80% probability that foreign bodies exist in the image. When the foreign body probability is not less than the preset probability threshold, the foreign body recognition result of the target foreign body image is judged to be correct. The preset probability threshold here is a standard value set in advance for judging whether the foreign body recognition result is reliable. For example, in a specific detection scenario, the preset probability threshold can be set to 0.7. When the judgment result is correct, a foreign body prompt will be given according to the target foreign body image. The foreign body prompt can be in the form of sounding an alarm, highlighting the foreign body area on the display screen, or sending a prompt message to relevant personnel. When the foreign body probability is less than the preset probability threshold, the foreign body recognition result of the target foreign body image is judged to be wrong. In this case, the foreign body prompt according to the target foreign body image is abandoned to avoid unnecessary interference and erroneous processing caused by erroneous prompts.

[0069] The technical solution of the embodiment of the present invention obtains the target foreign object image output by the foreign object recognition model, extracts the foreign object background image from the target foreign object image, then performs channel fusion processing on the target foreign object image and the foreign object background image based on a preset image channel fusion model to obtain a target fused image, then inputs the target fused image into the trained foreign object recognition result verification model to obtain a foreign object probability table of the target foreign object image, and finally obtains the verification result of the foreign object recognition result of the target foreign object image according to the foreign object probability table, thereby realizing the verification of the image foreign object recognition result, improving the accuracy of the foreign object recognition result, and at the same time improving the efficiency of the foreign object recognition result verification.

[0070] Embodiment 3

[0071] Figure 3 This is a schematic diagram of a structure of a device for constructing a foreign body recognition result verification model provided by Embodiment 3 of the present invention. Figure 3 As shown, the device comprises:

[0072] The original sample acquisition module 310 is used to acquire an original sample set, wherein the original sample includes at least one target image, a background image matching the target image, and an image label, wherein the target image is an image containing foreign matter identified by the foreign matter recognition model, and the image label indicates whether the recognition result of the foreign matter recognition model is a false alarm;

[0073] An image fusion module 320 is used to perform channel fusion processing on the target image and the background image in each original sample based on a preset image channel fusion model to obtain a fused image of each sample matching the original sample;

[0074] A training sample generation module 330 is used to form a training sample set according to each sample fusion image and image label matched with each original sample, and the image label is used as the annotation data in the training sample;

[0075] The model training module 340 is used to train the consistency classification network model using the training sample set to obtain a foreign body recognition result verification model.

[0076] The technical solution of the embodiment of the present invention first obtains the original sample set, and performs channel fusion processing on the target image and the background image in each original sample based on a preset image channel fusion model to obtain each sample fusion image matching each original sample, then forms a training sample set according to each sample fusion image matching each original sample and the image label, and finally uses the training sample set to train the consistency classification network model to obtain a foreign body recognition result verification model, thereby realizing the verification of the image foreign body recognition result, improving the accuracy of the foreign body recognition result, and at the same time improving the efficiency of the foreign body recognition result verification.

[0077] Based on the above embodiment, the model training module 340 includes:

[0078] A data input unit, used for obtaining a target training sample from a training sample set, and inputting the target training sample into a consistency classification network model;

[0079] A feature fusion unit is used to perform a feature extraction operation on the sample fusion image in the target training sample through the feature extraction layer in the consistency classification network model to obtain a target fusion feature;

[0080] A convolution calculation unit, used to process the target fusion feature through a convolution layer in a consistency classification network model to obtain a target consistency difference matching the target fusion feature;

[0081] A linear classification unit, used to process the target consistency difference through a linear classification layer in a consistency classification network model to obtain a detection classification result;

[0082] A parameter adjustment unit, configured to adjust the parameters of the consistency classification network model according to the detected classification result and the image labels in the target training sample through a parameter adjustment layer in the consistency classification network model;

[0083] The loop unit is used to return to execute the operation of obtaining the target training sample in the training sample set until the training end condition is met, and the consistency classification network model obtained by training is determined as the foreign body recognition result verification model.

[0084] On the basis of the above-mentioned embodiment, the original sample acquisition module 310 is further used to: after acquiring the original sample set, based on the preset data enhancement rules, perform data enhancement processing on the target image and the background image in each original sample to obtain each target image and the background image after data enhancement; and / or based on the preset image scaling rules, resize the target image and the background image in each original sample to obtain each target image and the background image that meets the set size.

[0085] A device for constructing a foreign body identification result verification model provided by an embodiment of the present invention can execute a method for constructing a foreign body identification result verification model provided by any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0086] Embodiment 4

[0087] Figure 4 A schematic diagram of the structure of a device for verifying image foreign body recognition results provided in Embodiment 4 of the present invention.

[0088] like Figure 4 As shown, the device comprises:

[0089] The foreign body recognition module 410 is used to obtain the target foreign body image output by the foreign body recognition model and extract the foreign body background image from the target foreign body image;

[0090] A channel fusion module 420 is used to perform channel fusion processing on the target foreign body image and the foreign body background image based on a preset image channel fusion model to obtain a target fused image;

[0091] A model processing module 430, configured to input the target fused image into a foreign body recognition result verification model trained by the method according to any one of claims 1 to 4, to obtain a foreign body probability table of the target foreign body image;

[0092] The result acquisition module 440 is used to obtain the verification result of the foreign object recognition result of the target foreign object image according to the foreign object probability table.

[0093] The technical solution of the embodiment of the present invention obtains the target foreign object image output by the foreign object recognition model, extracts the foreign object background image from the target foreign object image, then performs channel fusion processing on the target foreign object image and the foreign object background image based on a preset image channel fusion model to obtain a target fused image, then inputs the target fused image into the trained foreign object recognition result verification model to obtain a foreign object probability table of the target foreign object image, and finally obtains the verification result of the foreign object recognition result of the target foreign object image according to the foreign object probability table, thereby realizing the verification of the image foreign object recognition result, improving the accuracy of the foreign object recognition result, and at the same time improving the efficiency of the foreign object recognition result verification.

[0094] Based on the above embodiment, the result acquisition module 440 includes:

[0095] A probability acquisition unit, configured to acquire the probability of a foreign object matching the target foreign object image according to the foreign object probability table;

[0096] A first judgment unit is used to determine that the foreign object recognition result of the target foreign object image is correct when the foreign object probability is not less than a preset probability threshold, and to provide a foreign object prompt according to the target foreign object image;

[0097] The second judgment unit is used to determine that the foreign object recognition result of the target foreign object image is wrong when the foreign object probability is less than a preset probability threshold, and give up the foreign object prompt according to the target foreign object image.

[0098] An image foreign body recognition result verification device provided by an embodiment of the present invention can execute an image foreign body recognition result verification method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects of the execution method.

[0099] Embodiment 5

[0100] Figure 5 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0101] like Figure 5As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0102] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0103] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a method for constructing a foreign body recognition result verification model and a method for verifying an image foreign body recognition result.

[0104] Accordingly, a method for constructing a foreign body recognition result verification model includes:

[0105] Acquire an original sample set, wherein the original sample includes at least one target image, a background image matching the target image, and an image label, wherein the target image is an image containing a foreign object identified by a foreign object recognition model, and the image label indicates whether the recognition result of the foreign object recognition model is a false alarm;

[0106] Based on a preset image channel fusion model, channel fusion processing is performed on the target image and the background image in each original sample to obtain a fused image of each sample that matches each original sample;

[0107] A training sample set is formed according to each sample fusion image and image label matched with each original sample, and the image label is used as the annotation data in the training sample;

[0108] The consistent classification network model is trained using the training sample set to obtain a foreign object recognition result verification model.

[0109] In some embodiments, a method for constructing a foreign body identification result verification model and a method for verifying an image foreign body identification result can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via ROM12 and / or communication unit 19. When the computer program is loaded into RAM13 and executed by processor 11, one or more steps of the method for constructing a foreign body identification result verification model and a method for verifying an image foreign body identification result described above can be executed. Alternatively, in other embodiments, processor 11 can be configured to execute a method for constructing a foreign body identification result verification model and a method for verifying an image foreign body identification result by any other appropriate means (for example, by means of firmware).

[0110] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0111] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0112] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0113] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0114] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0115] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0116] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

Claims

1. A method for constructing a foreign body recognition result verification model, characterized in that: include: Acquire an original sample set, wherein the original sample includes at least one target image, a background image matching the target image, and an image label, wherein the target image is an image containing a foreign object identified by a foreign object recognition model, and the image label indicates whether the recognition result of the foreign object recognition model is a false alarm; Based on a preset image channel fusion model, channel fusion processing is performed on the target image and the background image in each original sample to obtain a fused image of each sample that matches each original sample; A training sample set is formed according to each sample fusion image and image label matched with each original sample, and the image label is used as the annotation data in the training sample; The consistent classification network model is trained using the training sample set to obtain a foreign object recognition result verification model.

2. The method according to claim 1, characterized in that The consistency classification network model includes a feature extraction layer, a convolution layer, a linear classification layer and a parameter adjustment layer which are connected in sequence.

3. The method according to claim 2, characterized in that The consistent classification network model is trained using the training sample set to obtain a foreign body recognition result verification model, including: Obtain target training samples from the training sample set, and input the target training samples into the consistency classification network model; Perform feature extraction operation on the sample fusion image in the target training sample through the feature extraction layer in the consistency classification network model to obtain the target fusion feature; Processing the target fusion feature through a convolutional layer in a consistency classification network model to obtain a target consistency difference that matches the target fusion feature; The target consistency difference is processed through a linear classification layer in a consistency classification network model to obtain a detection classification result; Adjusting the parameters of the consistency classification network model according to the detection classification results and the image labels in the target training samples through a parameter adjustment layer in the consistency classification network model; Return to execute the operation of obtaining the target training sample in the training sample set until the training end condition is met, and the consistency classification network model obtained by training is determined as the foreign body recognition result verification model.

4. The method according to claim 1, characterized in that After obtaining the original sample set, it also includes: Based on a preset data enhancement rule, data enhancement processing is performed on the target image and the background image in each original sample to obtain each target image and background image after data enhancement; and / or Based on the preset image scaling rules, the target image and the background image in each original sample are resized to obtain each target image and the background image that meet the set size.

5. A method for verifying image foreign body recognition results, characterized in that: include: Obtaining a target foreign object image output by a foreign object recognition model, and extracting a foreign object background image from the target foreign object image; Based on a preset image channel fusion model, channel fusion processing is performed on the target foreign body image and the foreign body background image to obtain a target fused image; Inputting the target fused image into a foreign body recognition result verification model trained by the method according to any one of claims 1 to 4 to obtain a foreign body probability table of the target foreign body image; According to the foreign object probability table, a verification result of the foreign object recognition result of the target foreign object image is obtained.

6. The method according to claim 5, characterized in that According to the foreign body probability table, a verification result of the foreign body recognition result of the target foreign body image is obtained, including: Acquire the probability of a foreign object matching the target foreign object image according to the foreign object probability table; When the foreign object probability is not less than a preset probability threshold, determining that the foreign object recognition result of the target foreign object image is correct, and providing a foreign object prompt according to the target foreign object image; When the foreign object probability is less than a preset probability threshold, it is determined that the foreign object recognition result of the target foreign object image is wrong, and the foreign object prompt according to the target foreign object image is abandoned.

7. A device for constructing a foreign body recognition result verification model, characterized in that: include: The original sample acquisition module is used to acquire an original sample set, wherein the original sample includes at least one target image, a background image matching the target image, and an image label, the target image is an image containing foreign matter identified by the foreign matter recognition model, and the image label marks whether the recognition result of the foreign matter recognition model is a false alarm; An image fusion module is used to perform channel fusion processing on the target image and the background image in each original sample based on a preset image channel fusion model to obtain a fused image of each sample that matches each original sample; A training sample generation module, used to form a training sample set according to each sample fusion image and image label matching each original sample, and the image label is used as the annotation data in the training sample; The model training module is used to train the consistency classification network model using the training sample set to obtain a foreign body recognition result verification model.

8. An image foreign body recognition result verification device, characterized in that: include: A foreign body recognition module is used to obtain a target foreign body image output by a foreign body recognition model and extract a foreign body background image from the target foreign body image; A channel fusion module is used to perform channel fusion processing on the target foreign body image and the foreign body background image based on a preset image channel fusion model to obtain a target fused image; A model processing module, used for inputting the target fusion image into a foreign body recognition result verification model trained by the method according to any one of claims 1 to 4, to obtain a foreign body probability table of the target foreign body image; The result acquisition module is used to obtain the verification result of the foreign body recognition result of the target foreign body image according to the foreign body probability table.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a method for constructing a foreign body recognition result verification model and an image foreign body recognition result verification method as described in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a method for constructing a foreign body recognition result verification model and an image foreign body recognition result verification method as described in any one of claims 1 to 7 when executed.

Images