Image detection method, device, electronic device and storage medium
By using feature extraction networks and coordinate prediction networks in image detection to determine image vertex coordinates and perform perspective transformation, the image distortion problem caused by shooting angle deviation is solved, and the efficiency and accuracy of image detection are improved.
Patent Information
- Application Number
- CN202111262362.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-28
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2041-10-28
AI Technical Summary
In the prior art, image distortion caused by shooting angle deviation during image detection can only obtain the smallest external rectangular area, which cannot be recognized, affecting the efficiency of image resolution.
By acquiring the original image, inputting the feature extraction network and coordinate prediction network of the pre-trained image detection model, determining the vertex coordinates of the image to be detected, and performing perspective transformation to eliminate image distortion to obtain the target image.
The shooting angle is not required to be adjusted multiple times, which improves the efficiency of image detection and ensures the accuracy of image recognition.
Smart Images

Figure CN113971738B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image detection, and in particular to an image detection method, device, electronic equipment and storage medium. Background Art
[0002] With the development of digital technology, many life scenarios involve image analysis, for example, in mobile payment, payment is completed by scanning a QR code. Generally speaking, the image analysis process includes two steps: detection and recognition. However, due to the deviation of the shooting angle, the shape of the image is distorted. The detection process only obtains the minimum circumscribed rectangular area of the distorted image and cannot be recognized. The only way is to continuously adjust the shooting angle until a recognizable image is detected, which seriously affects the efficiency of image analysis. Summary of the invention
[0003] The embodiments of the present invention provide an image detection method, an apparatus, an electronic device and a storage medium, which can overcome the deficiencies of the prior art.
[0004] In order to achieve the above purpose, the technical solution adopted by the embodiment of the present invention is as follows:
[0005] In a first aspect, an embodiment of the present invention provides an image detection method, the image detection method comprising:
[0006] Acquire an original image, wherein the original image includes an image to be detected, and the image to be detected is a polygon containing at least four vertices;
[0007] Inputting the original image into a feature extraction network of a pre-trained image detection model to obtain a feature map of the original image;
[0008] Inputting the feature map into the coordinate prediction network of the image detection model to obtain the vertex coordinates of the image to be detected;
[0009] Determine the image to be detected from the original image according to the vertex coordinates;
[0010] Perform perspective transformation on the image to be detected to obtain a target image.
[0011] As a possible implementation, the coordinate prediction network includes a thermal layer, a relative offset layer, a vertex offset layer and a decoding layer, and the step of inputting the feature map into the coordinate prediction network of the image detection model to obtain the vertex coordinates of the image to be detected includes:
[0012] Inputting the characteristic map into the thermodynamic layer to obtain the coordinates of the reference point;
[0013] Inputting the feature map into the relative offset layer to obtain a first offset of the coordinates of the center point of the image to be detected relative to the coordinates of the reference point;
[0014] Inputting the feature map into the vertex offset layer to obtain a second offset of the vertex coordinates of the image to be detected relative to the center point coordinates;
[0015] The coordinates of the reference point, the first offset and the second offset are used as decoding parameters of the feature map, and the decoding parameters are input into the decoding layer to obtain the vertex coordinates of the image to be detected.
[0016] As a possible implementation manner, the step of inputting the decoding parameters into the decoding layer to obtain the vertex coordinates of the image to be detected includes:
[0017] Using the decoding layer, according to the coordinates of the reference point and the first offset, the coordinates of the center point of the image to be detected are obtained;
[0018] The vertex coordinates of the image to be detected are obtained according to the center point coordinates and the second offset.
[0019] As a possible implementation, there are multiple feature maps, the coordinate prediction network further includes a suppression layer, and the step of inputting the feature map into the coordinate prediction network of the image detection model to obtain the vertex coordinates of the image to be detected further includes:
[0020] Input each of the feature maps into the thermal layer, the relative offset layer and the vertex offset layer respectively to obtain decoding parameters of each of the feature maps;
[0021] Inputting the decoding parameters of each of the feature maps into the decoding layer to obtain the vertex coordinates corresponding to each of the feature maps;
[0022] The vertex coordinates corresponding to all the feature maps are input into the suppression layer to obtain the vertex coordinates of the image to be detected.
[0023] As a possible implementation, the thermal layer includes a convolution module, and the step of inputting the feature map into the thermal layer to obtain the reference point coordinates includes:
[0024] Inputting the feature map into the convolution module to obtain a thermal feature map;
[0025] The coordinates of the pixel points in the thermal characteristic map whose thermal values are greater than the preset values are used as the coordinates of the reference points.
[0026] As a possible implementation manner, the step of performing perspective transformation on the image to be detected to obtain a target image includes:
[0027] Solving the perspective transformation matrix according to the preset point coordinates and the vertex coordinates, wherein the preset point coordinates are the vertex coordinates of the target image, and the target image is an image after perspective transformation of the image to be detected;
[0028] The coordinates of each pixel in the image to be detected are transformed according to the perspective transformation matrix to obtain the target image.
[0029] As a possible implementation method, the method of training the image detection model is:
[0030] Get training data and labels;
[0031] Inputting the training data into the image detection model to obtain predicted vertex coordinates;
[0032] According to the predicted vertex coordinates and the labels, the parameters of the image detection model are updated using a preset loss function until a preset termination condition is met, thereby obtaining a trained image detection model, wherein the preset loss function is determined based on the loss function of the thermal layer, the loss function of the relative offset layer, the loss function of the vertex offset layer, and the loss function of the size regression layer.
[0033] As a possible implementation, the preset loss function is based on the loss function of the thermal layer, the loss function of the relative offset layer, the loss function of the size regression layer and the loss function of the vertex offset layer using the formula L total =aL cls +bL off +cL size +dL cor Determine, where the L total is the loss function of the image detection model, the L cls is the loss function of the thermodynamic layer, the L off is the loss function of the relative offset layer, and the L size is the loss function of the size regression layer, the L cor is the loss function of the vertex offset layer, and a, b, c, and d are weight factors of the thermal layer, the relative offset layer, the size regression layer, and the vertex offset layer, respectively.
[0034] In a second aspect, an embodiment of the present invention further provides an image detection device, the image detection device comprising:
[0035] An acquisition module, used for acquiring an original image, wherein the original image includes an image to be detected, and the image to be detected is a polygon containing at least four vertices;
[0036] A detection module, used for inputting the original image into a feature extraction network of a pre-trained image detection model to obtain a feature map of the original image;
[0037] The detection module is further used to input the feature map into the coordinate prediction network of the image detection model to obtain the vertex coordinates of the image to be detected;
[0038] The detection module is further used to determine the image to be detected from the original image according to the vertex coordinates;
[0039] The correction module is used to perform perspective transformation on the image to be detected according to the vertex coordinates to obtain a target image.
[0040] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the image detection method provided in the first aspect of the embodiment of the present invention when calling the computer program.
[0041] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the image detection method provided in the first aspect of the embodiment of the present invention is implemented.
[0042] Compared with the prior art, an image detection method, device, electronic device and storage medium provided in an embodiment of the present invention first obtain an original image, wherein the original image includes an image to be detected, and the image to be detected is a polygon containing at least four vertices; then, the original image is input into a feature extraction network of a pre-trained image detection model to obtain a feature map of the original image; then, the feature map is input into a coordinate prediction network of the image detection model to obtain the vertex coordinates of the image to be detected, and the image to be detected is determined from the original image according to the vertex coordinates; finally, the image to be detected is perspective transformed according to the vertex coordinates to obtain a target image. Since the embodiment of the present invention determines the vertex coordinates of the image to be detected by inputting the original image into a pre-trained image detection model containing a feature extraction network and a coordinate prediction network, and obtains the image to be detected from the original image according to the vertex coordinates, and then perspective transforms the image to be detected according to the vertex coordinates to obtain the target image, the influence of image distortion caused by the shooting angle on image detection and recognition is eliminated, and the shooting angle does not need to be adjusted multiple times, thereby improving the efficiency of image detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.
[0044] Figure 1 A flow chart of an image detection method provided by an embodiment of the present invention;
[0045] Figure 2 A schematic diagram of an image detection model structure provided by an embodiment of the present invention;
[0046] Figure 3 A flow chart of a method for determining vertex coordinates of an image to be detected provided by an embodiment of the present invention;
[0047] Figure 4 A schematic diagram of the structure of an image detection model in a training phase provided by an embodiment of the present invention;
[0048] Figure 5 A flowchart of perspective transformation of an image to be detected provided by an embodiment of the present invention;
[0049] Figure 6 A comparison diagram of an image to be detected before and after perspective transformation provided by an embodiment of the present invention;
[0050] Figure 7 A block diagram of an image detection device provided by an embodiment of the present invention;
[0051] Figure 8 A schematic block diagram of the structure of an electronic device provided by an embodiment of the present invention.
[0052] Icon: 100 - image detection device; 101 - acquisition module; 102 - detection module; 103 - correction module; 200 - electronic device; 210 - memory; 220 - processor. DETAILED DESCRIPTION
[0053] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.
[0054] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0055] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.
[0056] In the description of the present invention, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. appear to indicate an orientation or position relationship, they are based on the orientation or position relationship shown in the accompanying drawings, or are the orientation or position relationship in which the product of the invention is usually placed when used. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.
[0057] In addition, the terms “first”, “second”, etc., if used, are merely used to distinguish between the descriptions and should not be understood as indicating or implying relative importance.
[0058] It should be noted that, in the absence of conflict, the features in the embodiments of the present invention may be combined with each other.
[0059] In order to overcome the influence of image distortion caused by shooting angle on image detection and recognition, the embodiments of the present invention provide an image detection method, device, electronic device and storage medium, which will be described in detail below.
[0060] Please refer to Figure 1 , Figure 1 A flow chart of an image detection method provided by an embodiment of the present invention, the method comprises steps S101 to S105.
[0061] Step S101 : acquiring an original image, wherein the original image includes an image to be detected, and the image to be detected is a polygon containing at least four vertices.
[0062] In the embodiment of the present invention, the image to be detected is a component of the original image, and its shape is an irregular polygon with no less than four sides, that is, the image to be detected is at least a quadrilateral.
[0063] Step S102, inputting the original image into a feature extraction network of a pre-trained image detection model to obtain a feature map of the original image.
[0064] In an embodiment of the present invention, the feature map of the original image may be one or more, and the multiple feature maps respectively reflect spatial features of different scales. The feature extraction network may be a U-net structure or a HR-net structure, etc.
[0065] Step S103, inputting the feature map into the coordinate prediction network of the image detection model to obtain the vertex coordinates of the image to be detected.
[0066] In an embodiment of the present invention, the image detection model includes a coordinate prediction network, and the feature map of the original image output by the feature extraction network is input into the coordinate prediction network to realize the detection of the coordinates of multiple vertices of the image to be detected.
[0067] Step S104, determining the image to be detected from the original image according to the vertex coordinates.
[0068] In the embodiment of the present invention, the position of the image to be detected in the original image and the area occupied by the image to be detected are determined according to the coordinates of multiple vertices of the image to be detected, and the image to be detected is cut out.
[0069] Step S105, performing perspective transformation on the image to be detected to obtain a target image.
[0070] In the embodiment of the present invention, the target image refers to an image that eliminates image distortion caused by the shooting angle in the image to be detected and can be used for subsequent image recognition. In other words, the image to be detected is the distorted target image.
[0071] The above method provided by the embodiment of the present invention has the beneficial effect that, by inputting the original image into a pre-trained image detection model including a feature extraction network kernel coordinate prediction network, the vertex coordinates of the image to be detected are determined, and the image to be detected is obtained from the original image according to the vertex coordinates, and then the image to be detected is perspective transformed according to the vertex coordinates to obtain the target image, thereby eliminating the influence of image distortion caused by the shooting angle on image detection and recognition, and there is no need to adjust the shooting angle multiple times, thereby improving the efficiency of image detection.
[0072] In order to better understand the steps of the above-mentioned image detection method, the embodiment of the present invention provides a specific image detection model, please refer to Figure 2 , Figure 2A schematic diagram of the structure of an image detection model provided in an embodiment of the present invention, the image detection model includes a feature extraction network of a U-net structure, which includes multiple downsampling layers and multiple upsampling layers connected to the multiple downsampling layers by jumps. The original image is input into the feature extraction network of the U-net structure, and spatial features of different scales are gradually extracted through layers of downsampling layers, and then the spatial features of the corresponding scales are restored through layers of upsampling layers, and the feature map of the original image is output. The image detection model also includes a coordinate prediction network, which includes a thermal layer, a relative offset layer, a vertex offset layer, a decoding layer, and a suppression layer.
[0073] based on Figure 1 and Figure 2 The embodiment of the present invention provides a specific implementation method for determining the vertex coordinates of the image to be detected by using a coordinate prediction network, please refer to Figure 3 , Figure 3 A flow chart of a method for determining vertex coordinates of an image to be detected provided by an embodiment of the present invention, step S103 includes sub-steps S103 - 1 to S103 - 4.
[0074] Sub-step S103-1, input the characteristic map into the thermodynamic layer to obtain the coordinates of the reference point.
[0075] In the embodiment of the present invention, the reference point is used to assist in locating the position of the image to be detected in the original image, and the coordinates of the reference point are determined by inputting the feature map of the original image into the thermodynamic layer and according to the output result of the thermodynamic layer.
[0076] As a specific implementation method, the specific steps of sub-step S103-1 include:
[0077] First, the feature map is input into the convolution module to obtain the thermal feature map.
[0078] Then, the coordinates of the pixel points in the thermal characteristic map whose thermal values are greater than the preset values are used as the coordinates of the reference points.
[0079] In an embodiment of the present invention, the thermal layer includes a convolution module, the feature map of the original image is input into the convolution module, and the convolution module outputs a thermal feature map. The above-mentioned preset value is a threshold value for classifying pixel points in the thermal feature map. When the thermal value of a pixel point is greater than the preset value, the pixel point is determined as a reference point, and its coordinates are the coordinates of the reference point.
[0080] Sub-step S103-2, inputting the feature map into the relative offset layer to obtain a first offset of the coordinates of the center point of the image to be detected relative to the coordinates of the reference point.
[0081] In an embodiment of the present invention, the first offset of the coordinates of the center point of the image to be detected relative to the coordinates of the reference point refers to the distance of the center point relative to the reference point in the x direction and the distance of the center point relative to the reference point in the y direction. The feature map of the original image is input into the relative offset layer, and the first offset of the coordinates of the center point of the image to be detected relative to the coordinates of the reference point is output.
[0082] Sub-step S103-3, inputting the feature map into the vertex offset layer to obtain a second offset of the vertex coordinates of the image to be detected relative to the center point coordinates.
[0083] In an embodiment of the present invention, the second offset of the vertex coordinates of the image to be detected relative to the center point coordinates refers to the distance of the vertex relative to the center point in the x direction and the distance of the vertex relative to the center point in the y direction. The feature map of the original image is input into the vertex offset layer, and the second offset of the vertex coordinates of the image to be detected relative to the center point coordinates is output.
[0084] Sub-step S103-4, using the coordinates of the reference point, the first offset and the second offset as decoding parameters of the feature map, and inputting the decoding parameters into a decoding layer to obtain the vertex coordinates of the image to be detected.
[0085] In an embodiment of the present invention, the decoding process of the decoding layer is the process of calculating the coordinates of the vertices of the image to be detected, and the decoding process requires the use of the coordinates of the reference point, the first offset and the second offset. The coordinates of the reference point, the first offset and the second offset obtained based on the feature map of the original image constitute the decoding parameters.
[0086] In the embodiment of the present invention, the feature map of the input coordinate prediction network may be one. In this case, the specific implementation of sub-step S103-4 is as follows:
[0087] First, using the decoding layer, the coordinates of the center point of the image to be detected are obtained according to the coordinates of the reference point and the first offset.
[0088] Then, the vertex coordinates of the image to be detected are obtained according to the center point coordinates and the second offset.
[0089] In the embodiment of the present invention, the specific implementation principle of the decoding process of the decoding layer is shown in the following formula:
[0090]
[0091] Among them, (x c ,y c ) represents the coordinates of the reference point, (δx c,i ,δy c,i ) is the first offset, is the second offset, is the center point coordinate of the image to be detected, are the vertex coordinates of the image to be detected.
[0092] In an embodiment of the present invention, the feature graphs input to the coordinate prediction network may also be multiple, which are output by the original image through the feature extraction network and have different scales. The thermal layer, relative offset layer and vertex offset layer of the coordinate prediction network output multiple sets of decoding parameters based on feature graphs of different scales, wherein each feature graph corresponds to a set of decoding parameters, and the decoding parameters of each feature graph can be calculated by using the decoding layer to obtain the vertex coordinates corresponding to each feature graph, and the vertex coordinates of the image to be detected can be determined from the vertex coordinates corresponding to the multiple feature graphs by using the suppression layer, that is, the specific implementation of sub-step S103-4 is as follows:
[0093] First, each feature map is input into the thermal layer, relative offset layer and vertex offset layer respectively to obtain the decoding parameters of each feature map.
[0094] Then, the decoding parameters of each feature map are input into the decoding layer to obtain the vertex coordinates corresponding to each feature map.
[0095] It can be understood that when there are multiple feature maps of the input coordinate prediction network, the specific implementation principles of the decoding parameters and the decoding process of the decoding layer for each feature map are the same as when there is only one feature map of the input coordinate prediction network.
[0096] Finally, the vertex coordinates corresponding to all feature maps are input into the suppression layer to obtain the vertex coordinates of the image to be detected.
[0097] As a specific implementation method, the suppression layer includes a classification module and a non-maximum suppression module. The vertex coordinates corresponding to each feature map and the original image are input into the classification module, and multiple candidate boxes and the scores of each candidate box are output. The non-maximum suppression module first arranges all candidate boxes from high to low according to the scores, and then selects the candidate box with the highest score as the processing box, traverses the remaining candidate boxes, and deletes the candidate boxes whose overlapping area with the processing box is greater than a preset threshold. Then, a candidate box with the highest score is selected from the unprocessed candidate boxes as the processing object, and the above traversal-deletion process is repeated until all candidate boxes are processed, and the coordinates of each vertex of the retained candidate boxes that have not been deleted are output as the vertex coordinates of the image to be detected.
[0098] In an embodiment of the present invention, before using the above-mentioned image detection method to process the original image, it is necessary to pre-train the image detection model. The embodiment of the present invention provides a specific implementation method for training the image detection model, and the steps are:
[0099] First, obtain training data and labels;
[0100] Then, the training data is input into the image detection model to obtain the predicted vertex coordinates;
[0101] Finally, according to the predicted vertex coordinates and labels, the preset loss function is used to update the parameters of the image detection model until the preset termination conditions are met to obtain the trained image detection model, wherein the preset loss function is determined based on the loss function of the thermal layer, the loss function of the relative offset layer, the loss function of the vertex offset layer and the loss function of the size regression layer.
[0102] See also Figure 4 , Figure 4 A schematic diagram of the structure of an image detection model in a training phase is provided for an embodiment of the present invention. In the embodiment of the present invention, for the image detection model training phase, its coordinate prediction network also includes a size regression layer, which is used to obtain the size information of the image, and uses the loss function of the thermal layer, the loss function of the relative offset layer, the loss function of the vertex offset layer and the loss function of the size regression layer to determine the preset loss function of the image detection model.
[0103] As a specific implementation method, the preset loss function is based on the loss function of the thermal layer, the loss function of the relative offset layer, the loss function of the size regression layer and the loss function of the vertex offset layer using the formula L total =aL cls +bL off +cL size +dL cor Determine, where L total is the loss function of the image detection model, L cls is the loss function of the thermodynamic layer, L off is the loss function of the relative offset layer, L size is the loss function of the size regression layer, L cor is the loss function of the vertex offset layer, a, b, c, d are the weight factors of the thermal layer, relative offset layer, size regression layer and vertex offset layer respectively.
[0104] based on Figure 1 The embodiment of the present invention provides a specific implementation method for perspective transformation of an image to be detected, please refer to Figure 5 , Figure 5 A flowchart of perspective transformation of an image to be detected is provided in an embodiment of the present invention, wherein step S105 includes sub-steps S105 - 1 to S105 - 2.
[0105] Sub-step S105-1, solving the perspective transformation matrix according to the preset point coordinates and the vertex coordinates, wherein the preset point coordinates are the vertex coordinates of the target image, and the target image is the image after the perspective transformation of the image to be detected.
[0106] Sub-step S105 - 2 , performing coordinate transformation on each pixel point in the image to be detected according to the perspective transformation matrix to obtain the target image.
[0107] In the embodiment of the present invention, the image to be detected is a distorted target image, and the image to be detected is subjected to perspective transformation to eliminate the distortion in the image to be detected, thereby obtaining the target image. In order to explain the above content more intuitively, the embodiment of the present invention takes the image to be detected in the shape of an irregular quadrilateral as an example, see Figure 6 , Figure 6 A comparison diagram of an image to be detected before and after perspective transformation provided by an embodiment of the present invention, wherein the coordinates of the four vertices of the image to be detected are (x 0 ,y 0 )、(x 1 ,y 1 )、(x 2 ,y 2 ) and (x 3 ,y 3 ), the shape of the target image is a square, and the coordinates of its four vertices are (u 0 , v 0 )、(u 1 , v 1 )、(u 2 , v 2 ) and (u 3 , v 3 ), the essence of perspective transformation is to transform the image to be detected from the current xy coordinate system to the uv coordinate system where the target image is located. The perspective transformation matrix describes the correspondence between the coordinate points in these two coordinate systems, where (x 0 ,y 0 ) and (u 0 , v 0 ) corresponds to (x 1 ,y 1 ) and (u 1 , v 1 ) corresponds to (x 2 ,y 2 ) and (u 2 , v 2 ) corresponds to, and (x 3 ,y 3 ) and (u 3 , v 3 ) correspondence, based on these four sets of correspondence, use the following formula to solve the perspective transformation matrix.
[0108]
[0109] Where (x, y) is the vertex coordinates of the image to be detected, (u, v) is the vertex coordinates of the target image, A, B, C, D, E, E, G and H are the perspective transformation matrices There are 8 elements to be solved in .
[0110] According to the perspective transformation matrix, the coordinates of each pixel point in the image to be detected are transformed, and the pixel value of each pixel point in the image to be detected is assigned to the corresponding pixel point in the target image to obtain the target image.
[0111] In order to execute the corresponding steps in the above embodiment and various possible implementations, an implementation method of the image detection device 100 is given below. Figure 7 , Figure 7 It should be noted that the basic principle and technical effects of the image detection device 100 provided by the embodiment of the present invention are the same as those of the above embodiment, and for the sake of brief description, the embodiment of the present invention is not mentioned.
[0112] The image detection device 100 includes an acquisition module 101 , a detection module 102 and a correction module 103 .
[0113] The acquisition module 101 is used to acquire an original image, where the original image includes an image to be detected, and the image to be detected is a polygon containing at least four vertices.
[0114] The detection module 102 is used to input the original image into a feature extraction network of a pre-trained image detection model to obtain a feature map of the original image.
[0115] The detection module 102 is also used to input the feature map into the coordinate prediction network of the image detection model to obtain the vertex coordinates of the image to be detected.
[0116] As a specific implementation method, the detection module 102 is specifically used to input the feature map into the thermal layer to obtain the coordinates of the reference point; input the feature map into the relative offset layer to obtain the first offset of the center point coordinates of the image to be detected relative to the coordinates of the reference point; input the feature map into the vertex offset layer to obtain the second offset of the vertex coordinates of the image to be detected relative to the coordinates of the center point; use the coordinates of the reference point, the first offset and the second offset as decoding parameters of the feature map, and input the decoding parameters into the decoding layer to obtain the vertex coordinates of the image to be detected.
[0117] The detection module 102 is further used to determine the image to be detected from the original image according to the vertex coordinates.
[0118] The correction module 103 is used for the vertex coordinates to perform perspective transformation on the image to be detected to obtain a target image.
[0119] As a specific implementation method, the correction module 103 is specifically used to solve the perspective matrix according to the preset point coordinates and the vertex coordinates of the image to be detected, wherein the preset point coordinates are the vertex coordinates of the target image, and the target image is the image after correction of the image to be detected; coordinate transformation is performed on each pixel point in the image to be detected according to the perspective matrix to obtain the target image.
[0120] For further information, please refer to Figure 8 , Figure 8 The present invention provides a schematic structural block diagram of an electronic device 200 provided in an embodiment of the present invention. The electronic device 200 may include a memory 210 and a processor 220 .
[0121] Among them, the processor 220 can be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the program execution of the image detection method provided in the following method embodiments.
[0122] The memory 210 may be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 210 may exist independently and be connected to the processor 220 via a communication bus. The memory 210 may also be integrated with the processor 220. Among them, the memory 210 is used to store machine executable instructions for executing the scheme of the present application. The processor 220 is used to execute the machine executable instructions stored in the memory 210 to implement the aforementioned method embodiment.
[0123] Since the electronic device 200 provided in the embodiment of the present invention is another implementation form of the image detection method provided in the aforementioned method embodiment, the technical effects that can be obtained can refer to the aforementioned method embodiment and will not be described in detail here.
[0124] An embodiment of the present invention further provides a readable storage medium containing computer executable instructions, which, when executed, can be used to perform relevant operations in the image detection method provided by the aforementioned method embodiment.
[0125] In summary, an image detection method, device, electronic device and storage medium provided by an embodiment of the present invention first obtain an original image, wherein the original image includes an image to be detected, and the image to be detected is a polygon containing at least four vertices; then, the original image is input into a feature extraction network of a pre-trained image detection model to obtain a feature map of the original image; then, the feature map is input into a coordinate prediction network of the image detection model to obtain the vertex coordinates of the image to be detected, and the image to be detected is determined from the original image according to the vertex coordinates; finally, the image to be detected is perspectively transformed according to the vertex coordinates to obtain a target image. Compared with the prior art, the embodiment of the present invention determines the vertex coordinates of the image to be detected by inputting the original image into a pre-trained image detection model containing a feature extraction network and a coordinate telemetry network, and obtains the image to be detected from the original image according to the vertex coordinates, and then perspectively transforms the image to be detected according to the vertex coordinates to obtain the target image, thereby eliminating the influence of image distortion caused by the shooting angle on image detection and recognition, and does not need to adjust the shooting angle multiple times, thereby improving the efficiency of image detection.
[0126] The above description is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.
Claims
1. An image detection method, characterized in that: The method comprises: Acquire an original image, wherein the original image includes an image to be detected, and the image to be detected is a polygon containing at least four vertices; Inputting the original image into a feature extraction network of a pre-trained image detection model to obtain a feature map of the original image; Inputting the feature map into the coordinate prediction network of the image detection model to obtain the vertex coordinates of the image to be detected; Determining the image to be detected from the original image according to the vertex coordinates; Performing perspective transformation on the image to be detected to obtain a target image; The coordinate prediction network includes a thermal layer, a relative offset layer, a vertex offset layer and a decoding layer. The step of inputting the feature map into the coordinate prediction network of the image detection model to obtain the vertex coordinates of the image to be detected includes: Inputting the characteristic map into the thermodynamic layer to obtain the coordinates of the reference point; Inputting the feature map into the relative offset layer to obtain a first offset of the coordinates of the center point of the image to be detected relative to the coordinates of the reference point; Inputting the feature map into the vertex offset layer to obtain a second offset of the vertex coordinates of the image to be detected relative to the center point coordinates; The coordinates of the reference point, the first offset and the second offset are used as decoding parameters of the feature map, and the decoding parameters are input into the decoding layer to obtain the vertex coordinates of the image to be detected.
2. The method according to claim 1, characterized in that The step of inputting the decoding parameters into the decoding layer to obtain the vertex coordinates of the image to be detected includes: Using the decoding layer, according to the coordinates of the reference point and the first offset, the coordinates of the center point of the image to be detected are obtained; The vertex coordinates of the image to be detected are obtained according to the center point coordinates and the second offset.
3. The method according to claim 1, characterized in that There are multiple feature maps, the coordinate prediction network further includes a suppression layer, and the step of inputting the feature map into the coordinate prediction network of the image detection model to obtain the vertex coordinates of the image to be detected also includes: Input each of the feature maps into the thermal layer, the relative offset layer and the vertex offset layer respectively to obtain decoding parameters of each of the feature maps; Inputting the decoding parameters of each of the feature maps into the decoding layer to obtain the vertex coordinates corresponding to each of the feature maps; The vertex coordinates corresponding to all the feature maps are input into the suppression layer to obtain the vertex coordinates of the image to be detected.
4. The method according to claim 1, characterized in that The thermal layer includes a convolution module, and the step of inputting the feature map into the thermal layer to obtain the reference point coordinates includes: Inputting the feature map into the convolution module to obtain a thermal feature map; The coordinates of the pixel points in the thermal characteristic map whose thermal values are greater than the preset values are used as the coordinates of the reference points.
5. The method according to claim 1, characterized in that The step of performing perspective transformation on the image to be detected to obtain a target image comprises: Solving the perspective transformation matrix according to the preset point coordinates and the vertex coordinates, wherein the preset point coordinates are the vertex coordinates of the target image, and the target image is an image after perspective transformation of the image to be detected; The coordinates of each pixel in the image to be detected are transformed according to the perspective transformation matrix to obtain the target image.
6. The method according to claim 1, characterized in that The image detection model is trained in the following way: Get training data and labels; Inputting the training data into the image detection model to obtain predicted vertex coordinates; According to the predicted vertex coordinates and the labels, the parameters of the image detection model are updated using a preset loss function until a preset termination condition is met, thereby obtaining a trained image detection model, wherein the preset loss function is determined based on the loss function of the thermal layer, the loss function of the relative offset layer, the loss function of the vertex offset layer, and the loss function of the size regression layer.
7. The method according to claim 6, characterized in that The preset loss function is based on the loss function of the thermal layer, the loss function of the relative offset layer, the loss function of the size regression layer and the loss function of the vertex offset layer using the formula L total =aL cls +bL off +cL size +dL cor Determine, where the L total is the loss function of the image detection model, the L cls is the loss function of the thermodynamic layer, the L off is the loss function of the relative offset layer, and the L size is the loss function of the size regression layer, the L cor is the loss function of the vertex offset layer, and a, b, c, and d are weight factors of the thermal layer, the relative offset layer, the size regression layer, and the vertex offset layer, respectively.
8. An image detection device, characterized in that: include: An acquisition unit, used for acquiring an original image, wherein the original image includes an image to be detected, and the image to be detected is a polygon containing at least four vertices; A detection module, used for inputting the original image into a feature extraction network of a pre-trained image detection model to obtain a feature map of the original image; The detection module is further used to input the feature map into the coordinate prediction network of the image detection model to obtain the vertex coordinates of the image to be detected; The detection module is further used to determine the image to be detected from the original image according to the vertex coordinates; A correction module, used for performing perspective transformation on the image to be detected to obtain a target image; The coordinate prediction network includes a thermal layer, a relative offset layer, a vertex offset layer and a decoding layer. The detection module is specifically used to input the feature map into the thermal layer to obtain the coordinates of the reference point; input the feature map into the relative offset layer to obtain a first offset of the coordinates of the center point of the image to be detected relative to the coordinates of the reference point; input the feature map into the vertex offset layer to obtain a second offset of the vertex coordinates of the image to be detected relative to the coordinates of the center point; use the coordinates of the reference point, the first offset and the second offset as decoding parameters of the feature map, and input the decoding parameters into the decoding layer to obtain the vertex coordinates of the image to be detected.
9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the method according to any one of claims 1 to 7 when calling the computer program.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
