Image detection method, device, electronic device and storage medium
By extracting the feature map information of the target image and using two image detection branches to predict key point information, combining these key points for image detection, the accuracy problem caused by shape detection deviation in image detection technology is solved, and higher image content is achieved Detection accuracy and real-time.
Patent Information
- Application Number
- CN202010705981.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-21
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2040-07-21
AI Technical Summary
In the existing image detection technology, image shape detection results are prone to deviations, which affects the accuracy of image segmentation and content detection.
By extracting the feature map information of the target image, and using two image detection branches to predict the target key point information, combining these key point information for image detection operations, improving the accuracy of detection.
It improves the accuracy and real-timeness of image content detection, and can effectively handle image detection under different angles and lighting conditions.
Smart Images

Figure CN111899235B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and in particular to an image detection method, device, electronic equipment and storage medium. Background Art
[0002] At present, image detection technology mainly segments images based on their colors, and then detects image content based on the results of image segmentation. At present, image segmentation often relies on image shape detection results, such as Hough circle detection results. However, the current image shape detection results are prone to deviations. For example, Hough circle detection will detect many relatively close circles, but the centers of the circles are slightly deviated, which in turn affects the accuracy of image segmentation and makes the accuracy of image content detection relatively poor. Summary of the invention
[0003] Embodiments of the present invention provide an image detection method, device, electronic device and storage medium to solve the problem of poor accuracy in image content detection.
[0004] In a first aspect, an embodiment of the present invention provides an image detection method, comprising:
[0005] Extract feature map information of the target image;
[0006] Predicting first key point information of the feature map information according to the first image detection branch, and predicting second key point information of the feature map information according to the second image detection branch;
[0007] Determining target key point information of the target image according to the first key point information and the second key point information;
[0008] An image detection operation is performed on the target image based on the target key point information.
[0009] In a second aspect, an embodiment of the present invention provides an image detection device, including:
[0010] An extraction module, used to extract feature map information of a target image;
[0011] A prediction module, used to predict first key point information of the feature map information according to the first image detection branch, and predict second key point information of the feature map information according to the second image detection branch;
[0012] A determination module, configured to determine target key point information of the target image based on the first key point information and the second key point information;
[0013] A detection module is used to perform an image detection operation on the target image based on the target key point information.
[0014] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the image detection method provided in the embodiment of the present invention.
[0015] In a fourth aspect, an embodiment of the present invention provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps in the image detection method provided by the embodiment of the present invention are implemented.
[0016] In an embodiment of the present invention, feature map information of a target image is extracted; first key point information of the feature map information is predicted based on a first image detection branch, and second key point information of the feature map information is predicted based on a second image detection branch; target key point information of the target image is determined based on the first key point information and the second key point information; and an image detection operation is performed on the target image based on the target key point information. Since the target key point information is predicted by two image detection branches, and an image detection operation is performed on the target image based on the target key point information, compared with the prior art that relies on the shape detection result of the image to perform image detection, the embodiment of the present invention can improve the accuracy of image content detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 is a flow chart of an image detection method provided by an embodiment of the present invention;
[0018] Figure 2 is a schematic diagram of an image detection model provided by an embodiment of the present invention;
[0019] Figure 3 is a schematic diagram of key points of a target image provided by an embodiment of the present invention;
[0020] Figure 4 is a schematic diagram of an image detection provided by an embodiment of the present invention;
[0021] Figure 5 is a schematic diagram of rotating a training image sample provided by an embodiment of the present invention;
[0022] Figure 6 is a schematic diagram of training image sample fusion provided by an embodiment of the present invention;
[0023] Figure 7 is a flow chart of training image sample fusion provided by an embodiment of the present invention;
[0024] Figure 8 is another flow chart of training image sample fusion provided by an embodiment of the present invention;
[0025] Fig. 9 is a structural diagram of an image detection device provided by an embodiment of the present invention;
[0026] Fig.10 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0028] The terms "first", "second", etc. in the specification and claims of this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of this application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first" and "second" are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.
[0029] See also Figure 1 , Figure 1 is a flow chart of an image detection method provided by an embodiment of the present invention, such as Figure 1 As shown, the following steps are included:
[0030] Step 101: extract feature map information of the target image.
[0031] The target image may be an image captured by a camera of the electronic device, or may be an image transmitted by other devices and received by the electronic device.
[0032] The feature map information may be obtained by performing feature recognition on a target image to extract the feature map information. Furthermore, the feature map information may describe image features of the target image.
[0033] Step 102: predict first key point information of the feature map information based on the first image detection branch, and predict second key point information of the feature map information based on the second image detection branch.
[0034] The first image detection branch and the second image detection branch are two different branches for detecting image key point information, for example, two different key point detection technologies.
[0035] Through step 102, key point information detected by different detection branches for the same feature map information can be obtained.
[0036] Step 103: Determine target key point information of the target image according to the first key point information and the second key point information.
[0037] The above-mentioned determination of the target key point information of the target image based on the first key point information and the second key point information can be to fuse the first key point information and the second key point information to obtain the target key point information, or can be based on one of the key point information and adjust the key point information according to the other key point information to obtain the target key point information, etc., and there is no limitation to this.
[0038] Step 104: Perform an image detection operation on the target image based on the target key point information.
[0039] The above-mentioned image detection operation performed on the target image based on the target key point information may be to detect the gesture, behavior or image indication content corresponding to the above-mentioned target image based on the target key point information, etc. For example: taking the above-mentioned target image as a person image as an example, the above-mentioned target key point information may include the key point information of the person (for example: one or more key points of the head, hands, feet, torso and other parts), so as to determine the behavior of the person based on the key point information; for another example: taking the above-mentioned target image as a hand image as an example, the above-mentioned target key point information may include the key point information of the hand (for example: one or more key points of 5 fingers, wrists and other parts), so as to determine the movement of the hand based on the key point information. It should be noted that the above is only an example. In the embodiment of the present invention, there is no limitation on performing the image detection operation on the target image based on the target key point information.
[0040] In the embodiment of the present invention, the above steps can be used to predict the target key point information through two image detection branches, and perform image detection operations on the target image based on the target key point information, which can improve the accuracy of image content detection. Furthermore, since the image detection operation is performed based on the key point information, the key point information can be extracted from the target images collected at different angles and different lighting backgrounds, and then the image detection operation is performed based on the key point information to improve the real-time performance of image detection, and can also improve the accuracy of image detection.
[0041] It should be noted that the image detection method provided in the embodiment of the present invention can be applied to electronic devices, such as security detection electronic devices, mobile phones, computers and other electronic devices.
[0042] As an optional implementation manner, extracting feature map information of the target image includes:
[0043] Extracting multiple feature maps of different scales of the target image;
[0044] Converting the multiple feature maps of different scales into a feature map of a unified scale;
[0045] The converted multiple feature maps are concatenated to obtain the feature map information.
[0046] The above-mentioned extraction of multiple feature maps of different scales of the target image may be performed by sampling the target image multiple times to obtain feature maps of different scales, for example: performing multiple downsampling, and the feature maps are successively reduced to half of the original ones.
[0047] The above-mentioned splicing of the multiple feature maps after conversion to obtain the feature map information may be splicing the multiple feature maps after conversion to obtain the feature map information through a full connection (concat) operation of a neural network.
[0048] In this implementation, since the feature map information is obtained based on multiple feature maps of different scales, the feature map information is more accurate.
[0049] As an optional implementation, predicting the first key point information of the feature map information based on the first image detection branch, and predicting the second key point information of the feature map information based on the second image detection branch, includes:
[0050] The first key point information of the feature map information is predicted based on a heatmap detection technology, and the second key point information of the feature map information is predicted based on a keypoint detection technology.
[0051] The first key point information of the feature map information predicted by the thermal map detection technology can be that the thermal map detection technology extracts the thermal map auxiliary information of the feature map information, identifies the brightest feature point of the thermal map, and obtains the first key point information. Taking the target image as an instrument panel as an example, the upper left end point and the lower right end point of the instrument panel, as well as the pointer and the end points of each color area, each point can be regarded as the brightest feature point of each layer of the thermal map, so that the position information and key point information of the instrument panel can be predicted.
[0052] The second key point information of the feature map information predicted by the key point detection technology may be obtained by performing key point regression on the feature map information according to the key point detection technology to obtain the second key point information of the feature map information. Taking the target image as an instrument panel as an example, the key point regression is performed according to the key point detection technology to obtain the position deviation information of the center point of the instrument panel and the upper left, lower right and each color endpoint, so as to obtain the position of the instrument panel and the position of the key point, that is, the second key point information.
[0053] In this implementation, high-precision and high-accuracy key point information can be obtained through thermal map detection technology and key point detection technology.
[0054] Furthermore, in this embodiment, the first key point information and the second key point information may include the center position and key point position of the target image, for example: the endpoints of the pointer of the dashboard image and the endpoints of the different color areas of the dial. After obtaining the first key point information and the second key point information through the thermal map detection technology and the key point detection technology, the first key point information is combined with the second key point information for comprehensive processing, aligned with the center point of the target image, and then corrected for each key point to achieve high-precision key point information prediction. For example, high-precision dashboard detection and key point prediction can be achieved for the dashboard.
[0055] As an optional implementation manner, the target key point information includes: limit point coordinates, center point coordinates, target point coordinates and at least one separation point coordinate, and the separation point coordinates are used to indicate the separation position of different color areas in the target image;
[0056] The performing an image detection operation on the target image based on the target key point information includes:
[0057] An image detection operation is performed on the target image based on a relationship between a first angle and at least one second angle, wherein the first angle is an angle formed by the limit point coordinates, the center point coordinates and the target point coordinates, and the second angle is an angle formed by the limit point coordinates, the center point coordinates and the separation point coordinates.
[0058] In this embodiment, the key points included in the target image include limit point coordinates, center point coordinates, target point coordinates and at least one separation point coordinate, wherein the separation point coordinates may be the separation point coordinates of two adjacent color areas in the target image, for example, Figure 3 The points 1 and 2 are the coordinates of the separation points of the two different color areas. For example, the right side of point 1 is the green area and the left side is the red area. For example, the left side of point 2 is the green area and the right side is the red area. Figure 3The x1 and x2 in represent the scale information of the instrument panel. The above target point coordinates can represent the position of the detected object in the target image area, such as Figure 3 Point 4 shown.
[0059] Taking the target image as a dashboard image as an example, the target key point information may include: Figure 3 Points 0 to 5 are shown, where point 0 is the limit point (the location of a limit screw on the dial), point 1 is the color separation endpoint of the red alarm area and the green safety area close to the limit point, point 2 is the color separation endpoint of the red alarm area and the green safety area far from the limit point, point 3 is the tail end endpoint of the pointer, point 4 is the head end endpoint of the pointer, and point 5 is the center point of the pointer rotation, that is, the center point of the instrument panel.
[0060] The image detection operation performed on the target image based on the relationship between the first angle and at least one second angle may be to determine the position of the target key point in the target image based on the relationship between the first angle and at least one second angle. For example, when the target image is a dashboard image, the relative angle between the pointer endpoint and the different color endpoints can be detected through the relationship between the first angle and at least one second angle to determine the position of the pointer to implement a safety warning, for example, if the pointer tip is within the angle of the safety area, it is safe, and if it is in the warning area, an alarm is triggered.
[0061] by Figure 3 For example, the first angle may be the angle formed by points 0, 5 and 4. And it can include two second angles, one second angle can be the angle formed by point 0, point 5 and point 1 Another second angle can be the angle formed by point 0, point 5 and point 2 like Figure 4 As shown, calculation and When satisfied 5° is a reserved angle for the safety state, and there is no limitation on this. For example, other reserved angles can be set, or not set.
[0062] In this implementation, the image content detection can be more accurately achieved through the first angle and the second angle.
[0063] As an optional implementation manner, the above-mentioned extraction of feature map information of the target image includes:
[0064] Extract feature map information of the target image through the backbone network, deconvolution layer and fully connected layer of the pre-configured neural network model;
[0065] The predicting of the first key point information of the feature map information according to the first image detection branch, and the predicting of the second key point information of the feature map information according to the second image detection branch, include:
[0066] Predicting first key point information of the feature map information based on the first image detection branch of the neural network model, and predicting second key point information of the feature map information based on the second image detection branch of the neural network model.
[0067] In this implementation, the target image may be an original image. In this implementation, the input may be the original image and the output may be target key point information, thereby improving the efficiency of image content detection.
[0068] like Figure 2 As shown, the neural network model includes a backbone network, a deconvolution layer, a fully connected layer, a first image detection branch and a second image detection branch. For example, the target image is passed through the backbone network to obtain feature maps of different network depths and scales. For example, the feature map is successively reduced to half of the original one by performing multiple downsampling through the backbone network. Among them, the backbone network can be a convolutional neural network. Then, feature maps of different scales are fused to obtain feature maps of multiple scales, such as fusing network features of different scales through a feature pyramid, which can achieve better color feature extraction and spatial position feature extraction of key points, and improve the detection performance of different degrees of hue in the same color range (such as object color fading, etc.) and detection performance at different distances.
[0069] After that, deconvolution (deconv) is performed on the feature maps of different scales to uniformly transform them into feature maps of uniform scale, such as transforming them into feature maps of the largest scale, and then splicing the feature maps through a fully connected layer (concat layer) to extract the feature map information of the target image. Of course, the feature maps can also be returned to their original size through multiple residual skip connections, convolutions, and deconvolutions, that is, converted into feature maps of uniform scale, and then multi-level feature splicing and fusion are achieved through a fully connected layer to obtain high-dimensional feature map information.
[0070] Optionally, the training sample of the neural network model includes at least one of the following:
[0071] Multiple training image samples obtained by rotating the same training image at multiple angles;
[0072] The same training image is rotated at multiple angles and the obtained multiple training image samples are scaled;
[0073] A plurality of image samples obtained by rotating the same training image at multiple angles, wherein the plurality of image samples are respectively used as a plurality of training image samples obtained by fusing the foreground image with the background image;
[0074] The same training image is rotated at multiple angles and scaled to obtain multiple image samples, and the multiple image samples are used as multiple training image samples obtained by fusing the foreground image with the background image;
[0075] The training image and the target image are of the same type.
[0076] The multiple training image samples obtained by rotating the same training image at multiple angles can be obtained by rotating the same training image, which can enrich the training samples of the neural network model to improve the accuracy of the neural network model. Figure 5 As shown, multiple different training image samples can be obtained.
[0077] In addition, since the corresponding key point coordinates are also rotated when the training image samples are rotated, the key point coordinates can be calculated based on the set rotation angle and the center point of the image during the training process, that is, the key point information annotation of the training image samples is obtained, so that each key point on each training image sample obtained corresponds to the annotated key point information. In addition, the rotated image can be rotated by image affine transformation.
[0078] Furthermore, in this embodiment, if the training image is a dashboard image, when the image is rotated, the key point coordinates and the dashboard image are rotated based on the center point of the dial (which is also the rotation center of the pointer), ensuring that the key point coordinates and the image after rotation can be aligned. For example: for a training image with a rotation angle of 1° to 360°, 1 to 360 images can be generated, and a small amount of training data can generate a large amount of training data at different angles. Figure 5 Shown are the results of rotating an image and key points by 90°, 180°, 270°, and 330° respectively.
[0079] The multiple image samples obtained by rotating the same training image at multiple angles are used as multiple training image samples by fusing the foreground image with the background image. In this way, the foreground image and different image backgrounds can be fused for data enhancement. While retaining the foreground image, different background images can be replaced. A small amount of foreground annotation data can be used to synthesize multiple images with different backgrounds. A large amount of data expansion can be performed to enhance the algorithm's anti-interference ability against different backgrounds, which can improve the stability of key point prediction of the neural network model.
[0080] Furthermore, in the above implementation, the rotated image can be scaled, that is, the same training image is rotated at multiple angles and scaled to obtain multiple training image samples, and the same training image is rotated at multiple angles and scaled to obtain multiple image samples, and the multiple image samples are respectively used as the foreground image and the background image to obtain multiple training image samples. For example: the above rotated image is scaled and then used as the foreground image and the background image to obtain multiple training image samples. Figure 6 As shown in FIG. 1 , different foreground images are first generated by rotation and scaling, and then fused with different background images to generate new training image samples.
[0081] Furthermore, the key point information of the fused multiple training image samples can be calculated based on the key point information of the background image and the foreground image. For example, before training, the current coordinates and key point coordinates of the training image sample can be calculated in combination with the coordinates of the background fusion position, that is, the key point annotation information of the foreground image can be aligned to the background image as the annotation information corresponding to the new training image sample.
[0082] For example, the fusion process can be Figure 7 As shown in the figure, the foreground image after rotation is scaled and the image size is obtained, so that the size of the network input can be controlled during training. At the same time, the background image also obtains the image size, and the size of the rotated image is compared with that of the background image. If the size of the rotated image is larger than that of the background image, the scaling process continues. When the image size is smaller than that of the background image, the image and background are fused. During the fusion process, the position of the rotated image is updated, that is, the coordinates of the key point information of the rotated image in the background image are calculated, that is, the key point information of the fused training image sample is calculated. Finally, the fused image and key point coordinates are added to the training set as training information.
[0083] Furthermore, during the fusion process, the starting point of the fusion can be selected according to the relationship between the coordinate point and the rotated image, for example: Figure 8 As shown, a point is randomly selected on the background image as the starting point of the rotated image, and the width and height of the background image and the rotated image are obtained. Then, it is calculated whether the sum of the starting point and the height and width of the rotated image exceeds the height and width of the background image. If it exceeds, the starting point of the background image is reselected, because if it exceeds, after fusion, some content of the rotated image cannot appear in the background image.
[0084] When it is not exceeded, a fusion operation is performed.
[0085] In an embodiment of the present invention, feature map information of a target image is extracted; first key point information of the feature map information is predicted based on a first image detection branch, and second key point information of the feature map information is predicted based on a second image detection branch; target key point information of the target image is determined based on the first key point information and the second key point information; and an image detection operation is performed on the target image based on the target key point information. Since the target key point information is predicted by two image detection branches, and an image detection operation is performed on the target image based on the target key point information, compared with the prior art that relies on the shape detection result of the image to perform image detection, the embodiment of the present invention can improve the accuracy of image content detection.
[0086] See also Fig. 9 , Fig. 9 is a structural diagram of an image detection device provided by an embodiment of the present invention, such as Fig. 9 As shown, the image detection device 900 includes:
[0087] Extraction module 901, used to extract feature map information of the target image;
[0088] A prediction module 902, configured to predict first key point information of the feature map information according to the first image detection branch, and predict second key point information of the feature map information according to the second image detection branch;
[0089] A determination module 903, configured to determine target key point information of the target image according to the first key point information and the second key point information;
[0090] The detection module 904 is used to perform an image detection operation on the target image based on the target key point information.
[0091] Optionally, the extraction module 904 includes:
[0092] An extraction unit, used to extract feature maps of multiple different scales of the target image;
[0093] A conversion unit, configured to convert the plurality of feature maps of different scales into a feature map of a unified scale;
[0094] The splicing module is used to splice the multiple converted feature maps to obtain the feature map information.
[0095] Optionally, the prediction module 902 is used to predict the first key point information of the feature map information based on the heat map detection technology, and predict the second key point information of the feature map information based on the key point detection technology.
[0096] Optionally, the extraction module 901 is used to extract feature map information of the target image through a backbone network, a deconvolution layer, and a fully connected layer of a pre-configured neural network model;
[0097] The prediction module 904 is used to predict the first key point information of the feature map information based on the first image detection branch of the neural network model, and to predict the second key point information of the feature map information based on the second image detection branch of the neural network model.
[0098] Optionally, the training sample of the neural network model includes at least one of the following:
[0099] Multiple training image samples obtained by rotating the same training image at multiple angles;
[0100] The same training image is rotated at multiple angles and the obtained multiple training image samples are scaled;
[0101] A plurality of image samples obtained by rotating the same training image at multiple angles, wherein the plurality of image samples are respectively used as a plurality of training image samples obtained by fusing the foreground image with the background image;
[0102] The same training image is rotated at multiple angles and scaled to obtain multiple image samples, and the multiple image samples are used as multiple training image samples obtained by fusing the foreground image with the background image;
[0103] The training image and the target image are of the same type.
[0104] Optionally, the target key point information includes: limit point coordinates, center point coordinates, target point coordinates and at least one separation point coordinate, and the separation point coordinates are used to indicate the separation position of different color areas in the target image;
[0105] The detection module is used to perform an image detection operation on the target image based on the relationship between a first angle and at least one second angle, wherein the first angle is the angle formed by the limit point coordinates, the center point coordinates and the target point coordinates, and the second angle is the angle formed by the limit point coordinates, the center point coordinates and the separation point coordinates.
[0106] The image detection device provided by the embodiment of the present invention can achieve Figure 1 To avoid repetition, the various processes in the method embodiment are not described here.
[0107] It should be noted that the image content analysis device in the embodiment of the present invention may be a device, or a component, an integrated circuit, or a chip in an electronic device.
[0108] See also Fig.10 , Fig.10 is a structural diagram of an electronic device provided by an embodiment of the present invention, such as Figure 6 As shown, the electronic device 1000 includes: a memory 1001, a processor 1002, and a program or instruction stored in the memory 1001 and executable on the processor 1002, and the program or instruction implements the steps in the above-mentioned image detection method when executed by the processor 1002.
[0109] An embodiment of the present invention also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned image detection method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0110] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0111] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0112] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. An image detection method, characterized in that: include: Extract feature map information of the target image; Predicting first key point information of the feature map information according to the first image detection branch, and predicting second key point information of the feature map information according to the second image detection branch; Determining target key point information of the target image according to the first key point information and the second key point information; Performing an image detection operation on the target image based on the target key point information; The target key point information includes: limit point coordinates, center point coordinates, target point coordinates and at least one separation point coordinate, and the separation point coordinates are used to indicate the separation position of different color areas in the target image; The performing an image detection operation on the target image based on the target key point information includes: An image detection operation is performed on the target image based on a relationship between a first angle and at least one second angle, wherein the first angle is an angle formed by the limit point coordinates, the center point coordinates and the target point coordinates, and the second angle is an angle formed by the limit point coordinates, the center point coordinates and the separation point coordinates.
2. The method according to claim 1, characterized in that The step of extracting feature map information of a target image includes: Extracting multiple feature maps of different scales of the target image; Converting the multiple feature maps of different scales into a feature map of a unified scale; The converted multiple feature maps are concatenated to obtain the feature map information.
3. The method according to claim 1, characterized in that The predicting of the first key point information of the feature map information according to the first image detection branch, and the predicting of the second key point information of the feature map information according to the second image detection branch, include: Predicting first key point information of the feature map information based on a heat map detection technique, and predicting second key point information of the feature map information based on a key point detection technique.
4. The method according to claim 1, characterized in that The feature map information of the target image is extracted including: Extract feature map information of the target image through the backbone network, deconvolution layer and fully connected layer of the pre-configured neural network model; The predicting of the first key point information of the feature map information according to the first image detection branch, and the predicting of the second key point information of the feature map information according to the second image detection branch, include: Predicting first key point information of the feature map information based on the first image detection branch of the neural network model, and predicting second key point information of the feature map information based on the second image detection branch of the neural network model.
5. The method according to claim 4, characterized in that The training samples of the neural network model include at least one of the following: Multiple training image samples obtained by rotating the same training image at multiple angles; The same training image is rotated at multiple angles and the obtained multiple training image samples are scaled; A plurality of image samples obtained by rotating the same training image at multiple angles, wherein the plurality of image samples are respectively used as a plurality of training image samples obtained by fusing the foreground image with the background image; A plurality of image samples are obtained by rotating the same training image at multiple angles and scaling the same training image, and the plurality of image samples are respectively used as a plurality of training image samples obtained by fusing the foreground image with the background image; The training image and the target image are of the same type.
6. An image detection device, characterized in that: include: An extraction module, used to extract feature map information of a target image; A prediction module, used to predict first key point information of the feature map information according to the first image detection branch, and predict second key point information of the feature map information according to the second image detection branch; A determination module, configured to determine target key point information of the target image based on the first key point information and the second key point information; A detection module, used for performing an image detection operation on the target image based on the target key point information; The target key point information includes: limit point coordinates, center point coordinates, target point coordinates and at least one separation point coordinate, and the separation point coordinates are used to indicate the separation position of different color areas in the target image; The detection module is used to perform an image detection operation on the target image based on the relationship between a first angle and at least one second angle, wherein the first angle is the angle formed by the limit point coordinates, the center point coordinates and the target point coordinates, and the second angle is the angle formed by the limit point coordinates, the center point coordinates and the separation point coordinates.
7. The device according to claim 6, characterized in that The extraction module comprises: An extraction unit, used to extract feature maps of multiple different scales of the target image; A conversion unit, configured to convert the plurality of feature maps of different scales into a feature map of a unified scale; The splicing module is used to splice the multiple converted feature maps to obtain the feature map information.
8. The device according to claim 6, characterized in that The prediction module is used to predict the first key point information of the feature map information based on the heat map detection technology, and to predict the second key point information of the feature map information based on the key point detection technology.
9. The device according to claim 6, characterized in that The extraction module is used to extract feature map information of the target image through the backbone network, deconvolution layer and full connection layer of the pre-configured neural network model; The prediction module is used to predict the first key point information of the feature map information based on the first image detection branch of the neural network model, and to predict the second key point information of the feature map information based on the second image detection branch of the neural network model.
10. The device according to claim 9, characterized in that The training samples of the neural network model include at least one of the following: Multiple training image samples obtained by rotating the same training image at multiple angles; A plurality of training image samples obtained by rotating the same training image at multiple angles and scaling the same training image; a plurality of image samples obtained by rotating the same training image at multiple angles, the plurality of image samples being used as a foreground image and a background image for fusion to obtain a plurality of training image samples; A plurality of image samples are obtained by rotating the same training image at multiple angles and scaling the same training image, and the plurality of image samples are respectively used as a plurality of training image samples obtained by fusing the foreground image with the background image; The training image and the target image are of the same type.
11. An electronic device, characterized in that: include: A memory, a processor, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps in the image detection method as claimed in any one of claims 1 to 5.
12. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps in the image detection method according to any one of claims 1 to 5 are implemented.
Citation Information
Patent Citations
Human skeleton key point detection method and device, electronic device and storage medium
CN109670397A
Image key point extraction method and device, readable storage medium and electronic equipment
CN109711273A
Human body key point extraction method and device, readable storage medium and equipment
CN110532981A
Motion recognition method and device
CN111401318A
Method and apparatus for detecting living body, electronic device, and storage medium
US20190354746A1
