Image detection method, face detection method and device, equipment and storage medium

Abstract

The embodiments of the present application provide an image detection method and apparatus, a face detection method and apparatus, a terminal device and a computer storage medium. The image detection method comprises acquiring an image to be detected; detecting the image to be detected through a detection window, and determining the key areas in the image area covered by the sliding of the detection window in the image to be detected according to the positional relationship of the target reference point in the sample image, and extracting the image features of the key areas to determine the feature vectors of the corresponding image areas; wherein the sample image is an image for training a model for extracting the shape features of a target object in an image, and the target reference point is a key feature point in the sample image for defining the shape features of the target object; and detecting the target object in the image to be detected according to the feature vectors of each of the image areas.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image detection method and device, a face detection method and device, a terminal device, and a computer-readable storage medium. Background Art

[0002] With the widespread application of multimedia technology and computer equipment networks, a large amount of image data has appeared on the network. How to effectively manage these image files and automatically identify the content of the image has become increasingly necessary.

[0003] Among them, face detection in image recognition has attracted particular attention in the field of computer vision. The known face detection method is to use a detection window to slide in the image in a column-by-column and then row-by-row scanning manner to extract features from the image to be detected, so as to detect the face in the image. However, this requires feature extraction from the entire area of ​​the image to be detected, which is computationally intensive and the extracted features are easily interfered by non-face parts, resulting in a long detection time and low accuracy. Summary of the invention

[0004] In order to solve the existing technical problems, the embodiments of the present application provide a fast and highly accurate image detection method and device, a face detection method and device, a terminal device and a computer-readable storage medium.

[0005] To achieve the above purpose, the technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect of an embodiment of the present application, there is provided an image detection method, comprising:

[0007] Acquire the image to be detected;

[0008] The image to be detected is detected by using a detection window, and according to the positional relationship of the target reference point in the sample image, key areas in the image area covered by the sliding of the detection window in the image to be detected are determined respectively, and image features of the key areas are extracted to determine the feature vectors of the corresponding image areas; the sample image is an image used to train a model for extracting shape features of target objects in an image, and the target reference point is a key feature point in the sample image used to define the shape features of the target object;

[0009] The target object in the image to be detected is detected according to the feature vectors of each of the image regions.

[0010] According to a second aspect of the present application, a face detection method is provided, comprising:

[0011] Acquire the image to be detected;

[0012] The image to be detected is detected by using detection windows of different sizes respectively, and according to the positional relationship of facial features in the facial image, key areas in the image area covered by the sliding of the detection window in the image to be detected are determined respectively, and image features of the key areas are extracted to determine the feature vectors of the corresponding image areas;

[0013] When it is determined according to the feature vectors of each of the image areas that image areas covered by detection windows of different sizes overlap and both contain faces, the overlapping image areas covered by the detection windows of different sizes are fused to determine the face and its position in the image to be detected.

[0014] According to a third aspect of the present application, an image detection device is provided, including:

[0015] An image acquisition module, used for acquiring an image to be detected;

[0016] A feature extraction module is used to detect the image to be detected through a detection window, determine key areas within the image area covered by the sliding of the detection window in the image to be detected according to the positional relationship of the target reference point in the sample image, and extract image features of the key areas to determine the feature vectors of the corresponding image areas;

[0017] The detection module is used to detect the target object in the image to be detected according to the feature vectors of each image area.

[0018] According to a fourth aspect of the present application, a face detection device is provided, comprising:

[0019] An acquisition module, used for acquiring an image to be detected;

[0020] A feature module is used to detect the image to be detected through detection windows of different sizes, determine key areas in the image area covered by the sliding of the detection window in the image to be detected according to the positional relationship of the facial features in the face image, and extract image features of the key areas to determine the feature vectors of the corresponding image areas;

[0021] The face detection module is used to determine, based on the feature vectors of each of the image areas, that the image areas covered by detection windows of different sizes overlap and both contain faces, and then fuse the overlapping image areas covered by the detection windows of different sizes to determine the face and its position in the image to be detected.

[0022] In a fifth aspect of an embodiment of the present application, a terminal device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor runs the computer program, the image detection method described in any embodiment of the present application or the face detection method described in any embodiment of the present application is executed.

[0023] In a sixth aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program code is stored. When the computer program code is loaded and executed by a processor, the image detection method described in any embodiment of the present application or the face detection method described in any embodiment of the present application is executed.

[0024] The image detection method and apparatus, terminal device and computer-readable storage medium provided in the above embodiments, in the process of detecting the image to be detected through the detection window, select key feature points as target reference points according to the shape features of the target object in the sample image, analyze the positional relationship of the target reference points in the sample image, and determine the key areas of the image area covered by the sliding of the detection window in the image to be detected according to the positional relationship, extract the image features of the key areas to determine the corresponding feature vectors of the image area, and perform feature extraction by determining the key areas of the image area covered by the detection window, thereby improving the feature extraction speed and reducing the amount of calculation to improve the image detection efficiency; secondly, it can reduce the interference of non-valid feature areas, and improve the accuracy of image detection;

[0025] The face detection method and apparatus, terminal device and computer-readable storage medium provided in the above embodiments respectively detect the image to be detected through detection windows of different sizes, respectively determine the key areas in the image area covered by the detection window sliding in the image to be detected according to the positional relationship of the facial features in the face image, extract the image features of the key areas to determine the feature vectors of the corresponding image areas, and when it is determined according to the feature vectors of each image area that the image areas covered by detection windows of different sizes overlap and both contain faces, the overlapping image areas covered by the detection windows of different sizes are fused to determine the face in the image to be detected and its position, and feature extraction is performed by determining the key areas of the image area covered by the detection window, which can improve the feature extraction speed and reduce the amount of calculation, so as to improve the efficiency of face detection in the image; by using detection windows of different sizes for detection and fusing the detection results of the overlapping areas, it is possible to quickly detect faces of different sizes mixed in different images or faces of different sizes mixed in the same image, thereby improving the detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1This is an application scenario diagram of the image detection method in one embodiment of the present application;

[0027] Figure 2 This is a flow chart of an image detection method in one embodiment of the present application;

[0028] Figure 3 This is a flow chart of an image detection method in another embodiment of the present application;

[0029] Figure 4 A schematic diagram of the positional relationship of a target reference point relative to a target object in an embodiment of the present application;

[0030] Figure 5 This is a schematic diagram of determining a key area according to a target reference point in an embodiment of the present application;

[0031] Figure 6 A schematic diagram of calculating the characteristic value of an image feature in an embodiment of the present application;

[0032] Figure 7 This is a schematic diagram of feature value processing in an embodiment of the present application;

[0033] Figure 8 It is a feature graph after feature value processing in the embodiment of the present application;

[0034] Fig. 9 This is a flowchart of an image detection method in another embodiment of the present application;

[0035] Fig.10 Schematic diagram of the training principle of cascade classifier;

[0036] Fig.11 This is a flow chart of a face detection method in one embodiment of the present application;

[0037] Fig.12 A flow chart of a face detection method in another embodiment of the present application;

[0038] Fig.13 This is a flowchart of a face detection method in an optional specific implementation of the present application;

[0039] Fig.14 This is a schematic diagram of the structure of a terminal device in an embodiment of the present application;

[0040] Fig.15 This is a schematic diagram of a module of a pattern detection device in an embodiment of the present application;

[0041] Fig.16 This is a schematic diagram of the structure of a terminal device in another embodiment of the present application;

[0042] Fig.17 Schematic diagram of the modules of a face detection device in one embodiment of the present application. DETAILED DESCRIPTION

[0043] The technical solution of the present invention is further elaborated in detail below in conjunction with the accompanying drawings and specific embodiments of the specification. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the technical field of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0044] In the following description, reference is made to “some embodiments”, which describes a subset of all possible embodiments, but it should be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0045] See also Figure 1 , is an optional application scenario diagram of the image detection method provided in an embodiment of the present invention. The user can install an image detection application client in a terminal device, input the image to be detected in the image detection application, and extract features of the predicted image and detect the target object through the server. Among them, the image detection application can be an application program that detects objects contained in different images by training to determine a cascade classifier. Taking the object contained in the image as a face as an example, the image detection application refers to an application program that detects the face area contained in the image to be detected to determine whether the image to be detected contains a face and the position of the face in the image to be detected. As another optional implementation, the user can also directly extract features of the predicted image and detect the target object through the image detection application of the terminal device. Among them, the terminal device can be a mobile phone, a computer, an ipad, an attendance machine, a camera, etc., which can install the image detection application, to execute the image detection method or face detection method described in any embodiment of the present application.

[0046] See also Figure 2 , an image detection method provided in an embodiment of the present application, comprising the following steps:

[0047] Step S101, obtaining an image to be detected.

[0048] The image to be detected can be any image used to detect whether it contains a target object. The terminal device may obtain the image to be detected by at least one of the following: receiving an image uploaded by a user through an image acquisition interface of an image detection application; calling a camera module to capture an image through an image detection application; and obtaining an image transmitted by other terminal devices through an image detection application. The image to be detected can be obtained in real time or stored in advance on the terminal device.

[0049] Step S103, detect the image to be detected through the detection window, determine the key areas in the image area covered by the sliding of the detection window in the image to be detected according to the positional relationship of the target reference point in the sample image, extract the image features of the key areas to determine the feature vectors of the corresponding image areas. The sample image is an image used to train a model for extracting the shape features of the target object in the image, and the target reference point is a key feature point in the sample image used to define the shape features of the target object.

[0050] The target reference point refers to a key feature point determined according to the target object contained in different images and included in the range of the target object so as to be used to define the shape feature of the target object. The sample image is an image used to train a model for extracting the shape feature of the target object in the image. Specifically, the sample image refers to a standard image containing the target object for training. The shape feature of the target object in the image can be extracted by analyzing the sample image. The classifier model capable of extracting the shape feature of the target object in the image can be obtained by training the classifier using the sample image. By establishing a classifier model for analyzing the shape feature of the target object in the sample image, a rule for detecting whether the target object exists in other images using the classifier model is obtained. In this embodiment, the rule refers to the position relationship of the target reference point in the sample image. Taking the target object as a human face as an example, the target reference point refers to the position points corresponding to the five sense organs, such as the center of the eye, the tip of the nose, and the corners of the mouth at both ends of the mouth. The position points of the five sense organs are used to determine whether it is a human face. The position relationship of the target reference point in the sample image refers to the detection rule determined according to the position law of the target reference point in the target object, which can be used to determine whether it is a target object.

[0051] By sliding the detection window in the image to be detected, face detection is performed on the image areas respectively covered by the detection window during the sliding process, wherein the steps of performing face detection on the image areas sequentially covered during the sliding process of the detection window are the same, including: determining the key area in the image area covered by the detection window, extracting the image features of the key area to determine the feature vector of the image area. Taking the detection window initially located at position one in the image to be detected and sliding from position one to position two as an example, according to the positional relationship of the target reference point in the sample image, the key area one of the image area covered by the detection window at position one and the key area two of the image area covered by the detection window at position two are respectively determined, the image features of the key area one are extracted to determine the feature vector of the image area corresponding to position one, and the image features of the key area two are extracted to determine the feature vector of the image area corresponding to position two. In the image area covered by the detection window, the key area is determined according to the target reference point, and the image features of the key area are extracted to determine the feature vector of the image area covered by the detection window sliding in the image to be detected. In this way, the scope of feature extraction in the image area covered by the detection window can be narrowed, the feature extraction speed can be improved, and the amount of calculation can be reduced to improve the image detection efficiency; secondly, by determining the key area for feature extraction, the interference of non-valid feature areas can be reduced, and the accuracy of image detection can be improved.

[0052] Step S105 , detecting the target object in the image to be detected according to the feature vectors of each of the image regions.

[0053] By respectively extracting image features of key areas in the image area covered by the detection window, it is determined whether the image area contains a target object.

[0054] In the above-mentioned embodiments of the present application, in the process of detecting the image to be detected through the detection window, the key areas within the image area covered by the detection window are respectively determined according to the positional relationship of the target reference point in the sample image, and the image features of the key areas are extracted to determine the corresponding feature vectors of the image area, wherein the determination of the key areas is conducive to reducing the amount of image feature extraction of the image area covered by each frame selection during the sliding process of the detection window, thereby improving the detection efficiency. By sequentially framing the detection area of ​​the image to be detected by the detection window and extracting features from the key areas of the image area covered by it, the feature extraction speed can be improved, the amount of calculation can be reduced, and the image detection efficiency can be improved; secondly, the interference of non-valid feature areas can be reduced, and the accuracy of image detection can be improved.

[0055] In some embodiments, see Figure 3The step S105, before detecting the target object in the image to be detected according to the feature vectors of each of the image regions, comprises:

[0056] Step S1041, adjusting the size of the detection window according to a set rule to obtain an updated detection window;

[0057] Step S1042, detect the image to be detected through the updated detection window, determine the key areas in the image area covered by the sliding of the updated detection window in the image to be detected according to the positional relationship of the target reference point in the sample image, and extract the image features of the key areas to determine the corresponding feature vectors of the image area.

[0058] The size of the detection window is adjusted according to the set rules, which can be increased or decreased in proportion to the size of the current detection window, or gradually increased or decreased in accordance with the size of the current detection window. The set rules refer to the rules of increasing or decreasing in proportion, or gradually increasing or decreasing the specified size. By adjusting the size of the detection window according to the set rules, the image to be detected is scanned and detected by detection windows of different sizes respectively, and the image features of the image area covered by the sliding of the detection windows of different sizes in the image to be detected are respectively extracted to determine the feature vector of the corresponding image area. Usually, the target object has different sizes in the image to be detected, and the image area range covered by the detection windows of different sizes is different. The image to be detected is detected in turn by the detection windows of different sizes to ensure that the target object can fall within the image area covered by the detection window, thereby improving the accuracy of detection.

[0059] Among them, the maximum size and minimum size of the detection window are pre-set according to the size range of the target object in the image to be detected. In the process of adjusting the size of the detection window according to the set rules, the image to be detected can be first detected through the detection window of the minimum size to confirm whether the target object is included in the image area covered each time during the sliding process of the detection window; then, according to the preset rules, the size of the detection window is increased by a specified size, and the image to be detected is detected respectively through the detection window with the increased size to confirm whether the target object is included in the image area covered each time during the sliding process of the detection window; and so on, until the size of the detection window is equal to or greater than the pre-set maximum size. By detecting the image to be detected respectively through detection windows of different sizes, it can be ensured that target objects of different sizes can be accurately detected.

[0060] In some embodiments, the step S105, detecting the target object in the image to be detected according to the feature vectors of each of the image regions, includes:

[0061] Determining whether the image to be detected contains a target object according to the feature vectors of each of the image regions;

[0062] In the case that the image area covered by the updated detection window and the image area covered by the detection window both contain the target object, the image area covered by the updated detection window and the image area covered by the detection window are fused.

[0063] The image to be detected is detected respectively by detection windows of different sizes. When the image area covered by the updated detection window and the image area covered by the detection window both contain the target object, that is, when the same area in the image to be detected is detected to contain the target object under detection windows of different sizes, or when the periphery of the same area in the detection image is detected to contain the target object multiple times, the image area covered by the updated detection window and the image area covered by the detection window are fused to accurately determine the position of the target object in the image to be detected.

[0064] Optionally, fusion refers to coordinate merging of the image area covered by the updated detection window and the image area covered by the detection window using a non-maximum suppression algorithm. For example, a first value based on which a face is detected and determined to be a face in the image area covered by the updated detection window is compared with a second value based on which a face is detected and determined to be a face in the image area covered by the detection window, and the image area corresponding to the relatively smaller value between the first value and the second value is determined as a redundant image area and discarded.

[0065] Optionally, fusing the image area covered by the updated detection window with the image area covered by the detection window includes:

[0066] In the case where one of the image area covered by the updated detection window and the image area covered by the detection window is contained in the other, or,

[0067] In a case where the overlap between the image area covered by the updated detection window and the image area covered by the detection window is greater than a threshold;

[0068] The reserved image region is selected according to the feature vector of the image region, and the coordinates of the target object are determined.

[0069] Wherein, the image area covered by the updated detection window and the image area covered by the detection window may be in a containment relationship or a partially overlapping relationship, and the image area fusion may be performed by using a non-maximum suppression algorithm to merge coordinates. In the case where one of the image area covered by the updated detection window and the image area covered by the detection window is contained in the other, the numerical value of the face is determined according to the feature vector of the image area, and the redundant image area with a relatively small numerical value is discarded. The redundant image area may be an image area with a larger size located outside or an image area with a smaller size located inside. The coordinates of the target object are determined according to the retained image area to achieve the purpose of target object coordinate size fusion; or, in the case where the overlap between the image area covered by the updated detection window and the image area covered by the detection window is greater than a threshold, the numerical value of the face is determined according to the feature vector of the image area, and the redundant image area with a relatively small numerical value is discarded. The redundant image area may be an image area on the left or an image area on the right. The coordinates of the target object are determined according to the retained image area to achieve the purpose of target object coordinate offset fusion.

[0070] Optionally, adjusting the size of the detection window according to a set rule to obtain an updated detection window includes:

[0071] The detection window is scaled according to a set ratio, and when the size of the scaled detection window is within a set size range, the scaled detection window is used as an updated detection window.

[0072] The rule for adjusting the size of the detection window can be to scale the detection window according to a set ratio. As an optional implementation, the set ratio can be 1.2 times, the minimum size of the detection window is 20*20, and the maximum size of the detection window is 600*600. In the process of detecting the image to be detected through the detection window, the image to be detected can be first detected with a detection window of size 20*20 to respectively determine the feature vectors corresponding to the image area covered by the detection window in the sliding of the image to be detected, and respectively detect whether the image area covered by the detection window in the sliding contains the target object; then the detection window is increased by 1.2 times to obtain an updated detection window, and the image to be detected is detected through the increased detection window until the size of the detection window is greater than 600*600.

[0073] By determining the appropriate set size range according to the possible sizes of the target object to be detected in different images or in the same image, the detection window is scaled within the set size range, and detection is performed separately using detection windows of different sizes to ensure that all target objects whose sizes fall within the set size range can be effectively detected, thereby improving the detection accuracy.

[0074] In some embodiments, determining the key areas in the image area covered by the detection window according to the positional relationship of the target reference point in the sample image includes:

[0075] Determining the proportional coordinates of the target reference point relative to the target object area according to the coordinates of the target reference point in the sample image and the size of the target object area in the sample image;

[0076] The position of the target reference point in the image area covered by the detection window is determined according to the proportional coordinates, and the key area is determined according to the position of the target reference point.

[0077] The positional relationship of the target reference point in the sample image refers to the detection rule that can determine whether it is the target object based on the position regularity of the target reference point in the target object. In this embodiment, based on the proportional coordinates of the target reference point relative to the target object area, the image area framed by the detection window each time during the sliding process is used as the image area to be detected, and the position of the target reference point in the image area is determined respectively, and the key area is further determined based on the position of the target reference point. By extracting the features of the key area to detect whether it contains the target object, the key area is used as the extraction object of the effective features, which can improve the detection efficiency and reduce the missed detection rate.

[0078] Optionally, there are multiple target reference points, and determining the key area according to the positions of the target reference points includes:

[0079] The minimum circumscribed square area of ​​a circle with the coordinates of the target reference point as the center and a preset value as the radius is the key area corresponding to the target reference point.

[0080] In this embodiment, the key area is determined by the minimum circumscribed square area of ​​a circle with the target reference point as the center and a preset value as the radius. When the detection window is used to detect whether the target object exists in the image to be detected, the detection window mainly extracts features from the key area so as to more quickly and accurately extract effective image features that can be used to determine the target object. Taking the target object as a human face, the target reference points can refer to five points: the center of the eye, the tip of the nose, and the corner of the mouth. The distance between the left and right eye centers and the distance between the corners of the mouth are used as a reference to determine the facial imaging area. The facial imaging area can be determined as follows: Figure 4As shown. The distance between the left and right eye centers is d1, the distance between the two points of the mouth corners is d2, the distance between the nose tip and the two eye centers and the two mouth corners is d3, and the width of the face imaging area is D. According to the positional relationship of the target reference point relative to the target object, at this time, the coordinates of the five points of the left and right eye centers, nose tip, and left and right mouth corners in the face are approximately (0.2459, 0.2531), (0.7341, 0.2531), (0.5, 0.5864), (0.2963, 0.8333), (0.7037, 0.8333) in proportion to the size of the face area.

[0081] According to the proportional coordinates of the target reference point relative to the target object, if there is a face in the image area covered by the detection window, the concentrated area of ​​the effective facial feature information of the face, that is, the key area corresponding to the target reference point, is determined, such as Figure 5 As shown, according to the positional relationship of the facial features in the human face, the positions of the five target reference points in the human face, namely the center of the two eyes, the tip of the nose, and the corners of the mouth at both ends, are determined respectively. The key area is a square image area with a side length of D, and the preset value of the radius is r=0.2D. Among them, the coordinates of the center of the left eye are (x1=0.2459D, y1=0.2531D), the coordinates of the center of the right eye are (x2=0.7451D, y3=0.2531D), the coordinates of the tip of the nose are (x3=0.5D, y3=0.5864D), the coordinates of the corner of the mouth at the left end are (x4=0.2963D, y4=0.8333D), and the coordinates of the corner of the mouth at the right end are (x5=0.7037D, y5=0.8333D). The coordinates of the target reference point in the image area covered by the detection window are determined according to the proportional coordinates of the target reference point relative to the target object. The key area corresponding to the target reference point is determined by the minimum circumscribed square area of ​​a circle with the coordinates (x1, y1) of the target reference point as the center and r as the radius. For example, a circle is determined with the center of the left eye (x1=0.2459D, y1=0.2531D) as the center and r=0.2D as the radius, and the minimum circumscribed square area of ​​the circle is determined as the key area of ​​the target reference point where the center of the left eye is located.

[0082] By taking the target key point coordinates as the center, determining a circle within a preset radius and using the circumscribed square area of ​​the circle as the key area, the area in the image that contains the most concentrated valid image features for determining whether it is a face can be defined as the key area, thereby avoiding missing valid features and improving the feature extraction speed and reducing the amount of calculation.

[0083] In some embodiments, extracting the image features of the key area to determine the feature vector of the corresponding image area includes:

[0084] Dividing the key area into a plurality of image blocks;

[0085] Determine a feature code corresponding to the image block according to the magnitude of the pixel value of each pixel point of each image block and the pixel value of its adjacent pixel point;

[0086] A feature vector of the image region is obtained according to the feature code corresponding to the image block.

[0087] In this embodiment, the feature vector of the image region adopts LBP (Local Binary Pattern) value, and the extracted feature is the texture feature of the local area of ​​the image. Figure 6 , the key area can be divided into multiple image blocks, and the pixel values ​​of the central pixel of the image block are compared with those of the adjacent pixels. If the pixel value of the adjacent pixel is greater than the pixel value of the central pixel, the position of the adjacent pixel is marked with 1, and if the pixel value of the adjacent pixel is less than the pixel value of the central pixel, the position of the adjacent pixel is marked with 0, thereby determining the characteristic value of the central pixel of the image block. The characteristic code is a field composed of characteristic values ​​to characterize the image characteristics of the corresponding image block. In this embodiment, the characteristic value of the central pixel of each image block can be used as the characteristic code corresponding to the image block. Optionally, the characteristic value of the pixel at other positions of the image block can be used as the characteristic code corresponding to the image block, or the average of the characteristic values ​​of multiple pixels located in the middle can be used as the characteristic code corresponding to the image block. Please refer to Figure 7 and Figure 8 , after dividing all key areas into image blocks and performing LBP value processing on the image blocks, an LBP feature map is obtained, the number of occurrences of different LBP values ​​in each key area is counted, and then the LBP values ​​of the key areas are connected in series to form an LBP feature vector corresponding to the image area currently covered by the detection window. Taking the image block as a 3*3 pixel block as an example, the central pixel of the image block is used as the threshold, and the grayscale values ​​of the 8 adjacent pixels are compared with it. If the pixel value of the surrounding pixels is greater than the central pixel value of the block, the position of the pixel is marked as 1, otherwise it is 0. In this way, the 8 points in the 3*3 neighborhood can generate an 8-bit binary number after comparison, that is, the LBP value of the central pixel of the image block is obtained as the texture information of the image block area.

[0088] By dividing the key area into image blocks to determine the feature code, and obtaining the feature vector of the image area corresponding to the key area from the feature code of the image block, a relatively simpler method of extracting the feature vector of the image area is provided, which can effectively reduce the dimension of the feature vector representing the image area and improve the detection efficiency.

[0089] Optionally, dividing the key area into a plurality of image blocks includes:

[0090] The target overlap is determined according to the feature quantity in the key area, and the key area is divided into a plurality of image blocks according to the target overlap, wherein the size of the target overlap is proportional to the size of the feature quantity.

[0091] The feature quantity refers to the number of valid image features that can determine whether it is a human face. The feature code is a field composed of feature values ​​to characterize the image features of the corresponding image block. The size of the feature quantity refers to the number of valid image features that can determine whether it is a human face. The larger the feature quantity, the more valid image features it means. Conversely, the smaller the feature quantity, the fewer valid image features it means. Dividing the key area into multiple image blocks with a certain degree of overlap between adjacent image blocks can reduce missed detection. The higher the overlap, the more feature quantities are extracted in the same area, which can reduce the probability of missed detection. Taking the target overlap of 50% as an example, the key area can be divided into 4*4 image blocks, and the rows or columns between adjacent image blocks overlap by 50% to ensure the extraction of feature quantities in the key area. The target overlap is proportional to the size of the feature quantity, which can reduce the feature extraction of information-free areas and improve image detection efficiency.

[0092] In some embodiments, see Fig. 9 , before acquiring the image to be detected, it also includes:

[0093] Step S1011, obtaining an original image containing a target object, adjusting the original image according to the calibration of the target reference point and the size of the target object area in the original image, until the display parameters of the original image meet the value conditions, and using the adjusted original image as a positive sample image;

[0094] Step S1012, acquiring an image that does not contain the target object, taking the image that does not contain the target object as a negative sample image, and forming a training image set according to the positive sample image and the negative sample image;

[0095] Step S1013: training the cascade classifier based on the training image set.

[0096] Display parameters refer to the numerical information carried by the image that can be used to adjust the display effect of the image, mainly including the direction, size, brightness, contrast, aspect ratio, resolution, etc. of the image. Value conditions refer to the conditions that need to be met for different parameters that are pre-set, such as image size, aspect ratio, size of the target object area, etc. The original image refers to the image taken or drawn for the target object to be detected, which can be collected based on the currently public image library on the Internet. In this embodiment, image detection is achieved by machine learning. By adjusting the original image, an image whose size range of the target object in the image meets the requirements is formed as a positive training sample, and an image that does not include the target object is obtained to form a negative training sample. The initial classifier is iteratively trained based on the positive training sample and the negative training sample to obtain a trained cascade classifier.

[0097] Taking the Adaboost iterative algorithm as an example, the process of iteratively training the cascade classifier mainly refers to the process of using a data set to train multiple weak classifiers, and then forming a strong classifier from the weak classifier set. Fig.10 , is a schematic diagram of the iterative training process of the cascade classifier, where data is feature data, W is sample weight, WeakClassifier represents weak classifier, and alpha represents the weight of each weak classifier when building a strong classifier. The training process includes: first assigning an initial weight W1 to each sample in data, then training a weak classifier with data (finding a certain dimension of feature from the M-dimensional feature data of data, which can minimize the classification error rate of samples in data, and thus obtaining weakClassifier1), then calculating the classification error rate E1 based on the weight W1 and the classification situation, and then calculating the weight alpha1 based on the error rate E1, and then calculating W2 based on the weight W1 and the weight alpha1 (that is, updating the weight, increasing the weight of samples that were misclassified by weakClassifier1), repeating the above process, multiple weak classifiers can be obtained, and finally these weak classifiers are weighted to obtain the final strong classifier, i.e., the trained cascade classifier.

[0098] The process of detecting the image to be detected by the trained cascade classifier includes sliding the detection window in the image to be detected, extracting the feature vectors of the image areas covered by the detection window during the sliding process, and classifying and judging the feature vectors through weak classifiers. According to the features of the specified dimensions of each weak classifier after training, the corresponding score values ​​val can be obtained after classification and judgment by these weak classifiers respectively. These score values ​​are multiplied and accumulated with the weight alpha of the corresponding weak classifier to obtain the score value of whether the image area is a face. The score value of the image area is compared with the judgment threshold of the cascade classifier, wherein the judgment threshold is obtained after training the cascade classifier. When the score value is higher than the judgment threshold, it means that a face is detected; otherwise, it means that a face is not detected.

[0099] In some embodiments, adjusting the original image according to the calibration of the target reference point and the size of the target object area in the original image until the display parameters of the original image meet the value conditions includes:

[0100] Cropping the original image according to the coordinates of the target reference point in the original image and the size relationship of the target reference point in the target object area;

[0101] The target object in the cropped image is adjusted to conform to a preset size range, where the preset size range corresponds to the detection window size.

[0102] Among them, the preset size range corresponds to the target objects of different sizes that can be detected by the graphic detection method, that is, the minimum size of the target object to be detected is 20*20, and the maximum size is 600*600. Then, the size of the original image is cropped according to the coordinates of the target reference point contained in the target object in the original image and the size relationship of the target reference point in the imaging area of ​​the target object, so as to form a sample image whose size meets the size range of the target object to be detected. By forming a standard sample image as a training sample, it is beneficial to improve the training speed and training accuracy of the cascade classifier, and improve the detection accuracy of the trained cascade classifier.

[0103] In some embodiments, detecting the target object in the image to be detected according to the feature vectors of each of the image regions includes:

[0104] splicing the feature vectors of each of the image regions;

[0105] According to the features of the spliced ​​images, a cascade classifier is used to determine whether the image to be detected contains the target object and the position of the target object.

[0106] The target object contained in the image to be predicted may be one or more. By extracting image features in the image area covered by the detection window, the features of the target object extracted at different positions of the detection window are spliced, and the spliced ​​features are judged by a cascade classifier, thereby obtaining a detection result of whether the image to be detected contains the target object and the position of the target object. Still taking the Adaboost iterative algorithm as an example, the classification layers correspond to the weak classifiers respectively.

[0107] In image detection methods, face is a common visual perception object, which is widely used in video surveillance, access control, video conferencing, mobile payment, human-computer interaction and other fields. The processing and analysis of face images usually includes face detection, face tracking, posture estimation and expression recognition, among which the face detection method is the basis of the above face information processing. Face detection refers to the process of determining the position and size of all faces in the input image. Fig.11 , and also provides a face detection method, comprising the following steps:

[0108] Step S201, obtaining an image to be detected.

[0109] The image to be detected may be any image that contains or does not contain a face, such as a large number of photos taken by cameras installed in various scenes, a large number of images uploaded to the Internet image database, etc. When it is necessary to detect and distinguish whether it is a face image according to image classification requirements, the face detection method provided in the embodiment of the present application can be used.

[0110] Step S203, respectively detect the image to be detected through detection windows of different sizes, determine the key areas in the image area covered by the sliding of the detection window in the image to be detected according to the positional relationship of the facial features in the facial image, extract the image features of the key areas to determine the corresponding feature vectors of the image area.

[0111] The image to be detected is detected by using detection windows of different sizes, and the image features of the image areas covered by the detection windows of different sizes are extracted to determine the feature vectors of the corresponding image areas. Usually, the face sizes in the image to be detected are different. The image to be detected is detected in turn by using detection windows of different sizes to ensure that the target object can fall within the image area covered by the detection window, achieve the purpose of detecting faces of mixed sizes, and improve the accuracy of detection.

[0112] According to the positional relationship of the facial features in the face image, the key areas in the image area covered by the detection window are determined respectively, and the image features of the key areas are extracted to determine the feature vectors of the corresponding image areas, so as to narrow the scope of feature extraction in the image area covered by the detection window, improve the feature extraction speed, reduce the amount of calculation, and improve the image detection efficiency; secondly, by determining the key areas for feature extraction, the interference of non-effective feature areas can be reduced, and the accuracy of image detection can be improved. The positional relationship of the facial features in the face image mainly refers to the positional relationship of the five position points of the center of the eyes, the tip of the nose, and the corners of the mouth at both ends of the face in the face image. In the process of detecting faces, the facial features are the main parts that effectively distinguish faces. Using the positions of the facial features as reference points to determine the key areas for feature extraction in the image detection process can both ensure detection accuracy and improve detection efficiency.

[0113] Step S205, when it is determined, based on the feature vectors of each of the image areas, that the image areas covered by the detection windows of different sizes overlap and both contain faces, the overlapping image areas covered by the detection windows of different sizes sliding in the image to be detected are merged to determine the face in the image to be detected and its position.

[0114] During the face detection process, detection windows of different sizes slide in the image to be detected to frame the image area to be detected. When the image areas covered by detection windows of different sizes overlap and both include faces, the overlapping image areas covered by detection windows of different sizes are fused to determine the coordinate position of the face.

[0115] In some embodiments, see Fig.12 The step S203, detecting the image to be detected by using detection windows of different sizes, determining key areas in the image area covered by the detection windows according to the positional relationship of the facial features in the face image, extracting image features of the key areas to determine the feature vectors of the corresponding image areas, includes:

[0116] Step S2031, detecting the image to be detected through a detection window, determining key areas in the image area covered by the sliding of the detection window in the image to be detected according to the positional relationship of the facial features in the face image, extracting image features of the key areas to determine feature vectors of the corresponding image areas;

[0117] Step S2032, adjusting the size of the detection window according to the set rule, and determining whether the adjusted detection window exceeds the set size range;

[0118] If not, executing step S2033, obtaining an updated detection window according to the adjusted detection window, and returning to the step of detecting the image to be detected through the detection window;

[0119] If so, execute step S205.

[0120] Among them, facial features are mainly concentrated in the areas of eyes, nose, mouth, etc. The key area is the feature extraction area in the image to be detected determined according to the positional relationship of the facial features in the face image. The position of the target reference point in the image area covered by the detection window can be determined based on the positional relationship of the five position points in the face image, namely the center of the eye, the tip of the nose, and the corners of the mouth at both ends, and then the minimum circumscribed square area of ​​the circle with the target reference point as the center and the preset value as the radius is used as the key area. When the detection window is used to detect whether the image to be detected exists, the detection window mainly extracts features from the key area, so as to more quickly and accurately extract effective image features that can be used to determine the target object.

[0121] The size of the detection window is adjusted according to the set rule. When the size of the adjusted detection window is within the set size range, an updated detection window is obtained, and the step of detecting the image to be detected through the detection window is returned. The adjustment of the set rule can be a rule for scaling the detection window according to a set ratio.

[0122] As an optional implementation, the setting ratio can be 1.2 times, the minimum size of the detection window is 20*20, and the maximum size of the detection window is 600*600. In the process of detecting the image to be detected through detection windows of different sizes, the image to be detected can be first detected with a detection window of size 20*20, and the key areas in the image area covered by the detection window are determined according to the positional relationship of the facial features in the face image, so as to respectively determine the feature vectors corresponding to the image area covered by the detection window, and detect whether the image area covered by the detection window contains the target object; then, the detection window is increased by 1.2 times. Get an updated detection window, and detect the image to be detected through the detection window with the updated size. According to the positional relationship of the facial features in the face image, determine the key areas in the image area covered by the detection window respectively, so as to determine the feature vectors corresponding to the image area covered by the detection window respectively, and detect whether the image area covered by the detection window contains the target object respectively; then, increase the updated detection window by 1.2 times again, and detect whether the image area covered by the detection window contains the target object respectively by determining the key areas for feature extraction, and so on, until the size of the detection window is greater than 600*600. After detecting the image to be detected through different detection windows, obtain the detection results of whether the image area covered by each detection window during the detection of the image to be detected contains a face, and merge according to whether there is overlap between the image areas containing faces, and finally obtain the face detection result in the entire image to be detected.

[0123] In some embodiments, extracting the image features of the key area to determine the feature vector of the corresponding image area includes:

[0124] Determine a target overlap degree according to a feature quantity in the key area, and divide the key area into a plurality of image blocks according to the target overlap degree;

[0125] Determine a feature code corresponding to the image block according to the magnitude of the pixel value of each pixel point of each image block and the pixel value of its adjacent pixel point;

[0126] A feature vector of the image region is obtained according to the feature code corresponding to the image block.

[0127] Dividing the key area into multiple image blocks, with a certain degree of overlap between adjacent image blocks, can reduce missed detection. The higher the overlap, the more features are extracted in the same area, which can reduce the probability of missed detection. Taking the target overlap of 50% as an example, the key area can be divided into 4*4 image blocks, and the rows or columns between two adjacent image blocks overlap by 50% to ensure the extraction of feature quantities in the key area. The target overlap is proportional to the size of the feature quantity, which can reduce the feature extraction of the information-free area and improve the image detection efficiency. Among them, the feature vector of the image area can use the LBP (Local Binary Pattern) value. Dividing the key area into multiple image blocks, according to the size comparison of the pixel value of the central pixel point of the image block and the adjacent pixel point, if the pixel value of the adjacent pixel point is greater than the pixel value of the central pixel point, the position of the adjacent pixel point is marked as 1, and if the pixel value of the adjacent pixel point is less than the pixel value of the central pixel point, the position of the adjacent pixel point is marked as 0, so that the feature value of the pixel point at the center of the image block can be determined. After performing LBP value processing on the image blocks in the key areas, an LBP feature map is obtained, the number of occurrences of different LBP values ​​in each key area is counted, and then the LBP values ​​of the key areas are connected in series to form an LBP feature vector corresponding to the image area currently covered by the detection window.

[0128] In order to have a more comprehensive understanding of the image detection method and face detection method provided in the embodiments of the present application, please refer to Fig.13 The following is a more specific exemplary description of the implementation of a face detection method for applying the image detection method to face detection. The face detection method includes the following steps:

[0129] S11, obtaining a first original image containing a human face, calibrating the coordinates of a target reference point based on the position of the facial features according to the size range of the human face in the image, and cropping the first original image according to the positional relationship of the target reference point in the facial imaging area to form a positive training sample;

[0130] S12, obtaining a second original image that does not contain a face, and cropping the second original image to form a negative training sample;

[0131] S13, forming a training sample set with positive training samples and negative training samples, training a classifier, determining the coordinates of the target reference point according to the positional relationship of the target reference point in the face imaging area of ​​the sample image, determining the key area according to the target reference point, extracting the image features of the key area to obtain a feature vector corresponding to the sample image, detecting the target object in the sample image according to the feature vector, and iteratively training with the positive training samples and the negative training samples to obtain a trained cascade classifier;

[0132] S14, obtaining an image to be detected through a trained cascade classifier, determining the size of a detection window according to the size range of a face in the image, sliding detection windows of different sizes in the image to be detected, and during the sliding detection process of detection windows of each size in the image to be detected, determining the coordinates of a target reference point for the image area covered by the detection window according to the positional relationship of the target reference point in the face imaging area, determining a key area according to the target reference point, extracting image features of the key area to obtain a feature vector corresponding to the image area, and detecting a face in the corresponding image area according to the feature vector;

[0133] Among them, in the process of detecting with different detection windows, the detection window with the smallest detection size can be used for detection first, and then the detection window can be increased by 1.2 times, and the above detection process can be repeated until the size of the detection window is larger than the maximum detection size. The minimum detection size is 20*20, and the maximum detection size is 600**600.

[0134] S15, when faces are detected simultaneously in image areas covered by detection windows of different sizes, fusing the image areas corresponding to the detection windows of different sizes;

[0135] S16, determining whether the detected image contains a face and corresponding coordinates according to the face detection results in the image areas respectively covered by the detection windows.

[0136] In the embodiment of the present application, by calibrating the target reference point based on the position of the facial features, determining the key area for feature extraction based on the target reference point, and using the feature extraction of the key area for face detection, the missed detection rate can be reduced, the detection time can be shortened, and the training time can be shortened to improve the detection speed. In addition, the memory usage during training can be reduced. As shown in the following table, a comparison is made between the method of extracting features based on the target reference point provided in the embodiment of the present application and the method of extracting features based on the entire face area:

[0137]

[0138] Secondly, when extracting features from a key area, the key area is divided into multiple image blocks with a certain degree of overlap. The degree of overlap between adjacent image blocks increases accordingly as the amount of features increases, thereby reducing missed detection and improving detection speed. Furthermore, through a dynamic adjustment strategy for the degree of overlap, a balance can be struck between missed detection and detection speed. For example, for an area with more features, the degree of overlap between adjacent image blocks can be larger when dividing image blocks; for an area with fewer features, the degree of overlap between adjacent image blocks can be smaller when dividing image blocks.

[0139] Another aspect of the present application provides a terminal device, see Fig.14, is an optional hardware structure diagram of a terminal device, the terminal device includes a memory 11 and a processor 12, and the memory 11 stores a computer program for implementing the image detection method provided in any embodiment of the present application. The terminal device also includes a network interface 13, a user interface 14, and a bus system 15 that couples and connects various components. It can be understood that the bus system 15 is used to realize the connection and communication between various components, and in addition to the data bus, it can also include a power bus, a control bus, and a status signal bus.

[0140] The user interface 13 may include a display, a keyboard, a mouse, a trackball, a click wheel, keys, buttons, a touch pad or a touch screen.

[0141] Among them, the memory 11 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of exemplary but not restrictive explanation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiment of the present application is intended to include but is not limited to these and any other suitable categories of memory. The memory is used to store various categories of data to support the operation of the image detection method. Examples of these data may include: any executable program for operating on the image detection device, such as an operating system and an application; original image; training sample set, etc.; wherein the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application may include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The image classification method provided by the embodiment of the present invention may be included in an application program.

[0142] Among them, the processor 12 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method provided in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory. The processor reads the information in the memory and completes the steps of the face detection method in any of the above embodiments in combination with its hardware, and can achieve the same technical effect. To avoid repetition, it is not repeated here.

[0143] See also Fig.15 On the other hand, the present application also provides a face detection device. In an exemplary embodiment, the image detection device can be implemented on the terminal device side. The image detection device includes an image acquisition module 111, which is used to acquire the image to be detected; a feature extraction module 112, which is used to detect the image to be detected through a detection window, and according to the positional relationship of the target reference point in the sample image, respectively determine the key areas in the image area covered by the sliding of the detection window in the image to be detected, and extract the image features of the key areas to determine the corresponding feature vectors of the image area; a detection module 113, which is used to detect the target object in the image to be detected according to the feature vectors of each of the image areas.

[0144] In some embodiments, it also includes a feature extraction module 112, which is also used to adjust the size of the detection window according to a set rule to obtain an updated detection window;

[0145] The image to be detected is detected through the updated detection window, and according to the positional relationship of the target reference point in the sample image, the key areas in the image area covered by the sliding of the updated detection window in the image to be detected are determined respectively, and the image features of the key areas are extracted to determine the feature vectors of the corresponding image areas.

[0146] In some embodiments, the feature extraction module 112 is specifically used to determine whether the image to be detected contains a target object based on the feature vectors of each of the image regions;

[0147] In the case that the image area covered by the updated detection window and the image area covered by the detection window both contain the target object, the image area covered by the updated detection window and the image area covered by the detection window are fused.

[0148] In some embodiments, the feature extraction module 112 is specifically configured to, when one of the image area covered by the updated detection window and the image area covered by the detection window is included in the other, or,

[0149] In a case where the overlap between the image area covered by the updated detection window and the image area covered by the detection window is greater than a threshold;

[0150] The reserved image region is selected according to the feature vector of the image region, and the coordinates of the target object are determined.

[0151] In some embodiments, the feature extraction module 112 is specifically configured to scale the detection window according to a set ratio, and when the size of the scaled detection window is within a set size range, use the scaled detection window as an updated detection window.

[0152] In some embodiments, the feature extraction module 112 is specifically used to determine the proportional coordinates of the target reference point relative to the target object area according to the coordinates of the target reference point in the sample image and the size of the target object area in the sample image;

[0153] The position of the target reference point in the image area covered by the detection window is determined according to the proportional coordinates, and the key area is determined according to the position of the target reference point.

[0154] In some embodiments, the feature extraction module 112 is specifically configured to define the minimum circumscribed square area of ​​a circle with the coordinates of the target reference point as the center and a preset value as the radius as the key area corresponding to the target reference point.

[0155] In some embodiments, the feature extraction module 112 is specifically used to divide the key area into a plurality of image blocks;

[0156] Determine a feature code corresponding to the image block according to the magnitude of the pixel value of each pixel point of each image block and the pixel value of its adjacent pixel point;

[0157] A feature vector of the image region is obtained according to the feature code corresponding to the image block.

[0158] In some embodiments, the feature extraction module 112 is specifically used to determine the target overlap based on the feature quantity in the key area, and divide the key area into multiple image blocks based on the target overlap, wherein the size of the target overlap is proportional to the size of the feature quantity.

[0159] In some embodiments, a training module 114 is further included, which is used to obtain an original image containing a target object, adjust the original image according to the calibration of the target reference point and the size of the target object area in the original image, until the display parameter of the original image meets the value condition, and use the adjusted original image as a positive sample image;

[0160] Acquire an image that does not contain the target object, use the image that does not contain the target object as a negative sample image, and form a training image set according to the positive sample image and the negative sample image;

[0161] The cascade classifier is trained based on the training image set.

[0162] In some embodiments, the training module is further used to crop the original image according to the coordinates of the target reference point in the original image and the size relationship of the target reference point in the target object area;

[0163] The target object in the cropped image is adjusted to conform to a preset size range, where the preset size range corresponds to the detection window size.

[0164] In some embodiments, the training module is further used to splice the feature vectors of each of the image regions;

[0165] According to the features of the spliced ​​images, a cascade classifier is used to determine whether the image to be detected contains the target object and the position of the target object.

[0166] It should be noted that: when the image detection device provided in the above embodiment realizes the detection of target objects in the image to be detected, only the division of the above program modules is used as an example. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the image detection device provided in the above embodiment belongs to the same concept as the image detection method embodiment. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0167] Another aspect of the present application provides a terminal device, see Fig.16, is an optional hardware structure diagram of a terminal device, the terminal device includes a memory 21, a processor 22, a network interface 23, a user interface 24 and a bus system 25 for coupling the various components. It can be understood that the bus system 25 is used to realize the connection and communication between the various components, and in addition to including a data bus, it can also include a power bus, a control bus and a status signal bus. The memory 21 stores a computer program for implementing the face detection method provided in any embodiment of the present application.

[0168] The user interface 13 may include a display, a keyboard, a mouse, a trackball, a click wheel, keys, buttons, a touch pad or a touch screen.

[0169] Among them, the memory 11 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of exemplary but not restrictive explanation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiment of the present application is intended to include but is not limited to these and any other suitable categories of memory. The memory is used to store various categories of data to support the operation of the image detection method. Examples of these data may include: any executable program for operating on the image detection device, such as an operating system and an application; original image; training sample set, etc.; wherein the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application may include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The image classification method provided by the embodiment of the present invention may be included in an application program.

[0170] Among them, the processor 12 can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method provided in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory. The processor reads the information in the memory and completes the steps of the face detection method in any of the above embodiments in combination with its hardware, and can achieve the same technical effect. To avoid repetition, it is not repeated here.

[0171] See also Fig.17 On the other hand, the present application also provides a face detection device. In an exemplary embodiment, the face detection device can be implemented on the terminal device side. The face detection device includes: an acquisition module 211, which is used to acquire the image to be detected; a feature module 212, which is used to detect the image to be detected through detection windows of different sizes, and determine the key areas in the image area covered by the detection window according to the positional relationship of the facial features in the face image, and extract the image features of the key areas to determine the corresponding feature vectors of the image area; a face detection module 213, which is used to determine that the image areas covered by detection windows of different sizes overlap and contain faces according to the feature vectors of each image area, and then fuse the overlapping image areas covered by the detection windows of different sizes to determine the face and its position in the image to be detected.

[0172] In some embodiments, the feature module 212 is specifically used to detect the image to be detected through a detection window, determine the key areas within the image area covered by the detection window according to the positional relationship of the facial features in the face image, and extract the image features of the key areas to determine the feature vectors of the corresponding image areas;

[0173] The size of the detection window is adjusted according to the set rules. When the size of the adjusted detection window is within the set size range, an updated detection window is obtained, and the step of detecting the image to be detected through the detection window is returned.

[0174] In some embodiments, the feature module 212 is specifically used to determine the target overlap according to the feature quantity in the key area, and divide the key area into multiple image blocks according to the target overlap;

[0175] Determine a feature code corresponding to the image block according to the magnitude of the pixel value of each pixel point of each image block and the pixel value of its adjacent pixel point;

[0176] A feature vector of the image region is obtained according to the feature code corresponding to the image block.

[0177] It should be noted that: when the face detection device provided in the above embodiment implements face detection in the image to be detected, only the division of the above program modules is used as an example. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the face detection device provided in the above embodiment and the face detection method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0178] In an illustrative embodiment, an embodiment of the present invention further provides a computer-readable storage medium, such as a memory including an executable program, the executable program is executed by a processor to complete the steps of the image detection method described in any embodiment of the present application or the steps of the face detection method described in any embodiment of the present application, and can achieve the same technical effect, to avoid repetition, no further description is given here. Among them, the computer-readable storage medium, such as a read-only memory (Read-Only Memory, referred to as ROM), a random access memory (Random Access Memory, referred to as RAM), a disk or an optical disk, etc.

[0179] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0180] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0181] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. The protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An image detection method, characterized in that: include: Acquire the image to be detected; The image to be detected is detected by using detection windows of different sizes, and according to the positional relationship of the target reference point in the sample image, the key areas in the image area covered by the sliding of the detection window in the image to be detected are determined respectively, and the image features of the key areas are extracted to determine the feature vectors of the corresponding image areas; the sample image is an image used to train a model for extracting shape features of target objects in an image, and the target reference point is a key feature point in the sample image used to define the shape features of the target object; When it is determined based on the feature vectors of each of the image areas that the image areas covered by detection windows of different sizes overlap and both contain the target object, the overlapping image areas covered by the detection windows of different sizes are fused to determine the target object and its position in the image to be detected.

2. The image detection method according to claim 1, characterized in that: When it is determined according to the feature vectors of each of the image regions that image regions covered by detection windows of different sizes overlap and both contain the target object, the overlapping image regions covered by the detection windows of different sizes are merged to determine the target object and its position in the image to be detected, including: Adjusting the size of the detection window according to a set rule to obtain an updated detection window; The image to be detected is detected through the updated detection window, and according to the positional relationship of the target reference point in the sample image, the key areas in the image area covered by the sliding of the updated detection window in the image to be detected are determined respectively, and the image features of the key areas are extracted to determine the feature vectors of the corresponding image areas.

3. The image detection method according to claim 2, characterized in that: When it is determined according to the feature vectors of the respective image regions that image regions covered by detection windows of different sizes overlap and both contain the target object, the overlapping image regions covered by the detection windows of different sizes are merged, including: Determining whether the image to be detected contains a target object according to the feature vectors of each of the image regions; In the case that the image area covered by the updated detection window and the image area covered by the detection window both contain the target object, the image area covered by the updated detection window and the image area covered by the detection window are fused.

4. The image detection method according to claim 3, characterized in that: The fusing the image area covered by the updated detection window and the image area covered by the detection window comprises: In the case where one of the image area covered by the updated detection window and the image area covered by the detection window is contained in the other, or, In a case where the overlap between the image area covered by the updated detection window and the image area covered by the detection window is greater than a threshold; The reserved image region is selected according to the feature vector of the image region, and the coordinates of the target object are determined.

5. The image detection method according to claim 2, characterized in that: The step of adjusting the size of the detection window according to the set rule to obtain an updated detection window includes: The detection window is scaled according to a set ratio, and when the size of the scaled detection window is within a set size range, the scaled detection window is used as an updated detection window.

6. The image detection method according to claim 1, characterized in that: Determining the key areas in the image area covered by the detection window according to the positional relationship of the target reference point in the sample image includes: Determining the proportional coordinates of the target reference point relative to the target object area according to the coordinates of the target reference point in the sample image and the size of the target object area in the sample image; The position of the target reference point in the image area covered by the detection window is determined according to the proportional coordinates, and the key area is determined according to the position of the target reference point.

7. The image detection method according to claim 6, characterized in that: There are multiple target reference points, and determining the key area according to the positions of the target reference points includes: The minimum circumscribed square area of ​​a circle with the coordinates of the target reference point as the center and a preset value as the radius is the key area corresponding to the target reference point.

8. The image detection method according to claim 1, characterized in that: The extracting the image features of the key area to determine the feature vector of the corresponding image area includes: Dividing the key area into a plurality of image blocks; Determine a feature code corresponding to the image block according to the magnitude of the pixel value of each pixel point of each image block and the pixel value of its adjacent pixel point; A feature vector of the image region is obtained according to the feature code corresponding to the image block.

9. The image detection method according to claim 8, characterized in that: The step of dividing the key area into a plurality of image blocks comprises: The target overlap is determined according to the feature quantity in the key area, and the key area is divided into a plurality of image blocks according to the target overlap, wherein the size of the target overlap is proportional to the size of the feature quantity.

10. The image detection method according to claim 1, characterized in that: Before acquiring the image to be detected, the method further includes: Acquire an original image containing a target object, adjust the original image according to the calibration of the target reference point and the size of the target object area in the original image, until the display parameters of the original image meet the value conditions, and use the adjusted original image as a positive sample image; Acquire an image that does not contain the target object, use the image that does not contain the target object as a negative sample image, and form a training image set according to the positive sample image and the negative sample image; The cascade classifier is trained based on the training image set.

11. The image detection method according to claim 10, characterized in that: The adjusting the original image according to the calibration of the target reference point and the size of the target object area in the original image until the display parameter of the original image meets the value condition includes: Cropping the original image according to the coordinates of the target reference point in the original image and the size relationship of the target reference point in the target object area; The target object in the cropped image is adjusted to conform to a preset size range, where the preset size range corresponds to the detection window size.

12. The image detection method according to any one of claims 1 to 11, characterized in that: The detecting of the target object in the image to be detected according to the feature vectors of each of the image regions comprises: splicing the feature vectors of each of the image regions; According to the features of the spliced ​​images, a cascade classifier is used to determine whether the image to be detected contains the target object and the position of the target object.

13. A face detection method, characterized in that: include: Acquire the image to be detected; The image to be detected is detected by using detection windows of different sizes respectively, and according to the positional relationship of facial features in the facial image, key areas in the image area covered by the sliding of the detection window in the image to be detected are determined respectively, and image features of the key areas are extracted to determine the feature vectors of the corresponding image areas; When it is determined according to the feature vectors of each of the image areas that image areas covered by detection windows of different sizes overlap and both contain faces, the overlapping image areas covered by the detection windows of different sizes are fused to determine the face and its position in the image to be detected.

14. The face detection method according to claim 13, characterized in that: The method of detecting the image to be detected by using detection windows of different sizes respectively, determining key areas in the image area covered by the sliding of the detection window in the image to be detected according to the positional relationship of the facial features in the facial image, and extracting image features of the key areas to determine the feature vectors of the corresponding image areas includes: Detecting the image to be detected through a detection window, determining key areas within the image area covered by the sliding of the detection window in the image to be detected according to the positional relationship of the facial features in the face image, and extracting image features of the key areas to determine feature vectors of the corresponding image areas; The size of the detection window is adjusted according to the set rule. When the size of the adjusted detection window is within the set size range, an updated detection window is obtained, and the step of detecting the image to be detected through the detection window is returned.

15. The face detection method according to claim 13, characterized in that: The extracting the image features of the key area to determine the feature vector of the corresponding image area includes: Determine a target overlap degree according to a feature quantity in the key area, and divide the key area into a plurality of image blocks according to the target overlap degree; Determine a feature code corresponding to the image block according to the magnitude of the pixel value of each pixel point of each image block and the pixel value of its adjacent pixel point; A feature vector of the image region is obtained according to the feature code corresponding to the image block.

16. An image detection device, characterized in that: include: An image acquisition module, used for acquiring an image to be detected; A feature extraction module is used to detect the image to be detected through detection windows of different sizes, determine key areas in the image area covered by the detection windows according to the positional relationship of the target reference points in the sample image, and extract image features of the key areas to determine the corresponding feature vectors of the image area; The detection module is used to determine, based on the feature vectors of each of the image areas, that the image areas covered by detection windows of different sizes overlap and both contain the target object, and then fuse the overlapping image areas covered by the detection windows of different sizes to determine the target object and its position in the image to be detected.

17. A face detection device, characterized in that: include: An acquisition module, used for acquiring an image to be detected; A feature module is used to detect the image to be detected through detection windows of different sizes, determine key areas in the image area covered by the detection window according to the positional relationship of the facial features in the face image, and extract image features of the key areas to determine the feature vectors of the corresponding image areas; The face detection module is used to determine, based on the feature vectors of each of the image areas, that the image areas covered by detection windows of different sizes overlap and both contain faces, and then fuse the overlapping image areas covered by the detection windows of different sizes to determine the face and its position in the image to be detected.

18. A terminal device, characterized in that: The invention comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor runs the computer program, the image detection method according to any one of claims 1 to 12 or the face detection method according to any one of claims 13 to 15 is executed.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program code, and when the computer program code is loaded and executed by the processor, the image detection method according to any one of claims 1 to 12 or the face detection method according to any one of claims 13 to 15 is executed.

Images