Image detection method, device, storage medium and apparatus

Abstract

The application discloses an image detection method, device, storage medium and apparatus, and the method comprises the following steps: performing foreground recognition on an initial candidate region to obtain a to-be-detected candidate region, performing face and human shape detection on the to-be-detected candidate region to obtain position information and a category score of the to-be-detected candidate region, and determining a target face and human shape region according to the position information and the category score. Compared with the prior art of indiscriminately processing foreground and background, the initial candidate region is subjected to foreground recognition in the application to obtain the to-be-detected candidate region, and the to-be-detected candidate region is subjected to face and human shape recognition to obtain the target face and human shape region, so that the defect of long processing time in the prior art is overcome, and the image detection process can be optimized and the detection time can be reduced.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image detection method, device, storage medium and apparatus. Background Technology

[0002] Face and human detection refers to the detection of the location of a face or human shape in a video frame. It is a key technology in fields such as face recognition and video surveillance.

[0003] However, existing face and human detection methods process both foreground and background indiscriminately, resulting in excessively long processing times.

[0004] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this invention is to provide an image detection method, device, storage medium, and apparatus, aiming to solve the technical problem of how to optimize the image detection process.

[0006] To achieve the above objectives, the present invention provides an image detection method, the image detection method comprising the following steps:

[0007] Foreground identification is performed on the initial candidate regions to obtain the candidate regions to be detected;

[0008] Face and human figure detection is performed on the candidate region to be detected to obtain the location information and category score of the candidate region to be detected.

[0009] The target human face region is determined based on the location information and the category score.

[0010] Optionally, the step of performing foreground recognition on the initial candidate region to obtain the candidate region to be detected specifically includes:

[0011] Obtain the previous frame detection image corresponding to the initial candidate region, and find the previous frame candidate region corresponding to the initial candidate region in the previous frame detection image;

[0012] Pixel analysis is performed on the initial candidate region and the previous frame candidate region respectively to obtain the pixel values ​​of the initial candidate region and the pixel values ​​of the previous frame candidate region.

[0013] The candidate regions to be detected are determined based on the initial candidate region pixel values ​​and the candidate region pixel values ​​of the previous frame.

[0014] Optionally, the step of determining the candidate region to be detected based on the initial candidate region pixel values ​​and the candidate region pixel values ​​of the previous frame specifically includes:

[0015] The current difference value is determined based on the initial candidate region pixel values ​​and the previous frame candidate region pixel values;

[0016] The initial candidate region is binarized based on the current difference value to obtain a binarized image;

[0017] Foreground pixels are determined based on the binarized image, and candidate regions to be detected are determined based on the foreground pixels.

[0018] Optionally, the step of determining foreground pixels based on the binarized image and determining candidate regions to be detected based on the foreground pixels specifically includes:

[0019] Foreground pixels are determined based on the binarized image, and candidate foreground regions are determined based on the foreground pixels.

[0020] The candidate foreground regions are traversed, and the traversed candidate foreground regions are taken as the foreground regions to be detected.

[0021] The area of ​​the foreground region to be detected is obtained, and it is determined whether the area is greater than a preset area threshold to obtain a judgment result.

[0022] After traversing the candidate foreground regions, the candidate regions to be detected are determined based on the judgment results.

[0023] Optionally, the step of determining the target face region based on the location information and the category score specifically includes:

[0024] The candidate regions to be detected are sorted according to the category scores to obtain the sorting results, and a first candidate region set is generated based on the sorting results.

[0025] Select target candidate regions from the first candidate region set, and generate a target candidate region set based on the target candidate regions;

[0026] Obtain the number of regions in the first candidate region set, and determine the target human face region based on the number of regions, the target candidate region set, and the first candidate region set.

[0027] Optionally, the step of obtaining the number of regions in the first candidate region set and determining the target face region based on the number of regions, the target candidate region set, and the first candidate region set specifically includes:

[0028] Obtain the number of regions in the first candidate region set, and determine whether the number of regions is greater than a preset number threshold;

[0029] When the number of regions exceeds a preset threshold, the target candidate regions in the first candidate region set are deleted to obtain a second candidate region set.

[0030] The candidate regions in the second candidate region set are traversed, and the traversed candidate regions are taken as candidate regions to be judged.

[0031] The region overlap is determined based on the location information of the target candidate region and the location information of the candidate region to be judged.

[0032] The target human face region is determined based on the region overlap, the target candidate region set, and the first candidate region set.

[0033] Optionally, the step of determining the target human face region based on the region overlap, the target candidate region set, and the first candidate region set specifically includes:

[0034] Determine whether the overlap of the regions is greater than a preset overlap threshold;

[0035] When the overlap of the regions is greater than a preset overlap threshold, the candidate regions to be judged in the second candidate region set are deleted, a third candidate region set is obtained, and the step of traversing the candidate regions in the second candidate region set is returned until the traversal of the candidate regions in the second candidate region set is completed.

[0036] After traversing the candidate regions in the second candidate region set, the first candidate region set is updated according to the third candidate region set, and the step of selecting a target candidate region from the first candidate region set is returned until the number of regions is less than or equal to a preset number threshold.

[0037] If the number of regions is less than or equal to a preset threshold, the target human face region is determined based on the target candidate region set and the first candidate region set.

[0038] Optionally, before the step of performing foreground recognition on the initial candidate region to obtain the candidate region to be detected, the image detection method further includes:

[0039] The current input image is acquired and preprocessed to obtain the current image to be detected;

[0040] According to the preset detection window, the current image to be detected is subjected to face and human figure detection using a preset face and human figure detection model to obtain the initial candidate region.

[0041] Optionally, the step of acquiring the current input image and preprocessing the current input image to obtain the current image to be detected specifically includes:

[0042] Obtain the current input image and adjust its size to obtain the current image to be processed;

[0043] Perform pixel analysis on the current image to be processed to obtain the current pixel value of the current image to be processed;

[0044] The current image to be processed is normalized based on the current pixel value to obtain the current image to be detected.

[0045] Optionally, the step of obtaining the current input image and adjusting the size of the current input image to obtain the current image to be processed specifically includes:

[0046] Acquire the current input image and analyze it to obtain the current size information of the current input image;

[0047] The current input image is resized based on the current size information and the preset standard size information to obtain the current image to be processed.

[0048] Optionally, the step of normalizing the current image to be processed based on the current pixel value to obtain the current image to be detected specifically includes:

[0049] The current pixel values ​​are sorted to obtain a sorting result, and a reference pixel value is determined based on the sorting result.

[0050] The pixel values ​​of the current image to be processed are adjusted based on the reference pixel value and the current pixel value to obtain the current image to be detected.

[0051] Optionally, after the step of determining the target human face region based on the location information and the category score, the image detection method further includes:

[0052] Obtain the current device information and find the information display template corresponding to the current device information;

[0053] The target human face region is displayed based on the information display template.

[0054] Furthermore, to achieve the above objectives, the present invention also proposes an image detection device, which includes a memory, a processor, and an image detection program stored in the memory and executable on the processor, the image detection program being configured to implement the steps of the image detection method described above.

[0055] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing an image detection program, which, when executed by a processor, implements the steps of the image detection method described above.

[0056] Furthermore, to achieve the above objectives, the present invention also proposes an image detection device, which includes: an identification module, a detection module, and a determination module;

[0057] The recognition module is used to perform foreground recognition on the initial candidate region to obtain the candidate region to be detected;

[0058] The detection module is used to perform face and human figure detection on the candidate region to be detected, and obtain the location information and category score of the candidate region to be detected;

[0059] The determining module is used to determine the target human face region based on the location information and the category score.

[0060] Optionally, the recognition module is further configured to acquire the previous frame detection image corresponding to the initial candidate region, and search for the previous frame candidate region corresponding to the initial candidate region in the previous frame detection image;

[0061] The recognition module is further configured to perform pixel analysis on the initial candidate region and the previous frame candidate region respectively, and obtain the pixel values ​​of the initial candidate region and the pixel values ​​of the previous frame candidate region.

[0062] The recognition module is further configured to determine the candidate region to be detected based on the pixel values ​​of the initial candidate region and the pixel values ​​of the candidate region in the previous frame.

[0063] Optionally, the recognition module is further configured to determine the current difference value based on the initial candidate region pixel value and the previous frame candidate region pixel value;

[0064] The recognition module is further configured to perform binarization processing on the initial candidate region based on the current difference value to obtain a binarized image;

[0065] The recognition module is further configured to determine foreground pixels based on the binarized image, and to determine candidate regions to be detected based on the foreground pixels.

[0066] Optionally, the recognition module is further configured to determine foreground pixels based on the binarized image, and determine candidate foreground regions based on the foreground pixels;

[0067] The identification module is further configured to traverse the candidate foreground regions and use the traversed candidate foreground regions as the foreground regions to be detected.

[0068] The recognition module is further configured to obtain the area of ​​the foreground region to be detected, and determine whether the area is greater than a preset area threshold, thereby obtaining a judgment result.

[0069] The recognition module is further configured to determine the candidate region to be detected based on the judgment result after traversing the candidate foreground region.

[0070] Optionally, the determining module is further configured to sort the candidate regions to be detected according to the category score, obtain a sorting result, and generate a first candidate region set according to the sorting result;

[0071] The determining module is further configured to select target candidate regions from the first candidate region set, and generate a target candidate region set based on the target candidate regions;

[0072] The determining module is further configured to obtain the number of regions in the first candidate region set, and determine the target human face region based on the number of regions, the target candidate region set, and the first candidate region set.

[0073] Optionally, the determining module is further configured to obtain the number of regions in the first candidate region set and determine whether the number of regions is greater than a preset number threshold.

[0074] The determining module is further configured to delete the target candidate region in the first candidate region set when the number of regions is greater than a preset number threshold, and obtain a second candidate region set.

[0075] The determining module is further configured to traverse the candidate regions in the second candidate region set and use the traversed candidate regions as candidate regions to be judged.

[0076] The determining module is further configured to determine the region overlap based on the location information of the target candidate region and the location information of the candidate region to be judged;

[0077] The determining module is further configured to determine the target human face region based on the region overlap, the target candidate region set, and the first candidate region set.

[0078] In this invention, foreground recognition is performed on the initial candidate region to obtain the candidate region to be detected. Face and human figure detection is then performed on the candidate region to be detected to obtain the location information and category score of the candidate region to be detected. The target face and human figure region is determined based on the location information and the category score. Compared with the existing method of processing the foreground and background indiscriminately, this invention overcomes the defect of excessive processing time in the prior art by performing foreground recognition on the initial candidate region to obtain the candidate region to be detected and performing face and human figure recognition on the candidate region to be detected to obtain the target face and human figure region. This optimizes the image detection process and reduces the detection time. Attached Figure Description

[0079] Figure 1 This is a schematic diagram of the structure of the image detection device in the hardware operating environment involved in the embodiments of the present invention;

[0080] Figure 2 This is a flowchart illustrating the first embodiment of the image detection method of the present invention;

[0081] Figure 3 This is a flowchart illustrating the second embodiment of the image detection method of the present invention;

[0082] Figure 4 This is a flowchart illustrating the third embodiment of the image detection method of the present invention;

[0083] Figure 5 This is a structural block diagram of the first embodiment of the image detection device of the present invention.

[0084] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0085] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0086] Reference Figure 1 , Figure 1 This is a schematic diagram of the image detection device structure in the hardware operating environment involved in the embodiments of the present invention.

[0087] like Figure 1As shown, the image detection device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen, and optionally, it may also include a standard wired interface or a wireless interface. In this invention, the wired interface of the user interface 1003 may be a USB interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a non-volatile memory (NVM), such as a disk storage device. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0088] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the image detection device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0089] like Figure 1 As shown, the memory 1005, which is identified as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an image detection program.

[0090] exist Figure 1 In the image detection device shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the user equipment; the image detection device calls the image detection program stored in the memory 1005 through the processor 1001 and executes the image detection method provided in the embodiment of the present invention.

[0091] Based on the above hardware structure, an embodiment of the image detection method of the present invention is proposed.

[0092] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the image detection method of the present invention, which presents the first embodiment of the image detection method of the present invention.

[0093] In a first embodiment, the image detection method includes the following steps:

[0094] Step S10: Perform foreground recognition on the initial candidate region to obtain the candidate region to be detected.

[0095] It should be understood that the execution subject of this embodiment is the image detection device, which can be a terminal device such as a mobile phone or computer, or other devices that can achieve the same or similar functions. This embodiment does not limit this; in this embodiment, a terminal device is used as an example.

[0096] It is understood that foreground recognition of the initial candidate region to obtain the candidate region to be detected can be performed by performing foreground recognition of the initial candidate region through a preset foreground recognition model. The preset foreground recognition model can be a foreground recognition model pre-set by the user, such as the three-frame difference model, etc. This embodiment does not limit this.

[0097] Furthermore, in order to improve the accuracy of foreground recognition and obtain reliable candidate regions to be detected, the foreground recognition of the initial candidate regions to obtain candidate regions to be detected includes:

[0098] Obtain the previous frame detection image corresponding to the initial candidate region, and find the previous frame candidate region corresponding to the initial candidate region in the previous frame detection image. Perform pixel analysis on the initial candidate region and the previous frame candidate region respectively to obtain the pixel value of the initial candidate region and the pixel value of the previous frame candidate region. Determine the candidate region to be detected based on the pixel value of the initial candidate region and the pixel value of the previous frame candidate region.

[0099] Furthermore, in order to reduce the computational load of face and human detection and achieve real-time face and human detection on the terminal device, before performing foreground recognition on the initial candidate region to obtain the candidate region to be detected, the following steps are also included:

[0100] The current input image is acquired and preprocessed to obtain the current image to be detected. Based on the preset detection window, the current image to be detected is subjected to face and human detection using a preset face and human detection model to obtain the initial candidate region.

[0101] Step S20: Perform face and human figure detection on the candidate region to be detected to obtain the location information and category score of the candidate region to be detected.

[0102] It should be understood that performing face and human figure detection on the candidate region to be detected, and obtaining the location information and category score of the candidate region to be detected, can be done by performing face and human figure detection on the candidate region to be detected using a preset face and human figure detection model, and obtaining the location information and category score of the candidate region to be detected.

[0103] It should be noted that the preset face and human detection model can be a MobileNet network structure. In this embodiment, the output channels of each layer of the MobileNet network structure can be reduced by half to reduce the number of parameters of the detection model, thereby realizing real-time face and human detection on the terminal device. The MobileNet network structure can detect faces and human figures simultaneously, thereby reducing detection time and system resource consumption.

[0104] It should be noted that the category score can be the score of the candidate region to be detected, which may be a face, a human figure, or a background. This embodiment does not limit this.

[0105] Step S30: Determine the target human face region based on the location information and the category score.

[0106] It should be understood that determining the target face region based on location information and category score can be done by taking the candidate regions to be detected that are not located at the edge of the image and whose category score is greater than a preset threshold as the target face region. The preset threshold can be a value set by the user in advance, and this embodiment does not limit it.

[0107] Furthermore, in order to remove overlapping candidate regions and obtain the target face region, determining the target face region based on the location information and the category score includes:

[0108] The candidate regions to be detected are sorted according to the category score to obtain a sorting result, and a first candidate region set is generated according to the sorting result. A target candidate region is selected from the first candidate region set, and a target candidate region set is generated according to the target candidate region. The number of regions in the first candidate region set is obtained, and the target human face region is determined according to the number of regions, the target candidate region set, and the first candidate region set.

[0109] In the first embodiment, foreground recognition is performed on the initial candidate region to obtain a candidate region to be detected. Face and human figure detection is then performed on the candidate region to be detected to obtain the location information and category score of the candidate region to be detected. The target face and human figure region is determined based on the location information and the category score. Compared with the existing method of processing the foreground and background indiscriminately, in this embodiment, foreground recognition is performed on the initial candidate region to obtain a candidate region to be detected, and face and human figure detection is performed on the candidate region to be detected to obtain the target face and human figure region. This overcomes the defect of excessive processing time in the prior art, thereby optimizing the image detection process and reducing detection time.

[0110] Reference Figure 3 , Figure 3This is a flowchart illustrating the second embodiment of the image detection method of the present invention, based on the above. Figure 2 The first embodiment shown is followed by a second embodiment of the image detection method of the present invention.

[0111] In the second embodiment, step S10 includes:

[0112] Step S101: Obtain the previous frame detection image corresponding to the initial candidate region, and find the previous frame candidate region corresponding to the initial candidate region in the previous frame detection image.

[0113] It should be understood that obtaining the previous frame detection image corresponding to the initial candidate region can be done by obtaining the previous frame image of the current input image containing the initial candidate region and using the previous frame image of the current input image as the previous frame detection image.

[0114] Step S102: Perform pixel analysis on the initial candidate region and the previous frame candidate region respectively to obtain the pixel values ​​of the initial candidate region and the pixel values ​​of the previous frame candidate region.

[0115] It is understood that performing pixel analysis on the initial candidate region and the candidate region of the previous frame to obtain the pixel values ​​of the initial candidate region and the candidate region of the previous frame can be achieved by inputting the initial candidate region and the candidate region of the previous frame into a preset pixel analysis model, obtaining the analysis results, and determining the pixel values ​​of the initial candidate region and the candidate region of the previous frame based on the analysis results. The preset pixel analysis model can be an analysis model pre-set by the user, and this embodiment does not limit it.

[0116] Step S103: Determine the candidate region to be detected based on the initial candidate region pixel values ​​and the previous frame candidate region pixel values.

[0117] Furthermore, to improve the reliability of the candidate region to be detected, step S103 includes:

[0118] The current difference value is determined based on the initial candidate region pixel values ​​and the previous frame candidate region pixel values;

[0119] The initial candidate region is binarized based on the current difference value to obtain a binarized image;

[0120] Foreground pixels are determined based on the binarized image, and candidate regions to be detected are determined based on the foreground pixels.

[0121] It should be understood that determining the current difference value based on the initial candidate region pixel values ​​and the previous frame candidate region pixel values ​​can be achieved by calculating the current difference value using a preset difference formula based on the initial candidate region pixel values ​​and the previous frame candidate region pixel values.

[0122] The preset difference formula is as follows:

[0123]

[0124] In the formula, This is the current difference value. These are the pixel values ​​of the initial candidate region. The pixel values ​​of the candidate region in the previous frame.

[0125] It is understandable that binarizing the initial candidate region based on the current difference value to obtain a binarized image can be achieved by binarizing the current difference value using a preset binarization formula, obtaining a binarized value, and then generating a binarized image based on the binarized value.

[0126]

[0127] In the formula, The value is a binary value. Preset pixel threshold.

[0128] It should be understood that determining the foreground pixels based on the binarized image and determining the candidate region to be detected based on the foreground pixels can be done by taking the pixels with a binarization value of 1 as the foreground points, judging whether the initial candidate region contains the foreground points, and determining the initial candidate region as the candidate region to be detected if the initial candidate region contains the foreground points.

[0129] Furthermore, considering practical applications, directly using the initial candidate region containing foreground points as the candidate region to be detected would inevitably lead to too few objects involved in the judgment process and low accuracy. To overcome this deficiency, the step of determining foreground pixels based on the binarized image and determining the candidate region to be detected based on the foreground pixels includes:

[0130] Foreground pixels are determined based on the binarized image, and candidate foreground regions are determined based on the foreground pixels.

[0131] The candidate foreground regions are traversed, and the traversed candidate foreground regions are taken as the foreground regions to be detected.

[0132] The area of ​​the foreground region to be detected is obtained, and it is determined whether the area is greater than a preset area threshold to obtain a judgment result.

[0133] After traversing the candidate foreground regions, the candidate regions to be detected are determined based on the judgment results.

[0134] It should be understood that determining the candidate foreground region based on the foreground pixels can be done by performing region connectivity based on the foreground pixels to obtain a connected region, and then using the connected region as the candidate foreground region.

[0135] It is understandable that the determination of whether the area of ​​the region is greater than the preset area threshold can result in the following: when the area of ​​the region is greater than the preset area threshold, the candidate foreground region is taken as the target foreground region; when the area of ​​the region is less than or equal to the preset area threshold, the candidate foreground region is taken as the noise region. The preset area threshold can be a value preset by the user according to the actual situation, and this embodiment does not limit it.

[0136] It should be understood that, after traversing the candidate foreground regions, determining the candidate region to be detected based on the judgment result can be done by taking the initial candidate region containing the target foreground region as the candidate region to be detected after traversing the candidate foreground regions.

[0137] In the second embodiment, by acquiring the previous frame detection image corresponding to the initial candidate region, and searching for the previous frame candidate region corresponding to the initial candidate region in the previous frame detection image, performing pixel analysis on the initial candidate region and the previous frame candidate region respectively, obtaining the pixel values ​​of the initial candidate region and the previous frame candidate region, and determining the candidate region to be detected based on the pixel values ​​of the initial candidate region and the previous frame candidate region, the accuracy of foreground recognition can be improved, thereby obtaining reliable candidate regions to be detected.

[0138] In the second embodiment, step S30 includes:

[0139] Step S301: Sort the candidate regions to be detected according to the category score, obtain the sorting result, and generate a first candidate region set according to the sorting result.

[0140] It should be understood that sorting the candidate regions to be detected according to the category score, obtaining the sorting result, and generating the first candidate region set according to the sorting result can be done by sorting the candidate regions to be detected from largest to smallest according to the face score and / or human shape score in the category score, and using the sorted candidate regions to be detected as the first candidate region set.

[0141] Step S302: Select target candidate regions from the first candidate region set, and generate a target candidate region set based on the target candidate regions.

[0142] It is understandable that selecting a target candidate region from the first candidate region set can be done by taking the first candidate region in the first candidate region set as the target candidate region.

[0143] It should be understood that generating a target candidate region set based on the target candidate region can be achieved by storing the target candidate regions into a preset empty set to obtain the target candidate region set.

[0144] Step S303: Obtain the number of regions in the first candidate region set, and determine the target human face region based on the number of regions, the target candidate region set, and the first candidate region set.

[0145] Furthermore, in order to reduce the computational load in the process of determining the target face shape region, step S303 includes:

[0146] Obtain the number of regions in the first candidate region set, and determine whether the number of regions is greater than a preset number threshold;

[0147] When the number of regions exceeds a preset threshold, the target candidate regions in the first candidate region set are deleted to obtain a second candidate region set.

[0148] The candidate regions in the second candidate region set are traversed, and the traversed candidate regions are taken as candidate regions to be judged.

[0149] The region overlap is determined based on the location information of the target candidate region and the location information of the candidate region to be judged.

[0150] The target human face region is determined based on the region overlap, the target candidate region set, and the first candidate region set.

[0151] It should be noted that the number of regions in the first candidate region set can be the number of candidate regions in the first candidate region set; the preset number threshold can be a value preset by the user, and 1 is used as an example in this embodiment.

[0152] It should be understood that when the number of regions exceeds the preset threshold, it indicates that there are multiple candidate regions in the first candidate region, and subsequent steps need to be performed.

[0153] Furthermore, to ensure the accuracy and reliability of the target face region, determining the target face region based on the region overlap, the target candidate region set, and the first candidate region set includes:

[0154] Determine whether the overlap of the regions is greater than a preset overlap threshold;

[0155] When the overlap of the regions is greater than a preset overlap threshold, the candidate regions to be judged in the second candidate region set are deleted, a third candidate region set is obtained, and the step of traversing the candidate regions in the second candidate region set is returned until the traversal of the candidate regions in the second candidate region set is completed.

[0156] After traversing the candidate regions in the second candidate region set, the first candidate region set is updated according to the third candidate region set, and the step of selecting a target candidate region from the first candidate region set is returned until the number of regions is less than or equal to a preset number threshold.

[0157] If the number of regions is less than or equal to a preset threshold, the target human face region is determined based on the target candidate region set and the first candidate region set.

[0158] It should be noted that the preset overlap threshold can be a value set by the user in advance. In this embodiment, 0.2 is used as an example for explanation.

[0159] Furthermore, for ease of understanding, the following examples are provided:

[0160] 1. The candidate regions A, B, C, D, E, and F to be detected are sorted according to their category scores to obtain the first candidate region set ABCDEF;

[0161] 2. Starting from candidate region A with the highest category score, determine whether the overlap between regions B through F and A is greater than a preset threshold;

[0162] 3. If the overlap between regions C, D and A is greater than a preset threshold, and the overlap between regions B, E, F and A is less than or equal to a preset threshold, then C and D are removed from the first candidate region set, and A is stored in the target candidate region set.

[0163] 4. From the second candidate region set BEF, select B with the largest category score, and then determine whether the overlap between B and E and F is greater than the preset threshold.

[0164] 5. If the overlap between regions E and B is greater than a preset threshold, and the overlap between regions F and B is less than or equal to a preset threshold, then E will be removed from the first candidate region set, and B will be stored in the target candidate region set.

[0165] 6. At this point, only E remains in the first candidate region set, and the number of regions is equal to 1. The target candidate region set contains A and B. A, B, and E are taken as the target human face region.

[0166] In the second embodiment, the candidate regions to be detected are sorted according to the category score to obtain a sorting result, and a first candidate region set is generated according to the sorting result. A target candidate region is selected from the first candidate region set, and a target candidate region set is generated according to the target candidate region. The number of regions in the first candidate region set is obtained, and a target face shape region is determined according to the number of regions, the target candidate region set, and the first candidate region set. This allows overlapping candidate regions to be detected to be removed, thereby obtaining the target face shape region.

[0167] Reference Figure 4 , Figure 4 This is a flowchart illustrating the third embodiment of the image detection method of the present invention, based on the above. Figure 2 The first embodiment shown is followed by a third embodiment of the image detection method of the present invention.

[0168] In the third embodiment, before step S10, the method further includes:

[0169] Step S01: Obtain the current input image and preprocess the current input image to obtain the current image to be detected.

[0170] It should be noted that the current input image can be an image entered by the user through the user interface of the terminal device, or it can be an image of the current frame selected from the currently detected video. The currently detected video can be the real-time video detected by the image sensor of the terminal device, and this embodiment does not limit it.

[0171] It should be understood that preprocessing the current input image can be performed by preprocessing the current input image using a preset preprocessing script to obtain the current image to be detected. The preset preprocessing script can be an image processing script set by the user in advance, and this embodiment does not limit it.

[0172] Furthermore, in order to reduce the computational load of preprocessing the current input image while ensuring that the generated current image to be detected has high accuracy and reliability, step S01 includes:

[0173] Obtain the current input image and adjust its size to obtain the current image to be processed;

[0174] Perform pixel analysis on the current image to be processed to obtain the current pixel value of the current image to be processed;

[0175] The current image to be processed is normalized based on the current pixel value to obtain the current image to be detected.

[0176] It should be understood that resizing the current input image can be done by resizing the current input image according to preset standard size information to obtain the current image to be processed. The preset standard size can be the image size set by the user in advance, and this embodiment does not limit it.

[0177] It is understandable that normalizing the current image to be processed based on the current pixel value to obtain the current image to be detected can be achieved by dividing the current pixel value by a preset pixel value to obtain the target pixel value, and generating the image to be detected based on the target pixel value. The preset pixel value can be set by the user in advance. In this embodiment, 255 is used as an example for explanation.

[0178] Furthermore, to ensure the accuracy and reliability of the current image to be processed, the step of acquiring the current input image and adjusting the size of the current input image to obtain the current image to be processed includes:

[0179] Acquire the current input image and analyze it to obtain the current size information of the current input image;

[0180] The current input image is resized based on the current size information and the preset standard size information to obtain the current image to be processed.

[0181] It is understandable that adjusting the size of the current input image based on the current size information and the preset standard size information to obtain the current image to be processed can be achieved by determining the size difference information based on the current size information and the preset standard size information, and then adjusting the size of the current input image based on the size difference information to obtain the current image to be processed.

[0182] Furthermore, considering practical applications, directly normalizing the current image to be processed based on the current pixel value to obtain the current image to be detected would inevitably lead to too few objects involved in the judgment process, resulting in poor reliability. To overcome this deficiency, the step of normalizing the current image to be processed based on the current pixel value to obtain the current image to be detected includes:

[0183] The current pixel values ​​are sorted to obtain a sorting result, and a reference pixel value is determined based on the sorting result.

[0184] The pixel values ​​of the current image to be processed are adjusted based on the reference pixel value and the current pixel value to obtain the current image to be detected.

[0185] It should be understood that sorting the current pixel value, obtaining the sorting result, and determining the reference pixel value based on the sorting result can be done by sorting the current pixel value from largest to smallest and taking the current pixel value that appears first in the sort as the reference pixel value.

[0186] It is understandable that the pixel values ​​of the current image to be processed are adjusted according to the reference pixel value and the current pixel value. The current image to be detected can be obtained by dividing the current pixel value by the reference pixel value to obtain the target pixel value, and then generating the current image to be detected based on the target pixel value.

[0187] Step S02: Perform face and human detection on the current image to be detected using a preset face and human detection model based on the preset detection window to obtain the initial candidate region.

[0188] It should be noted that the preset face and human detection model can be a MobileNet network structure. In this embodiment, the output channels of each layer of the MobileNet network structure can be reduced by half to reduce the number of parameters of the detection model, thereby realizing real-time face and human detection on the terminal device. The preset detection window can be a detection area preset by the user. The detection area can be multiple rectangular areas of different sizes and positions. This embodiment does not limit this.

[0189] In the third embodiment, by acquiring the current input image and preprocessing the current input image to obtain the current image to be detected, and by performing face and human detection on the current image to be detected according to the preset detection window and the preset face and human detection model, the initial candidate region is obtained, thereby reducing the computational load of face and human detection and realizing real-time face and human detection on the terminal device.

[0190] In the third embodiment, after step S30, the method further includes:

[0191] Step S40: Obtain the current device information and find the information display template corresponding to the current device information.

[0192] It should be noted that the current device information may be device identification information, etc., and this embodiment does not impose any restrictions on it.

[0193] It should be understood that finding the information display template corresponding to the current device information can be done by searching for the information display template corresponding to the current device information in a preset mapping relationship table. The preset mapping relationship contains the correspondence between the current device information and the information display template. The correspondence between the current device information and the information display template can be preset by the user, and this embodiment does not limit this.

[0194] Step S50: Display the target human face region based on the information display template.

[0195] It is understandable that displaying the target face / human shape region based on the information display template can involve aggregating the location information and category score of the target face / human shape region into the information display template for display.

[0196] In the third embodiment, by obtaining the current device information and finding the information display template corresponding to the current device information, the target face shape area is displayed based on the information display template, thereby enabling the target face shape area to be displayed to the user in a timely manner to improve the user experience.

[0197] Furthermore, embodiments of the present invention also propose a storage medium storing an image detection program, which, when executed by a processor, implements the steps of the image detection method described above.

[0198] In addition, refer to Figure 5 The present invention also proposes an image detection device, which includes: an identification module 10, a detection module 20, and a determination module 30;

[0199] The recognition module 10 is used to perform foreground recognition on the initial candidate region to obtain the candidate region to be detected.

[0200] It is understood that foreground recognition of the initial candidate region to obtain the candidate region to be detected can be performed by performing foreground recognition of the initial candidate region through a preset foreground recognition model. The preset foreground recognition model can be a foreground recognition model pre-set by the user, such as the three-frame difference model, etc. This embodiment does not limit this.

[0201] Furthermore, in order to improve the accuracy of foreground recognition and obtain reliable candidate regions to be detected, the foreground recognition of the initial candidate regions to obtain candidate regions to be detected includes:

[0202] Obtain the previous frame detection image corresponding to the initial candidate region, and find the previous frame candidate region corresponding to the initial candidate region in the previous frame detection image. Perform pixel analysis on the initial candidate region and the previous frame candidate region respectively to obtain the pixel value of the initial candidate region and the pixel value of the previous frame candidate region. Determine the candidate region to be detected based on the pixel value of the initial candidate region and the pixel value of the previous frame candidate region.

[0203] Furthermore, in order to reduce the computational load of face and human detection and achieve real-time face and human detection on the terminal device, before performing foreground recognition on the initial candidate region to obtain the candidate region to be detected, the following steps are also included:

[0204] The current input image is acquired and preprocessed to obtain the current image to be detected. Based on the preset detection window, the current image to be detected is subjected to face and human detection using a preset face and human detection model to obtain the initial candidate region.

[0205] The detection module 20 is used to perform face and human figure detection on the candidate region to be detected, and obtain the location information and category score of the candidate region to be detected.

[0206] It should be understood that performing face and human figure detection on the candidate region to be detected, and obtaining the location information and category score of the candidate region to be detected, can be done by performing face and human figure detection on the candidate region to be detected using a preset face and human figure detection model, and obtaining the location information and category score of the candidate region to be detected.

[0207] It should be noted that the preset face and human detection model can be a MobileNet network structure. In this embodiment, the output channels of each layer of the MobileNet network structure can be reduced by half to reduce the number of parameters of the detection model, thereby realizing real-time face and human detection on the terminal device. The MobileNet network structure can detect faces and human figures simultaneously, thereby reducing detection time and system resource consumption.

[0208] It should be noted that the category score can be the score of the candidate region to be detected, which may be a face, a human figure, or a background. This embodiment does not limit this.

[0209] The determining module 30 is used to determine the target human face region based on the location information and the category score.

[0210] It should be understood that determining the target face region based on location information and category score can be done by taking the candidate regions to be detected that are not located at the edge of the image and whose category score is greater than a preset threshold as the target face region. The preset threshold can be a value set by the user in advance, and this embodiment does not limit it.

[0211] Furthermore, in order to remove overlapping candidate regions and obtain the target face region, determining the target face region based on the location information and the category score includes:

[0212] The candidate regions to be detected are sorted according to the category score to obtain a sorting result, and a first candidate region set is generated according to the sorting result. A target candidate region is selected from the first candidate region set, and a target candidate region set is generated according to the target candidate region. The number of regions in the first candidate region set is obtained, and the target human face region is determined according to the number of regions, the target candidate region set, and the first candidate region set.

[0213] In this embodiment, foreground recognition is performed on the initial candidate region to obtain the candidate region to be detected. Face and human figure detection is then performed on the candidate region to be detected to obtain the location information and category score of the candidate region to be detected. The target face and human figure region is determined based on the location information and the category score. Compared with the existing method of processing the foreground and background indiscriminately, this embodiment overcomes the defect of excessive processing time in the prior art by performing foreground recognition on the initial candidate region to obtain the candidate region to be detected and performing face and human figure recognition on the candidate region to be detected to obtain the target face and human figure region. This optimizes the image detection process and reduces the detection time.

[0214] Other embodiments or specific implementations of the image detection device of the present invention can be referred to the above-described method embodiments, and will not be repeated here.

[0215] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0216] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the unit claims listing several devices, several of these devices may be embodied by the same hardware item. The use of the terms first, second, and third, etc., does not indicate any order and can be interpreted as names.

[0217] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as a read-only memory image (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0218] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. An image detection method, characterized in that, The image detection method includes the following steps: Foreground identification is performed on the initial candidate regions to obtain the candidate regions to be detected; Face and human figure detection is performed on the candidate region to be detected to obtain the location information and category score of the candidate region to be detected. The target human face region is determined based on the location information and the category score. The step of determining the target human face region based on the location information and the category score specifically includes: The candidate regions to be detected are sorted according to the category scores to obtain the sorting results, and a first candidate region set is generated based on the sorting results. Select target candidate regions from the first candidate region set, and generate a target candidate region set based on the target candidate regions; Obtain the number of regions in the first candidate region set, and determine the target human face region based on the number of regions, the target candidate region set, and the first candidate region set.

2. The image detection method as described in claim 1, characterized in that, The step of performing foreground recognition on the initial candidate region to obtain the candidate region to be detected specifically includes: Obtain the previous frame detection image corresponding to the initial candidate region, and find the previous frame candidate region corresponding to the initial candidate region in the previous frame detection image; Pixel analysis is performed on the initial candidate region and the previous frame candidate region respectively to obtain the pixel values ​​of the initial candidate region and the pixel values ​​of the previous frame candidate region. The candidate regions to be detected are determined based on the initial candidate region pixel values ​​and the candidate region pixel values ​​of the previous frame.

3. The image detection method as described in claim 2, characterized in that, The step of determining the candidate region to be detected based on the initial candidate region pixel values ​​and the candidate region pixel values ​​of the previous frame specifically includes: The current difference value is determined based on the initial candidate region pixel values ​​and the previous frame candidate region pixel values; The initial candidate region is binarized based on the current difference value to obtain a binarized image; Foreground pixels are determined based on the binarized image, and candidate regions to be detected are determined based on the foreground pixels.

4. The image detection method as described in claim 3, characterized in that, The steps of determining foreground pixels based on the binarized image and determining candidate regions to be detected based on the foreground pixels specifically include: Foreground pixels are determined based on the binarized image, and candidate foreground regions are determined based on the foreground pixels. The candidate foreground regions are traversed, and the traversed candidate foreground regions are taken as the foreground regions to be detected. The area of ​​the foreground region to be detected is obtained, and it is determined whether the area is greater than a preset area threshold to obtain a judgment result. After traversing the candidate foreground regions, the candidate regions to be detected are determined based on the judgment results.

5. The image detection method as described in claim 1, characterized in that, The step of obtaining the number of regions in the first candidate region set and determining the target human face region based on the number of regions, the target candidate region set, and the first candidate region set specifically includes: Obtain the number of regions in the first candidate region set, and determine whether the number of regions is greater than a preset number threshold; When the number of regions exceeds a preset threshold, the target candidate regions in the first candidate region set are deleted to obtain a second candidate region set. The candidate regions in the second candidate region set are traversed, and the traversed candidate regions are taken as candidate regions to be judged. The region overlap is determined based on the location information of the target candidate region and the location information of the candidate region to be judged. The target human face region is determined based on the region overlap, the target candidate region set, and the first candidate region set.

6. The image detection method as described in claim 5, characterized in that, The step of determining the target human face region based on the region overlap, the target candidate region set, and the first candidate region set specifically includes: Determine whether the overlap of the regions is greater than a preset overlap threshold; When the overlap of the regions is greater than a preset overlap threshold, the candidate regions to be judged in the second candidate region set are deleted, a third candidate region set is obtained, and the step of traversing the candidate regions in the second candidate region set is returned until the traversal of the candidate regions in the second candidate region set is completed. After traversing the candidate regions in the second candidate region set, the first candidate region set is updated according to the third candidate region set, and the step of selecting a target candidate region from the first candidate region set is returned until the number of regions is less than or equal to a preset number threshold. If the number of regions is less than or equal to a preset threshold, the target human face region is determined based on the target candidate region set and the first candidate region set.

7. The image detection method as described in claim 1, characterized in that, Before the step of performing foreground recognition on the initial candidate region to obtain the candidate region to be detected, the image detection method further includes: The current input image is acquired and preprocessed to obtain the current image to be detected; According to the preset detection window, the current image to be detected is subjected to face and human figure detection using a preset face and human figure detection model to obtain the initial candidate region.

8. The image detection method as described in claim 7, characterized in that, The steps of acquiring the current input image and preprocessing the current input image to obtain the current image to be detected specifically include: Obtain the current input image and resize the current input image to obtain the current image to be processed; Perform pixel analysis on the current image to be processed to obtain the current pixel value of the current image to be processed; The current image to be processed is normalized based on the current pixel value to obtain the current image to be detected.

9. The image detection method as described in claim 8, characterized in that, The steps of acquiring the current input image and adjusting its size to obtain the current image to be processed specifically include: Acquire the current input image and analyze it to obtain the current size information of the current input image; The current input image is resized based on the current size information and the preset standard size information to obtain the current image to be processed.

10. The image detection method as described in claim 8, characterized in that, The step of normalizing the current image to be processed based on the current pixel value to obtain the current image to be detected specifically includes: The current pixel values ​​are sorted to obtain a sorting result, and a reference pixel value is determined based on the sorting result. The pixel values ​​of the current image to be processed are adjusted based on the reference pixel value and the current pixel value to obtain the current image to be detected.

11. The image detection method according to any one of claims 1-10, characterized in that, After the step of determining the target human face region based on the location information and the category score, the image detection method further includes: Obtain the current device information and find the information display template corresponding to the current device information; The target human face region is displayed based on the information display template.

12. An image detection device, characterized in that, The image detection device includes: a memory, a processor, and an image detection program stored in the memory and executable on the processor, wherein the image detection program, when executed by the processor, implements the steps of the image detection method as described in any one of claims 1 to 11.

13. A storage medium, characterized in that, The storage medium stores an image detection program, which, when executed by a processor, implements the steps of the image detection method as described in any one of claims 1 to 11.

14. An image detection device, characterized in that, The image detection device includes: a recognition module, a detection module, and a determination module; The recognition module is used to perform foreground recognition on the initial candidate region to obtain the candidate region to be detected; The detection module is used to perform face and human figure detection on the candidate region to be detected, and obtain the location information and category score of the candidate region to be detected; The determining module is used to determine the target human face region based on the location information and the category score; The determining module is further configured to sort the candidate regions to be detected according to the category score, obtain a sorting result, and generate a first candidate region set according to the sorting result; The determining module is further configured to select target candidate regions from the first candidate region set, and generate a target candidate region set based on the target candidate regions; The determining module is further configured to obtain the number of regions in the first candidate region set, and determine the target human face region based on the number of regions, the target candidate region set, and the first candidate region set.

15. The image detection apparatus as described in claim 14, characterized in that, The recognition module is also used to obtain the previous frame detection image corresponding to the initial candidate region, and to find the previous frame candidate region corresponding to the initial candidate region in the previous frame detection image. The recognition module is further configured to perform pixel analysis on the initial candidate region and the previous frame candidate region respectively, and obtain the pixel values ​​of the initial candidate region and the pixel values ​​of the previous frame candidate region. The recognition module is further configured to determine the candidate region to be detected based on the pixel values ​​of the initial candidate region and the pixel values ​​of the candidate region in the previous frame.

16. The image detection apparatus as described in claim 15, characterized in that, The recognition module is further configured to determine the current difference value based on the initial candidate region pixel value and the previous frame candidate region pixel value; The recognition module is further configured to perform binarization processing on the initial candidate region based on the current difference value to obtain a binarized image; The recognition module is further configured to determine foreground pixels based on the binarized image, and to determine candidate regions to be detected based on the foreground pixels.

17. The image detection apparatus as described in claim 16, characterized in that, The recognition module is further configured to determine foreground pixels based on the binarized image, and to determine candidate foreground regions based on the foreground pixels; The identification module is further configured to traverse the candidate foreground regions and use the traversed candidate foreground regions as the foreground regions to be detected. The recognition module is further configured to obtain the area of ​​the foreground region to be detected, and determine whether the area is greater than a preset area threshold, thereby obtaining a judgment result. The recognition module is further configured to determine the candidate region to be detected based on the judgment result after traversing the candidate foreground region.

18. The image detection apparatus as described in claim 14, characterized in that, The determining module is further configured to obtain the number of regions in the first candidate region set and determine whether the number of regions is greater than a preset number threshold. The determining module is further configured to delete the target candidate region in the first candidate region set when the number of regions is greater than a preset number threshold, and obtain a second candidate region set. The determining module is further configured to traverse the candidate regions in the second candidate region set and use the traversed candidate regions as candidate regions to be judged. The determining module is further configured to determine the region overlap based on the location information of the target candidate region and the location information of the candidate region to be judged; The determining module is further configured to determine the target human face region based on the region overlap, the target candidate region set, and the first candidate region set.

Images