Image processing method and device based on human body recognition and facial recognition

Abstract

The present application relates to an image processing method and device based on human body recognition and facial recognition, and a computer-readable storage medium. The image processing method comprises: a normal facial recognition step, comprising: performing facial recognition on an image to be subjected to recognition, and when it is recognized that there is a missing human face in the current image to be subjected to recognition, using a facial box, which corresponds to the missing human face and is in the previous frame of said image, as a facial box for the missing human face in the current frame and a subsequent frame, and proceeding to a human-face-missed recognition step; and the human-face-missed recognition step, comprising: for said image, performing target recognition in a mode where facial recognition of several frames and human body recognition of one frame are alternately performed, and when it is recognized that there is a human body, in said current image, which corresponds to the missing human face, updating the facial box for the missing human face on the basis of a human body box which corresponds to the missing human face and is in said current image, and continuing the human-face-missed recognition step. By means of the present invention, the stability and reliability of a facial display picture can be improved, thereby improving the user experience.

Description

Image processing method and device based on human body and face recognition

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent application No. 202311519720.4, filed on November 14, 2023, entitled “Image processing method and device based on human body and face recognition,” and the entire contents of that application are incorporated herein by reference. Technical Field

[0003] The present application relates to the field of image processing technology, and in particular to an image processing method, device, and computer-readable storage medium for human body and face recognition. Background Art

[0004] With the development of image processing technology, faces are recognized and tracked in many application scenarios. In existing technologies, when performing target recognition on the acquired image, a face frame is obtained through a face recognition model. Then, an image of the corresponding size is cropped from the face frame and displayed directly or after being spliced ​​with other images. In face tracking applications, a tracking algorithm is also performed based on the face frame of the current frame and the face frames in the historical data. For example, each face frame is marked with an ID. If the face frame in the current frame does not match the face frame in the historical data, it is added as a new face frame and a new ID is added. If the face disappears in a frame, the face frame corresponding to the face in the historical frame is first cropped. If the face is still not detected after a delay of several frames, the processing and display of the face are canceled.

[0005] However, in the case of a face disappearing temporarily, such as lowering the head or turning around, but the person corresponding to the face is still in the original position, some frames or even several consecutive frames may not recognize the face. When using this existing image processing method, it will cause unstable face display and discontinuous scene display problems, which will reduce the user or observer experience. For example, in the intelligent framing application scenario, the framing frame will be determined based on the recognized face image. If two faces are recognized, the framing frame that can contain the two faces is determined based on the positions of the two faces, and then the image of the corresponding size is cropped for display. If the disappeared face happens to be at the boundary of the historical framing frame, the framing frame will become smaller. When the face reappears later, the framing frame will return to its original size. In this way, the displayed image will be larger and smaller at times. For example, in scenarios where the speaker is locked and tracked during teaching or meetings, or the displayed speaker is manually switched, the image corresponding to the speaker will be selected and displayed based on the recognized facial image; if the face is not recognized for several consecutive frames, the speaker's portrait will disappear, but the speaker may still be in the same position, and the face is not recognized only due to actions such as lowering the head or turning around. When the face is restored later, the speaker's face can be recognized again, and the image will also display the speaker's portrait, causing the image to be unstable and intermittent. For example, in the application scenario of smart puzzles, the display area is divided into a corresponding number (which can be equal to or greater than the number of face images) of square frame layouts according to the number of face images currently detected. After each face image generates a framing boundary frame to capture the image, it is displayed in the corresponding square frame. For example, a single face is displayed in a panoramic frame, two faces are displayed in a left-right two-square grid, three faces are displayed in a left-center-right three-square grid, four faces are displayed in a field four-square grid, 5 to 6 faces are displayed in a 2*3 six-square grid, and 7 to 9 faces are displayed in a nine-square grid. When a face is temporarily not recognized, the number and layout of the square frames need to be readjusted according to the number of recognized faces. After the face is restored, the original square frame layout will be restored. This will cause an unstable image layout.

[0006] Therefore, it is urgent to propose a more stable and reliable image processing method based on face recognition.

[0007] Summary of the Invention

[0008] Based on the above situation, the main purpose of this application is to provide an image processing method, device and computer-readable storage medium based on human body and face recognition, which can increase the stability and reliability of face display images and enhance user experience.

[0009] To achieve the above objectives, the technical solutions adopted in this application are as follows:

[0010] The first aspect of the present application provides an image processing method based on human body and face recognition, comprising:

[0011] In the normal face recognition step, face recognition is performed on the image to be recognized to obtain a recognition result. When a missing face is detected in the current image to be recognized, the face frame corresponding to the missing face in the previous frame of the image to be recognized is used as the face frame of the missing face in the current frame and subsequent frames, and the process proceeds to the missing face recognition step.

[0012] The face disappearance recognition step performs target recognition on the image to be recognized in an alternating manner of several frames of face recognition and one frame of human body recognition to obtain a recognition result. When a human body corresponding to a disappeared face is recognized in the current image to be recognized, the face frame of the disappeared face is updated based on the human body frame corresponding to the disappeared face in the current image to be recognized, and the face disappearance recognition step is continued. When the disappeared face is recognized in the current image to be recognized, the process proceeds to the normal face recognition step.

[0013] When a face is recognized, the recognition result includes a face frame; when a body is recognized, the recognition result includes a body frame.

[0014] A second aspect of the present application provides an image processing device based on human body and face recognition, comprising:

[0015] A normal face recognition module is configured to perform a normal face recognition step, i.e., perform face recognition on the image to be recognized to obtain a recognition result. When a missing face is detected in the current image to be recognized, the face frame corresponding to the missing face in the previous frame of the image to be recognized is used as the face frame of the missing face in the current frame and subsequent frames, and the process proceeds to the missing face recognition step;

[0016] The face disappearance recognition module is used to perform the face disappearance recognition step, that is, to perform target recognition on the image to be recognized in an alternating manner of several frames of face recognition and one frame of human body recognition to obtain a recognition result; wherein, when a human body corresponding to the disappeared face is recognized in the current image to be recognized, the face frame of the disappeared face is updated based on the human body frame corresponding to the disappeared face in the current image to be recognized, and the face disappearance recognition step is continued; when the disappeared face is recognized in the current image to be recognized, the normal face recognition step is entered;

[0017] When a face is recognized, the recognition result includes a face frame; when a body is recognized, the recognition result includes a body frame.

[0018] The third aspect of the present application provides an image processing device based on human body and face recognition, including a processor and a memory, wherein the memory stores executable instructions, and the processor can execute the executable instructions to implement any image processing method described above.

[0019] A fourth aspect of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the image processing method as described in any one of the above items.

[0020] The present application alternately performs face recognition and body recognition when a face disappears. When a face disappears but the body corresponding to the face is still in place, the face replacement position of the disappeared face can be updated according to the corresponding body frame. For example, the disappeared face frame is replaced by the corresponding body frame or part of the body frame, and then the viewfinder or the size of the captured image is determined according to each face frame to obtain the current image to be displayed for display. The face can be tracked, thereby improving the stability of the displayed image and the continuity of the scene display, improving the user experience, and avoiding image layout or display instability caused by the temporary disappearance of the face. Body recognition is the processing of images to be recognized in different frames separately. That is, for each frame, either face recognition or body recognition is performed, and when the face disappears, several frames of face recognition and one frame of body recognition are used alternately. Therefore, even if the CPU operating efficiency is not high during body recognition and it occupies a lot of resources, for the entire image processing, compared with using body recognition and face recognition for each frame, it can greatly improve the CPU operating efficiency and reduce the CPU occupancy rate. Compared with the processing method of using only face recognition, it increases the system overhead less and greatly improves the user experience.

[0021] Other beneficial effects of the present application will be explained through the introduction of specific technical features and technical solutions in the specific implementation methods. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The preferred embodiments of the present application will be described below with reference to the accompanying drawings.

[0023] FIG1 is a flowchart of a preferred embodiment of an image processing method of the present application;

[0024] FIG2 is a flowchart of another preferred embodiment of an image processing method of the present application;

[0025] 3 to 5 are schematic diagrams of matching a face frame with a body frame in a preferred embodiment of an image processing method of the present application;

[0026] FIG6 and FIG7 are schematic diagrams of application scenarios of a preferred embodiment of an image processing method of the present application;

[0027] FIG8 is a flowchart of another preferred embodiment of an image processing method of the present application;

[0028] FIG9 is a system block diagram of a preferred embodiment of an image processing device of the present application;

[0029] FIG10 is a system block diagram of another preferred embodiment of an image processing device of the present application;

[0030] FIG11 is a system block diagram of a preferred embodiment of an image processing device of the present application. DETAILED DESCRIPTION

[0031] The present application is described below based on examples, but the present application is not limited to these examples. In the detailed description of the present application below, some specific details are described in detail. In order to avoid obscuring the essence of the present application, well-known methods, processes, procedures, and components are not described in detail.

[0032] Furthermore, persons of ordinary skill in the art will appreciate that the figures provided herein are for illustration purposes only and are not necessarily drawn to scale.

[0033] Unless the context clearly requires otherwise, throughout the specification and claims, the words "include," "comprising," and similar words should be construed in an inclusive sense rather than an exclusive or exhaustive sense; that is, in the sense of "including but not limited to."

[0034] In the description of this application, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood to indicate or imply relative importance. In addition, in the description of this application, unless otherwise specified, "plurality" means two or more.

[0035] This application provides an image processing method based on human body and face recognition, which uses human body recognition to assist face recognition for target recognition, including:

[0036] Normal face recognition step S20: performing face recognition on the image to be recognized to obtain a recognition result. If a face is recognized in the current image to be recognized, the recognition result includes a face frame corresponding to the face. If a missing face is recognized in the current image to be recognized, the face frame corresponding to the missing face in the previous frame of the image to be recognized is used as the face frame of the missing face in the current frame and subsequent frames, and the process proceeds to missing face recognition step S40.

[0037] In this step, face recognition is performed on each frame of the image to be processed. This is specifically performed through a face recognition neural network. That is, the image to be processed is input into the face recognition neural network, and the neural network outputs a recognition result corresponding to the frame of the image to be processed. If the neural network recognizes a face, the recognition result includes face information. This face information includes facial features and face location, which can be represented in the form of a face frame, specifically the area on the image to be recognized that corresponds to the face. It should be noted that the neural network may recognize only one face, in which case the recognition result only includes the face information of that face. Of course, it is also possible to recognize multiple faces, in which case the recognition result includes multiple faces, and each face information will correspond to a face frame.

[0038] For ease of operation, when a face is detected in the image to be recognized, an ID is assigned to each face to represent its facial features. If multiple faces are detected, an ID is assigned to each face. Faces recognized in subsequent frames are matched against the stored faces. Faces that do not match are considered new faces and a new ID is assigned to them. The face ID and the corresponding face frame are stored.

[0039] When the face recognition in the current image to be recognized does not include missing faces, that is, there are no missing faces relative to the existing IDs, the face frames corresponding to the faces in the recognition results are directly used as the face frames of the current frame. When missing faces are recognized in the current image to be recognized, that is, there are missing faces in the current image to be recognized compared to the previous frame to be recognized, the recognition results of the current image to be recognized are specifically matched with the stored faces. If there is no matching face in the recognition results for some IDs, the face corresponding to the ID is considered missing and recorded as a missing face. For the disappeared face, the processing is not directly exited. Instead, the face frame corresponding to the disappeared face in the previous frame of the image to be recognized is used as the face frame of the disappeared face in the current frame and subsequent frames. In this way, in the case of a face that disappears briefly, each frame still has a corresponding face frame. That is to say, for the image to be recognized with a disappeared face, the other face frames are updated in real time and are all taken from the area in the image to be recognized in the current frame (that is, the face frame obtained in the recognition result), while the face frame of the disappeared face is selected from the area of ​​the image to be recognized in the last frame where the disappeared face exists as the face frame of its current frame.

[0040] In step S40, the face disappearance recognition step is performed on the image to be recognized by alternating between several frames of face recognition and one frame of human body recognition to obtain the face recognition position of each frame. When a human body corresponding to the disappeared face is recognized in the current image to be recognized, it is considered that the disappeared face is merely obscured. The face frame of the disappeared face is updated based on the human body frame corresponding to the disappeared face in the current image to be recognized, and the face disappearance recognition step is continued. When the disappeared face is recognized in the current image to be recognized, the process proceeds to the normal face recognition step.

[0041] In this step, face recognition and body recognition are performed alternately on multiple frames of the image to be processed. The number of frames for face recognition is greater than the number of frames for body recognition. Specifically, after performing face recognition on several frames, body recognition is performed on one frame. Several more frames of face recognition are then performed on another frame of body recognition, and this cycle repeats. Face recognition is still performed using a face recognition neural network, in the same manner as face recognition in normal face recognition step S20, and will not be further described here. Body recognition is performed using a body recognition neural network. Specifically, the image to be processed is input to the body recognition neural network, which outputs a recognition result corresponding to the image to be processed. If the neural network recognizes a person, the recognition result includes body information. This body information includes the person's location, which can be represented by a body frame, specifically the area on the image to be recognized that corresponds to the person. It should be noted that the neural network may recognize only one person, in which case the recognition result only includes that person's information. Alternatively, the neural network may recognize multiple people, in which case the recognition result includes multiple body information, each corresponding to a body frame.

[0042] When a human body is detected in the current image to be recognized, a determination is made as to whether the human body corresponds to the missing face. If so, the face frame of the missing face is determined based on the human body frame corresponding to the missing face. This can be done by directly using the human body frame as the face frame of the missing face, or by selecting a portion of the human body frame, such as the area corresponding to the position of the missing face. That is, when executing step S40 of identifying the missing face, for each face frame (a frame undergoing face recognition), the face frame of the missing face is determined based on the face frame corresponding to the face in the previous frame, while the other face frames are determined based on the face frame identified in the current frame. For each human body frame (a frame undergoing human body recognition), if a human body corresponds to the missing face, the face frame of the missing face is determined based on the human body frame corresponding to the missing face, while the other face frames are determined based on the face frame of the previous frame. If no human body corresponds to the missing face, the face frame of the previous frame is determined based on the face frame of the previous frame. In this case, the face is still considered to be in the missing state, i.e., it is still in the face disappearance mode, and step S40 of identifying the missing face is still executed for subsequent frames.

[0043] It is worth noting that in this application, for each frame of the image to be recognized, either face recognition or body recognition is performed. For the convenience of expression, the image to be recognized for face recognition can be recorded as a face frame, and the image to be recognized for body recognition can be recorded as a body frame. The face frame uses the neural network corresponding to the face, and the body frame uses the neural network corresponding to the body. Among them, when a face is recognized, the recognition result includes a face frame, and when a body is recognized, the recognition result includes a body frame. That is, if a face is recognized in a face frame, the recognition result of the face frame includes a face frame; if a body is recognized in a body frame, the recognition result of the body frame includes a body frame.

[0044] The present application sets a normal face mode and a face disappearance mode. In the normal face mode, a normal face recognition step S20 is performed, and in the face disappearance mode, a face disappearance recognition step S40 is performed. By alternating face recognition and body recognition when a face disappears, the face frame of the disappeared face can be updated according to the corresponding body frame when a face disappears but the body corresponding to the face is still in place. For example, the disappeared face frame is replaced by the corresponding body frame or part of the body frame, and then the framing frame or the size of the captured image is determined according to each face frame to obtain the current image to be displayed for display. The face can be tracked, thereby improving the stability of the displayed image and the continuity of the scene display, improving the user experience, and avoiding the problem of missing images due to the short face. The image layout or display instability caused by the temporary disappearance of the human face; and face recognition and body recognition are processed on different frames of the image to be recognized respectively, that is, for each frame, either face recognition or body recognition is performed, and when the face disappears, several frames of face recognition and one frame of body recognition are used alternately. Therefore, even if the CPU operating efficiency is not high during body recognition and it occupies a lot of resources, for the entire image processing, compared with using body recognition and face recognition for each frame, it can greatly improve the CPU operating efficiency and reduce the CPU occupancy rate. Compared with the processing method of using face recognition alone, it increases the system overhead less and greatly improves the user experience.

[0045] It can be understood that in the above image processing method, the image to be identified is first obtained, and then the image to be identified is subjected to target recognition through a normal face recognition step or a face disappearance recognition step.

[0046] In the face disappearance recognition step S40, when the human body frame recognizes that the current image to be recognized includes a human body, it is determined whether the human body is the human body corresponding to the disappeared face. In some embodiments, the face frame of the disappeared face is combined with the human body frame of the current human body frame for judgment. Specifically, it can be directly determined whether the face frame of the disappeared face is located within the human body frame. If so, it is considered that the disappeared face has a corresponding human body, and the human body frame is the human body frame corresponding to the disappeared face. In other embodiments, when determining whether the human body is the human body corresponding to the disappeared face, the recognition result of the previous face frame is also combined. Specifically, in the face disappearance recognition step S40,

[0047] When a human body is detected in the current image to be identified, the human body frame is identified, combined with the face frame of the previous frame to be identified and the face frame of the missing face, to determine whether the missing face has a corresponding human body.

[0048] If the face frame of the missing face can be successfully matched with at least one body frame, it is considered that the missing face has a corresponding body.

[0049] That is to say, when the current human body frame recognizes a human body, it may be one human body or multiple human bodies. In this embodiment, the human body frames corresponding to these human bodies are all used as matching objects, and the face frame in the previous frame recognition result and the face frame of the disappeared face are matched with these matching objects. Depending on the matching rules, the same face frame may match multiple human body frames. If the face frame of the disappeared face in the matching result can be successfully matched with at least one human body frame, it is considered that the disappeared face has a corresponding human body, and then the corresponding face frame in the current frame is determined based on the human body frame that successfully matches the disappeared face frame.

[0050] By combining the method of judging whether the disappeared face has a corresponding human body with the face frame recognized in the previous frame, the matching accuracy of the disappeared face and the human body frame can be improved, thereby making it more accurate to judge whether the human body corresponding to the disappeared face has truly disappeared or has disappeared temporarily. At the same time, in the display of two adjacent frames of images, it can be ensured that the picture displayed at the place where the face disappeared in the previous frame and the picture displayed by the face frame determined by the human body frame in the next frame can be more coordinated, that is, the actual face is displayed at the place where the face disappeared in the previous frame, and the human body corresponding to the face is lowering or turning the head is displayed at the place where the face disappeared in the next frame. By accurately matching the disappeared face with the human body frame that replaces its face frame, it can appear to the user that the human body whose face disappeared is indeed lowering or turning the head, thereby further improving the continuity of the picture.

[0051] In some embodiments of the present application, when matching a face frame and a body frame, the face frame and the body frame may be considered to be successfully matched as long as most of the face frame, such as 80%, 90% or a larger area, is located in the body frame; or the face frame and the body frame may be considered to be successfully matched only when the entire face frame is located in the body frame; or the face frame and the body frame may be considered to be successfully matched only when the face frame is located in the upper middle area of ​​the body frame. During the matching, each face frame, including the face frame corresponding to the missing face, can be matched once with each body frame as the matching object, that is, each face frame is matched once with each body frame, and then the matched face frames and body frames are determined, and then it is judged from these matched face frames and body frames whether there is a face frame corresponding to the missing face; face frames and body frames in a smaller range can also be selected for matching. For example, if the distribution range of each face frame and body frame is relatively large, in order to reduce the burden of data processing and improve the efficiency of matching judgment, the area including the face frame corresponding to the missing face can be selected as the judgment area. Of course, the area of ​​the judgment area is larger than the area of ​​a single face frame and the area of ​​the body frame. An area containing multiple body frames can be selected, such as an area containing three, four or more body frames, but smaller than the area of ​​the entire image to be recognized. In this way, only the multiple face frames and body frames in the judgment area are matched.

[0052] In a preferred embodiment of the present application, in the face disappearance recognition step S40, when a human body is recognized in the current image to be recognized, each face frame of the previous frame is matched with each body frame of the current frame. If there is a complete face frame in a body frame, the face frame is considered to match the body frame. As shown in Figure 3, there is only one face frame a in the body frame A, and the face frame a is considered to match the body frame A; if there are multiple face frames in a body frame, the face frame located in the body frame and close to the upper center is selected to match the body frame. As shown in Figure 4, there are two face frames in the body frame A, namely face frame a and face frame b, and the face frame a located in the body frame A close to the upper center is selected to match the body frame A. Among them, in the above matching, if the face frame selected to match the human body frame already has another matching human body frame, it means that the face frame falls into multiple human body frames at the same time. In this case, the human body frame is re-matched to the face frame. According to the positional relationship between the human body features and the facial features, one of the multiple human body frames in which the face frame is located at the upper center is selected to match the face frame, and the remaining human body frames are re-matched to the face frames. As shown in Figure 5, there is only one face frame in human body frame A, namely, face frame a. According to the above rules, face frame a should be used as the face frame matching with human body frame A, but face frame a has been matched with human body frame B before, namely, face frame a falls into human body frame A and human body frame B at the same time. In this case, face frame a is re-matched to the human body frame. In human body frame A and human body frame B, face frame a is obviously located at the upper center of human body frame B. Therefore, face frame a is selected to match with human body frame B, and human body frame A is re-matched to the face frame. With this matching method, each face frame can only be matched with one human frame. If a face frame corresponding to a missing face exists among these matching face frames and human frames, the face frame corresponding to the missing face in the current frame is determined based on the matching human frame. This method can further improve the accuracy of determining the face frame of the missing face, thereby making the display of each frame in the picture more reasonable.

[0053] In the face disappearance recognition step S40, the face frame may also recognize other disappeared faces, that is, not the disappeared faces recognized in the normal face recognition step S20. If a newly disappeared face is recognized in the current image to be recognized, the face frame corresponding to the newly disappeared face in the previous frame of the image to be recognized is used as the face frame of the newly added disappeared face in the current frame and subsequent frames, and the face disappearance recognition step is continued. For ease of description, the disappeared face recognized in the normal face recognition step S20 is referred to as the first face, and the newly disappeared face recognized in the face disappearance recognition step S40 is referred to as the second face. Then, in the face disappearance recognition step S40, when the face frame recognizes the presence of the second face, for the current face frame, the face frames of the first face and the second face are both selected from the corresponding face frames in the previous frame, and the face frames of the remaining faces are selected from the face frames in the recognition result of the current face frame. That is to say, as long as there are new missing faces in the current frame, the face frames of the new missing faces will select the corresponding face frames in the previous frame. When the face frames are matched with the human frames when the human body is recognized subsequently, the face frames of the missing faces will also be matched.

[0054] In some embodiments of the present application, in the matching of a human body frame and a face frame, due to the limitations of various matching rules, it is allowed that there are face frames or human body frames exist alone, that is, there may be face frames that are not matched to human body frames, and there may also be human body frames that are not matched to face frames.

[0055] In one embodiment of the present application, as shown in FIG2 , the image processing method further includes:

[0056] No face and no body recognition step S60, performing target recognition on the image to be recognized in an alternating manner of one frame of face recognition and one frame of body recognition to obtain a recognition result, and executing normal face recognition step S20 when a face is recognized; and executing human body and no face recognition step S80 when a human body is recognized;

[0057] In the step S80 of human-face recognition, target recognition is performed on the image to be recognized in an alternating manner of one frame of face recognition and several frames of human body recognition to obtain a recognition result; when a face is recognized in the current image to be recognized, the process goes to the normal face recognition step S20.

[0058] In this embodiment, similarly, either face recognition or body recognition is performed on each frame of the image to be processed, and the methods of performing face recognition and body recognition and the obtained recognition results can refer to the face recognition and body recognition in the normal face recognition step S20 and the face disappearance recognition step S40, and will not be repeated here.

[0059] In some embodiments of the present application, in the step S60 for recognizing faces and bodies without faces, face recognition and body recognition are performed alternately on multiple frames of images to be processed. The number of face frames and body frames in a cycle is equal. Specifically, one frame of face recognition is performed after one frame of body recognition, and the cycle repeats. In the step S80 for recognizing faces with or without bodies, face recognition and body recognition are still performed alternately on multiple frames of images to be processed. However, the number of face frames in a cycle is smaller than the number of body frames. Specifically, one frame of face recognition is performed after several frames of body recognition, and the cycle repeats.

[0060] In this embodiment, a no-face and no-body mode and a human-body and no-face mode are added. In the no-face and no-body mode, the no-face and no-body recognition step S60 is executed. In the human-body and no-face mode, the human-body and no-face recognition step S80 is executed. Specifically, at the beginning of target recognition, the no-face and no-body recognition step S60 is executed first. If a face is recognized, the process proceeds to the normal face recognition step S20. If a human body is recognized, the process proceeds to the human-body and no-face recognition step S80. Otherwise, the no-face and no-body recognition step S60 is executed continuously. In this way, even when a face is not recognized but the human body corresponding to the face actually exists, the presence of a human body can be promptly determined, thereby further increasing the continuity between image frames. Furthermore, by alternating between multiple human body frames and one face frame, the continuity of human body movements can be ensured while the face is detected promptly to accurately determine facial features.

[0061] In some embodiments of the present application, in the human-face-but-no-human face recognition step S80 , if no human body is detected in the current image to be recognized, the process proceeds to the human-face-but-no-human face recognition step S60 .

[0062] In some embodiments of the present application, the step S80 of recognizing a human body with or without a face further includes: after performing human body recognition on the current image to be recognized, that is, after performing human body recognition on the human body frame, determining a face frame corresponding to the human face based on the human body frame of the current image to be recognized. That is, after performing human body recognition on the human body frame, although the recognition result includes a human body frame, the human body frame must correspond to a human face. In order to increase the continuity of the displayed image, the face frame corresponding to the human face is determined based on the human body frame as the face frame of the current frame. Specifically, the entire human body frame can be selected as the face frame of the current frame. In some embodiments of the present application, the upper center area of ​​the human body frame is selected as the face frame of the current frame. After performing face recognition on the current image to be recognized, that is, after performing face recognition on the face frame, if no face is recognized, the face frame corresponding to the previous frame to be recognized is used as the face frame of the current frame, that is, the face frame after processing the previous human body frame; if a face is recognized, the face frame corresponding to the recognition result is used as the face frame of the current frame.

[0063] It should be noted that in step S60 of recognizing a person without a face, when a person is recognized, an identification number, such as an ID number, is also assigned to the person. After proceeding to step S80 of recognizing a person without a face, if person recognition is performed on the current image to be recognized and a person is recognized, it is determined whether the person is an existing person. If so, step S80 of recognizing a person without a face is continued. If the person is a newly added person, a new identification number is assigned to the newly added person. If face recognition is performed on the current image to be recognized and a face frame is recognized, it is determined whether the face frame corresponds to an existing person. If so, the face frame is used as the face frame of the current frame, i.e., replacing the face frame previously determined by the body frame, retaining the identification number corresponding to the existing person, and then proceeding to step S20 of recognizing a normal face. If not, the face frame is used as the face frame of the current frame, the identification number corresponding to the existing person is deleted, and a new identification number is assigned to the face frame, and then proceeding to step S20 of recognizing a normal face.

[0064] The no-face and no-body recognition step S60 also includes: after performing face recognition or body recognition on each frame, if the face disappearance recognition step S40 has not been performed before this step or the corresponding face frame does not exist in the historical frame, then the face frame is not output; if the corresponding face frame exists in the historical frame, then the face frame of the most recent historical frame is used as the face frame of the current frame.

[0065] In some embodiments of the present application, in the face disappearance recognition step S40, a preset number of face recognition frames and a frame of human body recognition may be cycled, but the human body corresponding to the disappeared face has not been recognized. In this case, it is considered that the human body corresponding to the disappeared face has disappeared, and the processing of the disappeared face in subsequent frames can be canceled. Based on the removal of the disappeared face, other faces other than the disappeared face are processed according to a normal image processing method. The face disappearance recognition step S40 may be exited and the normal face recognition step S20 or the no-face and no-human body recognition step S60, or other steps, may be directly entered. Which step to execute for the subsequent frame is determined based on the recognition result of the face frame of the previous frame. Specifically, the face disappearance recognition step S40 further includes:

[0066] When several frames of face recognition and one frame of body recognition have been performed for a preset number of times and the body corresponding to the disappeared face is still not recognized, the step to be executed is determined based on the recognition results of the current image to be recognized and the recognition results of the previous frame to be recognized. If the previous frame to be recognized did not recognize a face and the current frame did not recognize a body, the process proceeds to step S60 of no face and no body; if the previous frame to be recognized did not recognize a face and the current frame recognizes the body corresponding to the disappeared face in the abnormal face recognition step S20, the process proceeds to step S80 of recognition of body but no face; if the previous frame to be recognized recognizes other faces in addition to the disappeared face in the normal face recognition step S20, the process proceeds to step S20 of recognition of normal face; if the previous frame to be recognized recognizes other disappeared faces in addition to the disappeared face in the normal face recognition step S20, the process continues to execute step S40 of recognition of the disappeared face.

[0067] That is to say, after looping a preset number of face frames and a body frame, such as three, four or more times, if the body corresponding to the disappeared face is still not recognized, it means that the face has indeed exited normally. For other faces, normal image processing can be used, and the face disappearance recognition step S40 may be exited. The specific step or mode to enter needs to be determined in combination with the recognition result of the face frame of the previous frame. If the previous face frame did not recognize a face and the current body frame also did not recognize a body, it means that the state is no face and no body, and the subsequent frame executes the no face and no body step S60; if the previous face frame did not recognize a face and the current body frame recognizes the body corresponding to the disappeared face in the abnormal face recognition step S20, then the process goes to the body and no face step S80; if the previous face frame recognizes all other faces except the disappeared face in the normal face recognition step S20, since the disappeared face exited normally, its processing is canceled. Therefore, it means that all current faces have been recognized, and the process goes to the normal face recognition step S20 for execution. If the previous face frame recognizes that other faces have disappeared (i.e., the second face) in addition to the face that disappeared in the normal face recognition step S20 (i.e., the first face), then this indicates that there are still missing faces in the current face recognition step S40. Therefore, the face disappearance recognition step S40 is still executed. In this way, the high CPU usage and image processing complexity caused by the long execution of the face disappearance recognition step S40 can be avoided. In this step, when a human body is recognized, it is necessary to match the human body frame with the human body frame, and even determine the corresponding face frame based on the human body frame, which undoubtedly increases the system burden.

[0068] In some embodiments of the present application, in the face disappearance recognition step S40, i.e., the face disappearance mode, even if a human body that matches the disappearing face (i.e., the disappearing face in the normal face recognition step S20) is recognized, as long as the disappearing face is not recognized, regardless of whether other faces in the normal face recognition step S20 are recognized in the previous frame or the next frame, or other newly added faces are recognized, the face disappearance recognition step S40 continues to be executed, i.e., it remains in the face disappearance mode, until the disappearing face is recognized, then it switches to the normal face mode and executes the normal face recognition step S20.

[0069] Specifically, in this embodiment, the image processing method of the present application is shown in FIG8 ,

[0070] In the no-face and no-body mode, that is, executing the no-face and no-body recognition step S60, when a face is recognized, it switches to the normal face recognition mode, that is, executing the normal face recognition step S20; when a human body is recognized, it switches to the human body and no-face mode, that is, executing the human body and no-face recognition step S80.

[0071] In the human body but no face mode, that is, executing the human body but no face recognition step S80, when a face is recognized, it switches to the normal face mode, that is, executing the normal face recognition step S20; when no human body is recognized in the human body frame, it switches to the no face and no human body mode, that is, executing the no face and no human body recognition step S60.

[0072] In the normal face mode, that is, performing the normal face recognition step S20, when a disappeared face is recognized, the mode is switched to the disappeared face mode, that is, performing the disappeared face recognition step S40.

[0073] In the face disappearance mode, i.e., the face disappearance recognition step S40, when a disappearing face identified in the normal face mode is recognized, the process switches to the normal face mode, i.e., the normal face recognition step S20 is executed. When a human body corresponding to the disappearing face is recognized (i.e., in the human body frame), regardless of whether the previous or next frame (i.e., the face frame) contains other faces besides the face identified in the normal face recognition step S20, the process remains in the face disappearance mode, and the disappearance face recognition step S40 is continued. When a preset number of face recognition frames and a body recognition frame are cycled through, but the disappeared face identified in the normal face mode is still not identified, if the previous frame did not recognize a face and the current frame does not recognize a body, then the mode is switched to no face and no body mode, i.e., the no face and no body step S60 is executed; if the previous frame did not recognize a face and the current frame recognizes a body corresponding to the disappeared face in the abnormal face recognition step S20 (i.e., another body), then the mode is switched to body with body but no face mode, i.e., the body with body but no face recognition step S80 is executed; if the previous frame of the image to be recognized recognizes other faces in addition to the disappeared face in the normal face recognition step S20, then the mode is switched to normal face mode, i.e., the normal face recognition step S20 is executed; if the previous frame of the image to be recognized recognizes other disappeared faces in addition to the disappeared face in the normal face recognition step S20, then the mode is still in the disappeared face mode, and the disappeared face recognition step S40 is continued to be executed.

[0074] It should be noted that the number of cycles is also recorded when executing the face disappearance recognition step S40. Specifically, the number of cycles of several face frames and one body frame can be directly recorded, or the number of frames of executing the face disappearance recognition step S40 can be directly recorded.

[0075] In the above embodiments, after target recognition is performed on each frame, the image to be displayed is determined based on the face frame corresponding to the current frame, and then the image to be displayed is displayed through the display device. When the image processing method of the above embodiments is not run on the display device, the image to be displayed needs to be sent to the display device.

[0076] In one embodiment, the normal face recognition step S20, the disappeared face recognition step S40, and the human body and non-human face recognition step S80 also include: after target recognition is performed on each frame of the image to be recognized, the current image to be displayed is determined according to the face frame corresponding to the current image to be recognized, that is, in these steps, after the face frame corresponding to each frame is obtained, the image to be displayed will be determined according to these face frames.

[0077] The non-face and non-human body recognition step S60 also includes: after target recognition is performed on each frame of the image to be recognized, the current image to be displayed is determined according to the face frame corresponding to the image to be recognized in the most recent historical frame with a face, which is equivalent to using the image to be displayed corresponding to the image to be recognized in the most recent historical frame with a face as the current image to be displayed.

[0078] Specifically, the method for determining the image to be displayed based on the face frame can be processed according to different application scenarios. For example, when there are multiple face frames, a union or multi-grid splicing can be selected. For example, when there is only a single face frame, the face frame can be directly used as the image to be displayed. Alternatively, a larger or smaller framing frame can be determined based on the face frame as the image to be displayed. As shown in Figure 6, in the smart framing application scenario, two face frames, face frame c and face frame d, are obtained in the current frame. Based on these two face frames, the larger area can be selected as the image to be displayed, area E. As shown in Figure 7, in the intelligent puzzle application scenario, when the current frame has only one face frame, the face frame is displayed in a single grid, that is, a single grid image is generated, as shown in the upper left figure. When the current frame has two face frames, the two face frames are spliced ​​in a two-grid manner according to the size of the display area, that is, a two-grid image is generated, as shown in the upper right figure. When the current frame has only four face frames, the face frames are displayed in a four-grid manner, that is, a four-grid image is generated, as shown in the lower left figure. When the current frame has five or six face frames, the two face frames are spliced ​​in a six-grid manner according to the size of the display area, that is, a six-grid image is generated, as shown in the lower right figure. Of course, there are other application scenarios, which will not be listed here one by one.

[0079] Obviously, by adopting the image processing method of the present application, in the above-mentioned scenarios, even if the face disappears briefly, the face frame corresponding to the position determined by the human body frame will be displayed at the disappearing face. Therefore, it will not cause the image to be displayed to fluctuate in size, or the grid layouts to switch back and forth, and the display images of each face have good continuity in each frame, thereby improving the user experience.

[0080] The present application also provides an image processing device based on human body and face recognition. As shown in FIG9 , the image processing device 100 includes:

[0081] Normal face recognition module 102 is configured to execute normal face recognition step S20, i.e., perform face recognition on the image to be recognized to obtain a recognition result. When a missing face is detected in the current image to be recognized, the face frame corresponding to the missing face in the previous frame of the image to be recognized is used as the face frame of the missing face in the current frame and subsequent frames, and the process proceeds to the missing face recognition step.

[0082] The face disappearance recognition module 104 is configured to execute the face disappearance recognition step S40, i.e., perform target recognition on the image to be recognized by alternating between several frames of face recognition and one frame of human body recognition to obtain a recognition result. When a human body corresponding to the disappeared face is detected in the current image to be recognized, the disappeared face is deemed to be merely occluded. The human body frame corresponding to the disappeared face in the current image to be recognized is updated, and the face disappearance recognition step is continued. When the disappeared face is recognized in the current image to be recognized, the process proceeds to the normal face recognition step.

[0083] In some embodiments of the present application, when a face is recognized, the recognition result includes a face frame; when a body is recognized, the recognition result includes a body frame.

[0084] Referring to FIG10 , the image processing apparatus further includes:

[0085] The no-face and no-body recognition module 106 is configured to perform the no-face and no-body recognition step S60, i.e., perform target recognition on each frame of the image to be recognized by alternating between one frame of face recognition and one frame of body recognition, and perform the normal face recognition step when a face is recognized; and perform the human body and no-face recognition step when a human body is recognized;

[0086] The human-face recognition module 108 is used to execute the human-face recognition step S80, that is, to perform target recognition on each frame of the image to be recognized in a manner of alternating one frame of face recognition and several frames of human recognition to obtain the human recognition position; when a face is recognized in the current image to be recognized, the normal face recognition step is entered.

[0087] When the normal face recognition step S20 also includes other steps, the normal face recognition module 102 is also used to execute these steps. Similarly, when the face disappearance recognition step S40, the no-face and no-body recognition step S60, and the body and no-face recognition step S80 also include other steps, the face disappearance module 104, the no-face and no-body module 106, and the body and no-face module 108 are also used to execute other steps corresponding to these steps. In the image processing device, the methods of body recognition and face recognition are the same as in the aforementioned image processing method, and will not be elaborated here.

[0088] In some embodiments of the present application, “several frames” herein refers to two frames, three frames or more frames.

[0089] The present application also provides an image processing device based on human body and face recognition. Please refer to FIG11 . The device 200 includes a processor 202 and a memory 204 .

[0090] In some implementations of the present application, the processor 202 is configured to execute the program 206 , and specifically may execute the relevant steps in the above-mentioned embodiment of the image processing method.

[0091] Specifically, program 206 may include computer-executable instructions.

[0092] The processor 202 may be a central processing unit (CPU), a specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the image processing device may be processors of the same type, such as one or more CPUs, or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0093] The memory 204 is used to store the program 206. The memory 204 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0094] The present application also provides a chip suitable for an image processing system for target recognition. The chip stores an instruction set, which, when executed, can instruct a software update device to implement the operation of the image processing method as in any of the above embodiments.

[0095] In addition, the present application also provides a computer-readable storage medium, which stores a computer program, wherein when the computer program runs, the device where the computer-readable storage medium is located is controlled to execute the image processing method described in any of the aforementioned embodiments.

[0096] It should be noted that the computer-readable storage medium described in the embodiments of the present disclosure is not limited to the embodiments given above, and can also be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the embodiments of the present disclosure, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device.

[0097] It will be understood by those skilled in the art that, under the premise of no conflict, the above-mentioned preferred embodiments can be freely combined and superimposed. Among them, the flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or part of the code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. The numbering of each step in this article is only for the convenience of description and reference, and is not used to limit the order of execution. The specific execution order is determined by the technology itself, and those skilled in the art can determine various allowable and reasonable orders based on the technology itself.

[0098] It should be noted that the use of step numbers (letters or numbers) in this application to refer to certain specific method steps is solely for the purpose of descriptive convenience and brevity, and is in no way intended to limit the order of these method steps. Those skilled in the art will understand that the order of the relevant method steps should be determined by the technology itself and should not be unduly limited by the presence of step numbers. Those skilled in the art can determine various permissible and reasonable step orders based on the technology itself.

[0099] Those skilled in the art will appreciate that, provided there is no conflict, the above preferred solutions can be freely combined and superimposed.

[0100] It should be understood that the above-mentioned embodiments are merely illustrative and not restrictive. Without departing from the basic principles of the present application, various obvious or equivalent modifications or substitutions that can be made by those skilled in the art to the above-mentioned details will be included in the scope of the claims of the present application.

Claims

1. An image processing method based on human body and face recognition, comprising: Normal face recognition step, performing face recognition on the image to be recognized to obtain a recognition result. When it is recognized that there is a missing face in the current image to be recognized, the face frame corresponding to the missing face in the previous frame of the image to be recognized is used as the face frame of the missing face in the current frame and subsequent frames, and the process proceeds to the face disappearance recognition step; The face disappearance recognition step is to perform target recognition on the image to be recognized in an alternating manner of several frames of face recognition and one frame of human body recognition to obtain a recognition result; wherein, when it is recognized that there is a human body corresponding to the disappeared face in the current image to be recognized, the face frame of the disappeared face is updated according to the human body frame corresponding to the disappeared face in the current image to be recognized, and the face disappearance recognition step is continued; when the disappeared face is recognized in the current image to be recognized, the normal face recognition step is entered; When a face is recognized, the recognition result includes a face frame; when a body is recognized, the recognition result includes a body frame.

2. The image processing method according to claim 1, wherein: In the face disappearance recognition step, When a human body is identified in the current image to be identified, judging whether the disappeared face has a corresponding human body according to the identified human body frame, combined with the identified face frame of the previous frame of the image to be identified and the face frame of the disappeared face; If the face frame of the disappeared face can be successfully matched with at least one human body frame, it is considered that the disappeared face has a corresponding human body, and the face frame corresponding to the disappeared face in the current frame is determined according to the human body frame matching the disappeared face frame.

3. The image processing method according to claim 2, wherein: In the face disappearance recognition step, when a human body is recognized in the current image to be recognized, each face frame of the previous frame is matched with each human body frame of the current frame. If there is a complete face frame in a human frame, the face frame is considered to match the human frame; If there are multiple face frames in a human frame, a face frame located in the human frame and close to the upper center is selected to match the human frame; Among them, if the face frame selected to match the human body frame already has another matching human body frame, the human body frame is re-matched to the face frame, and one of the multiple human body frames in which the face frame is located in the upper center position is selected to match the face frame, and the remaining human body frames are re-matched to the face frame.

4. The image processing method according to claim 1, wherein: In the face disappearance recognition step, if a newly disappeared face is recognized in the current image to be recognized, the face frame corresponding to the newly disappeared face in the previous frame of the image to be recognized is used as the face frame of the newly disappeared face in the current frame and subsequent frames, and the face disappearance recognition step is continued.

5. The image processing method according to any one of claims 1 to 4, further comprising: In the step of recognizing a human body without a face, the target recognition is performed on the image to be recognized in an alternating manner of one frame of face recognition and one frame of human body recognition to obtain a recognition result. When a face is recognized, a normal face recognition step is performed; when a human body is recognized, a human body without a face recognition step is performed; In the step of face recognition with or without a human body, target recognition is performed on the image to be recognized in an alternating manner of one frame of face recognition and several frames of human body recognition to obtain a recognition result; when a face is recognized in the current image to be recognized, the normal face recognition step is entered.

6. The image processing method according to claim 5, wherein: The face disappearance recognition step also includes: When a preset number of face recognition frames and a body recognition frame have been performed and still no body corresponding to the disappeared face is recognized, if the previous frame of the image to be recognized did not recognize a face and the current frame did not recognize a body, then the process proceeds to the no face and no body step; if the previous frame of the image to be recognized did not recognize a face and the current frame recognizes a body that does not correspond to the disappeared face, then the process proceeds to the no face but body step; if the previous frame of the image to be recognized recognized other faces except the disappeared face, then the process proceeds to the normal face recognition step; if the previous frame of the image to be recognized recognized other disappeared faces except the disappeared face, then the process continues to execute the disappeared face recognition step.

7. The image processing method according to claim 5, wherein: The step of human-body-or-no-face recognition also includes: after performing human body recognition on the current image to be recognized, determining a face frame corresponding to the face based on the human body frame of the current image to be recognized; after performing face recognition on the current image to be recognized, if no face is recognized, using the face frame corresponding to the previous frame as the face frame of the current frame; if a face is recognized, using the face frame in the recognition result as the face frame of the current frame.

8. The image processing method according to claim 7, wherein: The normal face recognition step, the face disappearance recognition step, and the human body and non-human face recognition step further include: after performing target recognition on each frame of the image to be recognized, determining the current image to be displayed according to the face frame corresponding to the current image to be recognized; The step of identifying no human face or human body also includes: after performing target recognition on each frame of the image to be identified, determining the current image to be displayed according to the face frame corresponding to the image to be identified in the most recent historical frame with a human face.

9. An image processing device based on human body and face recognition, comprising: A normal face recognition module is used to perform a normal face recognition step, that is, to perform face recognition on the image to be recognized and obtain a recognition result. When a missing face is recognized in the current image to be recognized, a face frame corresponding to the missing face in the previous frame of the image to be recognized is used as a face frame of the missing face in the current frame and subsequent frames, and the process proceeds to the face disappearance recognition step; The face disappearance recognition module is used to perform the face disappearance recognition step, that is, to perform target recognition on the image to be recognized in an alternating manner of several frames of face recognition and one frame of human body recognition to obtain a recognition result; wherein, when it is recognized that there is a human body corresponding to the disappeared face in the current image to be recognized, the face frame of the disappeared face is updated according to the human body frame corresponding to the disappeared face in the current image to be recognized, and the face disappearance recognition step is continued; when the disappeared face is recognized in the current image to be recognized, the normal face recognition step is entered; When a face is recognized, the recognition result includes a face frame; when a body is recognized, the recognition result includes a body frame.

10. The image processing device according to claim 9, wherein: The face disappearance recognition module is also used for: When a human body is identified in the current image to be identified, judging whether the disappeared face has a corresponding human body according to the identified human body frame, combined with the identified face frame of the previous frame of the image to be identified and the face frame of the disappeared face; If the face frame of the disappeared face can be successfully matched with at least one human body frame, it is considered that the disappeared face has a corresponding human body, and the face frame corresponding to the disappeared face in the current frame is determined according to the human body frame matching the disappeared face frame.

11. The image processing apparatus according to claim 10, wherein: The face disappearance recognition module is also used for: When a human body is detected in the current image to be identified, each face frame of the previous frame is matched with each human body frame of the current frame; If there is a complete face frame in a human frame, the face frame is considered to match the human frame; If there are multiple face frames in a human frame, a face frame located in the human frame and close to the upper center is selected to match the human frame; Among them, if the face frame selected to match the human body frame already has another matching human body frame, the human body frame is re-matched to the face frame, and one of the multiple human body frames in which the face frame is located in the upper center position is selected to match the face frame, and the remaining human body frames are re-matched to the face frame.

12. The image processing apparatus according to claim 9, wherein: The face disappearance recognition module is also used for: If a newly added missing face is detected in the current image to be identified, the face frame corresponding to the newly added missing face in the previous frame of the image to be identified is used as the face frame of the newly added missing face in the current frame and subsequent frames, and the face disappearance identification step is continued.

13. The image processing device according to any one of claims 9 to 12, further comprising: The non-face and non-human body recognition module is used to perform the non-face and non-human body recognition steps, that is, to perform target recognition on the image to be recognized in an alternating manner of one frame of face recognition and one frame of human body recognition to obtain the recognition result, and to perform the normal face recognition step when a face is recognized; and to perform the human body and non-face recognition step when a human body is recognized; The face recognition module with or without a human body is used to execute the face recognition step with or without a human body, that is, target recognition is performed on the image to be recognized in an alternating manner of one frame of face recognition and several frames of human body recognition to obtain the recognition result; when it is recognized that there is a face in the current image to be recognized, it enters the normal face recognition step.

14. The image processing apparatus according to claim 13, wherein: The face disappearance recognition module is also used for: When a preset number of face recognition frames and a body recognition frame have been performed and the body corresponding to the disappeared face has not been recognized, if the previous frame of the image to be recognized did not recognize a face and the current frame did not recognize a body, then the process will be transferred to the step of no face and no body; if the previous frame of the image to be recognized did not recognize a face and the current frame recognized a body that does not correspond to the disappeared face, then the process will be transferred to the step of no face and body; if the previous frame of the image to be recognized recognized a face other than the disappeared face, then the process will be transferred to the step of normal face. Recognition step: If the previous frame of the image to be recognized recognizes other missing faces in addition to the missing face, continue to execute the face disappearance recognition step.

15. The image processing apparatus according to claim 13, wherein: The human face recognition module is also used for: After performing human body recognition on the current image to be recognized, a face frame corresponding to the face is determined based on the human body frame of the current image to be recognized; after performing face recognition on the current image to be recognized, if no face is recognized, the face frame corresponding to the previous frame is used as the face frame of the current frame; if a face is recognized, the face frame in the recognition result is used as the face frame of the current frame.

16. The image processing apparatus according to claim 15, wherein: The normal face recognition module, the face disappearance recognition module, and the human body and non-human face recognition module are also used to: after performing target recognition on each frame of the image to be recognized, determine the current image to be displayed according to the face frame corresponding to the current image to be recognized; The non-face and non-human body recognition module is further used to: after performing target recognition on each frame of the image to be recognized, determine the current image to be displayed according to the face frame corresponding to the image to be recognized in the most recent historical frame with a face.

17. An image processing device based on human body and face recognition, comprising a processor and a memory, wherein the memory stores executable instructions, and the processor can execute the executable instructions to implement the image processing method according to any one of claims 1 to 8.

18. A computer-readable storage medium storing a computer program, wherein: When the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the image processing method according to any one of claims 1 to 8.

Images