Image processing method and device, electronic device and computer-readable storage medium

Abstract

The present application discloses an image processing method and apparatus, an electronic device, and a computer-readable storage medium. The method includes: acquiring a first image, in which a first head and shoulders and a first human body belong to the same object; acquiring a second image, in which a second head and shoulders and a first face belong to the same object; and, upon determining that the object corresponding to the first head and shoulders is the same as the object corresponding to the second head and shoulders, determining that the object corresponding to the first face is the same as the object corresponding to the first human body.

Description

Technical Field

[0001] The present application relates to the field of security technology, and in particular to an image processing method and device, an electronic device, and a computer-readable storage medium. Background Art

[0002] In recent years, with the maturity of facial recognition technology, facial recognition has become increasingly used in an increasing number of scenarios to determine the identity of individuals in an image. However, when a face is absent from the image to be identified or the image quality is low, it is necessary to establish a correlation between the person's body and a face in an image other than the image to be identified, and to determine the person's identity based on this correlation. Therefore, improving the accuracy of the correlation between faces and bodies is of paramount importance. Summary of the Invention

[0003] The present application provides an image processing method and apparatus, an electronic device, and a computer-readable storage medium.

[0004] The present application provides an image processing method, the method comprising:

[0005] Acquire a first image, in which the first head and shoulders and the first human body belong to the same object;

[0006] Acquire a second image, in which the second head and shoulders and the first face belong to the same object;

[0007] When it is determined that the object corresponding to the first head and shoulders is the same as the object corresponding to the second head and shoulders, it is determined that the object corresponding to the first face is the same as the object corresponding to the first body.

[0008] In conjunction with any embodiment of the present application, after acquiring the first image, and in the case of determining that the object corresponding to the first head and shoulders is the same as the object corresponding to the second head and shoulders, and before determining that the object corresponding to the first face is the same as the object corresponding to the first body, the method further includes:

[0009] If it is determined that the first degree of overlap between the first human body and the first head and shoulders is greater than or equal to a first degree of overlap threshold, it is determined that the first human body and the first head and shoulders belong to the same object.

[0010] In conjunction with any embodiment of the present application, when determining that the first degree of overlap between the first human body and the first head and shoulders is greater than or equal to a first overlap threshold, before determining that the first human body and the first head and shoulders belong to the same object, the method further includes:

[0011] When it is determined that the first human body performs the target behavior, the first overlap degree between the first human body and the first head and shoulders is determined.

[0012] In conjunction with any embodiment of the present application, after acquiring the second image, and in the case of determining that the object corresponding to the first head and shoulders is the same as the object corresponding to the second head and shoulders, and before determining that the object corresponding to the first face is the same as the object corresponding to the first body, the method further includes:

[0013] When it is determined that the second degree of overlap between the first face and the second head and shoulders is greater than or equal to a second overlap threshold, it is determined that the object corresponding to the first face is the same as the object corresponding to the second head and shoulders.

[0014] In combination with any embodiment of the present application, the second image further includes a second face, and a third degree of overlap between the second face and the second head and shoulders is greater than or equal to the second overlap threshold;

[0015] After acquiring the second image, the method further includes:

[0016] Acquire a third image; in the third image, the third head and shoulders and the third face belong to the same object;

[0017] The determining that the object corresponding to the first face is the same as the object corresponding to the second head and shoulders, when it is determined that the second overlap between the first face and the second head and shoulders is greater than or equal to a second overlap threshold, includes:

[0018] When it is determined that the second overlap between the first face and the second head and shoulder is greater than or equal to a second overlap threshold, and the second head and shoulder match the third head and shoulder, and the first face matches the third face, it is determined that the object corresponding to the first face is the same as the object corresponding to the second head and shoulder.

[0019] In conjunction with any embodiment of the present application, before determining that the object corresponding to the first face and the object corresponding to the second head and shoulders are the same when determining that the second degree of overlap between the first face and the second head and shoulders is greater than or equal to a second overlap threshold, the method further includes:

[0020] If it is determined that no head and shoulders matching the second head and shoulders exist in an image acquired earlier than the second image, it is determined that a second degree of overlap between the first face and the second head and shoulders is greater than or equal to a second degree of overlap threshold.

[0021] In combination with any embodiment of the present application, the first image further includes a fourth face, the fourth face and the first head and shoulders belong to the same subject, and no face matching the fourth face exists in any image acquired earlier than the first image;

[0022] When it is determined that the first head-shoulder and the second head-shoulder match, the method further includes:

[0023] It is determined that the fourth face matches the first face.

[0024] The present application also provides an image processing device, which includes:

[0025] an acquisition unit, configured to acquire a first image, in which the first head and shoulders and the first human body belong to the same object;

[0026] The acquisition unit is further configured to acquire a second image, in which the second head and shoulders and the first face belong to the same object;

[0027] The processing unit is configured to determine that the object corresponding to the first face is the same as the object corresponding to the first body when it is determined that the object corresponding to the first head and shoulders is the same as the object corresponding to the second head and shoulders.

[0028] In combination with any embodiment of the present application, the processing unit is further configured to:

[0029] If it is determined that the first degree of overlap between the first human body and the first head and shoulders is greater than or equal to a first degree of overlap threshold, it is determined that the first human body and the first head and shoulders belong to the same object.

[0030] In combination with any embodiment of the present application, the processing unit is further configured to:

[0031] When it is determined that the first human body performs the target behavior, the first overlap degree between the first human body and the first head and shoulders is determined.

[0032] In combination with any embodiment of the present application, the processing unit is further configured to:

[0033] When it is determined that the second degree of overlap between the first face and the second head and shoulders is greater than or equal to a second overlap threshold, it is determined that the object corresponding to the first face is the same as the object corresponding to the second head and shoulders.

[0034] In combination with any embodiment of the present application, the second image further includes a second face, and a third degree of overlap between the second face and the second head and shoulders is greater than or equal to the second overlap threshold;

[0035] The acquisition unit is further configured to:

[0036] Acquire a third image; in the third image, the third head and shoulders and the third face belong to the same object;

[0037] The processing unit is further configured to:

[0038] When it is determined that the second overlap between the first face and the second head and shoulder is greater than or equal to a second overlap threshold, and the second head and shoulder match the third head and shoulder, and the first face matches the third face, it is determined that the object corresponding to the first face is the same as the object corresponding to the second head and shoulder.

[0039] In combination with any embodiment of the present application, the processing unit is further configured to:

[0040] If it is determined that no head and shoulders matching the second head and shoulders exist in an image acquired earlier than the second image, it is determined that a second degree of overlap between the first face and the second head and shoulders is greater than or equal to a second degree of overlap threshold.

[0041] In combination with any embodiment of the present application, the first image further includes a fourth face, the fourth face and the first head and shoulders belong to the same subject, and no face matching the fourth face exists in any image acquired earlier than the first image;

[0042] When it is determined that the first head-shoulder and the second head-shoulder match, the processing unit is further configured to:

[0043] It is determined that the fourth face matches the first face.

[0044] The present application also provides an electronic device, characterized in that it includes: a processor and a memory, the memory is used to store computer program code, the computer program code includes computer instructions, and when the processor executes the computer instructions, the electronic device executes a method such as the above-mentioned image processing method and any possible implementation method thereof.

[0045] The present application also provides another electronic device, including: a processor, a sending device, an input device, an output device and a memory, wherein the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor executes the computer instructions, the electronic device performs a method such as the above-mentioned image processing method and any possible implementation method thereof.

[0046] The present application also provides a computer-readable storage medium, in which a computer program is stored. The computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute a method such as the above-mentioned image processing method and any possible implementation method thereof.

[0047] The present application also provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are run on a computer, the computer is enabled to execute the above-mentioned image processing method and any possible implementation thereof.

[0048] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0050] The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, are used to illustrate the technical solutions of the present application.

[0051] Figure 1 A schematic diagram of a face, head, shoulders, and body provided in an embodiment of the present application;

[0052] Figure 2 A flowchart of an image processing method provided in an embodiment of the present application;

[0053] Figure 3 A flowchart of another image processing method provided in an embodiment of the present application;

[0054] Figure 4 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0055] Figure 5 A schematic diagram of the hardware structure of an image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] The terms "first," "second," and the like in the specification and claims of this application and the accompanying drawings are used to distinguish between different objects, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may optionally include other steps or elements inherent to the process, method, product, or apparatus.

[0058] It should be understood that, in this application, "at least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three or more, and "and / or" is used to describe the relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: only A exists, only B exists, and both A and B exist, where A and B can be singular or plural. The character " / " can indicate that the associated objects are in an "or" relationship, referring to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or plural. The character " / " can also represent the division sign in mathematical operations, for example, a / b = a divided by b; 6 / 3 = 2. "At least one of the following" or similar expressions.

[0059] References herein to "embodiments" mean that a particular feature, structure, or characteristic described in connection with the embodiments may be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0060] Before proceeding to the following explanation, we first define the face, head and shoulders, and human body that appear in the embodiments of this application. In the embodiments of this application, head and shoulders include the head and shoulders of a person.

[0061] For example, in Figure 1 In the figure, the part surrounded by rectangular frame a is the face, the part surrounded by rectangular frame b is the head and shoulders, and the part surrounded by rectangular frame c is the human body.

[0062] The execution subject of the embodiments of the present application is an image processing device, wherein the image processing device can be any electronic device that can execute the technical solution disclosed in the method embodiments of the present application. Optionally, the image processing device can be one of the following: a mobile phone, a computer, a tablet computer, or a wearable smart device.

[0063] It should be understood that the method embodiment of the present application can also be implemented by a processor executing computer program code. The following describes the embodiment of the present application in conjunction with the drawings in the embodiment of the present application. Figure 2 , Figure 2 It is a flowchart of an image processing method provided in an embodiment of the present application.

[0064] 201. Acquire a first image.

[0065] In the embodiment of the present application, the first image includes a first head and shoulders and a first human body. The first human body and the first head and shoulders belong to the same object, that is, the object corresponding to the first human body is the same as the object corresponding to the first head and shoulders.

[0066] In one implementation of acquiring the first image, the image processing apparatus receives the first image input by a user through an input component, wherein the input component includes one of the following: a keyboard, a mouse, a touch screen, a touch pad, or an audio input device.

[0067] In another implementation of acquiring the first image, the image processing apparatus receives the first image sent by a terminal. The terminal includes one of the following: a mobile phone, a computer, a tablet computer, or a server.

[0068] In another implementation of acquiring the first image, a communication connection exists between the image processing apparatus and the imaging device, and the image processing apparatus acquires the first image captured by the imaging device through the communication connection.

[0069] 202. Acquire a second image.

[0070] In the embodiment of the present application, the second image includes a second head and shoulders and a first face. The first face and the second head and shoulders belong to the same object, that is, the object corresponding to the first face is the same as the object corresponding to the second head and shoulders.

[0071] For example, the first face is Zhang San's face, and the second head and shoulders are Zhang San's head and shoulders. In this case, the object corresponding to the first face is the same as the object corresponding to the second head and shoulders.

[0072] In an implementation manner of acquiring the second image, the image processing apparatus receives the second image input by a user through an input component.

[0073] In another implementation manner of acquiring the second image, the image processing apparatus receives the second image sent by the terminal.

[0074] In another implementation of acquiring the second image, a communication connection exists between the image processing apparatus and the imaging device, and the image processing apparatus acquires the second image captured by the imaging device through the communication connection.

[0075] 203. When it is determined that the object corresponding to the first head and shoulders is the same as the object corresponding to the second head and shoulders, it is determined that the object corresponding to the first face is the same as the object corresponding to the first body.

[0076] In the embodiment of the present application, the first head-shoulder and the second head-shoulder match, which means that the object corresponding to the first head-shoulder is the same as the object corresponding to the second head-shoulder.

[0077] Since the object corresponding to the first head and shoulders is the same as the object corresponding to the first body, and the object corresponding to the first face is the same as the object corresponding to the second head and shoulders, the object corresponding to the first body is the same as the object corresponding to the first face.

[0078] In the embodiment of the present application, when the image processing device determines that the first head and shoulder and the second head and shoulder match, the image processing device determines that the object corresponding to the first head and shoulder is the same as the object corresponding to the second head and shoulder; when the image processing device determines that the first head and shoulder and the second head and shoulder do not match, the image processing device determines that the object corresponding to the first head and shoulder is different from the object corresponding to the second head and shoulder.

[0079] In one possible implementation, the first image and the second image are two different frames of images in the target video, and a time difference between a capture time of the first image and a capture time of the second image is less than or equal to a time threshold.

[0080] If the time interval between acquiring the first image and acquiring the second image is referred to as a target time interval, the time threshold is positively correlated with the target time interval.

[0081] Because the same person moves a small distance in a short period of time, if the imaging device capturing the target video captures the same position in the first image as in the second image, then, if the target time interval is short, the difference between the positions of the same person in the first image and the second image is small. Accordingly, the difference between the positions of the same person's head and shoulders in the first image and the second image is small. In other words, if the time threshold is small, the difference between the positions of the same person's head and shoulders in the first image and the second image is small.

[0082] Therefore, the image processing device can determine whether the first head and shoulders match the second head and shoulders based on the difference between the position of the first head and shoulders in the first image and the position of the second head and shoulders in the second image. Specifically, if the difference is greater than or equal to a difference threshold, the first head and shoulders do not match; if the difference is less than the difference threshold, the first head and shoulders match.

[0083] In another possible implementation, the first image and the second image are two different frames of images in the target video, and a time difference between a capture time of the first image and a capture time of the second image is less than or equal to a time threshold.

[0084] The image processing apparatus determines an intersection over union (IoU) between the first head-shoulder and the second head-shoulder. If the IoU is greater than or equal to an IoU threshold, the first head-shoulder and the second head-shoulder are determined to match; if the IoU is less than the IoU threshold, the first head-shoulder and the second head-shoulder are determined to not match.

[0085] In an embodiment of the present application, the image processing device determines that the object corresponding to the first human body is the same as the object corresponding to the first face when the object corresponding to the first head and shoulders is the same as the object corresponding to the second head and shoulders, and then establishes an association relationship between the faces and bodies in different images through the head and shoulders, and improves the accuracy of the association relationship.

[0086] Optionally, the image quality of the pixel area covered by the first human body is less than or equal to a face recognition quality threshold, where the face recognition quality threshold is used to distinguish whether the image can be used for face recognition.

[0087] Specifically, if the image quality of the image is greater than the face recognition quality threshold, the accuracy of the recognition result obtained by performing face recognition on the image is high. If the image quality of the image is less than or equal to the face recognition quality threshold, the accuracy of the recognition result obtained by performing face recognition on the image is low.

[0088] Therefore, when the image quality of the pixel area covered by the first face is less than or equal to the face recognition quality threshold, the accuracy of the identity of the object corresponding to the first human body obtained by performing face recognition on the first face is low.

[0089] In an embodiment of the present application, the image processing device determines that the object corresponding to the first human body is the same as the object corresponding to the first face by determining that the object corresponding to the first head and shoulders is the same as the object corresponding to the second head and shoulders, when the first human body and the first head and shoulders belong to the same object, and the second head and shoulders and the first face belong to the same object.

[0090] In this way, the identity of the object corresponding to the first human body can be determined by performing face recognition on the first human face, thereby improving the accuracy of the identity of the object corresponding to the first human body.

[0091] In one possible scenario, the image processing device detects the video stream and determines that the person to be identified in the first image of the video stream is littering. The first image contains the first head and shoulders of the person to be identified and the first body of the person to be identified, but does not contain the face of the person to be identified.

[0092] The image processing device determines that the second head and shoulders match the first head and shoulders, and in the second image of the video stream, the second head and shoulders and the first face belong to the same person.

[0093] At this time, the image processing device can determine the identity of the person to be identified by performing face recognition on the first face, or can determine the identity of the person to be identified by performing body comparison on the human body.

[0094] Considering that the human body includes dynamic attributes, determining identity based on dynamic attributes can introduce significant errors. Dynamic attributes can change significantly over a short period of time. For example, consider clothing. Another example is posture. Another example is a handbag.

[0095] The proportion of dynamic attribute features in the information carried by the head and shoulders is lower than the proportion of dynamic attribute features in the information carried by the human body. Therefore, the accuracy of determining identity through head and shoulders comparison is higher than that through human body comparison.

[0096] Since the person corresponding to the first face is the person to be identified, it is determined when the first head and shoulders match the second head and shoulders. By performing facial recognition on the first face, the identity of the person to be identified is determined, which can improve the accuracy of the identity of the person to be identified.

[0097] Therefore, when the image processing device determines that the first head and shoulders match the second head and shoulders, the image processing device determines that the identity of the person to be identified who litters is the identity of the first face.

[0098] As an optional implementation manner, the image processing device further determines whether the first human body and the first head and shoulders belong to the same object by performing the following steps:

[0099] 301. When it is determined that a first degree of overlap between the first human body and the first head and shoulders is greater than or equal to a first degree of overlap threshold, determine that the first human body and the first head and shoulders belong to the same object.

[0100] 302. When it is determined that the first degree of overlap between the first human body and the first head and shoulders is less than a first degree of overlap threshold, determine that the first human body and the first head and shoulders do not belong to the same object.

[0101] In the embodiment of the present application, if the first overlap between the first person and the first head and shoulders is greater than or equal to a first overlap threshold, it indicates that the first overlap between the first person and the first head and shoulders is large; if the first overlap between the first person and the first head and shoulders is less than the first overlap threshold, it indicates that the first overlap between the first person and the first head and shoulders is small. In other words, the first overlap threshold is used to distinguish whether the first overlap is large or small.

[0102] A large first degree of overlap between the first person and the first head and shoulders indicates a high probability that the first person and the first head and shoulders belong to the same object. A small first degree of overlap between the first person and the first head and shoulders indicates a low probability that the first person and the first head and shoulders belong to the same object.

[0103] Therefore, when the image processing device determines that the first human body and the first head and shoulders are greater than or equal to the first overlap threshold, the image processing device determines that the first human body and the first head and shoulders belong to the same object; when the image processing device determines that the first human body and the first head and shoulders are less than the first overlap threshold, the image processing device determines that the first human body and the first head and shoulders do not belong to the same object.

[0104] In one implementation of determining a first degree of overlap between a first head and shoulders and a first person, an image processing device determines a first area of ​​the intersection of the first head and shoulders and the first person, and a second area of ​​the first head and shoulders, and determines a ratio of the first area to the second area to obtain the first degree of overlap.

[0105] Optionally, the image processing device performs head-shoulder detection on the first image to obtain a first head-shoulder frame including the first head and shoulders, wherein the head-shoulder detection is used to detect the head and shoulders in the image. The image processing device determines the area of ​​the first head-shoulder frame to obtain the second area.

[0106] In another implementation of determining a first degree of overlap between a first head and shoulders and a first person, an image processing device determines a first area of ​​the intersection of the first head and shoulders and the first person, and a third area of ​​the first person, and determines a ratio of the first area to the third area to obtain the first degree of overlap.

[0107] Optionally, the image processing device performs human body detection processing on the first image to obtain a first human body frame including the first human body, wherein the human body detection processing is used to detect the human body in the image. The image processing device determines the area of ​​the first human body frame to obtain the third area.

[0108] In another implementation of determining the first degree of overlap between the first head and shoulders and the first human body, the image processing device determines an intersection-over-union ratio of the first head and shoulders and the first human body to obtain the first degree of overlap.

[0109] In this embodiment, the image processing device can determine whether the first head and shoulders and the first human body belong to the same object based on the first head and shoulders and the first human body.

[0110] As an optional implementation manner, the image processing apparatus further performs the following steps:

[0111] 401. When it is determined that the first human body performs the target behavior, determine a first degree of overlap between the first human body and the first head and shoulders.

[0112] In the embodiments of the present application, the target behavior can be any behavior. For example, the target behavior is smoking. Another example is driving a vehicle and illegally parking. Another example is pushing an electric scooter into an elevator.

[0113] In the embodiment of the present application, the image processing device needs to determine the identity of the object that generates the target behavior. Therefore, when the image processing device determines that the first person generates the target behavior, it needs to determine the identity of the first person.

[0114] Considering that the accuracy of the identity of the first person obtained by performing facial recognition on the first person is low, when it is necessary to determine the identity of the first person, the image processing device determines the identity of the first person by performing facial recognition on the face of the object corresponding to the same object as the object corresponding to the first person, thereby improving the accuracy of the identity of the first person.

[0115] Therefore, when the image processing device determines that the first human body produces the target behavior, it determines the first overlap between the first human body and the first head and shoulders, and then, when it determines that the first overlap is greater than or equal to the first overlap threshold, it determines that the first human body and the first head and shoulders belong to the same object.

[0116] In this way, when it is determined that the first person and the first head and shoulders belong to the same object, the identity of the first person is determined by performing face recognition on the first face, thereby improving the accuracy of the identity of the first person.

[0117] In one possible scenario, the target behavior is littering. The image processing device detects a video stream and determines that a first person in a first image of the video stream has engaged in littering. The first image also contains a first head and shoulders. Both the first person and the first head and shoulders are objects to be identified, but the first image does not contain a face of the object to be identified.

[0118] At this time, when the image processing device determines that the first head and shoulders match the second head and shoulders, the image processing device determines that the identity of the object to be identified who is littering is the identity of the first human face.

[0119] As an optional implementation manner, when it is determined that the first head-shoulder and the second head-shoulder match, the image processing apparatus further performs the following steps:

[0120] 501. When it is determined that a second degree of overlap between the first face and the second head and shoulders is greater than or equal to a second overlap threshold, determine that the object corresponding to the first face is the same as the object corresponding to the second head and shoulders.

[0121] In this embodiment of the present application, if the second overlap between the first face and the second head and shoulders is greater than or equal to the second overlap threshold, it indicates that the second overlap between the first face and the second head and shoulders is large; if the second overlap between the first face and the second head and shoulders is less than the second overlap threshold, it indicates that the second overlap between the first face and the second head and shoulders is small. In other words, the second overlap threshold is used to distinguish whether the second overlap is large or small.

[0122] A large second overlap between the first face and the second head and shoulders indicates a high probability that the first face and the second head and shoulders belong to the same object. A small second overlap between the first face and the second head and shoulders indicates a low probability that the first face and the second head and shoulders belong to the same object.

[0123] Therefore, when the image processing device determines that the second degree of overlap between the first face and the second head and shoulders is greater than or equal to the second overlap threshold, the image processing device determines that the first face and the second head and shoulders belong to the same object; when the image processing device determines that the second degree of overlap between the first face and the second head and shoulders is less than the second overlap threshold, the image processing device determines that the first face and the second head and shoulders do not belong to the same object.

[0124] In one implementation of determining a second degree of overlap between the second head and shoulders and the first face, an image processing device determines a fourth area of ​​the intersection of the second head and shoulders and the first face, and a fifth area of ​​the second head and shoulders, and determines a ratio of the fourth area to the fifth area to obtain the second degree of overlap.

[0125] Optionally, the image processing device performs head-shoulder detection on the second image to obtain a second head-shoulder frame including the second head and shoulders, wherein the head-shoulder detection is used to detect the head and shoulders in the image. The image processing device determines the area of ​​the second head-shoulder frame to obtain a fifth area.

[0126] In another implementation of determining the second degree of overlap between the second head and shoulders and the first face, the image processing device determines a fourth area of ​​the intersection of the second head and shoulders and the first face, and a sixth area of ​​the first face, and determines a ratio of the fourth area to the sixth area to obtain the second degree of overlap.

[0127] Optionally, the image processing device performs face detection processing on the second image to obtain a first face frame including the first face, wherein the face detection processing is used to detect faces in the image. The image processing device determines the area of ​​the first face frame to obtain a sixth area.

[0128] In another implementation of determining the second degree of overlap between the second head and shoulders and the first face, the image processing device determines an intersection-over-union ratio of the second head and shoulders and the first face to obtain the second degree of overlap.

[0129] In this embodiment, the image processing device can determine, based on the second head and shoulders and the first face, whether the second head and shoulders and the first face belong to the same object. Furthermore, if it is determined that the first face belongs to the same object as the second head and shoulders, the image processing device can determine the identities of the head and shoulders of the corresponding object and the object corresponding to the second head and shoulders by performing facial recognition on the first face. Alternatively, if it is determined that the first face belongs to the same object as the second head and shoulders, the image processing device can determine the identities of the human body of the corresponding object and the object corresponding to the second head and shoulders by performing facial recognition on the first face.

[0130] As an optional embodiment, the second image further includes a second face, wherein a third degree of overlap between the second face and the second head and shoulders is greater than or equal to a second overlap threshold, i.e., there is a high probability that the second face and the second head and shoulders belong to the same subject. In other words, in the second image, there is a high probability that the first face, the second face, and the second head and shoulders belong to the same subject.

[0131] In a possible scenario, the object corresponding to the first face is close to the object corresponding to the second face. At this time, regardless of whether the second head and shoulders belong to the same object as the first face, or the second head and shoulders belong to the same object as the second face, the second head and shoulders have a high degree of overlap with the first face and the second face.

[0132] For example, if the first face is Zhang San's face and the second face is Li Si's face, if the second head and shoulders are Zhang San's head and shoulders, then when Zhang San and Li Si are close, the overlap between Zhang San's head and shoulders and Li Si's face may be greater than or equal to the second overlap threshold.

[0133] If the second head and shoulders are Li Si's head and shoulders, then when Zhang San and Li Si are close to each other, the degree of overlap between Li Si's head and shoulders and Zhang San's face may also be greater than or equal to the second overlap threshold.

[0134] In one possible implementation, the image processing device determines that the first face and the second head and shoulders belong to the same object when the second degree of overlap is greater than the third degree of overlap; determines that the first face and the second head and shoulders belong to the same object, or determines that the second face and the second head and shoulders belong to the same object when the second degree of overlap is equal to the third degree of overlap; and determines that the second face and the second head and shoulders belong to the same object when the second degree of overlap is less than the third degree of overlap.

[0135] In another possible implementation, the image processing device determines that the first face and the second head and shoulders belong to the same object, or determines that the second face and the second head and shoulders belong to the same object when the second overlap and the third overlap are both greater than or equal to the second overlap threshold.

[0136] However, through any of the above two possible implementation methods, there may be errors in determining the face of the same object as the second head and shoulders from the first face and the second face. Based on this, the embodiment of the present application provides a solution to reduce the error.

[0137] Specifically, the image processing device further performs the following steps:

[0138] 601. Acquire a third image.

[0139] In the embodiment of the present application, the third image is captured at a different time than the second image, and the third image includes a third face and a third head and shoulders, and the third head and shoulders and the third face belong to the same subject.

[0140] In an implementation manner of acquiring the third image, the image processing apparatus receives the third image input by a user through an input component.

[0141] In another implementation manner of acquiring the third image, the image processing apparatus receives the third image sent by the terminal.

[0142] In another implementation of acquiring the third image, a communication connection exists between the image processing apparatus and the imaging device, and the image processing apparatus acquires the third image captured by the imaging device through the communication connection.

[0143] When acquiring the third image, the image processing apparatus performs the following steps during step 501:

[0144] 602. When it is determined that the second overlap between the first face and the second head and shoulders is greater than or equal to a second overlap threshold, and the second head and shoulders match the third head and shoulders, and the first face matches the third face, it is determined that the object corresponding to the first face is the same as the object corresponding to the second head and shoulders.

[0145] The second head-shoulder matches the third head-shoulder, indicating that the object corresponding to the second head-shoulder is the same as the object corresponding to the third head-shoulder. Furthermore, since the third head-shoulder and the third face belong to the same object, the object corresponding to the second head-shoulder is the same as the object corresponding to the third face.

[0146] The first face matches the third face, indicating that the object corresponding to the first face is the same as the object corresponding to the third face. Therefore, the object corresponding to the second head and shoulders is the same as the object corresponding to the first face.

[0147] Optionally, if the third image includes a fifth face that matches the second face, a fourth degree of overlap between the fifth face and the third head and shoulders is less than the second overlap threshold. In this case, the third image can determine that the fifth face and the third head and shoulders do not belong to the same object. Furthermore, because the fifth face matches the second face and the third head and shoulders match the second head and shoulders, the third image can determine that the second head and shoulders do not belong to the same object as the second face. Therefore, the image processing device can improve the accuracy of identifying faces that belong to the same object as the second head and shoulders by determining that the object corresponding to the first face and the object corresponding to the second head and shoulders are the same.

[0148] Optionally, there is no face matching the second face in the third image. In this case, the third image can determine that the second head and shoulders do not belong to the same object as the second face. Therefore, the image processing device can determine that the object corresponding to the first face and the object corresponding to the second head and shoulders are the same, thereby improving the accuracy of the face belonging to the same object as the second head and shoulders.

[0149] In this embodiment, because the third image was captured at a different time than the second image, the probability that the object corresponding to the first face and the object corresponding to the second face are close in the third image is low. Therefore, when the image processing device determines from the third image that the third head and shoulders and the third face belong to the same object, it can determine whether the second head and shoulders and the first face belong to the same object by determining whether the first face matches the third face, and whether the second head and shoulders match the third head and shoulders.

[0150] Specifically, if it is determined that the second degree of overlap is greater than or equal to the second degree of overlap threshold, the first face matches the third face, and the second head and shoulders match the third head and shoulders, it is determined that the object corresponding to the first face is the same as the object corresponding to the second head and shoulders. This improves the accuracy of identifying faces that belong to the same object as the second head and shoulders.

[0151] As an optional embodiment, the first image also includes a second person, wherein a fourth degree of overlap between the second person and the first head and shoulders is greater than or equal to a first overlap threshold, i.e., there is a high probability that the second person and the first head and shoulders belong to the same subject. In other words, in the first image, there is a high probability that the first person, the second person, and the first head and shoulders belong to the same subject.

[0152] In a possible scenario, the object corresponding to the first human body is close to the object corresponding to the second human body. At this time, regardless of whether the first head and shoulders belong to the same object as the first human body, or the first head and shoulders belong to the same object as the second human body, the first head and shoulders have a large degree of overlap with the first human body and the second human body.

[0153] For example, if the first body is Zhang San's body and the second body is Li Si's body, if the first head and shoulders are Zhang San's head and shoulders, then when Zhang San and Li Si are close, the overlap between Zhang San's head and shoulders and Li Si's body may be greater than or equal to the first overlap threshold.

[0154] If the first head and shoulders are Li Si's head and shoulders, then when Zhang San and Li Si are close to each other, the degree of overlap between Li Si's body and Zhang San's body may also be greater than or equal to the first overlap threshold.

[0155] In one possible implementation, the image processing device determines that the first human body and the first head and shoulders belong to the same object when the first degree of overlap is greater than the fourth degree of overlap; determines that the first human body and the first head and shoulders belong to the same object, or determines that the second human body and the first head and shoulders belong to the same object when the first degree of overlap is equal to the fourth degree of overlap; and determines that the second human body and the first head and shoulders belong to the same object when the first degree of overlap is less than the fourth degree of overlap.

[0156] In another possible implementation, the image processing device determines that the first human body and the first head and shoulders belong to the same object, or determines that the second human body and the first head and shoulders belong to the same object when the first overlap and the fourth overlap are both greater than or equal to the first overlap threshold.

[0157] However, through any of the above two possible implementation methods, there may be errors in determining the human body belonging to the same object as the first head and shoulders from the first human body and the second human body. Based on this, the embodiment of the present application provides a solution to reduce the error.

[0158] Specifically, the image processing device further performs the following steps:

[0159] 701. Acquire a fourth image. In this embodiment of the present application, the fourth image is acquired at a different time than the first image. The fourth image includes a third person and a fourth head and shoulders, and the fourth head and shoulders and the third person are of the same subject.

[0160] In an implementation manner of acquiring the fourth image, the image processing apparatus receives the fourth image input by a user through an input component.

[0161] In another implementation manner of acquiring the fourth image, the image processing apparatus receives the fourth image sent by the terminal.

[0162] In yet another implementation of acquiring the fourth image, a communication connection exists between the image processing apparatus and the imaging device, and the image processing apparatus acquires the fourth image captured by the imaging device through the communication connection.

[0163] 702. When it is determined that a first degree of overlap between the first human body and the first head and shoulder is greater than or equal to a first degree of overlap threshold, and the first head and shoulder match the fourth head and shoulder, and the first human body matches the third human body, determine that the object corresponding to the first human body is the same as the object corresponding to the first head and shoulder.

[0164] The first head-shoulder matches the fourth head-shoulder, indicating that the object corresponding to the first head-shoulder is the same as the object corresponding to the fourth head-shoulder. Furthermore, because the fourth head-shoulder and the third person belong to the same object, the object corresponding to the first head-shoulder is the same as the object corresponding to the third person.

[0165] The first human body matches the third human body, indicating that the object corresponding to the first human body is the same as the object corresponding to the third human body. Therefore, the object corresponding to the first head and shoulders is the same as the object corresponding to the first human body.

[0166] Optionally, when the fourth image includes a fourth person matching the third person, a fifth degree of overlap between the fourth person and the fourth head and shoulders is less than the first overlap threshold. In this case, the fourth image can determine that the fourth person and the fourth head and shoulders do not belong to the same object. Furthermore, because the fourth person matches the third person and the fourth head and shoulders match the first head and shoulders, the fourth image can determine that the first head and shoulders do not belong to the same object as the third person. Therefore, the image processing device can determine that the object corresponding to the first person and the object corresponding to the first head and shoulders are the same, thereby improving the accuracy of identifying the person belonging to the same object as the first head and shoulders.

[0167] Optionally, no human body matching the third human body exists in the fourth image. In this case, the fourth image can determine that the first head and shoulders and the third human body do not belong to the same object. Therefore, the image processing device can determine that the object corresponding to the first human body is the same as the object corresponding to the first head and shoulders, thereby improving the accuracy of the human body that belongs to the same object as the first head and shoulders.

[0168] In this embodiment, because the fourth image was captured at a different time than the second image, the probability that the object corresponding to the first person and the object corresponding to the third person are close in the fourth image is low. Therefore, when the image processing device determines from the fourth image that the fourth head and shoulders and the third person belong to the same object, it can determine whether the first head and shoulders match the first person by determining whether the first person and the third person match, and whether the first head and shoulders match the fourth person.

[0169] Specifically, if it is determined that the first degree of overlap is greater than or equal to a first degree of overlap threshold, the first person matches the third person, and the first head and shoulders match the fourth head and shoulders, it is determined that the object corresponding to the first person is the same as the object corresponding to the first head and shoulders. This improves the accuracy of identifying the person as the same object as the first head and shoulders.

[0170] As an optional implementation, the image processing device further performs the following steps:

[0171] 801. When it is determined that a head and shoulder matching the second head and shoulder does not exist in an image acquired earlier than the second image, determine that a second degree of overlap between the first face and the second head and shoulder is greater than or equal to a second overlap threshold.

[0172] There is no head and shoulder matching the second head and shoulder in the image acquired earlier than the second image, indicating that the image processing device cannot determine the face of the same object as the second head and shoulder based on the image acquired earlier than the second image.

[0173] Therefore, the image processing device determines the face of the same object as the second head and shoulders based on the second image. When it is necessary to determine the identity of the head and shoulders in an image whose acquisition time is later than the second image, the image processing device can use the face of the same object as the second head and shoulders determined in the second image, and the matching relationship between the second head and shoulders and the head and shoulders in the image whose acquisition time is later than the second image, to determine the identity of the head and shoulders in the image whose acquisition time is later than the second image.

[0174] Optionally, the second image is captured earlier than the first image. In this way, the image processing device determines, based on the second image, that the first face and the second head and shoulders belong to the same subject. If the identity of the first head and shoulders in the first image needs to be determined, the matching relationship between the first and second head and shoulders can be used to perform facial recognition on the first face to determine the identity of the second head and shoulders.

[0175] As an optional embodiment, the first transformation relationship of the first image and the second transformation relationship of the second image are the same, wherein the first transformation relationship is the transformation relationship between the pixel coordinate system of the first image and the world coordinate system, and the second transformation relationship is the transformation relationship between the pixel coordinate system of the second image and the world coordinate system.

[0176] In a possible implementation scenario, both the first image and the second image are captured by a target imaging device, and the position of the target imaging device when capturing the first image is the same as the position when capturing the second image.

[0177] In this embodiment, the image processing device further performs the following steps:

[0178] 901. Determine a first position of the first head and shoulders in the first image and a second position of the second head and shoulders in the second image.

[0179] 902. When a position difference between the first position and the second position is less than or equal to a position difference threshold, determine that the first head-shoulder and the second head-shoulder match.

[0180] Since the distance a person moves in a short period of time is usually short, and the acquisition time of the first image is short compared to the acquisition time of the second image, if the same object appears in both the first image and the second image, then the difference between the position of the object in the first image and the position in the second image should be small.

[0181] Therefore, the image processing apparatus can determine whether the object corresponding to the first head and shoulders is the same as the object corresponding to the second head and shoulders based on the size of the positional difference between the first position and the second position.

[0182] In the embodiment of the present application, the position difference threshold is used to determine whether the position difference is large or small. Specifically, if the position difference between the first position and the second position is less than or equal to the position difference threshold, the position difference is small; if the position difference between the first position and the second position is greater than the position difference threshold, the position difference is large.

[0183] Therefore, the image processing apparatus determines that the first head and shoulders match the second head and shoulders when the position difference between the first position and the second position is less than or equal to the position difference threshold, and determines that the first head and shoulders do not match the second head and shoulders when the position difference between the first position and the second position is greater than the position difference threshold.

[0184] Optionally, the time difference between the acquisition time of the first image and the acquisition time of the second image is less than or equal to a time difference threshold. The time difference threshold serves as a basis for determining whether the time difference is large or small. Specifically, if the position difference between the first position and the second position is less than or equal to the position difference threshold, the position difference is small; if the position difference between the first position and the second position is greater than the position difference threshold, the position difference is large.

[0185] In this embodiment, the image processing device determines whether the first head and shoulders match the second head and shoulders based on the position difference between the first position and the second position, which can improve the accuracy of determining whether the first head and shoulders match the second head and shoulders.

[0186] As an optional implementation, the first image further includes a fourth face, wherein the fourth face and the first head and shoulders belong to the same object, and there is no face matching the fourth face in images acquired earlier than the first image.

[0187] There is no head and shoulders matching the first face in the image acquired earlier than the first image, indicating that the image processing device cannot determine, based on the image acquired earlier than the first image, whether there is head and shoulders belonging to the same object as the fourth face in the image acquired earlier than the first image.

[0188] Therefore, when determining that the first head and shoulders match the second head and shoulders, the image processing device further performs the following steps:

[0189] 1001. Determine whether the fourth face matches the first face.

[0190] The first head-and-shoulder matches the second head-and-shoulder, and the object corresponding to the fourth face is the same as the object corresponding to the first head-and-shoulder. This indicates that the object corresponding to the second head-and-shoulder is the same as the object corresponding to the fourth face. Furthermore, because the object corresponding to the first face is the same as the object corresponding to the second head-and-shoulder, the object corresponding to the first face is the same as the object corresponding to the fourth face, indicating that the first and fourth faces match.

[0191] Therefore, based on the first image, the image processing device can determine that the first face matches the fourth face if the first head and shoulders match the second head and shoulders. Furthermore, the image processing device can determine that any head and shoulders belonging to the same object as the first face also belong to the same object as the fourth face, or that any body belonging to the same object as the first face also belongs to the same object as the fourth face.

[0192] Optionally, the first face belongs to the second image, and the acquisition time of the second image is earlier than the acquisition time of the first image.

[0193] As an optional implementation, the first image further includes a second human body, wherein the second human body and the first head and shoulders belong to the same object, and no human face matching the second human body exists in images acquired earlier than the first image.

[0194] The fact that the head and shoulders matching the first face do not exist in the image acquired earlier than the first image indicates that the image processing device cannot determine, based on the image acquired earlier than the first image, whether the head and shoulders belonging to the same object as the second human body exist in the image acquired earlier than the first image.

[0195] Therefore, when determining that the first head and shoulders match the second head and shoulders, the image processing device further performs the following steps: determining that the second human body matches the first human face.

[0196] The first head and shoulders match the second head and shoulders, and the object corresponding to the second person is the same as the object corresponding to the first head and shoulders. This indicates that the object corresponding to the second head and shoulders is the same as the object corresponding to the second person. Furthermore, because the object corresponding to the first face is the same as the object corresponding to the second head and shoulders, the object corresponding to the first face is the same as the object corresponding to the second person, indicating that the first face and the second person match.

[0197] Therefore, based on the first image, the image processing device can determine that the first face matches the second person if the first head and shoulders match the second head and shoulders. Furthermore, the image processing device can determine that any head and shoulders belonging to the same object as the first face also belong to the same object as the second person, or that any person belonging to the same object as the first face also belongs to the same object as the second person.

[0198] Based on the technical solution provided in the embodiments of this application, this application also provides a possible application scenario.

[0199] As the number of surveillance cameras increases, related electronic equipment can process the video streams collected by the surveillance cameras to monitor whether there are any violations. Among them, violations include at least one of the following: illegal smoking, illegal phone calls, fighting, illegal intrusion, climbing over the fence, leaving the post without permission, and crossing the boundary line.

[0200] When a violation is detected, the identity of the person who caused the violation can be determined based on the technical solution provided in the embodiments of the present application so that relevant personnel can hold the person accountable.

[0201] Specifically, the management center at Location A has a server connected to a surveillance camera at Location A. Through this connection, the server receives the video stream captured by the surveillance camera. Management center staff want to use this server to monitor the camera footage for any instances of illegal phone calls.

[0202] The staff can use the server as an image processing device. Figure 3 The flow shown executes the image processing method.

[0203] like Figure 3 As shown, after the image processing device obtains the video stream captured by the surveillance camera in step 3001, it performs face tracking detection processing, human body tracking detection processing, and head and shoulder tracking detection processing on the video stream in step 3002 to obtain the position coordinates of the face frame, the position coordinates of the human body frame, the position coordinates of the head and shoulder frame in each frame image, as well as the identification number of each face frame, the identification number of each human body frame, and the identification number of each head and shoulder frame.

[0204] The image processing device determines the following coincidences in each frame of the image based on the position coordinates of the face frame, the position coordinates of the body frame, and the position coordinates of the head and shoulders frame: the coincidence between the face frame and the body frame, the coincidence between the face frame and the head and shoulders frame, and the coincidence between the body frame and the head and shoulders frame.

[0205] When the degree of overlap between the face frame and the head-shoulder frame is greater than or equal to a second overlap threshold, it is determined that the face in the face frame and the head and shoulders in the head-shoulder frame belong to the same object, and a face-head-shoulder mapping relationship is established between the identification number of the face frame and the identification number of the head-shoulder frame.

[0206] When the degree of overlap between the human body frame and the head-shoulder frame is greater than or equal to a first overlap threshold, it is determined that the human body in the human body frame and the head and shoulders in the head-shoulder frame belong to the same object, and a human head-shoulder mapping relationship is established between the identification number of the human body frame and the identification number of the head-shoulder frame.

[0207] Specifically, in step 3003, the face and head and shoulders belonging to the same object are determined, and an association is established between the identification numbers of the face frame and the identification numbers of the head and shoulders frame belonging to the same object. In step 3004, the head and shoulders and body belonging to the same object are determined, and an association is established between the identification numbers of the body frame and the identification numbers of the head and shoulders frame belonging to the same object. It should be understood that steps 3003 and 3004 can be performed in no particular order.

[0208] According to the head-shoulder mapping relationship of the human body and the head-shoulder mapping relationship of the face, a mapping is established between the identification numbers of the face frame, the body frame, and the head-shoulder frame belonging to the same object to obtain an identity mapping relationship.

[0209] The image processing device also detects the video stream to determine whether there is an illegal phone call event in the video stream. If it is determined that there is an illegal phone call event in the target image in the video stream, the identity of the person making the illegal phone call (hereinafter referred to as the illegal person) can be further determined.

[0210] However, if the target image only contains the body of the offending object but not the face of the offending object, it is impossible to determine the identity of the offending object by performing face recognition on the target image.

[0211] At this point, the image processing device can determine the head and shoulders of the offending object from the target image and determine an identity mapping relationship with a head and shoulders matching the head and shoulders of the offending object as the target identity mapping relationship in step 3005. By performing face recognition on the face in the target identity mapping relationship, the identity of the offending object can be obtained.

[0212] Optionally, the image processing device performs a first optimization process on the mapping relationship between the face and the head and shoulders in step 3006, i.e., corrects the face and the head and shoulders in the identity mapping relationship through the first optimization process. The implementation process of the first optimization process can be referred to steps 601 and 602.

[0213] Optionally, the image processing device performs a second optimization process on the mapping relationship between the human body and the head and shoulders in step 3007, that is, corrects the human body and the head and shoulders in the identity mapping relationship through the first optimization process, wherein the implementation process of the second optimization process can be referred to steps 701 and 702.

[0214] It should be understood that there is no particular order in which step 3006 and step 3007 are executed.

[0215] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0216] If the technical solution of this application involves personal information, the product that applies the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing personal information. If the technical solution of this application involves sensitive personal information, the product that applies the technical solution of this application has obtained the individual's separate consent before processing sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, a clear and prominent sign is set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that they agree to the collection of their personal information; or on the personal information processing device, when the personal information processing rules are notified by obvious signs / information, the individual's authorization is obtained through pop-up information or by asking the individual to upload their personal information; among which, personal information processing may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0217] The above describes in detail the method of the embodiment of the present application, and the following provides an apparatus of the embodiment of the present application.

[0218] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an image processing device provided in an embodiment of the present application. The image processing device 1 includes: an acquisition unit 11 and a processing unit 12. Specifically:

[0219] An acquisition unit 11 is configured to acquire a first image, in which the first head and shoulders and the first human body belong to the same object;

[0220] The acquisition unit 11 is further configured to acquire a second image, in which the second head and shoulders and the first face belong to the same object;

[0221] The processing unit 12 is configured to determine that the object corresponding to the first face is the same as the object corresponding to the first body, if it is determined that the object corresponding to the first head and shoulders is the same as the object corresponding to the second head and shoulders.

[0222] In combination with any embodiment of the present application, the processing unit 12 is further configured to:

[0223] If it is determined that the first degree of overlap between the first human body and the first head and shoulders is greater than or equal to a first degree of overlap threshold, it is determined that the first human body and the first head and shoulders belong to the same object.

[0224] In combination with any embodiment of the present application, the processing unit 12 is further configured to:

[0225] When it is determined that the first human body performs the target behavior, the first overlap degree between the first human body and the first head and shoulders is determined.

[0226] In combination with any embodiment of the present application, the processing unit 12 is further configured to:

[0227] When it is determined that the second degree of overlap between the first face and the second head and shoulders is greater than or equal to a second overlap threshold, it is determined that the object corresponding to the first face is the same as the object corresponding to the second head and shoulders.

[0228] In combination with any embodiment of the present application, the second image further includes a second face, and a third degree of overlap between the second face and the second head and shoulders is greater than or equal to the second overlap threshold;

[0229] The acquisition unit 11 is further configured to:

[0230] Acquire a third image; in the third image, the third head and shoulders and the third face belong to the same object;

[0231] The processing unit 12 is further configured to:

[0232] When it is determined that the second overlap between the first face and the second head and shoulder is greater than or equal to a second overlap threshold, and the second head and shoulder match the third head and shoulder, and the first face matches the third face, it is determined that the object corresponding to the first face is the same as the object corresponding to the second head and shoulder.

[0233] In combination with any embodiment of the present application, the processing unit 12 is further configured to:

[0234] If it is determined that no head and shoulders matching the second head and shoulders exist in an image acquired earlier than the second image, it is determined that a second degree of overlap between the first face and the second head and shoulders is greater than or equal to a second degree of overlap threshold.

[0235] In combination with any embodiment of the present application, the first image further includes a fourth face, the fourth face and the first head and shoulders belong to the same subject, and no face matching the fourth face exists in any image acquired earlier than the first image;

[0236] When it is determined that the first head-shoulder and the second head-shoulder match, the processing unit 12 is further configured to:

[0237] It is determined that the fourth face matches the first face.

[0238] In some embodiments, the functions or modules included in the device provided in the embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0239] Figure 5 Schematic diagram of the hardware structure of an image processing device provided in an embodiment of the present application. The image processing device 2 includes a processor 21, a memory 22, an input device 23, and an output device 24. The processor 21, the memory 22, the input device 23, and the output device 24 are coupled via a connector, and the connector includes various interfaces, transmission lines, or buses, etc., which are not limited in the embodiments of the present application. It should be understood that in each embodiment of the present application, coupling refers to mutual connection in a specific manner, including direct connection or indirect connection through other devices, for example, through various interfaces, transmission lines, buses, etc.

[0240] The processor 21 may be one or more graphics processing units (GPUs). If the processor 21 is a GPU, the GPU may be a single-core GPU or a multi-core GPU. Alternatively, the processor 21 may be a processor group consisting of multiple GPUs, with the multiple processors coupled to each other via one or more buses. Alternatively, the processor may be another type of processor, etc., which is not limited in the present embodiment.

[0241] The memory 22 can be used to store computer program instructions and various computer program codes, including program codes for executing the solution of the present application. Optionally, the memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or portable compact disc read-only memory (CD-ROM), which is used for related instructions and data.

[0242] The input device 23 is used to input data and / or signals, and the output device 24 is used to output data and / or signals. The input device 23 and the output device 24 can be independent devices or an integrated device.

[0243] It can be understood that in the embodiment of the present application, the memory 22 can be used not only to store relevant instructions, but also to store relevant data. For example, the memory 22 can be used to store the first image obtained through the input device 23, etc. The embodiment of the present application does not limit the specific data stored in the memory.

[0244] It is understandable that Figure 5 Only a simplified design of an image processing device is shown. In actual applications, the image processing device may further include other necessary components, including but not limited to any number of input / output devices, processors, memories, etc., and all image processing devices that can implement the embodiments of this application are within the scope of protection of this application.

[0245] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0246] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here. Those skilled in the art will also clearly understand that the descriptions of the various embodiments of this application have different focuses. For the convenience and brevity of description, the same or similar parts may not be repeated in different embodiments. Therefore, for parts not described or not described in detail in a certain embodiment, reference can be made to the descriptions of other embodiments.

[0247] In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0248] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0249] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0250] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0251] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by a computer program instructing related hardware to perform the processes. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a first image, in which the first head and shoulders and the first human body belong to the same object; Acquire a second image, in which the second head and shoulders and the first face belong to the same object, the second image also includes a second face, and a third degree of overlap between the second face and the second head and shoulders is greater than or equal to a second overlap threshold; Acquire a third image; in the third image, the third head and shoulders and the third face belong to the same object; If it is determined that a second degree of overlap between the first face and the second head-shoulder is greater than or equal to a second overlap threshold, and the second head-shoulder matches the third head-shoulder, and the first face matches the third face, determining that the object corresponding to the first face is the same as the object corresponding to the second head-shoulder; When it is determined that the object corresponding to the first head and shoulders is the same as the object corresponding to the second head and shoulders, it is determined that the object corresponding to the first face is the same as the object corresponding to the first body.

2. The method according to claim 1, characterized in that After acquiring the first image, if it is determined that the object corresponding to the first head and shoulders is the same as the object corresponding to the second head and shoulders, and before determining that the object corresponding to the first face is the same as the object corresponding to the first body, the method further includes: If it is determined that the first degree of overlap between the first human body and the first head and shoulders is greater than or equal to a first degree of overlap threshold, it is determined that the first human body and the first head and shoulders belong to the same object.

3. The method according to claim 1 or 2, characterized in that In the case where it is determined that the first degree of overlap between the first human body and the first head and shoulders is greater than or equal to a first overlap threshold, before determining that the first human body and the first head and shoulders belong to the same object, the method further includes: When it is determined that the first human body performs the target behavior, the first overlap degree between the first human body and the first head and shoulders is determined.

4. The method according to claim 1, wherein Before determining that the object corresponding to the first face is the same as the object corresponding to the second head and shoulder, if it is determined that the second overlap between the first face and the second head and shoulder is greater than or equal to a second overlap threshold, the second head and shoulder match the third head and shoulder, and the first face matches the third face, the method further includes: If it is determined that no head and shoulders matching the second head and shoulders exist in an image acquired earlier than the second image, it is determined that a second degree of overlap between the first face and the second head and shoulders is greater than or equal to a second degree of overlap threshold.

5. The method according to claim 1 or 2, characterized in that The first image further includes a fourth face, the fourth face and the first head and shoulders belong to the same subject, and there is no face matching the fourth face in any image acquired earlier than the first image; When it is determined that the first head-shoulder and the second head-shoulder match, the method further includes: It is determined that the fourth face matches the first face.

6. An image processing device, characterized in that The device comprises: an acquisition unit, configured to acquire a first image, in which the first head and shoulders and the first human body belong to the same object; The acquisition unit is further configured to acquire a second image, wherein in the second image, the second head and shoulders and the first face belong to the same object, the second image also includes a second face, and a third degree of overlap between the second face and the second head and shoulders is greater than or equal to a second overlap threshold; The acquisition unit is further configured to acquire a third image; in the third image, the third head and shoulders and the third face belong to the same object; a processing unit configured to, if it is determined that a second degree of overlap between the first face and the second head-shoulder is greater than or equal to a second overlap threshold, the second head-shoulder matches the third head-shoulder, and the first face matches the third face, determine that the object corresponding to the first face is the same as the object corresponding to the second head-shoulder; The processing unit is configured to determine that the object corresponding to the first face is the same as the object corresponding to the first body when it is determined that the object corresponding to the first head and shoulders is the same as the object corresponding to the second head and shoulders.

7. An electronic device, characterized in that: include: A processor and a memory, the memory is used to store computer program code, the computer program code includes computer instructions, and when the processor executes the computer instructions, the electronic device executes the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 5.

Images