A computer vision-based group recognition method and a group recognition device

Abstract

The application discloses a group recognition method and device based on computer vision, and the group recognition method of one embodiment comprises the following steps: using a target detection model to sequentially perform target detection on collected images and output multiple target detection images; traversing each target detection image of each collected image, obtaining the same pedestrians of each target detection image, and storing the same pedestrians as group data; performing time sequence tracking on each target pedestrian, obtaining a pedestrian group according to the group data, and outputting the pedestrian group. The embodiment provided by the application is based on computer vision, the same pedestrians of target pedestrians are obtained by recognizing the target detection images, the pedestrian group among multiple pedestrians is obtained, the applicability and accuracy of customer group portrait analysis are improved, and the embodiment has practical application value.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a group recognition method and device based on computer vision. BACKGROUND

[0002] With the rapid development of computer vision technology, the demand for human body detection is increasing. Tracking based on human body detection and analyzing the group among multiple people become an important customer portrait information, which has strong analysis significance. However, it has not been paid much attention in recent years.

[0003] How to obtain the group information among multiple people based on computer vision technology and analyze and utilize the group information has become a technical problem to be solved by the technical personnel in the field. SUMMARY

[0004] In order to solve at least one of the above problems, the first aspect of the present application provides a group recognition method based on computer vision, comprising:

[0005] using a pre-set target detection model to sequentially perform target detection on the collected images and output multiple target detection images, the collected images being collected from the received target video according to a pre-set time interval;

[0006] traversing each target detection image of each frame of collected images, obtaining the same person of each target detection image corresponding to the target pedestrian, and storing it as group data, the same person being the target pedestrian corresponding to the target detection image closest to the current target pedestrian;

[0007] performing time sequence tracking on each target pedestrian, and in response to one target pedestrian leaving the target video, obtaining a pedestrian group according to the group data and outputting.

[0008] For example, in the group recognition method provided by some embodiments of the present application, the step of traversing each target detection image of each frame of collected images, obtaining the same person of each target detection image corresponding to the target pedestrian, and storing it as group data further comprises: traversing each target detection image of each frame of collected images, obtaining the pedestrian attribute of each target detection image corresponding to the target pedestrian and the same person of the target pedestrian, and storing it as group data;

[0009] The step of performing time sequence tracking on each target pedestrian, and in response to one target pedestrian leaving the target video, obtaining a pedestrian group according to the group data and outputting further comprises: performing time sequence tracking on each target pedestrian, and in response to one target pedestrian leaving the target video, obtaining a pedestrian group, and outputting the relationship attribute of the pedestrian group according to the pedestrian attribute of each target pedestrian in the pedestrian group.

[0010] For example, in the group recognition method provided in some embodiments of the present application, the step of traversing each target detection image of each frame of the collected images, obtaining the pedestrian attribute of the target pedestrian corresponding to each target detection image, and the same pedestrian of the target pedestrian, and storing as group data further comprises:

[0011] The following operation steps are performed until all target detection images of the current frame of the collected images are traversed:

[0012] A target detection image is taken out from the current frame of the collected images;

[0013] The target detection image is identified using a pre-set human attribute recognition model, and the pedestrian attribute of the corresponding target pedestrian is output;

[0014] The distance between the target pedestrian and other target detection images is calculated to obtain the same pedestrian of the target pedestrian, and is stored in the group data. For example, in the group recognition method provided in some embodiments of the present application, after the step of taking out a target detection image from the current frame of the collected images, and before the step of identifying the target detection image using a pre-set human attribute recognition model and outputting the pedestrian attribute of the corresponding target pedestrian, the group relationship recognition method further comprises:

[0015] It is judged whether the current target detection image meets the pre-set image size threshold, and if not, the operation steps of the current target detection image are exited;

[0016] It is judged whether the current target detection image meets the pre-set image quality threshold, and if not, the operation steps of the current target detection image are exited.

[0017] For example, in the group recognition method provided in some embodiments of the present application, the step of calculating the distance between the target pedestrian and other target detection images to obtain the same pedestrian of the target pedestrian, and storing in the group data further comprises:

[0018] The Euclidean distance between the center point of the target detection image of the target pedestrian and the center point of other target detection images is calculated, and the shortest distance is taken as the candidate distance;

[0019] It is judged whether the candidate distance is less than the pre-set distance threshold, and if less than, the target detection image corresponding to the candidate distance is taken as the same pedestrian of the target pedestrian, and if greater than, the operation steps of the current target detection image are exited;

[0020] The target pedestrian and the corresponding same pedestrian are stored as a same pedestrian record in the group data.

[0021] For example, in the group identification method provided by some embodiments of the present application, after the human attribute recognition model pre-set is used to identify the target detection image and output the pedestrian attribute of the target pedestrian, before the distance between the target pedestrian and other target detection images is calculated to obtain the same pedestrian of the target pedestrian and stored in the group data, the group relationship identification method further comprises:

[0022] determining whether the target pedestrian is a newly identified pedestrian, and if so, establishing pedestrian information of the target pedestrian, the pedestrian information comprising a current frame screenshot, a background image and a pedestrian attribute of the target pedestrian, and if not, storing the pedestrian attribute of the current frame image of the target pedestrian.

[0023] For example, in the group identification method provided by some embodiments of the present application, the time sequence tracking of each target pedestrian, in response to a target pedestrian leaving the target video, obtaining a pedestrian group from the group data, and outputting the relationship attribute of the pedestrian group according to the pedestrian attribute of each target pedestrian in the pedestrian group further comprises:

[0024] in response to the departure of a target pedestrian, obtaining the pedestrian group associated with the target pedestrian from the group data, the pedestrian group comprising the target pedestrian, the same pedestrian of the target pedestrian, and the same pedestrian of the same pedestrian of the target pedestrian;

[0025] determining the attribute result of each pedestrian in the pedestrian group according to the pedestrian information of each pedestrian in the pedestrian group;

[0026] outputting the relationship attribute of the pedestrian group according to the attribute result of each pedestrian in the pedestrian group.

[0027] For example, in the group identification method provided by some embodiments of the present application, the outputting of the relationship attribute of the pedestrian group according to the attribute result of each pedestrian in the pedestrian group further comprises:

[0028] deleting the same pedestrian record of the pedestrian group in the group data.

[0029] For example, in the group identification method provided by some embodiments of the present application, the image comprises a target region,

[0030] Before the time sequence tracking of each target pedestrian, in response to a target pedestrian leaving the target video, obtaining a pedestrian group from the group data, and outputting the relationship attribute of the pedestrian group according to the pedestrian attribute of each target pedestrian in the pedestrian group, the group relationship identification method further comprises: in response to a target pedestrian entering the target region, storing the current frame screenshot of the target pedestrian;

[0031] The time-series tracking of each target pedestrian, in response to one target pedestrian leaving from the target video, obtaining a pedestrian group according to the group data, and outputting a relationship attribute of the pedestrian group according to the pedestrian attribute of each target pedestrian in the pedestrian group further comprises: in response to one target pedestrian leaving from the target area, obtaining a pedestrian group according to the group data, and outputting a relationship attribute of the pedestrian group according to the pedestrian attribute of each target pedestrian in the pedestrian group.

[0032] The second aspect of the present application provides a group recognition device based on computer vision, comprising:

[0033] A target detection unit is configured to use a pre-set target detection model to sequentially perform target detection on the collected images and output a plurality of target detection images, wherein the collected images are collected from the received target video at a pre-set time interval;

[0034] A same-row obtaining unit is configured to traverse each target detection image of each frame of the collected images, obtain the same-row person of the target pedestrian corresponding to each target detection image, and store the same-row person as group data, wherein the same-row person is the target pedestrian corresponding to the target detection image closest to the current target pedestrian;

[0035] A group output unit is configured to perform time-series tracking on each target pedestrian, obtain a pedestrian group according to the group data in response to one target pedestrian leaving from the target video, and output.

[0036] The third aspect of the present application provides a group-based information pushing method, comprising:

[0037] The group recognition method according to the first aspect is used to obtain a pedestrian group;

[0038] According to the pedestrian group, a matching information is selected from a pre-set information set for pushing.

[0039] The fourth aspect of the present application provides a group-based information pushing device, characterized in that it comprises the group recognition device of the second aspect and a pushing unit, wherein the pushing unit is configured to:

[0040] According to the pedestrian group output by the group recognition device, a matching information is selected from a pre-set information set for pushing.

[0041] The fifth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the group recognition method according to the first aspect;

[0042] Or

[0043] The program is executed by a processor to implement the information pushing method according to the third aspect.

[0044] The sixth aspect of the present application provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor,

[0045] The processor implements the group identification method according to the first aspect when executing the program;

[0046] Or

[0047] The processor implements the information pushing method according to the third aspect when executing the program.

[0048] The present application has the following advantages:

[0049] The present application aims at the existing problems, and formulates a group identification method and a group identification device based on computer vision. In an embodiment of the group identification method, a collected pedestrian image is detected and identified through computer vision, a pedestrian image of each frame of collected image is traversed to obtain a same pedestrian, and the same pedestrian information of the obtained multiple frames of images is analyzed to obtain a group among multiple persons, so as to make up for the problems in the prior art, reduce calibration requirements, improve the applicability and accuracy of customer group portrait analysis, and have practical application value. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0051] Figure 1 A flow chart of the group identification method according to an embodiment of the present application is shown;

[0052] Figure 2 A schematic diagram of the multiple-person scene identification according to an embodiment of the present application is shown;

[0053] Figure 3 A structural block diagram of the group identification device according to an embodiment of the present application is shown;

[0054] Figure 4 A flow chart of the information pushing method according to an embodiment of the present application is shown;

[0055] Figure 5 A structural block diagram of the information pushing device according to an embodiment of the present application is shown;

[0056] Figure 6 A structural schematic diagram of a computer device according to another embodiment of the present application is shown;

[0057] Figure 7 A flow chart of a group identification method according to another embodiment of the present application is shown. DETAILED DESCRIPTION

[0058] In order to more clearly illustrate the present application, the present application will be further described below with reference to the preferred embodiments and the accompanying drawings. Like components are denoted by the same reference numerals in the drawings. It should be understood by those skilled in the art that the specific description below is illustrative rather than limiting, and should not limit the scope of protection of the present application.

[0059] In order to solve the problems in the prior art, such as Figure 1 An embodiment of the present application provides a group identification method based on computer vision, as shown in the drawings, comprising:

[0060] using a pre-set target detection model to sequentially perform target detection on the collected images and output a plurality of target detection images, the collected images being collected from the received target video at a pre-set time interval;

[0061] traversing each target detection image of each frame of the collected images to obtain the same pedestrian of the target pedestrian corresponding to each target detection image and store the same pedestrian as group data, the same pedestrian being the target pedestrian corresponding to the target detection image closest to the current target pedestrian;

[0062] performing time sequence tracking on each target pedestrian, and in response to one target pedestrian leaving the target video, obtaining a pedestrian group according to the group data and outputting the pedestrian group.

[0063] In the embodiment, for different groups existing among multiple people, the collected pedestrian images are detected and identified by computer vision, the pedestrian images of each frame of the collected images are traversed to obtain the same pedestrian of the target pedestrian, and the same pedestrian information of the obtained multiple frames of images is analyzed to obtain the groups among multiple people, thereby making up for the problems in the prior art, reducing the calibration requirement, improving the applicability and accuracy of customer group portrait analysis, and having practical application value.

[0064] Based on the different groups among multiple people obtained in the above embodiment, in an optional embodiment, the traversing each target detection image of each frame of the collected images to obtain the same pedestrian of the target pedestrian corresponding to each target detection image and store the same pedestrian as group data further comprises: traversing each target detection image of each frame of the collected images to obtain the pedestrian attribute of the target pedestrian corresponding to each target detection image and the same pedestrian of the target pedestrian, and store the same pedestrian as group data.

[0065] The time sequence tracking of each target pedestrian, in response to one target pedestrian leaving from the target video, acquires a pedestrian group according to the group data and outputs further comprises: the time sequence tracking of each target pedestrian, in response to one target pedestrian leaving from the target video, acquires a pedestrian group, and outputs the relationship attribute of the pedestrian group according to the pedestrian attribute of each target pedestrian in the pedestrian group.

[0066] In the embodiment, for different group relationships among multiple people, such as the couple relationship and the companion relationship of traveling together, and the family relationship based on the family, different group relationships are beneficial to the analysis and further use of the customer group portrait. Specifically, the continuous acquisition images are collected by the computer vision hardware device, such as the images continuously collected by one or more cameras, each target pedestrian is detected and recognized by the computer vision software algorithm, the pedestrian attribute of each target pedestrian is acquired, and the same pedestrian of each target pedestrian is acquired according to the pre-set distance threshold, the group data including the target pedestrian and the same pedestrian of each frame acquisition image is formed by the traversal operation, the pedestrian group including the target pedestrian is determined by the time sequence tracking of the target pedestrian in multiple acquisition images, and the relationship attribute of the pedestrian group is inferred according to the pedestrian attribute of each pedestrian in the pedestrian group.

[0067] In one specific example, as Figure 7 shown, a video collected by a camera set at the entrance of a park is taken as an example for group relationship recognition.

[0068] Firstly, a pre-set target detection model is used to sequentially perform target detection on the acquisition images and output multiple target detection images, the acquisition images being collected from the received target video according to a pre-set time interval.

[0069] In the embodiment, the target video collected by the camera is preprocessed, such as multiple acquisition images being cut from the target video according to a pre-set time interval, and 12 frames of images are continuously collected according to one minute in the embodiment, such as 360 acquisition images being collected for half a minute.

[0070] The acquisition images are taken out in time sequence, and target detection is performed on each acquisition image, and the target detection model is used to detect each acquisition image in the embodiment, as Figure 2 shown, the target detection model performs target detection on the frame acquisition image, and marks the rectangular box including the recognized pedestrians, each rectangular box being a target detection image, and each rectangular box marking an ID, and the same rectangular box having the same ID in the continuous frame acquisition image.

[0071] The target detection model used in this embodiment is a YOLOV5 model for target detection in the related art. The YOLOV5 model used in this application is pre-trained using a large number of labeled images to train the model until the detection accuracy reaches a detection threshold. The specific target detection model is not limited in this application. The person skilled in the art selects an appropriate target detection model according to the actual application requirements to achieve the design criterion of detecting pedestrians in the collected images, which will not be repeated here.

[0072] It is worth noting that the time interval of the intercepted image is not specifically limited in this application. Those skilled in the art should understand that the shorter the time interval, the more images collected, the higher the accuracy of identification, and the greater the computational load. Those skilled in the art should select an appropriate time interval according to the actual application requirements, which will not be repeated here.

[0073] Secondly, each target detection image of each frame of collected images is traversed to obtain the pedestrian attribute of each target detection image corresponding to the target pedestrian, and the same pedestrian as the target pedestrian is stored as group data. The pedestrian attribute includes the gender and age of the target pedestrian, and the same pedestrian is the closest pedestrian to the target pedestrian. The group data includes each target pedestrian in each frame of collected images and the corresponding same pedestrian.

[0074] In this embodiment, the pedestrian attribute recognition is further performed on each detection image, i.e., for each pedestrian rectangular box, the same pedestrian of each pedestrian is found and stored in the group data, in order to prepare for subsequent group relationship recognition.

[0075] Specifically, the following operation steps are performed until all target detection images of the current frame of collected images are traversed:

[0076] First, a target detection image is taken out from the current frame of collected images.

[0077] In this embodiment, in order to traverse each target detection image in the collected images, the rectangular boxes are sequentially taken out from the collected images.

[0078] Secondly, a pre-set human attribute recognition model is used to identify the target detection image and output the pedestrian attribute of the corresponding target pedestrian.

[0079] In this embodiment, a deep learning neural network for recognizing pedestrian attributes is used to identify the attributes of the screenshot of the rectangular box, and the attributes of the target pedestrian corresponding to the rectangular box are output. Specifically, the pedestrian attribute includes the gender and age of the target pedestrian.

[0080] The deep learning neural network used in this embodiment is a Resnet model. A large number of labeled training set images are used to train the model in advance, for example, images of different postures and angles of the same person, until the accuracy of the verification set reaches an accuracy threshold. The present application does not limit the specific human attribute recognition model, and the person skilled in the art selects an appropriate human attribute recognition model according to the actual application requirement, and takes the attribute recognition of the pedestrian in the rectangular frame as the design criterion, which will not be described here.

[0081] Considering that some of the target detection images in the collected images have a problem of too small image size, in an optional embodiment, the group relationship recognition method further includes: judging whether the current target detection image meets a pre-set image size threshold, and if not, exiting the operation step of the current target detection image.

[0082] In this embodiment, the size of the target detection image is measured. For example, when the width of the target detection image is greater than or equal to 40 pixels and the length of the target detection image is greater than or equal to 40 pixels, it is considered that the target detection image meets the recognition size threshold, and the subsequent recognition operation is continued, otherwise the target detection image is discarded and the operation step of the current target detection image is exited. This embodiment improves the recognition accuracy of the target detection image by screening the size of the target detection image.

[0083] Considering that some of the target detection images in the collected images are blocked, in an optional embodiment, it is judged whether the current target detection image meets a pre-set image quality threshold, and if not, the operation step of the current target detection image is exited.

[0084] In this embodiment, the quality of the target detection image is measured. For example, it is judged whether the current target detection image is blocked. If the blocked part of the target detection image is less than or equal to 20% of the whole, it is considered that the target detection image meets the recognition quality, and the subsequent recognition operation is continued, otherwise the target detection image is discarded and the operation step of the current target detection image is exited. This embodiment improves the recognition accuracy of the target detection image by screening the image quality of the target detection image.

[0085] Again, the distance between the target pedestrian and other target detection images is calculated to obtain the same pedestrians of the target pedestrian and stored in the group data.

[0086] In this embodiment, for the target pedestrian whose pedestrian attribute recognition is completed, the same pedestrians of the target pedestrian are further obtained and stored for subsequent group relationship recognition.

[0087] Specifically, in an optional embodiment, the Euclidean distance between the center point of the target detection image of the target pedestrian and the center points of other target detection images is calculated, and the shortest distance is taken as a candidate distance; it is judged whether the candidate distance is less than a preset distance threshold, and if so, the target detection image corresponding to the candidate distance is taken as a same pedestrian of the target pedestrian, and if not, the operation step of the current target detection image is exited; the target pedestrian and the corresponding same pedestrian are taken as a same record and stored in the group data.

[0088] In the embodiment, the distance between the center point of the target detection image and the center points of other target detection images, i.e., the distance between the current target detection image and other target detection images except the current target detection image, or in other words, the Euclidean distance between two different rectangular frames, is calculated, and the target detection image closest to the current target detection image is found, and then it is judged whether the closest distance meets the distance requirement of the same pedestrian through a preset distance threshold, and if so, the target pedestrian corresponding to the current target detection image and the target pedestrian corresponding to the target detection image closest to the current target detection image are considered as same companions of the same trip, i.e., the same pedestrian of the target pedestrian, and the current target pedestrian and the same pedestrian are taken as a same record and stored in the group data.

[0089] It is worth noting that, considering the rectangular frame for calculating the Euclidean distance between two different rectangular frames, those skilled in the art should understand that the Euclidean distance between different rectangular frames is calculated within a certain distance range, which will not be described here.

[0090] The group data includes each target pedestrian with recognized pedestrian attributes in each collected image and the same pedestrian of the target pedestrian, and there is no same record if the target pedestrian has no same pedestrian meeting the requirement. Table 1 is a record format of the group data, recording the same records of the first to fourth frames, wherein in the first frame, there are 4 same records, including: the same pedestrian of pedestrian 1 is pedestrian 2, the same pedestrian of pedestrian 2 is pedestrian 3, the same pedestrian of pedestrian 4 is pedestrian 2, and the same pedestrian of pedestrian 3 is pedestrian 4; in the second frame, there are 4 same records, including: the same pedestrian of pedestrian 1 is pedestrian 2, the same pedestrian of pedestrian 2 is pedestrian 3, the same pedestrian of pedestrian 4 is pedestrian 1, and the same pedestrian of pedestrian 3 is pedestrian 4; in the third frame, there are 4 same records, including: the same pedestrian of pedestrian 3 is pedestrian 4, the same pedestrian of pedestrian 4 is pedestrian 1, the same pedestrian of pedestrian 5 is pedestrian 4, and the same pedestrian of pedestrian 6 is pedestrian 5; in the third frame, there is only 1 same record, and the same pedestrian of pedestrian 6 is pedestrian 5. Through the group data, the same records in each collected image are recorded, so as to facilitate the subsequent confirmation of the pedestrian group.

[0091] Table 1 group data

[0092] Frame 1 Frame 2 Frame 3 Frame 4 [1,2] 1 1 [2,3] 1 1 [2,4] 1 [3,4] 1 1 1 [4,1] 1 1 [4,5] 1 [5,6] 1 1

[0093] In an optional embodiment, considering that there is a new pedestrian entering the camera monitoring range, after identifying the pedestrian attribute of the target pedestrian, it is determined whether the target pedestrian is a newly identified pedestrian, and if so, the pedestrian information of the target pedestrian is established, including the current frame screenshot, background image and pedestrian attribute of the target pedestrian, and if not, the pedestrian attribute of the current frame image of the target pedestrian is stored.

[0094] In this embodiment, for each target pedestrian of a new identified pedestrian attribute, for example, after meeting the size and quality requirements of the target detection image, and obtaining the pedestrian attribute of the target pedestrian through the human attribute recognition model, when it is found through the ID of the target pedestrian that there is no historical record, that is, the target pedestrian is a newly identified target pedestrian, for example, due to the shielding relationship, the target pedestrian is not identified in the previous frame, or the target pedestrian is a new target pedestrian entering the camera monitoring range, the pedestrian information of the target pedestrian is established, for example, the detailed record of the target pedestrian is established, including the screenshot of the target pedestrian in the current frame image, the background image and the pedestrian attribute of the target pedestrian, for example, taking the ID of the target pedestrian as the identity, taking the screenshot as the picture of the target pedestrian, retaining the background image of the current frame as the auxiliary material for further identification and matching, and the gender and age of the target pedestrian. By establishing the pedestrian information, the subsequent image frames are accumulated, so as to facilitate subsequent operations, for example, facilitating subsequent time sequence tracking of the target pedestrian, and adjusting the age of the target pedestrian, and improving the identification accuracy of the target pedestrian.

[0095] It is worth noting that for the target pedestrian with historical records, the image screenshot, background image and pedestrian attribute identified in the current frame are accumulated in the pedestrian information of the target pedestrian, so as to facilitate subsequent determination of the attribute result of the target pedestrian.

[0096] In this embodiment, when the identification operation of one target detection image is completed using the above operation steps, it is determined whether there is an un-identified target detection image in the current captured image, if so, the above operation steps are repeated, the un-identified target detection image is taken out, and the judgment and identification are performed, and the target pedestrian identified is recorded in the group data, until the identification operation of all target detection images in the current captured image is completed, so as to facilitate subsequent group relationship identification operation.

[0097] Thirdly, time sequence tracking is performed on each target pedestrian, the pedestrian group is obtained in response to a target pedestrian leaving the target video, and the relationship attribute of the pedestrian group is output according to the pedestrian attribute of each target pedestrian in the pedestrian group.

[0098] In the embodiment, each target pedestrian in the camera monitoring range is tracked in time sequence through the identification of continuous frames, for example, the action trajectory of the target pedestrian is obtained according to the identification of the target pedestrian with the same ID in the continuous frames, and the same pedestrian of the target pedestrian obtained in different frames, so as to identify the pedestrian group including the target pedestrian, and avoid identification errors caused by accidental factors.

[0099] Specifically, it includes:

[0100] Firstly, in response to a target pedestrian leaving the target video, the pedestrian group associated with the target pedestrian is obtained according to the group data, and the pedestrian group includes the target pedestrian, the same pedestrian of the target pedestrian, and the same pedestrian of the same pedestrian of the target pedestrian.

[0101] In the embodiment, the target pedestrian leaving the camera monitoring range is the trigger point for the identification of the pedestrian group, for example, after the traversal of all target detection images of a frame of collected images, compared with the target pedestrian of the previous frame, a certain target pedestrian leaves the monitoring range, at this time, the pedestrian group of the target pedestrian is determined by using the records in the group data.

[0102] Specifically, Table 2 is a statistical table of the target pedestrian leaving based on the group data. When pedestrian 1 and pedestrian 2 leave, according to the group data, it is known that there are 17 frames of collected images of the same record of pedestrian 1 and pedestrian 2, 18 frames of collected images of the same record of pedestrian 2 and pedestrian 3, 15 frames of collected images of the same record of pedestrian 4 and pedestrian 2, and 4 frames of collected images of the same record of pedestrian 3 and pedestrian 4. Therefore, when pedestrian 1 and pedestrian 2 leave, the judgment of the pedestrian group including pedestrian 1 and pedestrian 2 is triggered, and according to the content of the group data, pedestrian 1, pedestrian 2, pedestrian 3 and pedestrian 4 are judged as one pedestrian group.

[0103] Secondly, the attribute results of each pedestrian in the pedestrian group are determined according to the pedestrian information of each pedestrian.

[0104] In the embodiment, based on the identified pedestrian group, the attribute results of each pedestrian are determined according to the pedestrian information of each pedestrian, for example, the mode of the age of pedestrian 1 is determined according to the age of pedestrian 1 in multiple frames, and is taken as the attribute result of the age of pedestrian 1, so as to obtain the attribute results of each pedestrian in the pedestrian group. For example, the attribute result of pedestrian 1 is: male, 42 years old, and the attribute result of pedestrian 2 is: female, 40 years old, the attribute result of pedestrian 3 is: male, 10 years old, and the attribute result of pedestrian 4 is: female, 6 years old.

[0105] Finally, the relationship attribute of the pedestrian group is output according to the attribute results of each pedestrian in the pedestrian group.

[0106] Based on the obtained pedestrian group and the attribute structure of each pedestrian in the pedestrian group, according to a pre-set pedestrian group relationship model, it is determined that the pedestrian group is a family group, the relationship between each pedestrian is a family relationship, for example, the same group of parents and children, and the relationship attribute of the pedestrian group is output.

[0107] Table 1: Statistics table of target pedestrian leaving based on group data

[0108]

[0109] In view of the fact that the relationship recognition operation and the relationship attribute output will be triggered repeatedly when other pedestrians in the pedestrian group leave, for example, pedestrian 3 and pedestrian 4 leave the monitoring range, in an optional embodiment, the outputting of the relationship attribute of the pedestrian group according to the attribute structure of each pedestrian in the pedestrian group further comprises deleting the same group record of the pedestrian group in the group data.

[0110] In this embodiment, when it is determined that pedestrian 1, pedestrian 2, pedestrian 3 and pedestrian 4 are a pedestrian group, the recognition and determination of the pedestrian group are triggered when the first pedestrian leaves, and after the relationship attribute of the recognized pedestrian group is output, the same group record of the pedestrian group is deleted from the group data. As shown in Table 2, when pedestrian 3 leaves, the recognition operation is triggered, and since the relevant same group record has been deleted from the group data, the same group record cannot be recognized again, so the relationship attribute of the pedestrian group will not be output again; thereby, by deleting the same group record, the redundant recognition operation is avoided, and the recognition efficiency of the group relationship recognition method is improved.

[0111] It should be noted that those skilled in the art should understand that the operation on the group data can be operated on the basis of a backup of the group data, or the group data is pre-backuped to ensure the queryability of the original data, which is not limited in the present application, and those skilled in the art should select appropriate operation mode according to actual application requirements, which will not be described here.

[0112] Further, as shown in Table 2, when pedestrian 4 leaves, the recognition operation is triggered. At this time, according to the group data, although there is no record of pedestrians 1, 2, 3, and 4 traveling together, there is a record of pedestrians 4, 5, and 6 traveling together. Therefore, pedestrians 4, 5, and 6 are identified as a pedestrian group. Then, the attribute results of pedestrians 4, 5, and 6 are determined separately. For example, the attribute result of pedestrian 4 is: female, age 6 years old; the attribute result of pedestrian 5 is: female, age 68 years old; and the attribute result of pedestrian 6 is: male, age 70 years old. Based on the attribute structure of each pedestrian in this pedestrian group, according to the preset pedestrian group relationship model, it is determined that this pedestrian group is a family group, and the relationship between each pedestrian is a family relationship, such as grandparents and children traveling together. The relationship attributes of this pedestrian group are then output.

[0113] Similarly, when pedestrians 5 and 6 leave, the relationship attribute of the pedestrian group is triggered but not recognized or output.

[0114] As shown in Table 2, when pedestrian 7 leaves, a new identification operation is triggered. Based on the group data, the records of pedestrian 7's companions and the records of each companion of pedestrian 7 are obtained, thereby determining the pedestrian group and the attribute results of each pedestrian in the group. Based on the pedestrian group relationship model, the relationship attributes of the pedestrian group are output.

[0115] Thus, the task of... Figure 2 The method for identifying group relationships among multiple people at the park entrance involves using computer vision to detect pedestrians and identify their attributes among the numerous pedestrians moving around the park entrance. Based on a preset distance threshold, it identifies the companions of each target pedestrian and forms group data including target pedestrians and their companions through a traversal operation. By tracking the target pedestrians in multiple frames of images over time, it determines the pedestrian group including the target pedestrians and infers the relationship attributes of this group based on the pedestrian attributes of each individual pedestrian.

[0116] To ensure accurate pedestrian departure detection, in one optional embodiment, the tracker used in computer vision performs further analysis during temporal pedestrian tracking. For example, if the target pedestrian is not observed for several consecutive frames, a pedestrian group recognition operation is triggered to improve the accuracy of pedestrian departure detection. Specifically, in this embodiment, for example, if the target pedestrian is not observed for 50 consecutive frames, i.e., the pedestrian has not appeared within the camera's monitoring range for 4 consecutive seconds, it is confirmed that the pedestrian has left, thereby triggering the pedestrian group recognition operation.

[0117] In an optional embodiment, the acquired image includes a target region. Before performing temporal tracking of each target pedestrian, obtaining a pedestrian group based on group data in response to a target pedestrian leaving the target video, and outputting the relationship attributes of the pedestrian group based on the pedestrian attributes of each target pedestrian in the pedestrian group, the group relationship identification method further includes: storing a current frame screenshot of the target pedestrian in response to a target pedestrian entering the target region; the step of performing temporal tracking of each target pedestrian, obtaining a pedestrian group based on group data in response to a target pedestrian leaving the target video, and outputting the relationship attributes of the pedestrian group based on the pedestrian attributes of each target pedestrian in the pedestrian group further includes: obtaining a pedestrian group based on group data in response to a target pedestrian leaving the target region, and outputting the relationship attributes of the pedestrian group based on the pedestrian attributes of each target pedestrian in the pedestrian group.

[0118] In this embodiment, as Figure 2 As shown, the monitoring range of the camera is divided, with range 10 being the target area. Pedestrians within this target area are identified and analyzed as a group. Specifically, when the center point between the feet of a target pedestrian enters the target area, that pedestrian is considered a target pedestrian to be identified. Figure 2 As shown, multiple rectangles 20 represent target pedestrians within the target area.

[0119] Specifically, when a target pedestrian enters the target area, a screenshot of the current frame is stored for display purposes. The process of identifying the pedestrian group, determining the target pedestrian's attributes, and outputting the relationship attributes of the pedestrian group is triggered when the target pedestrian leaves the target area. For example, Figure 2 In the three groups of pedestrians on the left, middle, and right, the two target pedestrians were identified as classmates, companions, and lovers of different genders.

[0120] It should be noted that those skilled in the art should understand that this embodiment is only used to illustrate the specific implementation of this application. This application does not specifically limit the pedestrian group relationship model. Those skilled in the art can further identify the relationship of pedestrian groups based on this application, such as by further identifying the relationship of pedestrian groups through pedestrian attributes, pedestrian action tracking, etc. All of these are within the protection scope of this application and will not be elaborated here.

[0121] Corresponding to the group identification method provided in the above embodiments, one embodiment of this application also provides a group identification device that applies the above group identification method, such as... Figure 3 As shown, it includes:

[0122] The target detection unit is configured to sequentially perform target detection on the collected images using a preset target detection model and output a plurality of target detection images, the collected images being collected from the received target video at preset time intervals;

[0123] The same person acquisition unit is configured to traverse each target detection image of each frame of the collected images, acquire the same person of each target detection image corresponding to the target pedestrian, and store the same person as group data, the same person being the target pedestrian corresponding to the target detection image closest to the current target pedestrian;

[0124] The group output unit is configured to perform time sequence tracking on each target pedestrian, acquire a pedestrian group according to the group data in response to one target pedestrian leaving the target video, and output the pedestrian group.

[0125] In the embodiment, for different groups existing among multiple people, the collected pedestrian images are detected and recognized by computer vision, the pedestrian images of each frame of the collected images are traversed to acquire the same person of the target pedestrian, and the same person information of the acquired multiple frames of images is analyzed to acquire the group among multiple people, thereby making up for the problems in the prior art, reducing the calibration requirement, improving the applicability and accuracy of customer group portrait analysis, and having practical application value.

[0126] Since the group recognition device provided in the embodiment of the application corresponds to the group recognition method provided in the above several embodiments, the preceding embodiments are also applicable to the group recognition device provided in the embodiment, and will not be described in detail in the embodiment.

[0127] Based on the above group recognition method, as shown in Figure 4 One embodiment of the application further provides a group-based information pushing method, comprising:

[0128] Acquiring a pedestrian group according to the group recognition method;

[0129] Selecting matched information from a preset information set according to the pedestrian group to push the matched information.

[0130] In the embodiment, based on the recognition of the group among multiple people, matched information is selected according to the recognized group information to push, thereby realizing accurate information pushing on the basis of customer group portrait analysis, for example, according to the group information recognition in the flowing crowd at the park entrance, appropriate pushing information is matched according to different pedestrian groups, and the efficiency of information pushing is effectively improved.

[0131] Further, an embodiment of the present application can further acquire and analyze the pedestrian attributes of each target pedestrian, and perform matching and pushing according to the relationship attributes of each pedestrian group, for example, for a pedestrian group with identified family relationship attributes, information content with family outing as the theme or related theme is pushed; for a pedestrian group with identified companion relationship attributes, information content with companion outing as the theme or related theme is pushed; further, information filtering can be performed according to the gender and age attributes of the identified pedestrians, effectively improving the applicability and practicality of information pushing.

[0132] Corresponding to the information pushing method provided by the above embodiment, an embodiment of the present application further provides an information pushing device applying the above information pushing method, as shown in the following table, comprising a group identification device, and a pushing unit, the pushing unit is configured to select and push matching information from a pre-set information set according to the pedestrian group output by the group identification device. Figure 5

[0133] In the present embodiment, based on the group information identification of the information pushing device among multiple persons, matching information is selected and pushed by the pushing unit according to the identified group information, so as to realize accurate information pushing on the basis of customer group portrait analysis, effectively improving the applicability and practicality of information pushing.

[0134] Another embodiment of the present application provides a computer readable storage medium, which stores a computer program, the program is executed by a processor to realize the group identification method based on computer vision, or realize the information pushing method based on groups.

[0135] In actual application, the computer readable storage medium can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present embodiment, the computer readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, device or component.

[0136] ​Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0137] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0138] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0139] like Figure 6 As shown, another embodiment of the present invention provides a structural schematic diagram of a computer device. Figure 6 The computer device 12 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0140] like Figure 6 As shown, the computer device 12 is represented in the form of a general-purpose computing device. The components of the computer device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and a bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0141] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0142] Computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0143] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (…). Figure 6 Not shown; usually referred to as a "hard drive"). Although Figure 6 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. Memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0144] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0145] Computer device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with the computer device 12, and / or with any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through input / output (I / O) interface 22. Furthermore, computer device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) through network adapter 20. Figure 6 As shown, network adapter 20 communicates with other modules of computer device 12 via bus 18. It should be understood that, although... Figure 6 As not shown, it can be used in conjunction with computer device 12 with other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0146] The processor unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a computer vision-based group recognition method or a group-based information push method provided in the embodiments of the present invention.

[0147] This invention addresses existing problems by developing a group recognition method and device based on computer vision. In one embodiment of the group recognition method, for different groups existing among multiple people, computer vision is used to detect and identify the collected pedestrian images. The pedestrian images in each frame of the collected images are traversed to obtain the companions of the target pedestrian. The companion information of the acquired multi-frame images is analyzed to obtain the group among multiple people. This overcomes the problems existing in the prior art, reduces the calibration requirements, and improves the applicability and accuracy of customer group profile analysis, thus having practical application value.

[0148] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. For those skilled in the art, other variations or modifications can be made based on the above description. It is impossible to exhaustively list all the implementation methods here. All obvious variations or modifications derived from the technical solutions of the present invention are still within the protection scope of the present invention.

Claims

1. A group recognition method based on computer vision, characterized in that, include: The pre-set target detection model is used to sequentially perform target detection on the acquired images and output multiple target detection images. The acquired images are collected from the received target video at preset time intervals. Traverse each target detection image in each frame of captured images, obtain the companions of the target pedestrian corresponding to each target detection image, and store them as group data. The companions are the target pedestrians corresponding to the target detection images that are closest to the current target pedestrian. Perform time-series tracking for each target pedestrian; in response to a target pedestrian leaving the target video, obtain and output the pedestrian group based on the group data. The step of traversing each target detection image of each frame of captured images, obtaining the companions of the target pedestrian corresponding to each target detection image, and storing them as group data further includes: traversing each target detection image of each frame of captured images, obtaining the pedestrian attributes of the target pedestrian corresponding to each target detection image, and the companions of the target pedestrian, and storing them as group data; The step of performing time-series tracking of each target pedestrian, and in response to a target pedestrian leaving the target video, obtaining and outputting a pedestrian group based on group data, further includes: performing time-series tracking of each target pedestrian, obtaining a pedestrian group in response to a target pedestrian leaving the target video, and outputting the relationship attributes of the pedestrian group based on the pedestrian attributes of each target pedestrian in the pedestrian group; The step of outputting the relationship attributes of the pedestrian group based on the pedestrian attributes of each target pedestrian in the pedestrian group further includes: Delete the records of pedestrian groups in the group data; The group data includes each target pedestrian in each frame of the captured image, and the corresponding companion. The pedestrian group includes the target pedestrian, the companion of the target pedestrian, and the companion of the companion of the target pedestrian.

2. The group identification method according to claim 1, characterized in that, The step of traversing each target detection image in each frame of acquired images, obtaining the pedestrian attributes of the target pedestrian corresponding to each target detection image, and the companions of the target pedestrian, and storing them as group data further includes: Perform the following steps until all target detection images in the current frame have been traversed: Extract a target detection image from the currently acquired frame; The target detection image is identified using a pre-set human attribute recognition model, and the corresponding pedestrian attributes of the target pedestrian are output. The distance between the target pedestrian and other target detection images is calculated to obtain the pedestrian's companions, and stored in the group data.

3. The group identification method according to claim 2, characterized in that, After retrieving a target detection image from the currently captured image, and before recognizing the target detection image using a pre-set human attribute recognition model and outputting the corresponding pedestrian attributes of the target pedestrian, the group recognition method further includes: Determine whether the current target detection image meets the preset image size threshold; otherwise, exit the operation steps for the current target detection image. Determine whether the current target detection image meets the preset image quality threshold; otherwise, exit the operation steps for the current target detection image.

4. The group identification method according to claim 3, characterized in that, The step of calculating the distance between the target pedestrian and other target detection images to obtain the pedestrian's companions and storing them in the group data further includes: Calculate the Euclidean distance between the center point of the target pedestrian's target detection image and the center points of other target detection images, and use the shortest distance as the candidate distance; Determine whether the candidate distance is less than a preset distance threshold. If it is less, then the target detection image corresponding to the candidate distance is taken as the companion of the target pedestrian. If it is greater, exit the operation step of the current target detection image. The target pedestrian and their corresponding companion are stored as a single companion record in the group data.

5. The group identification method according to claim 4, characterized in that, After identifying the target detection image using a pre-set human attribute recognition model and outputting the corresponding pedestrian attributes of the target pedestrian, and before calculating the distance between the target pedestrian and other target detection images to obtain the pedestrian's companions and storing them in the group data, the group recognition method further includes: Determine whether the target pedestrian is a newly identified pedestrian. If so, establish pedestrian information for the target pedestrian, which includes the current frame screenshot, background image, and pedestrian attributes. Otherwise, store the pedestrian attributes of the current frame image of the target pedestrian.

6. The group identification method according to claim 5, characterized in that, The step of performing time-series tracking of each target pedestrian, obtaining a pedestrian group in response to a target pedestrian leaving the target video, and outputting the relationship attributes of the pedestrian group based on the pedestrian attributes of each target pedestrian in the pedestrian group further includes: In response to a target pedestrian leaving the target video, the pedestrian group associated with the target pedestrian is obtained based on the group data; Determine the attribute results of each pedestrian based on the pedestrian information of each pedestrian in the pedestrian group; Output the relationship attributes of the pedestrian group based on the attribute results of each pedestrian in the pedestrian group.

7. The group identification method according to any one of claims 2-6, characterized in that, The acquired image includes the target region. Before performing time-series tracking of each target pedestrian, obtaining a pedestrian group in response to a target pedestrian leaving the target video, and outputting the relationship attributes of the pedestrian group based on the pedestrian attributes of each target pedestrian in the pedestrian group, the group identification method further includes: storing a screenshot of the current frame of the target pedestrian in response to a target pedestrian entering the target area; The step of performing time-series tracking of each target pedestrian, obtaining a pedestrian group in response to a target pedestrian leaving the target video, and outputting the relationship attributes of the pedestrian group based on the pedestrian attributes of each target pedestrian in the pedestrian group further includes: in response to a target pedestrian leaving the target area, obtaining a pedestrian group based on group data, and outputting the relationship attributes of the pedestrian group based on the pedestrian attributes of each target pedestrian in the pedestrian group.

8. A group recognition device based on computer vision, characterized in that, include: The target detection unit is configured to sequentially perform target detection on the acquired images using a preset target detection model and output multiple target detection images, wherein the acquired images are acquired from the received target video at preset time intervals; The peer acquisition unit is configured to traverse each target detection image of each frame of captured images, acquire the peers of the target pedestrian corresponding to each target detection image, and store them as group data. The peers are the target pedestrians corresponding to the target detection images that are closest to the current target pedestrian. A group output unit is configured to perform temporal tracking of each target pedestrian, and in response to a target pedestrian leaving the target video, acquire and output the pedestrian group based on the group data. The peer acquisition unit is configured to: traverse each target detection image in each frame of captured images, acquire the pedestrian attributes of the target pedestrian corresponding to each target detection image, and the peers of the target pedestrian, and store them as group data. The group data includes each target pedestrian in each frame of captured images and the corresponding peers. The pedestrian group includes the target pedestrian, the peers of the target pedestrian, and the peers of the peers of the target pedestrian. The group output unit is configured to: perform time-series tracking of each target pedestrian; obtain a pedestrian group in response to a target pedestrian leaving the target video; output the relationship attributes of the pedestrian group based on the pedestrian attributes of each target pedestrian in the pedestrian group; and delete the peer records of the pedestrian group in the group data.

Images