Target matching method, event determination method, device and computer equipment

Abstract

The present application relates to a target matching method, an event determination method, an apparatus, a computer device, a storage medium, and a computer program product. The method comprises: obtaining a first object identifier for which an object identifier match fails and the time at which a first behavior corresponding to the first object identifier occurs, wherein the first object identifier is the identifier of the object that performed the first behavior; searching for a set of second object identifiers within a preset time period corresponding to the time at which the first behavior occurred, wherein the second object identifier is the identifier of an object that has a tendency to perform a second behavior, and the second behavior and the first behavior are opposite behaviors performed with respect to a target area; matching the first object identifier and the second object identifier set based on the similarity between the object image corresponding to the first object identifier and the object images corresponding to each second object identifier in the second object identifier set, and determining a second object identifier that successfully matches the first object identifier. This method can improve the accuracy of target matching.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a target matching method, an event determination method, an apparatus, a computer device, a storage medium, and a computer program product. Background Art

[0002] With the development of computer vision technology, object matching technology has emerged. Object matching technology matches detected objects by comparing their image similarities. Object matching technology has applications in areas such as passenger flow counting and traffic scheduling.

[0003] Traditional target matching methods, such as in the field of customer flow counting, typically involve installing a camera above a store entrance. Upon detecting a customer, the camera begins capturing their image. Based on these captured images of the customer at different times, the customer's trajectory can be determined, yielding a customer trajectory. If a customer's trajectory indicates entering the store from outside the store, the customer is considered to have entered the store; otherwise, the customer is considered to have exited the store. The image of the customer entering the store is associated with an entry ID, while the image of the customer exiting the store is associated with an exit ID. By calculating the similarity between the images of the customer entering and exiting the store, the entry and exit IDs corresponding to the images with the highest similarity are matched.

[0004] However, in traditional technologies, objects in images may be missed or misdetected due to reasons such as object occlusion, resulting in the loss of the object's trajectory. The object identifier corresponding to the missing trajectory will not be used in subsequent target matching, which will inevitably lead to the failure of successful matching of some object identifiers, resulting in low target matching accuracy. Summary of the Invention

[0005] Based on this, it is necessary to provide a target matching method, event determination method, device, computer equipment, storage medium and computer program product that can improve the accuracy of target matching in response to the above technical problems.

[0006] In a first aspect, the present application provides a target matching method. The method comprises:

[0007] Obtaining a first object identifier for which object identifier matching fails and a time at which a first behavior corresponding to the first object identifier occurs, wherein the first object identifier is an identifier of an object on which the first behavior occurs;

[0008] Searching for a set of second object identifiers within a preset time period corresponding to the time when the first behavior occurs, wherein the second object identifiers are identifiers of objects that have a tendency to perform a second behavior, and the second behavior and the first behavior are opposite behaviors performed with respect to a target area;

[0009] The first object identifier and the second object identifier set are matched based on the similarity between the object image corresponding to the first object identifier and the object images corresponding to each second object identifier in the second object identifier set to determine a second object identifier that successfully matches the first object identifier.

[0010] In one embodiment, the first behavior refers to entering a target area, and the second behavior refers to leaving the target area;

[0011] The searching for a second object identifier set within a preset time period corresponding to the time when the first behavior occurs includes:

[0012] Searching for an initial object identification set and an object trajectory set corresponding to the initial object identification set within a preset period of time after the first behavior occurs;

[0013] From the object trajectory set, a subset of object trajectories that meets a second behavior occurrence trend identification condition is selected;

[0014] The object identification subset corresponding to the object trajectory subset in the initial object identification set is determined as the second object identification set.

[0015] In one embodiment, the first behavior refers to leaving the target area, and the second behavior refers to entering the target area;

[0016] The searching for a second object identifier set within a preset time period corresponding to the time when the first behavior occurs includes:

[0017] Searching for an initial object identification set and an object trajectory set corresponding to the initial object identification set within a preset period before the occurrence time of the first behavior;

[0018] From the object trajectory set, a subset of object trajectories that meets a second behavior occurrence trend identification condition is selected;

[0019] The object identification subset corresponding to the object trajectory subset in the initial object identification set is determined as the second object identification set.

[0020] In one embodiment, selecting a subset of object trajectories from the object trajectory set that meets the second behavior occurrence trend identification condition includes:

[0021] Eliminating object trajectories associated with entry and exit behavior identifiers from the object trajectory set to obtain a retained object trajectory set, wherein the entry and exit behavior identifiers are used to indicate that the object trajectories are trajectories that enter and exit the target area;

[0022] In the retained object trajectory set, a set of object trajectories that conform to a preset trajectory rule is determined as an object trajectory subset that meets the second behavior occurrence trend recognition condition.

[0023] In one embodiment, obtaining the first object identifier for which the object identifier matching fails includes:

[0024] Acquire a third object identification set entering the target area and an entry image set corresponding to the third object identification set;

[0025] Acquire a fourth object identification set leaving the target area and an out-of-area image set corresponding to the fourth object identification set;

[0026] matching each third object identifier in the third object identifier set and each fourth object identifier in the fourth object identifier set based on similarities between each in-domain image in the in-domain image set and each out-domain image in the out-domain image set, to obtain third object identifiers and / or fourth object identifiers for which matching fails;

[0027] The third object identifier and / or the fourth object identifier that failed to match is determined as the first object identifier that failed to match the object identifier.

[0028] In a second aspect, the present application provides a method for determining an event. The method comprises:

[0029] Obtaining a first object identifier for which object identifier matching fails and a time at which a first behavior corresponding to the first object identifier occurs, wherein the first object identifier is the object identifier at which the first behavior occurs;

[0030] Searching for a set of second object identifiers within a preset time period corresponding to the time when the first behavior occurs, wherein the second object identifiers are object identifiers that have a trend of occurring a second behavior, and the second behavior and the first behavior are opposite behaviors performed with respect to the target area;

[0031] matching the first object identifier with the second object identifier set based on similarities between the object image corresponding to the first object identifier and the object images corresponding to each second object identifier in the second object identifier set, and determining a second object identifier that successfully matches the first object identifier;

[0032] determining, based on the object trajectory corresponding to the second object identifier that successfully matches the first object identifier, an occurrence time of a second behavior corresponding to the second object identifier;

[0033] Determining the occurrence time of the second behavior corresponding to the second object identifier as the occurrence time of the second behavior corresponding to the first object identifier;

[0034] A target area entry and exit event corresponding to the first object identifier is determined according to the first object identifier, the time at which the first behavior occurs, and the time at which the second behavior occurs.

[0035] In a third aspect, the present application further provides a target matching device. The device comprises:

[0036] A data acquisition module, configured to acquire a first object identifier for which object identifier matching fails and a time at which a first behavior corresponding to the first object identifier occurs, wherein the first object identifier is an identifier of an object on which the first behavior occurs;

[0037] an identifier search module, configured to search for a set of second object identifiers within a preset time period corresponding to the time when the first behavior occurred, wherein the second object identifiers are identifiers of objects that have a tendency to perform a second behavior, and the second behavior and the first behavior are opposite behaviors performed with respect to a target area;

[0038] an identifier matching module, configured to match the first object identifier with the second object identifier set based on similarities between the object image corresponding to the first object identifier and the object images corresponding to each second object identifier in the second object identifier set, and determine a second object identifier that successfully matches the first object identifier.

[0039] In a fourth aspect, the present application further provides a computer device. The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0040] Obtaining a first object identifier for which object identifier matching fails and a time at which a first behavior corresponding to the first object identifier occurs, wherein the first object identifier is an identifier of an object on which the first behavior occurs;

[0041] Searching for a set of second object identifiers within a preset time period corresponding to the time when the first behavior occurs, wherein the second object identifiers are identifiers of objects that have a tendency to perform a second behavior, and the second behavior and the first behavior are opposite behaviors performed with respect to a target area;

[0042] The first object identifier and the second object identifier set are matched based on the similarity between the object image corresponding to the first object identifier and the object images corresponding to each second object identifier in the second object identifier set to determine a second object identifier that successfully matches the first object identifier.

[0043] In a fifth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0044] Obtaining a first object identifier for which object identifier matching fails and a time at which a first behavior corresponding to the first object identifier occurs, wherein the first object identifier is an identifier of an object on which the first behavior occurs;

[0045] Searching for a set of second object identifiers within a preset time period corresponding to the time when the first behavior occurs, wherein the second object identifiers are identifiers of objects that have a tendency to perform a second behavior, and the second behavior and the first behavior are opposite behaviors performed with respect to a target area;

[0046] The first object identifier and the second object identifier set are matched based on the similarity between the object image corresponding to the first object identifier and the object images corresponding to each second object identifier in the second object identifier set to determine a second object identifier that successfully matches the first object identifier.

[0047] In a sixth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0048] Obtaining a first object identifier for which object identifier matching fails and a time at which a first behavior corresponding to the first object identifier occurs, wherein the first object identifier is an identifier of an object on which the first behavior occurs;

[0049] Searching for a set of second object identifiers within a preset time period corresponding to the time when the first behavior occurs, wherein the second object identifiers are identifiers of objects that have a tendency to perform a second behavior, and the second behavior and the first behavior are opposite behaviors performed with respect to a target area;

[0050] The first object identifier and the second object identifier set are matched based on the similarity between the object image corresponding to the first object identifier and the object images corresponding to each second object identifier in the second object identifier set to determine a second object identifier that successfully matches the first object identifier.

[0051] The above-mentioned target matching method, event determination method, apparatus, computer device, storage medium, and computer program product, for a first object identifier that fails to match an object identifier, the first object identifier being the identifier of an object that has performed a first behavior, searches for a set of second object identifiers within a preset time period corresponding to the time of occurrence of the first behavior corresponding to the first object identifier, the second object identifier being the identifier of an object with a tendency to perform the second behavior, and then performs identifier matching based on the similarity between the object image corresponding to the first object identifier and the object images corresponding to each second object identifier in the second object identifier set, to determine a second object identifier that successfully matches the first object identifier. It can be seen that the present application matches the first object identifier that fails to match an object identifier with a set of second object identifiers that have been missed or falsely detected, and then finds a second object identifier that matches the first object identifier in the second object identifier set, thereby improving the accuracy of target matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 A diagram of an application environment of a target matching method in one embodiment;

[0053] Figure 2 1 is a flow chart of a target matching method in one embodiment;

[0054] Figure 3 A schematic diagram of store entry and exit behavior recognition in one embodiment;

[0055] Figure 4 is a schematic diagram of target matching based on object trajectory in one embodiment;

[0056] Figure 5 is a structural block diagram of a target matching device in one embodiment;

[0057] Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0059] First, before specifically introducing the technical solutions of the embodiments of the present application, we will first introduce the technical background or technical evolution of the embodiments of the present application. Taking customer flow statistics as an example, under normal circumstances, because current computer vision tracking technology is difficult to achieve long-term continuous tracking, when a customer enters a store and then reappears in the camera image when leaving the store, his ID may change. In other words, the tracking algorithm identifies one person entering the store and another person leaving the store, that is, the customer IDs of the same person entering and leaving the store are different. By matching the IDs, we are actually matching which customer IDs of the customers entering the store and leaving the store are actually the same person. Because a person entering the store will inevitably leave the store, after the matching is completed, some customer IDs of customers entering the store may not be matched, and some customer IDs of customers leaving the store may not be matched. This may be caused by missed detection of events. In other words, the IDs of customers entering the store that are not matched are because their exit events were missed; the IDs of customers leaving the store that are not matched are because their entry events were missed. For example, when multiple customers enter a store together, because only one camera is installed at the store entrance to capture the store to save costs, these customers may block each other's view, resulting in missing tracks for some customers, and thus missed detection of store entry events. Conversely, missed detection of store exit events is also possible. Based on this background, the applicant, through long-term model simulation research and development, as well as the collection, demonstration, and verification of experimental data, invented and created the technical solutions described in the following embodiments, for which the applicant has devoted a great deal of creative effort.

[0060] The target matching method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown, the image acquisition device 102 communicates with the server 104 via a network. A data storage system can store data that the server 104 needs to process. The data storage system can be integrated with the server 104, or placed on a cloud or other network server. Specifically, the image acquisition device 102 captures an object image corresponding to a first object identifier that fails to match an object identifier near a target area, as well as object images corresponding to each second object identifier in a second object identifier set, and uploads them to the server 104. The server 104 obtains the first object identifier that fails to match an object identifier and the time of occurrence of a first behavior corresponding to the first object identifier. The first object identifier is the identifier of the object that performed the first behavior. The server 104 then searches for a set of second object identifiers within a preset time period corresponding to the time of occurrence of the first behavior. The second object identifier is the identifier of an object that has a tendency to perform a second behavior, which is the opposite of the first behavior and is performed with respect to the target area. Finally, based on the similarity between the object image corresponding to the first object identifier and the object images corresponding to each second object identifier in the second object identifier set, the server 104 matches the first object identifier with the set of second object identifiers, and determines a second object identifier that successfully matches the first object identifier. The server 104 can be implemented as a standalone server or a server cluster consisting of multiple servers.

[0061] In one embodiment, Figure 2 As shown, a target matching method is provided, which is applied to Figure 1 The following steps are used as an example to illustrate the server in the example:

[0062] Step S202: Acquire the first object identifier for which the object identifier matching fails and the time at which the first behavior corresponding to the first object identifier occurs.

[0063] The first object identifier is the identifier of the object that performed the first behavior. The object can be a person, a vehicle, or the like. The first behavior is an action performed against a target area. Alternatively, the first behavior can refer to the object entering the target area. The time at which the first behavior occurs refers to the time at which the object entered the target area.

[0064] Specifically, taking an object entering or exiting a target area as an example, before step S202, the process further includes: the server obtaining a third object identification set for entering the target area and an entry-domain image set corresponding to the third object identification set, and obtaining a fourth object identification set for leaving the target area and an exit-domain image set corresponding to the fourth object identification set; then, the server matches each third object identification in the third object identification set with each fourth object identification in the fourth object identification set based on the similarity between each entry-domain image in the entry-domain image set and each exit-domain image in the exit-domain image set, and obtains a third object identification and / or fourth object identification for which a match fails; finally, the server determines the third object identification and / or fourth object identification for which a match fails as the first object identification for which an object identification match fails.

[0065] Afterwards, the server obtains the time at which the first behavior corresponding to the first object identifier occurs. Optionally, taking the example of the first behavior being the object entering the target area, the image acquisition device uses target tracking technology to capture the object image of the object at each moment, and the object trajectory of the object is determined based on the position information of the object in each object image. The range of the target area is pre-set. If the object trajectory enters the target area from outside the target area, it is determined that the object has entered the target area. The time when the object trajectory first overlaps with the target area is determined as the time at which the first behavior occurs.

[0066] Step S204: searching for a second object identifier set within a preset time period corresponding to the time when the first behavior occurs.

[0067] Among them, the second object identifier is the identifier of an object with a tendency to perform a second behavior, and the second behavior and the first behavior are opposite behaviors performed on the target area. The tendency of the second behavior can be understood as that after the server obtains the object trajectory of an object, it identifies the object trajectory based on the second behavior identification condition and determines that it is not the second behavior. However, the actual situation is that the object has performed the second behavior. The reason for this problem may be that the image acquisition device has missed or misdetected a part of the object's trajectory, that is, the trajectory is missing, resulting in an incomplete object trajectory. Therefore, in order to successfully match the object identifier, it is necessary to find this missing object trajectory, and then find the relevant information of the second behavior that matches the first behavior. Since the object trajectory cannot be identified as the second behavior due to incomplete trajectory, it is called the tendency of the second behavior.

[0068] For example, the second behavior may be leaving the target area, and the method for identifying the second behavior may be: if the object trajectory steps out of the target area from the target area to the outside of the target area, then it is determined that the object has left the target area. However, due to reasons such as object occlusion, the image acquisition device (shooting the picture outside the target area) may not capture the image of the object stepping out of the target area. Therefore, the object trajectory only contains trajectory points that are getting farther and farther away from the store, and the trajectory points that step out of the target area are missing. In view of this, the method for identifying the trend of the second behavior may be: in the object trajectory that is not associated with the entry and exit behavior identifier, search for the object trajectory of "each trajectory point is getting farther and farther away from the store", and determine it as the object trajectory corresponding to the trend of the second behavior. Among them, the entry and exit behavior identifier is used to indicate that the object trajectory is a trajectory of entering and exiting the target area.

[0069] Specifically, the server searches for the second object identifier set within a preset time period corresponding to the time when the first behavior occurs.

[0070] Step S206 , matching the first object identifier with the second object identifier set based on the similarity between the object image corresponding to the first object identifier and the object images corresponding to each second object identifier in the second object identifier set, and determining a second object identifier that successfully matches the first object identifier.

[0071] Specifically, the server obtains an object image corresponding to the first object identifier and an object image corresponding to each second object identifier in the second object identifier set. The object image can be acquired by an image acquisition device installed near the target area. The server then calculates the similarity between the object image corresponding to the first object identifier and the object images corresponding to each second object identifier in the second object identifier set. For example, cosin is used to calculate the similarity between the images. Finally, based on the similarity, the server matches the first object identifier with the second object identifier set to determine the second object identifier that successfully matches the first object identifier. Optionally, if the similarity is greater than a similarity threshold, the corresponding first object identifier and the second object identifier are matched.

[0072] In the above-mentioned target matching method, for a first object identifier for which an object identifier match fails, the first object identifier is the identifier of an object that has performed a first behavior, and a second object identifier set within a preset time period corresponding to the time of occurrence of the first behavior corresponding to the first object identifier is searched. The second object identifier is the identifier of an object with a tendency to perform the second behavior, and then identifier matching is performed based on the similarity between the object image corresponding to the first object identifier and the object images corresponding to each second object identifier in the second object identifier set, to determine a second object identifier that successfully matches the first object identifier. It can be seen that the present application matches the first object identifier for which an object identifier match fails with the set of second object identifiers that have been missed or falsely detected, and then finds the second object identifier in the second object identifier set that matches the first object identifier, thereby improving the accuracy of target matching.

[0073] In one embodiment, the first behavior refers to entering the target area, and the second behavior refers to leaving the target area. Based on this, step S204 includes the following steps:

[0074] Step S2042, searching for an initial object identification set and an object trajectory set corresponding to the initial object identification set within a preset period after the time when the first behavior occurs;

[0075] Step S2044: selecting a subset of object trajectories that meet the second behavior occurrence trend identification condition from the object trajectory set;

[0076] Step S2046 : Determine the object identification subset corresponding to the object trajectory subset in the initial object identification set as the second object identification set.

[0077] Specifically, taking the target area as a store area as an example, the first behavior refers to entering the store, and the second behavior refers to leaving the store. The time when the first behavior occurs refers to the time of entering the store. Since entering the store comes first and leaving the store comes later, the server searches for the initial object identification set and the object trajectory set corresponding to the initial object identification set within a preset time period after the time of entering the store. The preset time period can be determined according to the business format of the store. The initial object identification set can be a collection of identifications of all objects detected by the image acquisition device above the store door. Each initial object identification corresponds to an object trajectory. It can be understood that the initial object identification set may include the identification of objects entering the store, the identification of objects leaving the store, the identification of objects passing by the store, the identification of objects with a tendency to enter the store, and the identification of objects with a tendency to leave the store.

[0078] Then, the server searches for object trajectories in the object trajectory set that are not associated with entry and exit behavior identifiers. The object trajectory that is not associated with entry and exit behavior identifiers refers to an object trajectory that cannot be identified as entering or exiting a store. It can be understood that the object trajectory that is not associated with entry and exit behavior identifiers may include object trajectories corresponding to the identifiers of objects passing by the store, object trajectories corresponding to the identifiers of objects with a trend of entering the store, and object trajectories corresponding to the identifiers of objects with a trend of exiting the store. Based on this, among the object trajectories that are not associated with entry and exit behavior identifiers, the server searches for object trajectories that meet the law of the trend of exiting the store, and then constructs a subset of object trajectories that meet the conditions for identifying the trend of exiting the store based on the object trajectories found.

[0079] Finally, the server determines the object identification subset corresponding to the object trajectory subset in the initial object identification set as the second object identification set.

[0080] In this embodiment, for an inbound object identifier whose object identifier matching fails, a matching object identifier can be found among object identifiers with an incomplete outbound behavior trend, thereby improving the accuracy of matching the inbound object identifier and the outbound object identifier.

[0081] In one embodiment, the first behavior refers to leaving the target area, and the second behavior refers to entering the target area. Based on this, step S204 includes the following steps:

[0082] Step S2041, searching for an initial object identification set and an object trajectory set corresponding to the initial object identification set within a preset period before the first behavior occurs;

[0083] Step S2044: selecting a subset of object trajectories that meet the second behavior occurrence trend identification condition from the object trajectory set;

[0084] Step S2046 : Determine the object identification subset corresponding to the object trajectory subset in the initial object identification set as the second object identification set.

[0085] Specifically, taking the target area as a store area, the first behavior indicates entering the store, and the second behavior indicates entering the store. The time the first behavior occurs indicates the time of exiting the store. Since entering the store comes first and exiting the store comes later, the server searches for the initial object identification set and the object trajectory set corresponding to the initial object identification set within a preset period before the exit time.

[0086] Then, the server searches for object trajectories in the object trajectory set that are not associated with entry and exit behavior identifiers. The object trajectories that are not associated with entry and exit behavior identifiers refer to object trajectories that cannot be identified as entering or exiting a store. It can be understood that the object trajectories that are not associated with entry and exit behavior identifiers may include object trajectories corresponding to the identifiers of objects passing by the store, object trajectories corresponding to the identifiers of objects with a trend of entering the store, and object trajectories corresponding to the identifiers of objects with a trend of exiting the store. Based on this, among the object trajectories that are not associated with entry and exit behavior identifiers, the server searches for object trajectories that meet the rules of the trend of entering the store, and then constructs a subset of object trajectories that meet the conditions for identifying the trend of entering the store based on the object trajectories found.

[0087] Finally, the server determines the object identification subset corresponding to the object trajectory subset in the initial object identification set as the second object identification set.

[0088] In this embodiment, for outbound object identifiers whose object identifiers fail to match, matching object identifiers can be found among object identifiers with incomplete inbound behavior trends, thereby improving the accuracy of matching outbound object identifiers and inbound object identifiers.

[0089] Alternatively, in one embodiment, a camera is mounted above the store entrance, and computer vision object detection and tracking techniques are used to capture customer trajectories. A trajectory is a sequence of coordinate points over time. By marking the store entrance area in the camera's image, if a customer's trajectory crosses from outside the store entrance into the store entrance, it can be considered an entry. Conversely, if the trajectory extends from inside the store entrance to outside the store entrance, it can be considered an exit. These store entry and exit events constitute customer flow information.

[0090] In one embodiment, step S2044 includes the following steps:

[0091] Step S204a, removing object trajectories associated with entry and exit behavior identifiers from the object trajectory set to obtain a retained object trajectory set;

[0092] Step S204b: In the retained object trajectory set, a set of object trajectories that conform to a preset trajectory rule is determined as an object trajectory subset that meets the second behavior occurrence trend recognition condition.

[0093] Specifically, the server first removes the object trajectories associated with the entry and exit behavior identifiers from the object trajectory set to obtain the retained object trajectory set, and then determines the set of object trajectories that conform to the preset trajectory law in the retained object trajectory set as the object trajectory subset that meets the second behavior occurrence trend identification condition. Optionally, if the second behavior refers to entering a store and the image acquisition device is installed outside the store, the object trajectory that conforms to the preset trajectory law refers to an object trajectory in which each trajectory point is getting farther and farther away from the store. If the second behavior refers to entering a store and the image acquisition device is installed outside the store, the object trajectory that conforms to the preset trajectory law refers to an object trajectory in which each trajectory point is getting closer and closer to the store. The relative distance between each trajectory point and the store entrance is calculated in turn to determine whether the trajectory point is getting closer and closer to the store.

[0094] In this embodiment, object trajectories associated with entry and exit behavior identifiers are first eliminated, and then object trajectories that meet the preset trajectory rules are searched to obtain a subset of object trajectories that meet the second behavior occurrence trend identification condition, which is conducive to improving the efficiency of trajectory search.

[0095] Based on the same inventive concept, an embodiment of the present application further provides an event determination method including the target matching method of any one of the above embodiments, the event determination method further including:

[0096] Step S212: determining the occurrence time of the second behavior corresponding to the second object identifier based on the object trajectory corresponding to the second object identifier that successfully matches the first object identifier;

[0097] Step S214, determining the occurrence time of the second behavior corresponding to the second object identifier as the occurrence time of the second behavior corresponding to the first object identifier;

[0098] Step S216 , determining a target area entry and exit event corresponding to the first object identifier according to the first object identifier, the first behavior occurrence time, and the second behavior occurrence time.

[0099] In this embodiment, based on the improved accuracy of target matching, events determined by applying the target matching method also improve the accuracy of event information.

[0100] The following describes an embodiment of the present application in conjunction with a specific application scenario. First, it should be noted that this embodiment relies on trajectories and store area annotation for detecting store entry and exit. Trajectories may be incomplete due to occlusion by dense crowds, so it may not be possible to perfectly achieve the following: in the case of store entry, the trajectory extends from the marked outside the door to the inside, and in the case of store exit, the trajectory extends from the marked inside the door to the outside. Instead, some store entry trajectories disappear before extending into the door, or some store exit trajectories start outside the door and continue to extend further. These trajectories may cause missed detections. Therefore, for customer IDs with unmatched unilateral events (only entering or exiting the store), a search is conducted over a wider time and space range to find a customer ID whose trajectory shows a trend of leaving or entering the store (if the unilateral event is entering the store, the search is conducted for trajectories with an exit trend; if the unilateral event is exiting the store, the search is conducted for trajectories with an entry trend). The image features corresponding to the searched trajectory must be highly similar to the image features to be searched. These events found based on trajectory trends are actually the missed events that we want to know about.

[0101] The method comprises the following steps:

[0102] Step S222: passenger flow statistics.

[0103] Specifically, if Figure 3 As shown, if the customer's trajectory crosses from outside the store door into the store door (or from outside the store area into the store area), it can be understood as entering the store, and vice versa, if the trajectory extends from the area inside the store door to the area outside the store door, it can be understood as leaving the store.

[0104] Step S224: track pairing.

[0105] Specifically, each trajectory contains information including the coordinates of the point at each moment and the image of the customer within the corresponding detection box. A ResNet50 network pre-trained on ImageNet is used as an image feature extractor to extract features from all images of the customer corresponding to each customer ID. The features obtained for each image are a one-dimensional vector of length 2048. Because target tracking technology generates a series of detection box images for each customer ID, features are extracted from all these detection box images and then averaged to form the final feature vector.

[0106] Suppose that on a given day, a store has M customer IDs entering and N customer IDs leaving. Since target tracking-based entry and exit detection methods cannot guarantee complete detection of entry and exit events, and may result in missed or false detections, M is not equal to N. Assuming that every customer entering the store inevitably leaves, the similarity between the M customer IDs entering and the N customer IDs leaving the store is calculated using cosine based on the visual feature vectors of their detection boxes. The similarity is normalized between 0 and 1, with the closer the value is to 1, the more likely the two customer IDs are the same person. Identity matching can be performed by sequentially calculating the similarity between the M customer IDs entering the store and the N customer IDs leaving the store. If the similarity between the customer ID leaving the store that has the highest similarity to a customer ID entering the store also exceeds a certain similarity threshold (for example, 0.8), the customer ID leaving the store is considered a match with the customer ID entering the store. Then, the ID of the customer leaving the store will no longer be matched with other unmatched IDs of the customer entering the store, because the ID of the customer leaving the store has already been successfully matched with the ID of a customer entering the store. The matching result is: some customer IDs of the customer entering and leaving the store are successfully matched, while some customer IDs of the customer entering the store and some customer IDs of the customer leaving the store are not matched, that is, the matching fails, and the first object ID for which the object ID matching fails is determined.

[0107] Step S226, enter the store unilaterally and find the store.

[0108] Specifically, for the customer IDs entering the store that failed to match, it may be because the missing trajectory caused the missed detection, resulting in the fact that they do not have the customer IDs of the other half of the store exit events, so the trajectory trend is used to find the trajectories of their other half of the store exit events. First, find the trajectories corresponding to all customer IDs that appeared within one hour after the entry time of the customer ID entering the store. Then, remove the customer IDs that have already generated events from these trajectories, that is, the customer IDs with object trajectories associated with entry and exit behavior identifiers. Afterwards, among the trajectories corresponding to the remaining customer IDs, retain the trajectories that meet the requirement of "if a trajectory has each trajectory point farther and farther away from the store". Because a customer who wants to leave the store will simply leave the store farther and farther away. The trajectory passing by the door may be at a distance from the door that is sometimes far and sometimes near. Finally, among all the retained customer IDs, the customer ID with the highest visual feature similarity to the customer ID entering the store is searched. If the similarity is higher than the similarity threshold (for example, 0.8), it is considered that the customer ID is the customer ID of the same person leaving the store corresponding to the customer ID entering the store. Then, the starting time of the trajectory corresponding to the customer ID suspected of leaving the store is taken as the store exit time, and thus an exit event is retrieved.

[0109] Step S228, exit the store unilaterally and find the store again.

[0110] Specifically, for the customer IDs that failed to match the store exit, it may be because of the missing trajectory that led to missed detection, resulting in the absence of the customer IDs of the other half of the store entry event. Therefore, the trajectory trend is used to find the trajectory of their other half of the store entry event. First, find the trajectories corresponding to all customer IDs that appeared within an hour before the store exit time of the customer ID that left the store. Then, from these trajectories, remove the customer IDs that have already generated events, that is, the customer IDs with object trajectories associated with entry and exit behavior identifiers. Afterwards, among the trajectories corresponding to the remaining customer IDs, retain the trajectories that meet the requirement that "if a trajectory has each trajectory point getting closer and closer to the store." Because a customer who wants to enter the store will directly enter the store and get closer and closer. The trajectory passing by the door may be at a distance from the door that is sometimes far and sometimes near. Finally, among all the retained customer IDs, the customer ID with the highest visual feature similarity to the customer ID of the customer leaving the store is searched. If the similarity is higher than the similarity threshold (for example, 0.8), it is considered that the customer ID is the customer ID of the same person entering the store as the customer ID of the customer leaving the store. Then, the end time of the trajectory corresponding to the customer ID suspected to be entering the store is taken as the entry time, and thus an entry event is retrieved.

[0111] like Figure 4 As shown, the two tracks on the left represent that the customer IDs of customers leaving the store want to find the customer IDs of customers entering the store, the two tracks on the right represent that the customer IDs of customers entering the store want to find the customer IDs of customers leaving the store, and the horizontal track represents the track of customer IDs passing by.

[0112] In this embodiment, we use trajectory matching to identify potential missed events. We then use trajectory trends to identify those missed events. By combining these two technologies, we achieve more accurate store entry and exit event detection, or customer flow counting.

[0113] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0114] Based on the same inventive concept, embodiments of the present application also provide a target matching device for implementing the target matching method described above. The solution provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations in one or more target matching device embodiments provided below can be found in the limitations of the target matching method described above and will not be repeated here.

[0115] In one embodiment, Figure 5 As shown, a target matching device is provided, including: a data acquisition module 302, an identification search module 304 and an identification matching module 306, wherein:

[0116] The data acquisition module 302 is configured to acquire a first object identifier for which an object identifier match fails and a time at which a first behavior corresponding to the first object identifier occurs, wherein the first object identifier is an identifier of an object on which the first behavior occurs;

[0117] an identifier search module 304 configured to search for a set of second object identifiers within a preset time period corresponding to the time when the first behavior occurred, wherein the second object identifiers are identifiers of objects that have a tendency to perform a second behavior, and the second behavior is an opposite behavior to the first behavior performed with respect to the target area;

[0118] The identifier matching module 306 is used to match the first object identifier with the second object identifier set based on the similarity between the object image corresponding to the first object identifier and the object images corresponding to each second object identifier in the second object identifier set, and determine the second object identifier that successfully matches the first object identifier.

[0119] In one embodiment, the first behavior refers to entering a target area, and the second behavior refers to leaving the target area; the identification search module 304 is specifically used to search for an initial object identification set and an object trajectory set corresponding to the initial object identification set within a preset time period after the first behavior occurs; from the object trajectory set, select an object trajectory subset that meets the second behavior occurrence trend identification condition; and determine the object identification subset corresponding to the object trajectory subset in the initial object identification set as the second object identification set.

[0120] In one embodiment, the first behavior refers to leaving the target area, and the second behavior refers to entering the target area; the identification search module 304 is specifically used to search for an initial object identification set and an object trajectory set corresponding to the initial object identification set within a preset time period before the occurrence of the first behavior; in the object trajectory set, select an object trajectory subset that meets the second behavior occurrence trend identification condition; and determine the object identification subset corresponding to the object trajectory subset in the initial object identification set as the second object identification set.

[0121] In one embodiment, the identification search module 304 is specifically used to eliminate object trajectories associated with entry and exit behavior identifiers from the object trajectory set to obtain a retained object trajectory set, wherein the entry and exit behavior identifiers are used to indicate that the object trajectory is a trajectory entering and exiting the target area; in the retained object trajectory set, a set of object trajectories that conform to a preset trajectory law is determined as an object trajectory subset that meets the second behavior occurrence trend identification condition.

[0122] In one embodiment, the data acquisition module 302 is specifically used to obtain a third object identification set entering the target area and an entry-domain image set corresponding to the third object identification set; obtain a fourth object identification set leaving the target area and an exit-domain image set corresponding to the fourth object identification set; match each third object identification in the third object identification set with each fourth object identification in the fourth object identification set based on the similarity between each entry-domain image in the entry-domain image set and each exit-domain image in the exit-domain image set to obtain a third object identification and / or a fourth object identification for which the matching fails; and determine the third object identification and / or the fourth object identification for which the matching fails as the first object identification for which the object identification matching fails.

[0123] Each module in the target matching device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0124] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it implements a target matching method.

[0125] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0126] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0127] Obtaining a first object identifier for which object identifier matching fails and a time at which a first behavior corresponding to the first object identifier occurs, wherein the first object identifier is an identifier of an object on which the first behavior occurs;

[0128] Searching for a set of second object identifiers within a preset time period corresponding to the time when the first behavior occurs, wherein the second object identifiers are identifiers of objects that have a tendency to perform a second behavior, and the second behavior and the first behavior are opposite behaviors performed with respect to a target area;

[0129] The first object identifier and the second object identifier set are matched based on the similarity between the object image corresponding to the first object identifier and the object images corresponding to each second object identifier in the second object identifier set to determine a second object identifier that successfully matches the first object identifier.

[0130] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0131] Searching for an initial object identification set and an object trajectory set corresponding to the initial object identification set within a preset period of time after the first behavior occurs;

[0132] From the object trajectory set, a subset of object trajectories that meets a second behavior occurrence trend identification condition is selected;

[0133] The object identification subset corresponding to the object trajectory subset in the initial object identification set is determined as the second object identification set.

[0134] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0135] Searching for an initial object identification set and an object trajectory set corresponding to the initial object identification set within a preset period before the occurrence time of the first behavior;

[0136] From the object trajectory set, a subset of object trajectories that meets a second behavior occurrence trend identification condition is selected;

[0137] The object identification subset corresponding to the object trajectory subset in the initial object identification set is determined as the second object identification set.

[0138] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0139] Eliminating object trajectories associated with entry and exit behavior identifiers from the object trajectory set to obtain a retained object trajectory set, wherein the entry and exit behavior identifiers are used to indicate that the object trajectories are trajectories that enter and exit the target area;

[0140] In the retained object trajectory set, a set of object trajectories that conform to a preset trajectory rule is determined as an object trajectory subset that meets the second behavior occurrence trend recognition condition.

[0141] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0142] Acquire a third object identification set entering the target area and an entry image set corresponding to the third object identification set;

[0143] Acquire a fourth object identification set leaving the target area and an out-of-area image set corresponding to the fourth object identification set;

[0144] matching each third object identifier in the third object identifier set and each fourth object identifier in the fourth object identifier set based on similarities between each in-domain image in the in-domain image set and each out-domain image in the out-domain image set, to obtain third object identifiers and / or fourth object identifiers for which matching fails;

[0145] The third object identifier and / or the fourth object identifier that failed to match is determined as the first object identifier that failed to match the object identifier.

[0146] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0147] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0148] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0149] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0150] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0151] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A target matching method, characterized in that: The method comprises: Obtaining a first object identifier for which object identifier matching fails and a time at which a first behavior corresponding to the first object identifier occurs, wherein the first object identifier is an identifier of an object for which the first behavior occurs, and the object identifier matching failure means that the first object identifier does not match an object identifier for a behavior opposite to the first behavior; Searching for a set of second object identifiers within a preset time period corresponding to the time when the first behavior occurred, wherein the second object identifiers are identifiers of objects that have a trend of occurring a second behavior, the second behavior being opposite to the first behavior and directed at a target area, and the trend of the second behavior being a behavior whose trajectory satisfies a second behavior trend identification condition; The first object identifier and the second object identifier set are matched based on the similarity between the object image corresponding to the first object identifier and the object images corresponding to each second object identifier in the second object identifier set to determine a second object identifier that successfully matches the first object identifier.

2. The method according to claim 1, characterized in that The first behavior refers to entering a target area, and the second behavior refers to leaving the target area; The searching for a second object identifier set within a preset time period corresponding to the time when the first behavior occurs includes: Searching for an initial object identification set and an object trajectory set corresponding to the initial object identification set within a preset period of time after the first behavior occurs; From the object trajectory set, a subset of object trajectories that meets a second behavior occurrence trend identification condition is selected; The object identification subset corresponding to the object trajectory subset in the initial object identification set is determined as the second object identification set.

3. The method according to claim 1, characterized in that The first behavior refers to leaving the target area, and the second behavior refers to entering the target area; The searching for a second object identifier set within a preset time period corresponding to the time when the first behavior occurs includes: Searching for an initial object identification set and an object trajectory set corresponding to the initial object identification set within a preset period before the occurrence time of the first behavior; From the object trajectory set, a subset of object trajectories that meets a second behavior occurrence trend identification condition is selected; The object identification subset corresponding to the object trajectory subset in the initial object identification set is determined as the second object identification set.

4. The method according to claim 2 or 3, characterized in that The selecting, from the object trajectory set, a subset of object trajectories that meets the second behavior occurrence trend identification condition comprises: Eliminating object trajectories associated with entry and exit behavior identifiers from the object trajectory set to obtain a retained object trajectory set, wherein the entry and exit behavior identifiers are used to indicate that the object trajectories are trajectories that enter and exit the target area; In the retained object trajectory set, a set of object trajectories that conform to a preset trajectory rule is determined as an object trajectory subset that meets the second behavior occurrence trend recognition condition.

5. The method according to claim 1, wherein The obtaining of the first object identifier for which the object identifier matching fails includes: Acquire a third object identification set entering the target area and an entry image set corresponding to the third object identification set; Acquire a fourth object identification set leaving the target area and an out-of-area image set corresponding to the fourth object identification set; matching each third object identifier in the third object identifier set and each fourth object identifier in the fourth object identifier set based on similarities between each in-domain image in the in-domain image set and each out-domain image in the out-domain image set, to obtain third object identifiers and / or fourth object identifiers for which matching fails; The third object identifier and / or the fourth object identifier that failed to match is determined as the first object identifier that failed to match the object identifier.

6. An event determination method comprising the target matching method according to any one of claims 2 to 5, characterized in that: The method further comprises: determining, based on the object trajectory corresponding to the second object identifier that successfully matches the first object identifier, an occurrence time of a second behavior corresponding to the second object identifier; Determining the occurrence time of the second behavior corresponding to the second object identifier as the occurrence time of the second behavior corresponding to the first object identifier; A target area entry and exit event corresponding to the first object identifier is determined according to the first object identifier, the time at which the first behavior occurs, and the time at which the second behavior occurs.

7. A target matching device, characterized in that: The device comprises: A data acquisition module, configured to acquire a first object identifier for which an object identifier match fails and a time at which a first behavior corresponding to the first object identifier occurs, wherein the first object identifier is an identifier of an object for which the first behavior occurs, and the object identifier match failure means that the first object identifier does not match an object identifier for a behavior opposite to the first behavior; an identifier search module, configured to search for a set of second object identifiers within a preset time period corresponding to the time when the first behavior occurred, wherein the second object identifiers are identifiers of objects that have a trend of occurring a second behavior, the second behavior being opposite to the first behavior and directed at a target area, and the trend of the second behavior being a behavior whose trajectory satisfies a second behavior trend identification condition; an identifier matching module, configured to match the first object identifier with the second object identifier set based on similarities between the object image corresponding to the first object identifier and the object images corresponding to each second object identifier in the second object identifier set, and determine a second object identifier that successfully matches the first object identifier.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images