A computer vision-based commercial space passenger flow event identification method

Abstract

The application discloses a kind of commercial space passenger flow event identification method based on computer vision, it is related to image recognition technical field, including, the entrance and exit video of commercial space is collected, fixed facility contour, reflective surface, display surface, display content and human form target are identified, divide outside the store area, entrance area and in-store area and determine the order of passage, according to the synchronous displacement mapping of human form target inside and outside reflective surface identification reflection portrait, according to the synchronous change relationship of human form target in display surface and display content and not along the order of passage movement screen portrait is identified, eliminate reflection portrait and screen portrait, form effective passenger flow target and scene structure.The application is analyzed by image recognition synchronous displacement mapping of human form target inside and outside reflective surface and the synchronous change relationship of human form target in display surface and display content, identifies and eliminates reflection portrait and screen portrait, reduces false target repeated count and real passenger flow misdeletion, improves the accuracy of effective passenger flow target identification.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method for recognizing passenger flow events in commercial spaces based on computer vision. Background Technology

[0002] With the development of digital operation of commercial spaces, the recognition of customer flow events based on computer vision and image recognition has gradually expanded from counting people to analyzing people's passage direction, area migration, and dwell behavior. Video target detection, multi-target tracking, and temporal image recognition technologies can extract human targets and movement trajectories from entrance and exit videos, providing data support for store customer flow statistics, spatial layout optimization, and operational resource allocation. With the popularization of glass decorations, electronic display devices, and open entrances and exits, the recognition of customer flow events in commercial spaces has placed higher demands on the ability to adapt to complex visual environments and the accuracy of event judgment.

[0003] Existing methods have several shortcomings. First, the morphological features of people reflected in glass and on display screens are similar to those of real customers. Image recognition methods that rely on fixed shielding areas, single-frame confidence, and target categories are difficult to accurately distinguish between fake images and real people, which can easily lead to double counting or false deletion of real customer flow. Second, people are prone to occlusion when passing through door frames, pillars, and intersecting crowds, which can cause continuous trajectories to split. Association methods based on location proximity, appearance similarity, and fixed counting lines are difficult to restore the target attribution before and after occlusion. It is also difficult to distinguish between brief passing, returning after entering the store, and actual staying behavior, which can lead to missed counts, double counting, and misjudgment of events. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a computer vision-based method for recognizing customer flow events in commercial spaces to solve the problems of inaccurate customer flow target recognition caused by false human image interference and unreliable determination of customer flow events caused by occlusion interruption.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] This invention provides a computer vision-based method for recognizing customer flow events in commercial spaces. The method includes: acquiring entrance and exit videos of the commercial space; identifying the outlines of fixed facilities, reflective surfaces, display surfaces, display content, and human targets; dividing the space into an outdoor area, an entrance area, and an indoor area and determining the passage order; recognizing reflected human images based on the synchronous displacement mapping of human targets inside and outside the reflective surfaces; recognizing screen human images based on the relationship between human targets inside the display surfaces and the synchronous changes in display content, and their non-movement along the passage order; eliminating reflected and screen human images to form effective customer flow targets and scene structures; forming customer flow evidence segments based on the continuous movement of effective customer flow targets within the scene structure; when an interruption occurs due to occlusion, determining the subsequent reachable range based on the termination position and movement direction of the customer flow evidence segment before the interruption; and determining the subsequent reachable range based on the reappearance and movement of the evidence segment within the subsequent reachable range. The continuous positional changes of the effective passenger flow target form subsequent passenger flow evidence segments, and these subsequent passenger flow evidence segments are matched with the passenger flow evidence segments before the interruption to form a segment connection relationship. Passenger flow evidence segments are set as evidence nodes, and candidate connections are established according to the time sequence and segment connection relationship. Candidate connections that deviate from the passage sequence are excluded. The one-to-one connection relationship between evidence nodes is determined according to the time interval between the preceding and following passenger flow evidence segments to form a candidate passenger flow event chain. The regional migration order of the candidate passenger flow event chain is tested. When the candidate passenger flow event chain passes through the outside area, the entrance area, and the inside area in sequence, and does not re-enter the entrance area after entering the inside area, and the continuous duration in the inside area exceeds the passage time from the entrance area to the inside area, the candidate passenger flow event chain is determined as an in-store passenger flow event, and passenger flow event identification information is output.

[0008] As a preferred embodiment of the computer vision-based commercial space customer flow event recognition method of the present invention, the specific steps of collecting entrance and exit videos of the commercial space, identifying the outlines of fixed facilities, reflective surfaces, display surfaces, display content, and human targets, dividing the area into an outdoor zone, an entrance zone, and an indoor zone, and determining the passage order are as follows:

[0009] Extract object boundaries from consecutive video frames of entrance and exit videos, and identify the boundaries of objects that are continuously located at the same image position and whose contour range does not change with human-shaped targets as fixed facility contours.

[0010] The passage opening is determined based on the outline of the fixed facilities corresponding to the door frame, wall and column. The area covering the passage opening is divided into the entrance area, the side of the passage opening facing the public passage is divided into the outside area, and the side of the passage opening facing the business space is divided into the inside area. The passage sequence is determined according to the spatial connection order of the outside area, entrance area and inside area.

[0011] The surface whose outer boundary position does not change with video frames and whose interior displays the mirror content of the surrounding scene is defined as the reflective surface. The surface whose outer boundary position does not change with video frames and whose interior image changes continuously or changes during overall switching is defined as the display surface. The image inside the outer boundary of the display surface is defined as the display content, and human-shaped targets are identified frame by frame.

[0012] As a preferred embodiment of the computer vision-based commercial space passenger flow event recognition method of the present invention, the specific steps of recognizing the reflected human image based on the synchronous displacement mapping of human-shaped targets inside and outside the reflective surface are as follows:

[0013] Compare the movement direction and displacement length of human-shaped targets inside and outside the reflective surface in the same pair of adjacent video frames, select the reflective surface boundary segment closest to the human-shaped target inside the reflective surface, and decompose the corresponding displacement into directions parallel to and perpendicular to the selected reflective surface boundary segment;

[0014] If the parallel directions of movement are consistent and the vertical directions of movement are opposite, and the sum of the absolute differences between the two displacement lengths in the parallel direction and the two displacement lengths in the vertical direction does not exceed the synchronous displacement difference limit, then the corresponding set of position changes is retained.

[0015] If at least two consecutive sets of position changes satisfy the following conditions: the movement direction in the parallel direction is consistent and the movement direction in the vertical direction is opposite; the sum of the absolute difference between the two displacement lengths in the parallel direction and the absolute difference between the two displacement lengths in the vertical direction does not exceed the synchronous displacement difference limit; and the time when the human-shaped target inside and outside the reflective surface begins to change position and the time when it stops changing position does not differ by more than one frame, then the synchronous displacement mapping is determined, and the corresponding human-shaped target inside the reflective surface is identified as a reflected human image.

[0016] As a preferred embodiment of the computer vision-based commercial space passenger flow event recognition method of the present invention, the specific steps for recognizing the screen image based on the relationship between the human-shaped target within the display surface and the display content changing synchronously and not moving along the passage sequence are as follows:

[0017] Compare the area of ​​the human-shaped target within the display surface frame by frame with the area where the displayed content changes;

[0018] The range of changes in the displayed content covers at least one of the changes in the appearance, disappearance, and outline of the human target. When the changes in the human target and the changes in the displayed content occur in the same video frame, it is determined that the human target and the displayed content change synchronously within the display surface.

[0019] The system checks the order in which human figures pass through the outdoor area, entrance area, and indoor area of ​​the store. Human figures that change synchronously with the displayed content but do not pass through the outdoor area, entrance area, and indoor area in sequence are identified as screen images.

[0020] As a preferred embodiment of the computer vision-based commercial space passenger flow event recognition method of the present invention, the specific steps for eliminating reflected human images and screen images to form effective passenger flow targets and scene structures are as follows:

[0021] Delete the target records and location change records corresponding to the reflected human figures and the screen human figures, and identify the retained human figures as valid passenger flow targets;

[0022] The outdoor area, entrance area, and indoor area are respectively associated with the corresponding image ranges in the entrance and exit videos. The passage sequence is recorded according to the spatial connection order from the outdoor area through the entrance area to the indoor area, thus forming a scene structure.

[0023] Compare the location of the effective customer flow target frame by frame with the image range of the outside area, entrance area and inside area to determine the area where the effective customer flow target is located.

[0024] When the location of the effective passenger flow target is located at the common boundary of adjacent areas, the effective passenger flow target is determined to be located in the entrance area.

[0025] As a preferred embodiment of the computer vision-based commercial space passenger flow event recognition method of the present invention, the specific steps of forming passenger flow evidence fragments according to the continuous movement of effective passenger flow targets in the scene structure are as follows:

[0026] When the same valid passenger flow target is present in consecutive video frames, the corresponding locations are sequentially assigned to the same passenger flow evidence segment;

[0027] If the same valid passenger flow target is not formed in the next video frame, the current passenger flow evidence segment is terminated, and subsequent continuous position changes are formed into a new passenger flow evidence segment.

[0028] The first position of the passenger flow evidence segment and the corresponding video capture time are determined as the starting position and starting time, the last position and the corresponding video capture time are determined as the ending position and ending time, and the direction of the last different position before the ending position pointing to the ending position is determined as the direction of movement.

[0029] Continuous recordings containing only one location record do not form a fragment of passenger flow evidence.

[0030] Continuous records containing multiple location records, but where the locations are identical, do not constitute a fragment of passenger flow evidence.

[0031] As a preferred embodiment of the computer vision-based commercial space passenger flow event recognition method of the present invention, the specific steps for determining the subsequent reachable range based on the termination position and movement direction of the passenger flow evidence fragment before the interruption are as follows:

[0032] If the effective passenger flow target's image range overlaps with the image range enclosed by the outline of the fixed facility before the termination of the passenger flow evidence segment, and the termination position does not reach the screen boundary of the entrance / exit video, it is determined that fixed facility occlusion has occurred.

[0033] If the image range of a valid passenger flow target overlaps with the image range of another valid passenger flow target before the termination of the passenger flow evidence segment, and the other valid passenger flow target continues to form a location record in subsequent video frames, it is determined that a valid passenger flow target occlusion has occurred.

[0034] Determine the average movement distance of the passenger flow evidence segment between adjacent video frames before the interruption, determine the number of video frames that do not form a location record after the termination of the passenger flow evidence segment as the number of obscured frames, and determine the extension distance based on the product of the average movement distance and the number of obscured frames.

[0035] Starting from the termination position, the extension distance is extended along the direction of movement. The two sides of the extension range are determined by the width of the image range where the effective passenger flow target is located before the termination of the passenger flow evidence segment. Only the part of the extension range that overlaps with the image ranges of the outside area, entrance area and inside area, is continuously connected from the termination position and can enter the adjacent area is retained to form the subsequent reachable range.

[0036] As a preferred embodiment of the computer vision-based commercial space passenger flow event recognition method of the present invention, the specific steps for forming segment connection relationships are as follows:

[0037] The location of the re-emerging effective passenger flow target is compared with the subsequent reachable range frame by frame. The location that first falls within the subsequent reachable range is determined as the re-emerging location, and the locations that are continuously formed from the re-emerging location are included in the subsequent passenger flow evidence segment.

[0038] Extend the evidence of passenger flow before the interruption along the direction of movement before the interruption to the area where the boundary of the area was first crossed, and determine the subsequent area entered by the direction of movement before the interruption.

[0039] Extend the subsequent movement direction along the starting position of the subsequent passenger flow evidence fragments to the first crossing of the area boundary to determine the subsequent area entered by the subsequent movement direction;

[0040] When two directions of movement enter the same subsequent area, and the subsequent passenger flow evidence segment does not first enter the area before the termination position of the passenger flow evidence segment before the interruption in the passage sequence, the subsequent passenger flow evidence segment is matched with the passenger flow evidence segment before the interruption to form a segment connection relationship.

[0041] As a preferred embodiment of the computer vision-based commercial space passenger flow event recognition method of the present invention, the specific steps for forming a candidate passenger flow event chain are as follows:

[0042] Each passenger flow evidence segment, along with its start time, end time, start position, end position, direction of movement, and the area where each position is located, is set as an evidence node. Evidence nodes are arranged in order of their start time from first to last. If the start times are the same, evidence nodes are arranged in order of their end time from first to last. Candidate connections from previous evidence nodes to subsequent evidence nodes are established based on the segment connection relationship.

[0043] If the region where the starting position of a subsequent evidence node is located is located before the region where the ending position of a preceding evidence node is located in the passage order, the corresponding candidate connection is excluded.

[0044] When the termination position of the preceding evidence node and the starting position of the following evidence node are in the same region, the corresponding candidate connection is excluded when the movement direction of the following evidence node extends to the first time it crosses the region boundary and enters the region before the termination position of the preceding evidence node in the passage sequence.

[0045] The remaining candidate connections are arranged in ascending order of the time interval between the end time of the preceding passenger flow evidence segment and the start time of the following passenger flow evidence segment. If the time intervals are the same, they are arranged in the order of the preceding evidence nodes. If the preceding evidence nodes are the same, they are arranged in the order of the following evidence nodes. Candidate connections in which the preceding evidence node has not yet established a following connection and the following evidence node has not yet established a preceding connection are retained in turn to form a one-to-one connection relationship.

[0046] Starting with evidence nodes that have not yet established a preceding connection, connect subsequent evidence nodes sequentially along the one-to-one connection relationship, and organize the continuously connected evidence nodes into a candidate passenger flow event chain.

[0047] As a preferred embodiment of the computer vision-based commercial space passenger flow event recognition method of the present invention, the specific steps for outputting passenger flow event recognition information are as follows:

[0048] Arrange the regions where each location is located according to the connection order of the evidence nodes in the candidate passenger flow event chain and the video capture time, merge the records of consecutive identical regions to form the region migration order, and retain the first and last video capture times of each region record.

[0049] According to the order of regional migration, the first group of adjacent records are selected: the external area record, the entrance area record, and the internal area record. The duration between the first video capture time of the entrance area record and the first video capture time of the internal area record is determined as the passage time from the entrance area to the internal area. The duration between the first video capture time and the last video capture time of the internal area record is determined as the continuous holding time.

[0050] If no new entry area records are found between the first video capture time of the in-store area record and the termination time of the last evidence node in the candidate customer flow event chain, and the duration of the continuous record exceeds the passage time from the entry area to the in-store area, the candidate customer flow event chain is determined as an in-store customer flow event. The first video capture time of the in-store area record is determined as the event occurrence time, and the effective customer flow target location corresponding to the event occurrence time is determined as the event occurrence location.

[0051] Organize the evidence nodes in the candidate passenger flow event chain according to the connection order, and form event chain evidence by combining the start time, end time, start position, end position, direction of movement, area where each position is located, and one-to-one connection relationship of each evidence node. Combine the event occurrence time, event occurrence location, and event chain evidence into passenger flow event identification information and output it.

[0052] The beneficial effects of this invention are as follows: By analyzing the synchronous displacement mapping of human-shaped targets inside and outside the reflective surface and the synchronous change relationship between human-shaped targets and display content on the display surface through image recognition, the invention identifies and eliminates reflected human images and screen human images, reduces the repeated counting of false targets and the false deletion of real customer flow, and improves the accuracy of effective customer flow target identification; by constructing subsequent reachable range, segment connection relationship and candidate customer flow event chain through image recognition, and combining the regional migration order, passage time and continuous holding time to determine the customer flow event entering the store, the invention restores the target association before and after the occlusion interruption, and reduces the omission, duplicate counting and misjudgment of the event entering the store. Attached Figure Description

[0053] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a flowchart of a computer vision-based method for recognizing passenger flow events in commercial spaces.

[0055] Figure 2 Flowchart for interference with facial recognition and scene structure.

[0056] Figure 3 A flowchart for connecting segments after an interruption caused by occlusion.

[0057] Figure 4 A flowchart for constructing candidate customer flow event chains and determining in-store events. Detailed Implementation

[0058] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0059] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0060] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0061] Reference Figures 1-4 As one embodiment of the present invention, this embodiment provides a method for recognizing passenger flow events in commercial spaces based on computer vision, comprising the following steps:

[0062] S1. Collect entrance and exit videos of commercial spaces, identify the outlines of fixed facilities, reflective surfaces, display surfaces, display content, and human targets, divide the store area into external area, entrance area, and internal area, and determine the passage order. Identify reflected human images based on the synchronous displacement mapping of human targets inside and outside the reflective surfaces, and identify screen human images based on the relationship between human targets inside the display surfaces and the synchronous changes in display content and their non-movement along the passage order. Eliminate reflected human images and screen human images to form effective customer flow targets and scene structure.

[0063] S1.1 Collect entrance and exit videos of commercial spaces, identify the outlines of fixed facilities, reflective surfaces, display surfaces, display content within display surfaces, and human targets, divide the store area into external area, entrance area, and internal area according to the outlines of fixed facilities, determine the passage sequence, and continuously determine the positional changes of human targets and the changes in display content.

[0064] It should be noted that object boundaries are extracted from consecutive video frames of entrance and exit videos. Object boundaries that are continuously located at the same image position and whose outline range does not change with moving targets in the image are identified as fixed facility outlines. Passage openings are determined based on the fixed facility outlines corresponding to door frames, walls, and columns. The area covering the passage opening is divided into the entrance area, the side of the passage opening facing the public passage is divided into the outside area, and the side of the passage opening facing the business space is divided into the inside area. The passage sequence is determined according to the spatial connection order of the outside area, entrance area, and inside area.

[0065] Within the image area corresponding to the outline of the fixed facility, the surface whose outer boundary position does not change with the video frame and whose interior displays the mirror content of the surrounding scene is defined as the reflective surface; the surface whose outer boundary position does not change with the video frame and whose interior image changes continuously or switches as a whole is defined as the display surface, and the image within the outer boundary of the display surface is defined as the display content.

[0066] Human targets are identified frame by frame. The position of the human target is represented by the midpoint of the bottom edge of the target box corresponding to the human target. The same human target is identified based on the overlap of the target boxes in adjacent video frames. When there are multiple candidate target boxes, the correspondence with the smallest center position distance is retained. The corresponding positions are recorded in chronological order to form a sequence of human target position changes. At the same time, the areas where the image changes on the display surface and the corresponding times are recorded to form a sequence of display content changes.

[0067] The sequence of changes in the position of the human-shaped target is used to compare the changes in the position of the human-shaped target inside and outside the reflective surface, while the sequence of changes in the displayed content is used to determine the relationship between the changes in the human-shaped target and the displayed content inside the display surface.

[0068] S1.2 Continuously compare the positional changes of human-shaped targets inside and outside the reflective surface, determine the synchronous displacement mapping between human-shaped targets inside and outside the reflective surface, identify the corresponding human-shaped target inside the reflective surface as a reflected human image; identify the human-shaped target inside the display surface that changes synchronously with the display content and does not move along the passage sequence as a screen human image.

[0069] It should be noted that, by utilizing the position change sequence of the reflective surface, the display surface, the human-shaped target, the display content change sequence, and the passage order, the human-shaped target outside the reflective surface, the human-shaped target inside the reflective surface, and the human-shaped target inside the display surface are determined according to the location of the human-shaped target.

[0070] For human-shaped targets outside and inside the reflective surface that change position in the same pair of adjacent video frames, the direction of movement and displacement length are determined according to the position from the previous position to the next position. The boundary segment of the reflective surface closest to the human-shaped target inside the reflective surface is selected, and the corresponding displacement is decomposed into directions parallel to and perpendicular to the selected boundary segment. When the movement directions in the parallel directions are consistent and the movement directions in the perpendicular directions are opposite, and the sum of the absolute differences between the two displacement lengths in the parallel direction and the two displacement lengths in the perpendicular direction does not exceed the synchronous displacement difference limit, a set of position changes is recorded. When at least two consecutive sets of position changes satisfy the above relationship, and the time when the human-shaped targets inside and outside the reflective surface begin to change position and the time when the position change stops is not more than one frame apart, it is determined that there is a synchronous displacement mapping between the human-shaped targets inside and outside the reflective surface.

[0071] When a human-shaped target inside a reflective surface corresponds to multiple human-shaped targets outside the reflective surface, the sum of the absolute differences in the parallel and vertical displacement lengths corresponding to each group of position changes during continuous movement time is calculated. The correspondence with the smallest statistical result is retained, and the human-shaped target inside the reflective surface is identified as a reflected human image. When the human-shaped targets inside and outside the reflective surface do not change position in the same pair of adjacent video frames, no comparison is made between the corresponding positions. During the deployment phase, the same real human-shaped target and its corresponding reflected human image are manually marked. The sum of the absolute differences between the two displacement lengths in the parallel direction and the two displacement lengths in the vertical direction is calculated for each group, and the maximum value obtained is determined as the synchronous displacement difference limit.

[0072] For human-shaped targets within the display surface, the corresponding target frame is matched with the changing area in the display content change sequence frame by frame. When the changing area of ​​the display content covers the area where the human-shaped target appears, disappears, or changes its outline, and the corresponding changes occur in the same video frame, it is determined that the human-shaped target and the display content change synchronously. Then, according to the human-shaped target's position change sequence, it is checked whether the human-shaped target passes through the outside area, entrance area, and inside area in sequence. Human-shaped targets that change synchronously with the display content but do not form the above movement sequence are identified as screen images.

[0073] S1.3. Remove reflected human figures and screen human figures from the human figures, determine the remaining human figures as effective customer flow targets, and divide the area of ​​the outside area, entrance area and inside area into corresponding traffic order to form effective customer flow targets and scene structure.

[0074] It should be noted that, based on the recognition results of reflected human images and screen human images, the target records and position change sequences corresponding to reflected human images and screen human images are deleted from the human targets, so that reflected human images and screen human images do not participate in the formation of passenger flow evidence fragments; the retained human targets are identified as valid passenger flow targets, and the position change sequences formed by each valid passenger flow target in chronological order are retained.

[0075] The outdoor area, entrance area, and indoor area are associated with the corresponding image ranges in the entrance and exit videos. The passage sequence is recorded according to the spatial connection order from the outdoor area through the entrance area to the indoor area, forming a scene structure. The scene structure includes the image ranges of the outdoor area, entrance area, and indoor area, as well as the passage sequence between the areas.

[0076] The location of the effective customer flow target is compared frame by frame with the image range of the outside area, entrance area and inside area to determine the area where the effective customer flow target is located in each video frame; when the location of the effective customer flow target falls on the common boundary of the adjacent area, the effective customer flow target is determined to be located in the entrance area; the area corresponding to each video frame is recorded to the position change sequence of the effective customer flow target, so that the position change sequence of the effective customer flow target corresponds to the scene structure.

[0077] It should also be noted that existing technologies often filter glass reflections and images of people on the display screen by pre-setting shielding areas. This can easily lead to the omission of false targets and the misdeletion of real people due to the fixed shielding range. This solution identifies and removes reflected images and images of people on the screen by synchronously changing human figures inside and outside the reflective surface, synchronously changing human figures and display content inside the display surface, and identifying the passage order. This forms an effective customer flow target and scene structure. It can get rid of the limitations of fixed shielding range, adapt to changes in reflection position, display screen, and personnel distribution, reduce glass reflections and images of people entering the customer flow statistics of the advertising screen, avoid the incorrect filtering of real people in front of the screen, and improve the authenticity of customer flow targets and the accuracy of area attribution.

[0078] S2. Passenger flow evidence segments are formed according to the continuous movement of effective passenger flow targets in the scene structure. When there is an obstruction, the subsequent reachable range is determined according to the termination position and movement direction of the passenger flow evidence segment before the interruption. The subsequent passenger flow evidence segments are formed according to the continuous position changes of effective passenger flow targets that reappear and move continuously within the subsequent reachable range. The subsequent passenger flow evidence segments are matched with the passenger flow evidence segments before the interruption to form a segment connection relationship.

[0079] S2.1. Form passenger flow evidence segments according to the continuous position changes of effective passenger flow targets in the scene structure, and determine the start time, end time, start position, end position and direction of movement of the passenger flow evidence segments.

[0080] It should be noted that the position change sequence of each effective passenger flow target is arranged according to the video capture time; when the same effective passenger flow target is present in consecutive video frames, the corresponding positions are sequentially assigned to the same passenger flow evidence segment; when an effective passenger flow target first forms a position, a passenger flow evidence segment is formed starting from the current position; when the same effective passenger flow target does not form a position in the next video frame, the current passenger flow evidence segment is terminated, and subsequent consecutive position changes are formed separately to form a new passenger flow evidence segment.

[0081] Frame by frame, the locations in the customer flow evidence segments are mapped to the image ranges of the outdoor area, entrance area, and indoor area, recording the area where each location is located. The video capture time and location corresponding to the first location are determined as the start time and start position, and the video capture time and location corresponding to the last location are determined as the end time and end position. The direction from the last different location before the end position to the end position is determined as the direction of movement. Continuous records containing only one location or all the same locations do not constitute a customer flow evidence segment.

[0082] S2.2 When the effective passenger flow target is interrupted due to scene occlusion, it extends along the direction of movement from the end position of the passenger flow evidence segment before the interruption, and the extension range expands with the duration of the occlusion, limiting the extension range to the area that the scene structure allows passage, forming the subsequent reachable range; scene occlusion includes occlusion caused by fixed facilities and occlusion caused by other effective passenger flow targets.

[0083] It should be noted that after the passenger flow evidence segment ends, the last target frame before the end is checked; if the target frame overlaps with the image range enclosed by the outline of the fixed facility, and the ending position does not reach the screen boundary of the entrance / exit video, it is determined that the effective passenger flow target is interrupted due to the obstruction of the fixed facility; if the target frame overlaps with the target frame of another effective passenger flow target, and the other effective passenger flow target continues to form a position record in subsequent video frames, it is determined that the effective passenger flow target is interrupted due to the obstruction of other effective passenger flow targets; if the above conditions are not met, no subsequent reachable range is formed.

[0084] Arrange the positions of passenger flow evidence segments before the video capture time according to the time of the interruption, and calculate the average movement distance between adjacent video frames: ;

[0085] in, Indicates the average distance traveled; This indicates the number of locations included in the passenger flow evidence fragment before the interruption; The location is indicated by its sequential number arranged according to the video capture time. Indicates the first One location; Indicates the first The adjacent positions after the first position; Indicates to From 1 to The distances between each adjacent position are summed. Indicates the first The position and the first Image distance between locations.

[0086] The number of video frames that, after the termination of a passenger flow evidence segment, fail to form a location record belonging to the valid passenger flow target where the interruption occurred is determined as the number of occlusion frames, and the extension distance is determined accordingly. ;

[0087] in, It represents the distance extended along the direction of motion from the termination position; Indicates the number of frames obscured.

[0088] Starting from the ending position, extend the distance along the direction of movement. The two sides of the extension range are determined by the width of the last target box before termination; the geometric overlap between the extension range and the image ranges of the outside area, entrance area and inside area is taken, and only the part that is continuously connected from the termination position and can enter the adjacent area is retained to form the subsequent reachable range; the extension range stops expanding after reaching the boundary of the image range that can be continuously passed along the direction of movement in the scene structure.

[0089] S2.3. Form subsequent passenger flow evidence segments based on the continuous position changes of effective passenger flow targets that reappear within the subsequent reachable range. Correspond the subsequent passenger flow evidence segments whose reappearance location is within the subsequent reachable range and whose movement direction continues the movement direction before the interruption with the passenger flow evidence segments before the interruption to form a segment connection relationship.

[0090] It should be noted that after the passenger flow evidence segment terminates due to scene occlusion, the position of the re-emerging valid passenger flow target is compared with the subsequent reachable range corresponding to the current video frame frame by frame; when the position of the first re-emergence of the valid passenger flow target is within the subsequent reachable range, the corresponding position is determined as the re-emergence position, and the position records formed in consecutive video frames from the re-emergence position are included in the same subsequent passenger flow evidence segment to determine the starting position, starting time and movement direction of the subsequent passenger flow evidence segment.

[0091] The subsequent area is determined by extending the movement direction along the pre-interruption direction from the end position of the passenger flow evidence segment before the interruption to the first crossing of the area boundary; the subsequent area is determined in the same way from the start position of the subsequent passenger flow evidence segment along the subsequent movement direction; if two movement directions enter the same subsequent area, and the subsequent passenger flow evidence segment has not entered the area that is ahead of the current area in the passage sequence, the subsequent movement direction is determined to continue the movement direction before the interruption.

[0092] When the reappearance location is within the subsequent reachable range and the direction of movement continues the direction of movement before the interruption, the subsequent passenger flow evidence fragments are matched with the passenger flow evidence fragments before the interruption to form a fragment connection relationship; when multiple subsequent passenger flow evidence fragments meet the above conditions, fragment connection relationships are established separately; when the reappearance location is outside the subsequent reachable range, or when the subsequent passenger flow evidence fragments enter the area before the current area in the passage sequence, no fragment connection relationship is established.

[0093] S3. Set passenger flow evidence fragments as evidence nodes, establish candidate connections according to time sequence and fragment connection relationship, exclude candidate connections whose migration direction deviates from the passage order, determine the one-to-one connection relationship between evidence nodes according to the time interval between consecutive passenger flow evidence fragments, and form a candidate passenger flow event chain.

[0094] S3.1 Set passenger flow evidence fragments as evidence nodes, arrange the evidence nodes according to the start time, and establish candidate connections between evidence nodes in sequence based on the connection relationship of the fragments.

[0095] It should be noted that each passenger flow evidence segment, along with its corresponding start time, end time, start position, end position, direction of movement, and the area where each position is located, is set as an evidence node.

[0096] Evidence nodes are arranged in order of their start time from first to last; if the start times are the same, they are arranged in order of their end time from first to last; the connection between segments is checked item by item, and the evidence node corresponding to the passenger flow evidence segment before the interruption is determined as the preceding evidence node, and the evidence node corresponding to the subsequent passenger flow evidence segment is determined as the following evidence node.

[0097] When the termination time of a preceding evidence node is earlier than the start time of a subsequent evidence node, a candidate connection is established from the preceding evidence node to the subsequent evidence node; when the same preceding evidence node has a fragment connection relationship with multiple subsequent evidence nodes, the corresponding candidate connections are retained respectively; when the same subsequent evidence node has a fragment connection relationship with multiple preceding evidence nodes, the corresponding candidate connections are retained respectively for subsequent determination of the one-to-one connection relationship between evidence nodes; no candidate connection is established for evidence nodes that do not have a fragment connection relationship.

[0098] S3.2. Eliminate candidate connections whose migration direction deviates from the passage order, and arrange the remaining candidate connections in ascending order of the interval between the end time of the preceding passenger flow evidence segment and the start time of the subsequent passenger flow evidence segment. In turn, retain candidate connections in which the preceding evidence node has not yet established a subsequent connection and the subsequent evidence node has not yet established a preceding connection, and eliminate other subsequent candidate connections with the same preceding evidence node and other preceding candidate connections with the same subsequent evidence node, thus forming a one-to-one connection relationship between evidence nodes.

[0099] It should be noted that, by examining each candidate connection, the direction of movement of the area formed by the termination position of the preceding evidence node pointing to the starting position of the subsequent evidence node is determined as the migration direction. If the area where the starting position of the subsequent evidence node is located is located before the area where the termination position of the preceding evidence node is located in the passage sequence, the corresponding candidate connection is excluded. If the termination position of the preceding evidence node and the starting position of the subsequent evidence node are located in the same area, and the movement direction of the subsequent passenger flow evidence fragment extends to the first crossing of the area boundary, and the extension direction enters the area where the termination position of the preceding evidence node is located in the passage sequence, the corresponding candidate connection is also excluded.

[0100] Determine the time interval for the retained candidate connections: ;

[0101] in, Indicates the sequence number of the preceding evidence node in the current candidate connection; Indicates the sequence number of the subsequent evidence node in the current candidate connection; Indicates the end time of the passenger flow evidence segment corresponding to the preceding evidence node; Indicates the start time of the passenger flow evidence segment corresponding to the subsequent evidence node; This indicates the time interval between two pieces of passenger flow evidence.

[0102] Candidate connections are arranged in ascending order of time interval. When time intervals are the same, they are first arranged according to the order of preceding evidence nodes, and then according to the order of subsequent evidence nodes. Each candidate connection is checked in turn. If a preceding evidence node has not yet retained a connection pointing to another subsequent evidence node, and a subsequent evidence node has not yet retained a connection from another preceding evidence node, the current candidate connection is retained, and other subsequent candidate connections with the same preceding evidence node and other preceding candidate connections with the same subsequent evidence node are excluded. The above screening does not exclude the establishment of preceding and subsequent connections for the same evidence node, and ensures that each evidence node has at most one preceding connection and one subsequent connection, forming a one-to-one connection relationship.

[0103] S3.3 Starting from an evidence node that has no preceding evidence node, connect subsequent evidence nodes in sequence according to the one-to-one connection relationship and time order, and organize the continuously connected evidence nodes into a candidate passenger flow event chain.

[0104] It should be noted that, based on the one-to-one connection relationship between evidence nodes, evidence nodes without prior connections are screened, arranged in order of their start time, and determined as the starting evidence nodes of the candidate passenger flow event chain.

[0105] Starting from each initial evidence node, subsequent evidence nodes are sequentially connected along the established subsequent connections. Each time a subsequent evidence node is connected, the subsequent evidence node is used as the current evidence node to continue checking the subsequent connections until the current evidence node has no subsequent connections established, thus completing a candidate passenger flow event chain. Evidence nodes that have already been added to a candidate passenger flow event chain are no longer added to other candidate passenger flow event chains.

[0106] Evidence nodes without prior and subsequent connections form a separate candidate passenger flow event chain. The start time, end time, start position, end position, direction of movement, and region of each evidence node are retained according to the connection order for subsequent determination of the regional migration order of the candidate passenger flow event chain.

[0107] S4. Examine the regional migration order of the candidate customer flow event chain. When the candidate customer flow event chain passes through the outside area, the entrance area and the inside area in sequence, and does not re-enter the entrance area after entering the inside area, and the continuous duration of the inside area exceeds the passage time from the entrance area to the inside area, the candidate customer flow event chain is determined as an in-store customer flow event, and the customer flow event identification information is output.

[0108] S4.1 Determine the region where each evidence node in the candidate passenger flow event chain is located according to the time sequence, merge consecutive records of the same region, and form the region migration order of the candidate passenger flow event chain.

[0109] It should be noted that the evidence nodes are arranged according to the connection order in the candidate customer flow event chain, and the location records within each evidence node are arranged according to the video capture time. The outdoor area, entrance area, and indoor area corresponding to each location are then organized into the area records.

[0110] Starting with the first record in a given region, compare adjacent records in that region one by one. If adjacent records in the same region are identical, merge consecutive identical records in that region and retain the video capture times corresponding to the first and last positions in the merged record. If the ending and starting positions of adjacent evidence nodes are in the same region, merge the corresponding records in the region where the end of the previous evidence node begins with the beginning of the next evidence node. If adjacent records in different regions are different, end the current merge and continue comparing from the next record in that region.

[0111] The regional migration sequence is formed according to the order of the merged regional records, and the first and last video capture times of each regional record are retained for subsequent determination of the passage time from the entrance area to the store area and the continuous holding time after entering the store area.

[0112] S4.2. Based on the regional migration order, filter the candidate customer flow event chain that passes through the outside area, entrance area and inside area in sequence, determine the time from the entry area to the inside area of ​​the candidate customer flow event chain, form the passage time from the entrance area to the inside area, and determine the continuous maintenance time based on the evidence nodes that are continuously located in the inside area after entering the inside area.

[0113] It should be noted that, according to the regional migration order, each segment of the regional records is checked from front to back. The first group of adjacent regional records, namely the outdoor area record, the entrance area record, and the indoor area record, is selected and the corresponding candidate customer flow event chain is retained. Candidate customer flow event chains that do not form the above adjacent arrangement relationship are not included in the subsequent judgment of in-store customer flow events.

[0114] The first video capture time recorded in the entrance area is determined as the entry time into the entrance area, and the first video capture time recorded in the store area immediately following the entrance area is determined as the entry time into the store area. The passage time from the entrance area to the store area is: ;

[0115] in, This indicates the time taken to travel from the entrance area to the store interior; Indicates the time of entry into the entrance area; This indicates the time of entry into the store area.

[0116] The last video capture time recorded in the store area is defined as the end time of continuous recording, and the continuous recording duration is: ;

[0117] in, Indicates the duration of continuous stay after entering the store area; This indicates the last video capture time recorded in the store's internal area.

[0118] If an entrance area record or an external area record appears after an in-store area record, the subsequent area record will not be counted in the continuous retention time.

[0119] S4.3 If a candidate customer flow event chain enters the store area but does not re-enter the entrance area, and the continuous duration of its stay in the store area exceeds the passage time from the entrance area to the store area, the candidate customer flow event chain will be identified as an in-store customer flow event, and the event occurrence time, event location, and event chain evidence will be output.

[0120] It should be noted that the continuous holding time is compared with the passage time from the entrance area to the store area, and the area records between the first video capture time of the store area record and the termination time of the last evidence node in the candidate customer flow event chain are checked; if no entrance area records appear again during the above check period, and the continuous holding time is greater than the passage time from the entrance area to the store area, the candidate customer flow event chain is determined as an in-store customer flow event; if entrance area records appear again during the above check period, an in-store customer flow event is not formed; if the continuous holding time is not greater than the passage time from the entrance area to the store area, an in-store customer flow event is also not formed.

[0121] The first video capture time of the in-store area recorded to determine the in-store customer flow event is determined as the event occurrence time, and the effective customer flow target location corresponding to the event occurrence time is determined as the event occurrence location. The evidence nodes in the candidate customer flow event chain are organized according to the connection order, and the start time, end time, start position, end position, direction of movement, area where each position is located, and one-to-one connection relationship of each evidence node are formed into event chain evidence. The event occurrence time, event occurrence location, and event chain evidence are combined into customer flow event identification information and output after corresponding to the in-store customer flow event.

[0122] In summary, this invention identifies and removes reflected human figures and screen images by analyzing the synchronous displacement mapping of human figures inside and outside the reflective surface and the synchronous change relationship between human figures and displayed content on the display surface through image recognition analysis. This reduces the duplicate counting of false targets and the false deletion of real customer flow, thereby improving the accuracy of effective customer flow target identification. Furthermore, it constructs subsequent reachable ranges, segment connection relationships, and candidate customer flow event chains through image recognition, and determines in-store customer flow events by combining regional migration order, passage duration, and continuous holding duration. This restores the target association before and after the occlusion interruption, reducing missed counting, duplicate counting, and misjudgment of in-store events.

[0123] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for recognizing customer flow events in commercial spaces based on computer vision, characterized in that, include: The process involves collecting video footage from the entrances and exits of commercial spaces, identifying the outlines of fixed facilities, reflective surfaces, display surfaces, display content, and human figures. The area is divided into an outdoor zone, an entrance zone, and an indoor zone, and the passage order is determined. Reflected human images are identified based on the synchronous displacement mapping of human figures inside and outside the reflective surfaces. The specific steps are as follows: Compare the movement direction and displacement length of human-shaped targets inside and outside the reflective surface in the same pair of adjacent video frames, select the reflective surface boundary segment closest to the human-shaped target inside the reflective surface, and decompose the corresponding displacement into directions parallel to and perpendicular to the selected reflective surface boundary segment; If the parallel directions of movement are consistent and the vertical directions of movement are opposite, and the sum of the absolute differences between the two displacement lengths in the parallel direction and the two displacement lengths in the vertical direction does not exceed the synchronous displacement difference limit, then the corresponding set of position changes is retained. If at least two consecutive sets of position changes satisfy the following conditions: the movement direction in the parallel direction is consistent and the movement direction in the vertical direction is opposite; the sum of the absolute difference between the two displacement lengths in the parallel direction and the absolute difference between the two displacement lengths in the vertical direction does not exceed the synchronous displacement difference limit; and the time when the human-shaped target inside and outside the reflective surface begins to change position and the time when it stops changing position does not differ by more than one frame, then the synchronous displacement mapping is determined, and the corresponding human-shaped target inside the reflective surface is identified as a reflected human image. Based on the relationship between human figures on the display surface and the display content changing synchronously and not moving along the passage sequence, screen images are identified, reflected images and screen images are eliminated, and effective passenger flow targets and scene structures are formed; Passenger flow evidence segments are formed based on the continuous movement of effective passenger flow targets in the scene structure. When an obstruction occurs, the subsequent reachable range is determined based on the termination position and movement direction of the passenger flow evidence segment before the interruption. Subsequent passenger flow evidence segments are formed based on the continuous position changes of effective passenger flow targets that reappear and move continuously within the subsequent reachable range. The subsequent passenger flow evidence segments are then matched with the passenger flow evidence segments before the interruption to form a segment connection relationship. Passenger flow evidence fragments are set as evidence nodes. Candidate connections are established according to the time sequence and the connection relationship between fragments. Candidate connections whose migration direction deviates from the passage order are excluded. The one-to-one connection relationship between evidence nodes is determined according to the time interval between consecutive passenger flow evidence fragments, forming a candidate passenger flow event chain. The regional migration order of the candidate customer flow event chain is examined. When the candidate customer flow event chain passes through the outside area, the entrance area and the inside area in sequence, and does not re-enter the entrance area after entering the inside area, and the continuous duration of the inside area exceeds the passage time from the entrance area to the inside area, the candidate customer flow event chain is identified as an in-store customer flow event, and the customer flow event identification information is output.

2. The commercial space passenger flow event recognition method based on computer vision as described in claim 1, characterized in that, The process involves collecting video footage from the entrances and exits of commercial spaces, identifying the outlines of fixed facilities, reflective surfaces, display surfaces, display content, and human figures, dividing the area into outdoor, entrance, and indoor zones, and determining the passage order. The specific steps are as follows: Extract object boundaries from consecutive video frames of entrance and exit videos, and identify the boundaries of objects that are continuously located at the same image position and whose contour range does not change with human-shaped targets as fixed facility contours. The passage opening is determined based on the outline of the fixed facilities corresponding to the door frame, wall and column. The area covering the passage opening is divided into the entrance area, the side of the passage opening facing the public passage is divided into the outside area, and the side of the passage opening facing the business space is divided into the inside area. The passage sequence is determined according to the spatial connection order of the outside area, entrance area and inside area. The surface whose outer boundary position does not change with video frames and whose interior displays the mirror content of the surrounding scene is defined as the reflective surface. The surface whose outer boundary position does not change with video frames and whose interior image changes continuously or changes during overall switching is defined as the display surface. The image inside the outer boundary of the display surface is defined as the display content, and human-shaped targets are identified frame by frame.

3. The commercial space passenger flow event recognition method based on computer vision as described in claim 1, characterized in that, The specific steps for identifying the screen image based on the relationship between the human-shaped target within the display surface and the displayed content changing synchronously and not moving along the passage sequence are as follows: Compare the area of ​​the human-shaped target within the display surface frame by frame with the area where the displayed content changes; The range of changes in the displayed content covers at least one of the changes in the appearance, disappearance, and outline of the human target. When the changes in the human target and the changes in the displayed content occur in the same video frame, it is determined that the human target and the displayed content change synchronously within the display surface. The system checks the order in which human figures pass through the outdoor area, entrance area, and indoor area of ​​the store. Human figures that change synchronously with the displayed content but do not pass through the outdoor area, entrance area, and indoor area in sequence are identified as screen images.

4. The commercial space passenger flow event recognition method based on computer vision as described in claim 3, characterized in that, The process of eliminating reflected and screen-based human images to create effective customer flow targets and scene structures involves the following steps: Delete the target records and location change records corresponding to the reflected human figures and the screen human figures, and identify the retained human figures as valid passenger flow targets; The outdoor area, entrance area, and indoor area are respectively associated with the corresponding image ranges in the entrance and exit videos. The passage sequence is recorded according to the spatial connection order from the outdoor area through the entrance area to the indoor area, thus forming a scene structure. Compare the location of the effective customer flow target frame by frame with the image range of the outside area, entrance area and inside area to determine the area where the effective customer flow target is located. When the location of the effective passenger flow target is located at the common boundary of adjacent areas, the effective passenger flow target is determined to be located in the entrance area.

5. The commercial space passenger flow event recognition method based on computer vision as described in claim 4, characterized in that, The specific steps for forming passenger flow evidence fragments based on the continuous movement of effective passenger flow targets within the scene structure are as follows: When the same valid passenger flow target is present in consecutive video frames, the corresponding locations are sequentially assigned to the same passenger flow evidence segment; If the same valid passenger flow target is not formed in the next video frame, the current passenger flow evidence segment is terminated, and subsequent continuous position changes are formed into a new passenger flow evidence segment. The first position of the passenger flow evidence segment and the corresponding video capture time are determined as the starting position and starting time, the last position and the corresponding video capture time are determined as the ending position and ending time, and the direction of the last different position before the ending position pointing to the ending position is determined as the direction of movement. Continuous recordings containing only one location record do not constitute a fragment of passenger flow evidence. Continuous records containing multiple location records, but where the locations are identical, do not constitute a fragment of passenger flow evidence.

6. The commercial space passenger flow event recognition method based on computer vision as described in claim 1, characterized in that, The steps for determining the subsequent reachable range based on the termination position and direction of movement of passenger flow evidence fragments before the interruption are as follows: If the effective passenger flow target's image range overlaps with the image range enclosed by the outline of the fixed facility before the termination of the passenger flow evidence segment, and the termination position does not reach the screen boundary of the entrance / exit video, it is determined that fixed facility occlusion has occurred. If the image range of a valid passenger flow target overlaps with the image range of another valid passenger flow target before the termination of the passenger flow evidence segment, and the other valid passenger flow target continues to form a location record in subsequent video frames, it is determined that a valid passenger flow target occlusion has occurred. Determine the average movement distance of the passenger flow evidence segment between adjacent video frames before the interruption, determine the number of video frames that do not form a location record after the termination of the passenger flow evidence segment as the number of obscured frames, and determine the extension distance based on the product of the average movement distance and the number of obscured frames. Starting from the termination position, the extension distance is extended along the direction of movement. The two sides of the extension range are determined by the width of the image range where the effective passenger flow target is located before the termination of the passenger flow evidence segment. Only the part of the extension range that overlaps with the image ranges of the outside area, entrance area and inside area, is continuously connected from the termination position and can enter the adjacent area is retained to form the subsequent reachable range.

7. The commercial space passenger flow event recognition method based on computer vision as described in claim 6, characterized in that, The specific steps for forming the segment connection relationship are as follows: The location of the re-emerging effective passenger flow target is compared with the subsequent reachable range frame by frame. The location that first falls within the subsequent reachable range is determined as the re-emerging location, and the locations that are continuously formed from the re-emerging location are included in the subsequent passenger flow evidence segment. Extend the evidence of passenger flow before the interruption along the direction of movement before the interruption to the area where the boundary of the area was first crossed, and determine the subsequent area entered by the direction of movement before the interruption. Extend the subsequent movement direction along the starting position of the subsequent passenger flow evidence fragments to the first crossing of the area boundary to determine the subsequent area entered by the subsequent movement direction; When two directions of movement enter the same subsequent area, and the subsequent passenger flow evidence segment does not first enter the area before the termination position of the passenger flow evidence segment before the interruption in the passage sequence, the subsequent passenger flow evidence segment is matched with the passenger flow evidence segment before the interruption to form a segment connection relationship.

8. The commercial space passenger flow event recognition method based on computer vision as described in claim 7, characterized in that, The specific steps for forming a candidate passenger flow event chain are as follows: Each passenger flow evidence segment, along with its start time, end time, start position, end position, direction of movement, and the area where each position is located, is set as an evidence node. Evidence nodes are arranged in order of their start time from first to last. If the start times are the same, evidence nodes are arranged in order of their end time from first to last. Candidate connections from previous evidence nodes to subsequent evidence nodes are established based on the segment connection relationship. If the region where the starting position of a subsequent evidence node is located is located before the region where the ending position of a preceding evidence node is located in the passage order, the corresponding candidate connection is excluded. When the termination position of the preceding evidence node and the starting position of the following evidence node are in the same region, the corresponding candidate connection is excluded when the movement direction of the following evidence node extends to the first time it crosses the region boundary and enters the region before the termination position of the preceding evidence node in the passage sequence. The remaining candidate connections are arranged in ascending order of the time interval between the end time of the preceding passenger flow evidence segment and the start time of the following passenger flow evidence segment. If the time intervals are the same, they are arranged in the order of the preceding evidence nodes. If the preceding evidence nodes are the same, they are arranged in the order of the following evidence nodes. Candidate connections in which the preceding evidence node has not yet established a following connection and the following evidence node has not yet established a preceding connection are retained in turn to form a one-to-one connection relationship. Starting with evidence nodes that have not yet established a preceding connection, connect subsequent evidence nodes sequentially along the one-to-one connection relationship, and organize the continuously connected evidence nodes into a candidate passenger flow event chain.

9. The commercial space passenger flow event recognition method based on computer vision as described in claim 8, characterized in that, The specific steps for outputting passenger flow event identification information are as follows: Arrange the regions where each location is located according to the connection order of the evidence nodes in the candidate passenger flow event chain and the video capture time, merge the records of consecutive identical regions to form the region migration order, and retain the first and last video capture times of each region record. According to the order of regional migration, the first group of adjacent records are selected: the external area record, the entrance area record, and the internal area record. The duration between the first video capture time of the entrance area record and the first video capture time of the internal area record is determined as the passage time from the entrance area to the internal area. The duration between the first video capture time and the last video capture time of the internal area record is determined as the continuous holding time. If no new entry area records are found between the first video capture time of the in-store area record and the termination time of the last evidence node in the candidate customer flow event chain, and the duration of the continuous record exceeds the passage time from the entry area to the in-store area, the candidate customer flow event chain is determined as an in-store customer flow event. The first video capture time of the in-store area record is determined as the event occurrence time, and the effective customer flow target location corresponding to the event occurrence time is determined as the event occurrence location. Organize the evidence nodes in the candidate passenger flow event chain according to the connection order, and form event chain evidence by combining the start time, end time, start position, end position, direction of movement, area where each position is located, and one-to-one connection relationship of each evidence node. Combine the event occurrence time, event occurrence location, and event chain evidence into passenger flow event identification information and output it.

Images