Method for processing a video and related product

Abstract

Embodiments of the present application provide a video processing method and related products, the method comprising: obtaining a first video stream containing a first object; performing feature extraction on video frames in the first video stream to obtain a first feature map set; obtaining a second feature map set stored in advance; comparing at least one first feature map in the first feature map set with at least one second feature map in the second feature map set to obtain a comparison result; and determining whether the first object attempts to break through a gate according to the comparison result, and sending a prompt message to a control center if the first object attempts to break through the gate. In the embodiments of the present application, the arm card swiping action is modeled, the passenger's card swiping action in the early stage is judged to predict whether the passenger will break through the gate, and the prediction information that the passenger will break through the gate is sent to a station attendant in advance, so that the station attendant can intervene in advance, which can effectively improve the ticket checking efficiency of the gate and the travel experience of the user.

Description

Technical Field

[0001] This application relates to the field of rail transit technology, and in particular to a video processing method and related products. Background Technology

[0002] With the country's vigorous development of rail transit, cities have rapidly established multiple subway lines, resulting in a surge in passenger data and a corresponding increase in the workload of ticket checking.

[0003] To alleviate the pressure on station staff during ticket checks, most ticket gates now use turnstiles. While turnstiles can manage passenger flow and regulate entry and exit, they are not effective in preventing passengers from jaywalking. When a passenger jaywalks, the turnstile only sounds an alarm after the passenger has entered the gate's passage. Upon receiving the alarm, staff must then persuade the passenger to leave the gate and re-swipe their card to enter, severely impacting the ticket checking process and travel experience for subsequent passengers. Summary of the Invention

[0004] This application discloses a video processing method and related products. The method is based on arm swiping action modeling. It predicts whether a passenger will attempt to force their way through the gate by judging the passenger's card swiping action in advance. If so, the gate will send an alarm to the station management center in advance. The station management center will send the prediction information of the passenger's possible attempt to force their way through the gate to the station staff through multicast, so that the station staff can intervene in advance, improve the gate's ticket checking efficiency and the user's travel experience.

[0005] In a first aspect, this application provides a video processing method, the method comprising: acquiring a first video stream containing a first object, the first video stream including a plurality of first video frames; performing feature extraction on each of the plurality of first video frames to obtain a first feature map set representing the arm movement features of the first object, the first feature map set including at least one first feature map; acquiring a pre-stored second feature map set, the second feature map set including at least one second feature map; comparing the at least one first feature map in the first feature map set with the at least one second feature map in the second feature map set to obtain a comparison result; determining, based on the comparison result, whether the first object has attempted to break through a gate; and sending a prompt message to a control center if the first object has attempted to break through a gate.

[0006] In this method, the first object is the passenger currently passing through the turnstile. This method compares the current passenger's action characteristics when passing through the turnstile with the action characteristics of passengers who have previously attempted to ram through the turnstile to predict whether the passenger will attempt to ram through the turnstile. This can effectively intervene in advance against passengers attempting to ram through the turnstile, improving the efficiency of passenger ticket checking and entry.

[0007] In an optional implementation of the first aspect, before acquiring the first video stream containing the first object, the method further includes: acquiring a historically acquired second video stream containing the second object, wherein the second video stream is a video stream of the second object's gate-breaking machine, and the second video stream includes a plurality of second video frames; performing feature extraction on each of the plurality of second video frames to obtain at least one second feature map representing the arm movement features of the second object; constructing the at least one second feature map into a second feature map set, and storing the second feature map set.

[0008] In this embodiment of the application, the second target is passengers who have historically attempted to breach the gate. By using the action characteristics of passengers who have historically breached the gate as reference data for the action characteristics of the current passenger, it is possible to determine more accurately whether the current passenger has attempted to breach the gate.

[0009] In an optional implementation of the first aspect, before comparing the first feature set with the second feature set to obtain a comparison result, the method further includes: calling a ticket checking interface to query whether the first object has completed ticket checking; comparing the first feature set with the second feature set to obtain a comparison result includes: if the first object has not completed ticket checking, comparing the first feature set with the second feature set to obtain the comparison result.

[0010] If a passenger has already completed ticket inspection, it can be determined that the passenger did not attempt to force their way through the gate. Therefore, in this embodiment, by pre-determining whether the passenger has completed ticket inspection, it is possible to decide whether to compare the first feature image with the second feature image. This reduces unnecessary operations, saves system resources, and further improves the efficiency of passenger ticket inspection and entry.

[0011] In an optional implementation of the first aspect, the step of comparing the first feature map set with the second feature map set to obtain a comparison result includes: calculating the similarity between each first feature map in the at least one first feature map and each second feature map in the at least one second feature map, and using the similarity between each first feature map and each second feature map as the comparison result.

[0012] In this embodiment, the comparison between the first feature map and the second feature map is completed by a pre-trained neural network model. This can efficiently and accurately determine the similarity between the current passenger's action features and the action features of passengers who have previously attempted to break through the gate, making the judgment on whether a passenger is attempting to break through the gate more accurate.

[0013] In an optional implementation of the first aspect, determining whether the first object has attempted to break through the gate based on the comparison result includes: if the similarity in the comparison result is greater than a first threshold, then determining that the first object has attempted to break through the gate.

[0014] Understandably, the similarity between two feature maps can reflect the similarity between the action features of the first object and the action features of the second object. When the similarity is greater than the first threshold, the first object is very likely to also be attempting to break through the gate. Therefore, in this embodiment, when the similarity between any first feature map in the first feature map set and any feature map in the second feature map set is greater than the first threshold, it can be roughly determined that the first object is attempting to break through the gate.

[0015] In an optional implementation of the first aspect, determining whether the first object has attempted to break through the gate based on the comparison result includes: if there is a similarity greater than a first threshold in the comparison result, obtaining the number of similarities greater than the first threshold in the comparison result; if the number is greater than a second threshold, determining that the first object has attempted to break through the gate.

[0016] In this embodiment, when the number of similarities is greater than a second threshold (assuming the second threshold is N, where N is an integer greater than 1), that is, when there are at least N images in the first feature map set, and for any frame of the N frames, there is at least one second feature map in the second map set, and the similarity of the at least one second feature map to any frame of the N frames is greater than the first threshold, then it is determined that the first object has attempted to break through the gate. This makes the determination result more accurate.

[0017] In an optional implementation of the first aspect, before acquiring the first video stream containing the first object, the method further includes: acquiring a target image via a camera, the target image containing a facial image of the first object, the camera being mounted on a turnstile; acquiring the first video stream containing the first object includes: if the facial image is a frontal image of the first object, capturing a picture of the first object via the camera to obtain the first video stream.

[0018] Understandably, in actual passenger ticket checking scenarios, other interfering objects may exist around the first person during ticket checking, such as pedestrians passing in front of the turnstile or station staff. Therefore, in this embodiment, when photographing the first person, it can be first determined whether the captured image contains a frontal image of the first person. If it does, then the camera photographs the first person. This avoids wasting system resources by also photographing and extracting motion features from interfering objects.

[0019] Secondly, this application provides a video processing apparatus, the apparatus comprising: a shooting unit, configured to acquire a first video stream containing a first object, the first video stream including a plurality of first video frames; an extraction unit, configured to extract features from each of the plurality of first video frames to obtain a first feature map set representing the arm movement features of the first object, the first feature map set including at least one first feature map; an acquisition unit, configured to acquire a pre-stored second feature map set, the second feature map set including at least one second feature map; a comparison unit, configured to compare the at least one first feature map in the first feature map set with the at least one second feature map in the second feature map set to obtain a comparison result; an analysis unit, configured to determine, based on the comparison result, whether the first object has attempted to break through the gate; and a sending unit, configured to send a prompt message to a control center if the first object has attempted to break through the gate.

[0020] In one possible implementation of the second aspect, the shooting unit is further configured to acquire a second video stream containing the second object from historical data, the second video stream being a video stream of the second object passing through a gate, the second video stream including multiple second video frames; the extraction unit is further configured to perform feature extraction on each of the multiple second video frames to obtain at least one second feature map representing the arm movement features of the second object; the at least one second feature map is used to form a second feature map set, and the second feature map set is stored.

[0021] In one possible implementation of the second aspect, the apparatus further includes: a calling unit for calling the ticket checking interface to query whether the first object has completed ticket checking; and a comparison unit specifically used to compare the first feature set with the second feature set when the first object has not completed ticket checking, to obtain the comparison result.

[0022] In one possible implementation of the second aspect, the comparison unit is specifically used to calculate the similarity between each first feature map in the at least one first feature map and each second feature map in the at least one second feature map, and to use the similarity between each first feature map and each second feature map as the comparison result.

[0023] In one possible implementation of the second aspect, the analysis unit is specifically used to determine that the first object has attempted to break through the gate if the similarity in the comparison result is greater than a first threshold.

[0024] In one possible implementation of the second aspect, the analysis unit is specifically used to obtain the number of similarities in the comparison results that are greater than the first threshold if there is a similarity greater than the first threshold; if the number is greater than the second threshold, then it is determined that the first object has attempted to break through the gate.

[0025] In one possible implementation of the second aspect, the shooting unit is further configured to capture a target image via a camera, the target image including a facial image of the first object, the camera being mounted on a turnstile; specifically, the shooting unit is configured to capture a picture of the first object via the camera when the facial image is a frontal image of the first object, thereby obtaining the first video stream.

[0026] Thirdly, this application provides a gate, the gate comprising: one or more processors and a memory; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, and the one or more processors invoking the computer instructions to cause the gate to perform the method as described in the first aspect and any possible method of the first aspect.

[0027] Fourthly, this application provides a computer-readable storage medium including instructions, characterized in that, when the instructions are executed on an electronic device, the electronic device performs the methods described in the first aspect and any possible method described in the first aspect. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments or background art of this application, the accompanying drawings used in the embodiments or background art of this application will be briefly introduced below.

[0029] Figure 1 This application provides a schematic diagram of a passenger gate breaching mechanism as an embodiment of the present application.

[0030] Figure 2 A flowchart illustrating a video processing method provided in an embodiment of this application;

[0031] Figure 3 A schematic diagram illustrating some arm movement features provided in the embodiments of this application;

[0032] Figure 4 A schematic diagram illustrating a scenario for preventing passengers from rushing through the gate, provided as an embodiment of this application;

[0033] Figure 5 This is a schematic diagram of the structure of a video processing apparatus provided in an embodiment of this application;

[0034] Figure 6 This is a schematic diagram of a gate provided in an embodiment of this application. Detailed Implementation

[0035] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described below in conjunction with the accompanying drawings.

[0036] The terms "first" and "second," etc., used in the specification, claims, and drawings of this application are used only to distinguish different objects and not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0037] The term "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0038] In this application, "at least one (item)" means one or more, "more than one" means two or more, "at least two (items)" means two or three or more, and "and / or" is used to describe the relationship between related objects, indicating that there can be three relationships. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items. For example, at least one (item) of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c".

[0039] This invention provides video processing methods and related products. To more clearly describe the solutions of this invention, the following section introduces some knowledge related to the video processing methods and related products provided in this application.

[0040] (1) Convolutional Neural Network

[0041] A convolutional neural network (CNN) is a deep neural network with a convolutional structure. A CNN contains a feature extractor consisting of convolutional layers and subsampling layers. This feature extractor can be viewed as a filter, and the convolution process can be seen as performing convolution between a trainable filter and an input image or feature map.

[0042] A convolutional layer is a layer of neurons in a convolutional neural network that performs convolutional processing on the input signal. In a convolutional layer of a convolutional neural network, a neuron may only be connected to some of the neurons in its neighboring layers. A convolutional layer typically contains several feature maps, each of which can be composed of a series of rectangularly arranged neural units.

[0043] (2) Feature Map

[0044] In each convolutional layer of a CNN, the data exists in a three-dimensional form. It can be viewed as many two-dimensional images stacked together, each of which is called a feature map.

[0045] In convolutional neural networks, a network can be used to simulate the characteristics of the visual pathway. The concept of hierarchy involves constructing neurons from simple to complex, starting from the bottom up. In a convolutional network, feature maps can form a complete description of an object, such as a person's height, build, or appearance.

[0046] Figure 1 This is a schematic diagram illustrating a passenger gate breaching mechanism as provided in an embodiment of this application. Figure 1 As shown, the turnstile 10 includes a card reader area 101, a barrier 102, and a buzzer alarm 103. It should be understood that the structures illustrated in this embodiment are for illustrative purposes only and do not constitute a specific limitation on the turnstile. In other embodiments of this application, the turnstile 10 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements; for example, the buzzer alarm 103 may be installed inside the turnstile. Furthermore, the illustrated components may be implemented in hardware, software, or a combination of both.

[0047] like Figure 1As shown in (A), passenger 20, who attempted to force their way through the gate, is in front of gate 10. At this point, the passenger has not yet passed through gate 10, barrier 102 is closed, and the buzzer alarm is in sleep mode. Subsequently, without swiping their card, passenger 20 forcibly passes through gate 10 by some means, such as stepping over barrier 102 or following the previous passenger who had their ticket checked correctly, and reaches behind barrier 102. (Reference) Figure 1 In scenario (B), passenger 20 forcibly passes through gate 10 without swiping their card. At this point, the barrier is still closed, and the card reader 101 of the gate does not detect the passenger's ticket or card. The buzzer alarm 103 then sounds. Subsequent intervention from the train crew is required to remove the passenger from the gate and remind them to obtain a replacement ticket or swipe their card again. While this intervention effectively prevents passengers from swiping their tickets, it significantly increases operational costs, burdens the crew, and severely impacts the ticket checking process and travel experience for subsequent passengers.

[0048] To address the aforementioned issues, this application provides a video processing method. This method can model the arm-swipe card action and predict whether a passenger intends to attempt to force their way through the gate based on the passenger's previous card-swipe actions. If so, the gate will send an alert to the station management center in advance. The station management center will then send the predicted information about the passenger's potential gate attempt to station staff via multicast, allowing staff to intervene in advance and improve the gate's ticket checking efficiency and the user's travel experience. Please refer to [link / reference] for details. Figure 2 .

[0049] Figure 2 This is a flowchart illustrating a video processing method provided in an embodiment of this application. Figure 2 As shown, the method may include the following steps:

[0050] 201. Obtain the first video stream containing the first object.

[0051] The camera on the turnstile captures images of the first object, obtaining the first video stream, which includes multiple first video frames. It should be understood that this application does not limit the type or structure of the turnstile; for example, the turnstile can be a swing gate, wing gate, tripod gate, or semi-automatic full-height turnstile. The camera can be mounted on the top of the turnstile's housing or in other locations on the turnstile; this application does not limit its implementation.

[0052] The first object is the passenger who currently wants to pass through the gate, and among all passengers who want to pass through the gate, the first object is closest to the gate opening.

[0053] Understandably, in actual ticket checking scenarios, other interfering objects may exist around the first person during ticket checking, such as pedestrians passing in front of the gate, station staff, and other passengers waiting to check tickets behind the first person. Furthermore, these interfering objects may not be clearly visible in the camera's view. Therefore, in some embodiments, before acquiring the first video stream, the gate can first use the camera to capture the environment surrounding the first person, obtaining an image containing the first person's face. Then, it can be determined whether the acquired image contains a frontal image of the first person. If it does, the camera then captures the first person's image to obtain the first video stream. This avoids wasting system resources by also capturing and extracting motion features from the interfering objects.

[0054] 202. Perform feature extraction on each of the above-mentioned multiple first video frames to obtain a first feature map set used to represent the arm movement features of the above-mentioned first object.

[0055] Each of the aforementioned video frames is a multi-frame image displaying the posture of the first object's human body. In this method, the aforementioned gate is equipped with host computer software, which is used to extract features from each of the aforementioned multiple first video frames to obtain the aforementioned first feature map set used to characterize the arm behavior features of the first object.

[0056] In the specific implementation process, the host computer software can extract features from each of the first video frames based on a convolutional neural network (CNN) and use LSTM to model the temporal relationship to obtain the first feature map set. Alternatively, the host computer software can also use models such as 3D convolution or a two-stream model to extract the first feature map set; this application embodiment does not limit this approach.

[0057] 203. Obtain the pre-stored second feature map set.

[0058] Understandably, prior to implementing this method, the aforementioned second feature map set was already stored in the turnstile. This second feature map set is a feature map set obtained by extracting features from multiple video frames in a second video stream captured from historical images of passengers attempting to breach the turnstile, and the second feature map set includes at least one second feature map that can represent the arm movement characteristics of the passengers attempting to breach the turnstile.

[0059] Specifically, in the initial acquisition of the second feature set, a large number of passengers can be photographed to obtain video streams of passengers checking tickets and entering the station. From these large video streams, corresponding video frames are extracted, and a convolutional neural network (CNN) is used to analyze and extract the actual card-swiping actions of passengers during ticket checking, creating a card-swiping behavior database. Then, the arm movement features of passengers attempting to bypass the gate are obtained from this database, and representative arm movement features are selected as the aforementioned second feature set.

[0060] Figure 3 Images (A)-(E) exemplarily illustrate some arm movement features of passengers attempting to breach the gate according to embodiments of this application. Wherein:

[0061] Figure 3 The arm feature diagram shown in (A) represents the motion characteristics of a passenger with their arm hanging down vertically, without any obvious card swiping action; Figure 3 The arm feature diagram shown in (B) represents the movement characteristics of a passenger's arm swinging normally, without any obvious card swiping action; Figure 3 The arm feature diagram shown in (C) represents the movement characteristics of a passenger raising their arm, having their palm above their chest, and not making any obvious card swiping motion. Figure 3 The arm feature diagram shown in (D) represents the action characteristics of a passenger raising their arm without any obvious object in their hand; Figure 3 The arm feature diagram shown in (E) indicates that the passenger's arm is raised and there is an object in his hand, but the object in the passenger's hand is clearly not a card.

[0062] also, Figure 3 The feature maps shown in (A)-(E) are presented in a form only to facilitate reader understanding and differentiation. In actual application scenarios, the feature maps in the second feature map set can also be presented in other forms, and this application embodiment does not limit this.

[0063] 204. Compare at least one first feature map in the first feature map set with at least one second feature map in the second feature map set to obtain a comparison result.

[0064] As explained above, both the first and second feature map sets can contain multiple feature maps. For ease of explanation, let's assume the first feature map set contains two feature maps, denoted as feature map A and feature map B; and the second feature map set contains three feature maps, denoted as feature map C and feature map D. The comparison can be performed in the following two ways:

[0065] ① Compare each first feature map in the first feature map set with each second feature map in the second feature map set in turn. If the similarity between a first feature map and a second feature map is greater than a preset threshold, the comparison is stopped; otherwise, compare each first feature map in the first feature map set with each second feature map in the second feature map set until the comparison is completed.

[0066] For example, the aforementioned gate can first compare feature map A in the first feature map set with feature map C in the second feature map set. If the similarity between the two feature maps is greater than a preset threshold, the comparison will stop; otherwise, the similarity between feature map A and feature map D will be compared. If the similarity between the two feature maps is greater than a preset first threshold, the comparison will stop; otherwise, the similarity between feature map B and feature map C will be compared, and so on.

[0067] ② Compare each first feature map in the first feature map set with each second feature map in the second feature map set in turn, and record the similarity between each first feature map and each second feature map until the comparison is completed.

[0068] For example, the aforementioned turnstile compares feature image A and feature image B with feature image C and feature image D in sequence, records the similarity between each pair of feature images, and counts the number of similarities greater than a preset first threshold. See Table 1 below for details:

[0069] Table 1

[0070] Feature Map A Feature map B Feature map C 60％ 70％ Feature map D 10％ 80％

[0071] As shown in Table 1, the similarity between feature map A and feature map C is 60%, the similarity between feature map A and feature map D is 10%, the similarity between feature map B and feature map C is 70%, and the similarity between feature map B and feature map D is 80%. Assuming the first threshold is 50%, it is easy to see that the number of features with a similarity greater than the first threshold is 3.

[0072] Furthermore, in some embodiments, before executing step 204, the gate can also call the ticket checking interface to query whether the first object has completed ticket checking. It is understandable that when a passenger has completed ticket checking, it can be determined that the passenger did not attempt to force their way through the gate, and there is no need to compare the feature images. Comparing the first feature image with the second feature image when the passenger has not yet completed ticket checking reduces unnecessary operations, saves system resources, and further improves the efficiency of passenger ticket checking and entry.

[0073] 205. Based on the above comparison results, determine whether the first object attempted to break through the gate.

[0074] Based on the two comparison methods in step 204, the specific determination method in this step can also be divided into the following two methods:

[0075] ① If at least one of the above comparison results has a similarity greater than the first threshold, then it is determined that the first object has attempted to break through the gate.

[0076] In other words, when there is at least one first feature map in the first feature map set, and the similarity between the first feature map and at least one second feature map in the second feature map is greater than the first threshold, the gate will consider that the current action of the first object is similar to the action characteristics of the passenger who previously tried to break through the gate, and the gate will determine that the first object has attempted to break through the gate.

[0077] ② When there is a similarity greater than the first threshold in the above comparison results, and the number of similarities greater than the first threshold is greater than the second threshold, it is determined that the first object has attempted to break through the gate.

[0078] In other words, after comparing each first feature map in the first feature map set with each second feature map in the second feature map set to obtain the similarity between each first feature map and each second feature map, if the number of similarities greater than the first threshold is greater than the preset second threshold, then the turnstile considers the current action of the first object to be highly similar to the action features of a passenger who previously attempted to break through the gate, and the turnstile will determine that the first object has attempted to break through the gate. Taking Table 1 as an example, assuming the first threshold is 50% and the second threshold is 2, then as shown in Table 1, the similarities greater than the first threshold are 60%, 70%, and 80%, totaling 3, which are greater than the second threshold. In this case, the turnstile will determine that the first object has attempted to break through the gate.

[0079] 206. If the first object attempts to break through the gate, send a notification message to the control center.

[0080] Once the aforementioned turnstile determines that the first individual is attempting to breach the gate, it will send a notification to the control center in advance. The control center will then send the notification to relevant station staff via multicast, enabling rapid prediction and location, and allowing for early intervention. This will improve the gate's ticket checking efficiency and enhance the user's travel experience.

[0081] Based on the above method, this application provides a schematic diagram of a scenario to prevent passengers from rushing through the gate, as detailed in 4.

[0082] like Figure 4 As shown, in Figure 4 In this context, the turnstile 40 can be the aforementioned... Figure 2 The gate in the description; passenger 41 is the passenger who wants to pass through the gate 40, i.e., the first object in the aforementioned description; the control center 42 is responsible for various application needs such as business processing, comprehensive data analysis, emergency handling, on-site command, and information release within the station, and the control center 42 and the gate 40 can be directly or indirectly connected through wired or wireless communication, which is not limited in this application.

[0083] The turnstile 40 may include: a camera 401, a card reader 403, a barrier 404, and a feature image library 406. The barrier 404 is closed by default. When the card reader 403 senses a passenger's ticket or valid identification information, the barrier 404 will automatically open, allowing the passenger to pass. The camera 401 can be used to capture images or video streams representing the passenger's actions and postures, for example... Figure 4 The dashed box 402 in the diagram represents the field of view of the camera 401 when capturing the passenger 402. After the camera acquires the video stream or image, the host computer software in the gate can extract features from the video frames of the video stream or the image to obtain a feature map representing the passenger's action features, for example... Figure 4 Feature map 405 is the feature map obtained by the host computer software in the turnstile from the image captured by the camera within the dashed frame 402; specifically, feature map 405 can be the first feature map described above. Feature map library 406 can be stored in the turnstile's memory, containing a set of feature maps representing the arm movement characteristics of passengers attempting to pass through the turnstile. Figure 4 Feature image 407 is the feature image in this feature image set. Feature image 407 represents the action feature of the passenger raising his arm and having no obvious object in his hand; specifically, feature image 406 can be the second feature image in the aforementioned description.

[0084] When passenger 41 attempts to pass through turnstile 40, camera 401 captures an image of passenger 41, generating a video stream containing multiple frames. The host computer software within the turnstile then extracts features from these frames in real time, producing a first feature map set representing passenger 41's actions. The turnstile then selects a first feature map from this set and compares it sequentially with second feature maps from feature map library 406 to determine the similarity between the two feature maps. When the similarity between a first feature map and a second feature map exceeds a preset first threshold, turnstile 40 sends a notification to the control center, indicating that a passenger is attempting to pass through. For example, feature map 407 represents the action of a passenger raising their arm without any obvious object in their hand, and feature map 405 also represents the same action. However, if the similarity between feature map 405 and feature map 407 in feature map library 406 obtained by gate 40 is greater than the first threshold mentioned above, gate 40 will send a prompt message to the control center, indicating that a passenger is attempting to pass through the gate. The control center can then send this message via multicast to relevant station staff for rapid prediction and location, allowing for early intervention. This prevents passenger 41 from attempting to pass through gate 40, thereby improving the gate's ticket checking efficiency and the user's travel experience.

[0085] The following is a schematic diagram of the structure of a video processing device provided in an embodiment of this application. Please refer to [link / reference]. Figure 5 .like Figure 5 As shown, Figure 5 The structural diagram of the video processing device in the diagram can execute... Figure 2 The video processing method includes the following apparatus: a shooting unit 501, configured to acquire a first video stream containing a first object, the first video stream comprising multiple first video frames; an extraction unit 502, configured to extract features from each of the multiple first video frames to obtain a first feature set representing the arm movement features of the first object, the first feature set comprising at least one first feature image; an acquisition unit 503, configured to acquire a pre-stored second feature set, the second feature set comprising at least one second feature image; a comparison unit 504, configured to compare the at least one first feature image in the first feature set with the at least one second feature image in the second feature set to obtain a comparison result; an analysis unit 505, configured to determine, based on the comparison result, whether the first object has attempted to break through the gate; and a sending unit 506, configured to send a prompt message to a control center if the first object has attempted to break through the gate.

[0086] In one possible implementation of the second aspect, the shooting unit 501 is further configured to acquire a second video stream containing the second object from historical data, the second video stream being a video stream of the second object passing through a gate, the second video stream including multiple second video frames; the extraction unit 502 is further configured to perform feature extraction on each of the multiple second video frames to obtain at least one second feature map representing the arm movement features of the second object; the at least one second feature map is used to form the second feature map set, and the second feature map set is stored.

[0087] In one possible implementation of the second aspect, the above-mentioned device further includes: a calling unit 507, used to call the ticket checking interface to query whether the first object has completed ticket checking; and a comparison unit 504, specifically used to compare the first feature set with the second feature set when the first object has not completed ticket checking, to obtain the comparison result.

[0088] In one possible implementation of the second aspect, the comparison unit 504 is specifically used to calculate the similarity between each first feature map in the at least one first feature map and each second feature map in the at least one second feature map, and to use the similarity between each first feature map and each second feature map as the comparison result.

[0089] In one possible implementation of the second aspect, the analysis unit 505 is specifically used to determine that the first object has attempted to break through the gate if there is a similarity greater than the first threshold in the comparison results.

[0090] In one possible implementation of the second aspect, the analysis unit 505 is specifically used to obtain the number of similarities in the comparison results that are greater than the first threshold if there is a similarity greater than the first threshold; if the number is greater than the second threshold, then it is determined that the first object has attempted to break through the gate.

[0091] In one possible implementation of the second aspect, the shooting unit 501 is further configured to capture a target image via a camera, the target image including a face image of the first object, and the camera being mounted on a turnstile; specifically, the shooting unit 501 is configured to capture the first object via the camera when the face image is a frontal face image of the first object, thereby obtaining the first video stream.

[0092] It should be understood that the division of the various units in the above video processing device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. For example, each of the above units can be a separately established processing element, or they can be integrated into the same chip. Alternatively, they can be stored as program code in the controller's storage element, and called and executed by a processing element of the processor. Furthermore, the units can be integrated together or implemented independently. The processing element here can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the method or each of the above units can be completed through hardware integrated logic circuits in the processor element or software instructions. The processing element can be a general-purpose processor, such as a CPU, or one or more integrated circuits configured to implement the above method, such as one or more application-specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), etc.

[0093] Figure 6 This is a schematic diagram of a gate provided in an embodiment of this application. Figure 6 As shown, the gate 60 includes a processor 601, a memory 602, a camera 603, and a communication interface 604; the processor 601, memory 602, camera 603, and communication interface 604 are interconnected via a bus 605, and the gate can be used in a railway ticket checking system.

[0094] The memory 602 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CDROM). The memory 602 is used for related instructions and data. Specifically, the memory can implement... Figure 5 The function of the acquisition unit 503 is described above. The camera 603 is used to capture images of the scene and transmit the captured video stream to the processor 501 via the bus 605. Specifically, the camera can achieve... Figure 5 The function of the imaging unit 501. The communication interface 604 is used to receive and send data, such as to... Figure 4 The control center sends notification messages, etc. Specifically, communication interface 604 can achieve this. Figure 5 The function of the sending unit 506 in the middle.

[0095] The processor 601 can be one or more central processing units (CPUs). If the processor 601 is a CPU, the CPU can be a single-core CPU or a multi-core CPU. The steps performed by the video processing device in the above embodiments can be based on the above... Figure 6 The structure of the turnstile is shown. Specifically, the processor 601 can implement... Figure 5 The functions of the extraction unit 502, comparison unit 504, analysis unit 505, and calling unit 507 are described.

[0096] In embodiments of this application, another computer-readable storage medium is provided, which stores a computer program that, when executed by a processor, performs the following: acquiring a first video stream containing a first object, the first video stream comprising a plurality of first video frames; performing feature extraction on each of the plurality of first video frames to obtain a first feature map set representing the arm movement features of the first object, the first feature map set comprising at least one first feature map; acquiring a pre-stored second feature map set, the second feature map set comprising at least one second feature map; comparing the at least one first feature map in the first feature map set with the at least one second feature map in the second feature map set to obtain a comparison result; determining, based on the comparison result, whether the first object has attempted to break through the gate; and, if the first object has attempted to break through the gate, sending a prompt message to a control center.

[0097] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the video processing method provided in the foregoing embodiments.

[0098] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0099] This invention is described in terms of flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowcharts and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0100] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0101] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0102] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A video processing method, characterized in that, include: Acquire a first video stream containing a first object, the first video stream comprising a plurality of first video frames; Feature extraction is performed on each of the plurality of first video frames to obtain a first feature map set representing the arm movement features of the first object. The first feature map set includes at least one first feature map. Obtain a pre-stored second feature map set, the second feature map set including at least one second feature map; The at least one first feature map in the first feature map set is compared with the at least one second feature map in the second feature map set to obtain a comparison result; Based on the comparison results, it is determined whether the first object attempted to break through the gate; If the first object attempts to break through the gate, a notification message is sent to the control center.

2. The method according to claim 1, characterized in that, Before acquiring the first video stream containing the first object, the method further includes: Acquire a second video stream containing the second object from historical data collection. The second video stream is the video stream of the second object, the gate-breaking machine. The second video stream includes multiple second video frames. Feature extraction is performed on each of the plurality of second video frames to obtain at least one second feature map representing the arm movement features of the second object; The at least one second feature map is used to form the second feature map set, and the second feature map set is stored.

3. The method according to claim 1, characterized in that, Before comparing the first feature map set with the second feature map set to obtain the comparison result, the method further includes: Call the ticket checking interface to check whether the first object has completed ticket checking; The step of comparing the first feature map set with the second feature map set to obtain the comparison result includes: If the first object has not completed ticket inspection, the first feature set is compared with the second feature set to obtain the comparison result.

4. The method according to any one of claims 1 to 3, characterized in that, The step of comparing the first feature map set with the second feature map set to obtain the comparison result includes: The similarity between each first feature map in the at least one first feature map and each second feature map in the at least one second feature map is calculated, and the similarity between each first feature map and each second feature map is used as the comparison result.

5. The method according to claim 4, characterized in that, The step of determining whether the first object attempted to break through the gate based on the comparison result includes: If the similarity in the comparison results is greater than the first threshold, it is determined that the first object has attempted to break through the gate.

6. The method according to claim 4, characterized in that, The step of determining whether the first object attempted to break through the gate based on the comparison result includes: If there is a similarity greater than the first threshold in the comparison results, obtain the number of similarities greater than the first threshold in the comparison results; If the number is greater than the second threshold, it is determined that the first object is attempting to break through the gate.

7. The method according to any one of claims 1 to 3, characterized in that, Prior to acquiring the first video stream containing the first object, the method further includes: The target image is captured by a camera, the target image containing the face image of the first object, and the camera is installed on the gate. The acquisition of the first video stream containing the first object includes: When the face image is a frontal image of the first object, the first object is captured by the camera to obtain the first video stream.

8. A video processing apparatus, characterized in that, include: A shooting unit is used to acquire a first video stream containing a first object, the first video stream including a plurality of first video frames; An extraction unit is used to extract features from each of the plurality of first video frames to obtain a first feature set representing the arm movement features of the first object. The first feature set includes at least one first feature map. An acquisition unit is configured to acquire a pre-stored second feature map set, the second feature map set including at least one second feature map; The comparison unit is used to compare at least one first feature map in the first feature map set with at least one second feature map in the second feature map set to obtain a comparison result; The analysis unit is used to determine, based on the comparison results, whether the first object has attempted to break through the gate; The sending unit is used to send a prompt message to the control center when the first object attempts to break through the gate.

9. A turnstile, characterized in that, The gate includes: one or more processors and a memory; The memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the gate to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium comprising instructions, characterized in that, When the instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method as described in any one of claims 1 to 7.

Images