A video stream recognition method and device

Abstract

One or more embodiments of the specification provide a video stream identification method and device, the method comprising: determining optical flow change information corresponding to a target video stream, wherein a theoretical collection time period of the target video stream is a preset time period and a theoretical collection device is a target user terminal; and determining pose change information of the target user terminal, wherein the pose change information is determined based on sensor detection information of the target user terminal within the preset time period. According to comparison information between the determined optical flow change information and the pose change information, a credibility identification result for the target video stream is determined.

Description

[0001] This application was filed with the Chinese Patent Office on December 31, 2019, with application number 2019114033183, for invention.

[0002] A divisional application of a Chinese patent application entitled “A video stream recognition method and apparatus”. Technical Field

[0003] This document relates to the field of Internet technology, and in particular to a video stream recognition method and apparatus. Background Technology

[0004] Currently, with the advent of the internet age and the rapid development of mobile internet technology, the internet is widely used in people's daily learning, work, and life. People can handle and present various daily tasks via the internet using user terminals (such as smartphones). Users can install corresponding applications on their smartphones according to their actual needs, such as payment applications, financial management applications, instant messaging applications, shopping applications, and so on.

[0005] Currently, if a user needs to complete a certain business within an application, they need to trigger the smartphone's camera to capture a target object (such as a user's face, bank card, ID card, or receipt) using an upload control within the application, and upload the video stream information of the target object so that the corresponding internet service can be activated for the user based on this video stream information. However, malicious users may use video stream injection attacks, that is, directly hooking the smartphone's hardware / driver / API layer to replace the real-time captured raw video stream with pre-stored video stream information, so as to use the pre-stored video stream information as the input source to maliciously trigger the execution of the target service.

[0006] For example, taking account login for payment applications based on facial recognition as an example, a malicious user can obtain the target user's facial capture video information in advance. Then, during facial image capture, the malicious user uploads the target user's facial capture video information to the identity verification server by replacing video frames. At this time, the identity verification server will verify the login user based on the facial capture video information, thereby determining that the user's identity verification is successful, allowing the malicious user to complete the identity verification and enter the user operation interface. This provides an entry point for malicious users to carry out illegal activities, and fails to achieve the purpose of ensuring account security through identity verification.

[0007] Therefore, there is a need to provide a technical solution that can identify video streams quickly, accurately, and reliably. Summary of the Invention

[0008] The purpose of one or more embodiments of this specification is to provide a video stream identification method. This video stream identification method includes:

[0009] The optical flow variation information corresponding to the target video stream is determined, wherein the theoretical acquisition time period of the target video stream is a preset time period and the theoretical acquisition device is the target user terminal. Furthermore, the pose variation information of the target user terminal is determined, wherein the pose variation information is determined based on sensor detection information of the target user terminal within the preset time period. Based on the comparison information between the optical flow variation information and the pose variation information, a credibility identification result for the target video stream is determined.

[0010] The purpose of one or more embodiments of this specification is to provide a video stream recognition device. The video stream recognition device includes:

[0011] The optical flow variation information corresponding to the target video stream is determined, wherein the theoretical acquisition time period of the target video stream is a preset time period and the theoretical acquisition device is the target user terminal. Furthermore, the pose variation information of the target user terminal is determined, wherein the pose variation information is determined based on sensor detection information of the target user terminal within the preset time period. Based on the comparison information between the optical flow variation information and the pose variation information, a credibility identification result for the target video stream is determined.

[0012] The purpose of one or more embodiments of this specification is to provide a video stream recognition device, including: a processor; and a memory arranged to store computer-executable instructions.

[0013] When executed, the computer-executable instructions cause the processor to determine optical flow change information corresponding to the target video stream, wherein the theoretical acquisition time period of the target video stream is a preset time period and the theoretical acquisition device is the target user terminal. Furthermore, the processor determines the pose change information of the target user terminal, wherein the pose change information is determined based on sensor detection information of the target user terminal within the preset time period. Based on the comparison information between the optical flow change information and the pose change information, a credibility identification result for the target video stream is determined.

[0014] The purpose of one or more embodiments of this specification is to provide a storage medium for storing computer-executable instructions. When executed by a processor, the executable instructions determine optical flow variation information corresponding to a target video stream, wherein the theoretical acquisition time period of the target video stream is a preset time period and the theoretical acquisition device is a target user terminal. Furthermore, they determine pose variation information of the target user terminal, wherein the pose variation information is determined based on sensor detection information of the target user terminal within the preset time period. Based on a comparison between the optical flow variation information and the pose variation information, a credibility identification result for the target video stream is determined. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in one or more embodiments of this specification or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in one or more of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 A schematic diagram of a first process for a video stream recognition method provided in one or more embodiments of this specification;

[0017] Figure 2 A schematic diagram illustrating the specific implementation principle of the video stream recognition method provided in one or more embodiments of this specification;

[0018] Figure 3 A second flowchart illustrating a video stream recognition method provided in one or more embodiments of this specification;

[0019] Figure 4 A schematic diagram illustrating the specific implementation principle of determining optical flow change information in a video stream recognition method provided in one or more embodiments of this specification;

[0020] Figure 5 A third flowchart illustrating a video stream recognition method provided in one or more embodiments of this specification;

[0021] Figure 6 A schematic diagram illustrating the specific implementation principle of determining pose change information in the video stream recognition method provided in one or more embodiments of this specification;

[0022] Figure 7 A schematic diagram of a fourth process for a video stream recognition method provided in one or more embodiments of this specification;

[0023] Figure 8A schematic diagram of the module composition of the video stream recognition device provided in one or more embodiments of this specification;

[0024] Figure 9 This is a schematic diagram of the structure of a video stream recognition device provided in one or more embodiments of this specification. Detailed Implementation

[0025] To enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of one or more embodiments of this specification, and not all embodiments. Based on the embodiments in one or more embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this document.

[0026] It should be noted that, unless otherwise specified, one or more embodiments and features described in this specification can be combined with each other. One or more embodiments of this specification will now be described in detail with reference to the accompanying drawings and examples.

[0027] This specification provides one or more embodiments of a video stream identification method and apparatus. By comparing and analyzing the optical flow change information determined based on the target video stream with the pose change information of the target user terminal, it identifies whether the target video stream is video stream information captured and uploaded in real time for the target object. This enables rapid identification of malicious video stream attacks that replace real-time video streams with pre-stored non-real-time video streams, allowing for timely interception of the non-real-time video stream injected by the malicious video stream attack and improving the accuracy of subsequent business processing.

[0028] Figure 1 This is a schematic diagram of a first process for a video stream recognition method provided in one or more embodiments of this specification. Figure 1The method described can be executed by a user terminal or a server. The user terminal can be a mobile terminal such as a smartphone or a terminal device such as an IoT device. Specifically, the user terminal can be used to collect video stream information of a target object and perform credibility identification on the video stream information. If the credibility identification is successful, it determines whether to perform corresponding control operations based on the target video stream information, or uploads the video stream information to the server so that the server can continue to perform user authentication based on the video stream information. The server can be a backend server or a cloud server. Specifically, the server is used to receive the video stream information uploaded by the user terminal, perform credibility identification on the video stream information, and if the credibility identification is successful, perform user authentication based on the video stream information, and provide a certain business service to the user if the authentication is successful.

[0029] This includes the process of verifying the credibility of video stream information of a target object by the user terminal or the server, such as... Figure 1 As shown, the above video stream recognition method includes at least the following steps:

[0030] S102, determine the optical flow change information corresponding to the target video stream, wherein the theoretical acquisition time period of the target video stream is a preset time period and the theoretical acquisition device is the target user terminal;

[0031] Specifically, after detecting a video stream acquisition request, the user terminal uses a camera device to acquire video stream information of the target object and obtains the target video stream used for user authentication. The theoretical acquisition time period of the target video stream includes the time period from the start time of video stream acquisition to the end time of acquisition.

[0032] In this regard, considering that the target video stream may be video stream information collected in real time by the camera device, or it may be non-real-time video stream information injected by replacing the real-time video stream information through a preset video stream attack method, before performing user authentication based on the target video stream, the optical flow change information corresponding to the target video stream is first determined so as to compare the optical flow change information with the pose change information of the target user terminal to identify whether the target video stream is a credible video stream information collected in real time by the target user terminal.

[0033] Specifically, for the case where the target video stream is real-time acquired video stream information, the theoretical acquisition time period and the actual acquisition time period of the target video stream are the same, and both the theoretical acquisition device and the actual acquisition device are the target user terminal, meaning that the target video stream is acquired by the target user terminal within the preset time period. However, for the case where the target video stream is non-real-time acquired video stream information injected by replacing real-time acquired video stream information through preset video stream attack methods, the theoretical acquisition time period and the actual acquisition time period of the target video stream are different, meaning that the actual acquisition time of the target video stream is not the preset time period, and the actual acquisition device may not be the target user terminal, but rather a video stream information pre-stored by the target user terminal to replace the real-time acquired video stream information.

[0034] S104, determine the pose change information of the target user terminal, wherein the pose change information is determined based on the sensor detection information of the target user terminal within a preset time period;

[0035] The target user terminal includes: a camera device and at least one preset sensor; the preset sensor may be an IMU sensor, such as a gyroscope, or an accelerometer, and the sensor detection information may include at least one of the sensor's three-axis attitude angle, angular rate, and acceleration;

[0036] Specifically, while the target user terminal uses a camera device to capture video streams of the target object, it also uses at least one preset sensor to collect sensor detection information in order to determine the pose change information of the target user terminal based on the sensor detection information; then, the pose change information is used as the basis for comparison with the optical flow change information of the target video stream, thereby identifying the credibility of the target video stream.

[0037] S106, Based on the comparison information between the determined optical flow change information and pose change information, determine the credibility recognition result for the target video stream.

[0038] In the case where the target video stream is real-time acquired video stream information, since the target video stream and sensor detection information are acquired synchronously when the target user terminal is under a certain jitter state, that is, when the target user terminal is not absolutely stationary, the target video stream and sensor detection information are acquired at the same time. Therefore, the direction of optical flow change naturally recorded in the target video stream and the direction of sensor spatial motion naturally recorded in the sensor detection information should have directional change consistency. Therefore, the comparison information between the optical flow change information determined based on the target video stream and the pose change information determined based on the sensor detection information should meet the preset change consistency condition.

[0039] In cases where the target video stream is not acquired in real time, the target video stream and the sensor detection information are not acquired synchronously under the same jitter state. Therefore, the direction of optical flow change naturally recorded in the target video stream and the direction of sensor spatial motion naturally recorded in the sensor detection information do not have directional change consistency. Consequently, the comparison information between the optical flow change information determined based on the target video stream and the pose change information determined based on the sensor detection information will not meet the preset change consistency condition.

[0040] Therefore, it can be seen that by comparing the optical flow change information determined based on the target video stream with the pose change information determined based on the sensor detection information, it can be identified whether the target video stream is a reliable video stream information collected in real time by the target user terminal.

[0041] In one or more embodiments of this specification, by comparing and analyzing the optical flow change information determined based on the target video stream with the pose change information of the target user terminal, it is possible to identify whether the target video stream is video stream information captured and uploaded in real time for the target object. This enables rapid identification of malicious video stream attacks that use pre-stored non-real-time acquired video streams to replace real-time acquired video streams, so as to promptly intercept the non-real-time acquired video stream injected by the malicious video stream attack and improve the accuracy of subsequent business processing.

[0042] Specifically, in the case where the user terminal performs credibility identification on the target video stream, the user terminal obtains the target video stream and sensor detection information collected within a preset time period, and determines the credibility identification result of the target video stream based on the above steps S102 to S106; when the credibility identification result indicates that the target video stream is a real-time acquired credible video stream, the user terminal identifies whether to perform corresponding control operations based on the target video stream information, or uploads the target video stream to the server so that the server can perform user authentication based on the target video stream.

[0043] Correspondingly, in the case where the server performs credibility identification on the target video stream, the user terminal obtains the target video stream and sensor detection information collected within a preset time period, and uploads the target video stream and sensor detection information to the server, so that the server determines the credibility identification result of the target video stream based on the above steps S102 to S106; and when the credibility identification result is that the target video stream is a real-time acquired credible video stream, user authentication is performed based on the target video stream.

[0044] In specific implementation, such as Figure 2 As shown, taking user identification documents as the target object and smartphones as the target user terminal, a schematic diagram illustrating the specific implementation principle of the video stream recognition method is presented, including:

[0045] (1) After the user terminal detects the video stream acquisition request, that is, after the user triggers the video stream upload control, it uses the camera device to acquire the video stream information of the target object; wherein, the actual acquisition time period of the video stream information of the target object is a preset time period, which includes: multiple specified time nodes;

[0046] (2) During the aforementioned preset time period, the user terminal uses at least one preset sensor to collect sensor detection information; wherein, the target user terminal includes: a camera device and at least one preset sensor;

[0047] (3) Acquire the target video stream to be identified; wherein the theoretical acquisition time period of the target video stream is a preset time period and the theoretical acquisition device is the target user terminal;

[0048] Specifically, if the theoretical acquisition time period and the actual acquisition time period of the target video stream are the same, and both the theoretical acquisition device and the actual acquisition device are the target user terminal, then the target video stream is the video stream information of the target object acquired by the camera device mentioned above; otherwise, the target video stream is an untrusted video stream that has been maliciously replaced.

[0049] (4) Acquire sensor detection information collected within a preset time period; wherein, the sensor detection information may include at least one of the three-axis attitude angle, angular rate, and acceleration of the sensor; while the target user terminal is collecting video stream from the target object using the camera device, it also uses at least one preset sensor to collect sensor detection information.

[0050] (5) Based on multiple object image frames corresponding to multiple specified time nodes in the acquired target video stream, determine the optical flow change information corresponding to the target video stream;

[0051] (6) Based on the sensor detection information obtained at each specified time point, determine the pose change information of the target user terminal;

[0052] (7) Based on the comparison information between the determined optical flow change information and pose change information, determine the credibility recognition result for the target video stream.

[0053] It should be noted that the processes (3) to (7) above can be executed by the user terminal, especially the information processing module in the user terminal, and can also be executed by the server.

[0054] In the process of determining optical flow change information based on the target video stream of the target object, the aforementioned preset time period includes multiple specified time nodes; wherein, the multiple specified time nodes can be obtained by dividing the preset time period according to a certain time interval.

[0055] Correspondingly, such as Figure 3 As shown, in S102 above, determining the optical flow change information corresponding to the target video stream specifically includes:

[0056] S1021, Obtain the target video stream to be identified, wherein the target video stream includes: multiple object image frames containing the target object corresponding to multiple specified time nodes respectively;

[0057] In the case where the server performs credibility identification on the target video stream, the target video stream is uploaded to the server by the target user terminal, which triggers the server to perform user authentication based on the target video stream.

[0058] S1022, determine the optical flow change information corresponding to the target video stream based on the object image frames at multiple specified time points.

[0059] Specifically, after receiving the target video stream uploaded by the target user terminal, the server does not directly perform user authentication based on the target video stream. Instead, it first determines the corresponding optical flow change information based on the target video stream, and then performs credibility identification on the target video stream based on the optical flow change information and the pose change information determined by the sensor detection information collected in the same preset time period. When the credibility identification result is that the target video stream is a real-time acquired credible video stream, user authentication is performed based on the target video stream.

[0060] Specifically, in S1022 above, the optical flow change information corresponding to the target video stream is determined based on object image frames at multiple specified time points, including:

[0061] Step 1: Determine the two object image frames corresponding to two adjacent specified time nodes as an image frame combination;

[0062] Step 2: For each image frame combination, the image optical flow information of the image frame combination is identified using a preset optical flow method to obtain the corresponding optical flow spatial motion matrix;

[0063] One approach is to use existing optical flow methods to analyze the optical flow direction of multiple object image frames in the target video stream, obtaining optical flow change information. Specifically, the optical flow method assigns a velocity vector to each pixel in each object image frame, forming a motion vector field. Based on the displacement direction of each pixel in the object image frames corresponding to two adjacent specified time nodes, the velocity vector characteristics of each pixel are obtained. Based on the velocity vector characteristics of each pixel, the corresponding optical flow spatial motion matrix is ​​determined.

[0064] Step 3: Determine the optical flow change information corresponding to the target video stream based on the optical flow spatial motion matrix corresponding to each image frame combination.

[0065] In this context, the optical flow naturally recorded in the video stream is generated by the movement of the camera device due to the jitter of the user terminal. Optical flow can express changes in the image, thus reflecting the amplitude of the user terminal's jitter. Simultaneously, if sensor detection information is acquired through a preset sensor in the user terminal, the spatial motion direction naturally recorded in this sensor detection information is also generated by the movement of the preset sensor caused by the jitter of the user terminal. This spatial motion direction can also reflect the amplitude of the user terminal's jitter. Since the jitter amplitudes are synchronous and consistent, the changing pattern of the optical flow direction naturally recorded in the video stream is directly correlated with the changing pattern of the spatial motion direction naturally recorded in the sensor detection information. The changing pattern of the optical flow direction and the changing pattern of the sensor's spatial motion direction should exhibit consistency.

[0066] Furthermore, considering that if the target object is relatively stationary, for example, a document placed on a fixed plane, the optical flow naturally recorded in the video stream is generated by the movement of the camera device caused by the user terminal's jitter. However, in reality, there may be situations where the target object itself moves, such as a user's face. In this case, the optical flow naturally recorded in the video stream is not only generated by the movement of the camera device caused by the user terminal's jitter but also includes the optical flow caused by the movement of the target itself. If the optical flow caused by the movement of the target itself is taken into account, it will affect the accuracy of the comparison information between optical flow change information and pose change information, thereby reducing the accuracy of the credibility recognition result of the target video stream.

[0067] Therefore, to further improve the accuracy of the credibility recognition results for target video streams, the image region containing the moving target can be removed from the object image frame. The removed object image frame can then be used as the basis for determining optical flow change information. Correspondingly, in step two above, for each image frame combination, the image optical flow information of the image frame combination is recognized using a preset optical flow method to obtain the corresponding optical flow spatial motion matrix, specifically including:

[0068] For each image frame combination, a specified region is removed from the two object image frames in the image frame combination to obtain two object image frames after removal; wherein, the specified region is the image region containing the moving target;

[0069] The optical flow information of the two object image frames after removal is identified using a preset optical flow method, and the corresponding optical flow spatial motion matrix is ​​obtained.

[0070] Furthermore, considering the need to compare the optical flow change information of the target video stream with the pose change information of the target user terminal to identify whether the target video stream is a reliable video stream information acquired in real time by the target user terminal, in order to improve the accuracy and reference value of the information comparison, it is necessary to determine the optical flow change information and pose change information in the same spatial coordinate system. Based on this, in step three above, the optical flow change information corresponding to the target video stream is determined according to the optical flow spatial motion matrix corresponding to each image frame combination, specifically including:

[0071] For each optical flow spatial motion matrix, a coordinate system transformation is performed on the optical flow spatial motion matrix under a preset spatial coordinate system to obtain the transformed optical flow spatial motion matrix;

[0072] Based on the transformed optical flow spatial motion matrices, the optical flow change information corresponding to the target video stream is determined.

[0073] In one specific embodiment, such as Figure 4 As shown, a schematic diagram illustrating the specific implementation principle of determining optical flow change information in video stream recognition methods is presented, specifically as follows:

[0074] (1) Obtain object image frames corresponding to multiple specified time nodes within a preset time period, for example, object image frame 1...object image frame i...object image frame n;

[0075] (2) Determine the two object image frames corresponding to two adjacent specified time nodes as an image frame combination;

[0076] (3) For each image frame combination, the image optical flow information of the image frame combination is identified using the preset optical flow method to obtain the corresponding optical flow spatial motion matrix. For example, the currently identified image frame combination includes: object image frame i and object image frame i+1, where i = 1…n-1;

[0077] (4) For each optical flow spatial motion matrix, perform coordinate system transformation on the optical flow spatial motion matrix under the preset spatial coordinate system to obtain the transformed optical flow spatial motion matrix;

[0078] (5) Determine the optical flow change information corresponding to the target video stream based on the transformed optical flow spatial motion matrix.

[0079] In this process, while the target user terminal uses a camera device to capture video streams of the target object, it also collects sensor detection information through at least one preset sensor in order to determine the pose change information of the target user terminal based on the sensor detection information. Specifically, for the process of determining pose change information based on sensor detection information, the preset time period includes multiple specified time nodes; wherein, the multiple specified time nodes can be obtained by dividing the preset time period according to a certain time interval, that is, the multiple specified time nodes are the same as the specified time nodes selected for determining optical flow change information.

[0080] Correspondingly, such as Figure 5 As shown, S104 above determines the pose change information of the target user terminal, specifically including:

[0081] S1041, Obtain sensor detection information collected by at least one preset sensor of the target user terminal at each specified time node;

[0082] In the case where the server performs credibility identification on the target video stream, the sensor detection information is uploaded to the server by the target user terminal, and the actual acquisition time of the sensor detection information is the same as the theoretical acquisition time of the target video stream.

[0083] S1042, determine the pose change information of the target user terminal based on sensor detection information at multiple specified time points.

[0084] Specifically, after obtaining the sensor detection information corresponding to each specified time node, the sensor spatial motion information of each specified time node is determined based on the sensor detection information, and then the pose change information of the target user terminal is determined.

[0085] Specifically, in S1042 above, based on sensor detection information at multiple specified time points, the pose change information of the target user terminal is determined, including:

[0086] Step 1: Determine the corresponding sensor spatial motion matrix based on the sensor detection information corresponding to each pair of adjacent specified time nodes;

[0087] Specifically, based on the sensor detection information collected by the preset sensors, the spatial state information of the preset sensors is identified; and the corresponding sensor spatial motion matrix is ​​determined according to the spatial state information corresponding to each pair of adjacent specified time nodes.

[0088] Step 2: Determine the pose change information of the target user terminal based on the spatial motion matrix of each sensor.

[0089] Furthermore, considering the need to compare the optical flow change information of the target video stream with the pose change information of the target user terminal to identify whether the target video stream is a reliable video stream information acquired in real time by the target user terminal, in order to improve the accuracy and reference value of the information comparison, it is necessary to determine the optical flow change information and pose change information in the same spatial coordinate system. Based on this, step two above, according to the spatial motion matrix of each sensor, determines the pose change information of the target user terminal, specifically including:

[0090] For each sensor spatial motion matrix, a coordinate system transformation is performed on the sensor spatial motion matrix under a preset spatial coordinate system to obtain the transformed sensor spatial motion matrix;

[0091] Based on the transformed sensor spatial motion matrices, the pose change information of the target user terminal is determined.

[0092] In one specific embodiment, such as Figure 6 As shown, a schematic diagram illustrating the specific implementation principle of determining pose change information in video stream recognition methods is presented, specifically as follows:

[0093] (1) Obtain sensor detection information corresponding to multiple specified time nodes within a preset time period, for example, sensor detection information 1... sensor detection information i... sensor detection information n;

[0094] (2) Determine the corresponding sensor spatial motion matrix based on the sensor detection information corresponding to each pair of adjacent specified time nodes. For example, the currently identified detection information combination includes: sensor detection information i and sensor detection information i+1, where i = 1…n-1;

[0095] (3) For each sensor spatial motion matrix, the sensor spatial motion matrix is ​​transformed in a preset spatial coordinate system to obtain the transformed sensor spatial motion matrix.

[0096] (4) Determine the pose change information of the target user terminal based on the transformed sensor spatial motion matrix.

[0097] In this regard, considering that if the target video stream is real-time acquired video stream information, the direction of optical flow change naturally recorded in the target video stream should have directional consistency with the direction of sensor spatial motion naturally recorded in the sensor detection information. Therefore, for the credibility identification process of the target video stream, such as... Figure 7 As shown, in S106 above, based on the comparison information between the determined optical flow change information and pose change information, the credibility recognition result for the target video stream is determined, specifically including:

[0098] S1061, compare the determined optical flow change information with the pose change information to obtain the corresponding optical flow pose comparison result;

[0099] The optical flow pose comparison results may include at least one of the following: the relative difference comparison results between optical flow change information and pose change information, the change similarity comparison results, or the relative change trend comparison results.

[0100] S1062, determine whether the determined optical flow pose comparison result meets the preset change consistency condition;

[0101] The preset consistency conditions include: the relative difference in optical flow pose is less than the preset maximum difference threshold, the similarity of optical flow pose changes is greater than the preset minimum similarity threshold, or the relative change trend of optical flow pose meets the preset trend consistency condition.

[0102] If the judgment result is yes, then S1063, determine that the target video stream is a real-time acquired reliable video stream information;

[0103] Specifically, if the optical flow pose comparison result is determined to meet the preset change consistency condition, it means that the theoretical acquisition time period and the actual acquisition time period of the target video stream are the same, and that both the theoretical acquisition device and the actual acquisition device are the target user terminal. That is, the target video stream was acquired by the target user terminal within the preset time period. Therefore, the target video stream has not been attacked by malicious users.

[0104] If the judgment result is negative, then in S1064, the target video stream is determined to be an untrusted video stream information that was not acquired in real time;

[0105] Specifically, if the optical flow pose comparison result does not meet the preset change consistency condition, it means that the theoretical acquisition time period of the target video stream is different from the actual acquisition time period. That is, the actual acquisition time of the target video stream is not the preset time period, and the actual acquisition device may not be the target user terminal. Therefore, the target video stream may be non-real-time acquired video stream information injected by a malicious user through preset video stream attack methods to replace the real-time acquired video stream information.

[0106] The optical flow change information includes multiple optical flow spatial motion matrices within a preset time period, and the pose change information includes multiple sensor spatial motion matrices within a preset time period. Preferably, the optical flow spatial motion matrix and the sensor spatial motion matrix are obtained by coordinate system transformation in the same spatial coordinate system.

[0107] Correspondingly, in S1061 above, the determined optical flow change information is compared with the pose change information to obtain the corresponding optical flow pose comparison result, specifically including:

[0108] Multiple optical flow spatial motion matrices are compared with multiple sensor spatial motion matrices to obtain the corresponding optical flow pose comparison results.

[0109] Specifically, the optical flow spatial motion matrix and the sensor spatial motion matrix are in one-to-one correspondence. For two adjacent specified time nodes (i.e., the same combination of specified time nodes), there is one optical flow spatial motion matrix and one sensor spatial motion matrix, respectively. Therefore, for each combination of specified time nodes, the difference between the corresponding optical flow spatial motion matrix and the sensor spatial motion matrix is ​​calculated to obtain the relative difference in optical flow pose corresponding to that combination of specified time nodes; or,

[0110] For each specified time node combination, the corresponding optical flow spatial motion matrix and sensor spatial motion matrix are used to calculate matrix similarity, thus obtaining the similarity of optical flow pose changes (e.g., matrix distance) for that specified time node combination; or,

[0111] A first trend of change is determined based on multiple optical flow spatial motion matrices, and a second trend of change is determined based on multiple sensor spatial motion matrices. Based on the first trend of change and the second trend of change, the relative trend of optical flow pose is determined.

[0112] Furthermore, to further improve the accuracy of target video stream credibility recognition, the initial change information can be preprocessed before comparing the optical flow change information and pose change information. Then, the preprocessed optical flow change information and the preprocessed pose change information are compared. Based on this, the above-mentioned comparison of multiple optical flow spatial motion matrices with multiple sensor spatial motion matrices yields the corresponding optical flow pose comparison results, specifically including:

[0113] Multiple optical flow spatial motion matrices and multiple sensor spatial motion matrices are preprocessed to obtain preprocessed multiple optical flow spatial motion matrices and multiple preprocessed sensor spatial motion matrices. The preprocessing includes at least one of the following: smoothing filtering, noise reduction processing, and alignment of comparison start points.

[0114] The preprocessed optical flow spatial motion matrices and the preprocessed sensor spatial motion matrices are compared to obtain the corresponding optical flow pose comparison results.

[0115] In this regard, considering the possible time delay between the acquisition time of video stream information and the acquisition time of sensor detection information, the comparison starting points of optical flow change information and pose change information are aligned; considering that there may be some abnormal points in the determined optical flow change information and pose change information, these abnormal points need to be removed; and considering that the acquired relevant information may be subject to external noise interference, the change information can be denoising processed; thereby further improving the accuracy of the reliable identification of the target video stream.

[0116] Specifically, regarding the determination process of optical flow pose comparison results, it is possible to analyze whether the difference between the optical flow spatial motion matrix and the sensor spatial motion matrix is ​​less than a preset threshold, or to analyze whether the relative change trends between the optical flow spatial motion matrix and the sensor spatial motion matrix are consistent. Based on this, the above-mentioned preprocessed multiple optical flow spatial motion matrices and preprocessed multiple sensor spatial motion matrices are compared to obtain the corresponding optical flow pose comparison results, specifically including:

[0117] For each pair of adjacent specified time nodes within a preset time period, the preprocessed optical flow spatial motion matrix corresponding to each pair of adjacent specified time nodes is subtracted from the preprocessed sensor spatial motion matrix to obtain the corresponding optical flow pose comparison result.

[0118] or,

[0119] The first change trend of multiple preprocessed optical flow spatial motion matrices and the second change trend of multiple preprocessed sensor spatial motion matrices are compared to obtain the corresponding optical flow pose comparison results.

[0120] The first and second trends can be represented by waveform curves. The amplitude of the first waveform curve corresponding to the first trend represents the direction of optical flow change in the image, and the amplitude of the second waveform curve corresponding to the second trend represents the direction of spatial motion of the sensor. Since both the direction of optical flow change in the image and the direction of spatial motion of the sensor are caused by the same jitter of the target user terminal, if the amplitude changes of the first and second waveform curves are consistent, the target video stream is determined to be a reliable video stream information collected in real time by the target user terminal. In addition, considering the possibility of individual anomalies or external noise, the first and second waveform curves can be smoothed and filtered first, and then the amplitude changes of the smoothed first and second waveform curves can be compared. If the amplitude changes are consistent, the target video stream is determined to be a reliable video stream information collected in real time by the target user terminal.

[0121] The video stream identification method in one or more embodiments of this specification determines optical flow change information corresponding to a target video stream, wherein the theoretical acquisition time period of the target video stream is a preset time period and the theoretical acquisition device is a target user terminal; and determines the pose change information of the target user terminal, wherein the pose change information is determined based on sensor detection information of the target user terminal within the preset time period; and determines the credibility identification result of the target video stream based on the comparison information between the determined optical flow change information and the pose change information. By comparing and analyzing the optical flow change information determined based on the target video stream with the pose change information of the target user terminal, it identifies whether the target video stream is video stream information captured and uploaded in real time for the target object, thereby enabling rapid identification of malicious video stream attacks that replace real-time acquired video streams with pre-stored non-real-time acquired video streams, so as to promptly intercept the non-real-time acquired video stream injected by the malicious video stream attack and improve the accuracy of subsequent business processing.

[0122] Corresponding to the above Figures 1 to 7 Based on the same technical concept, one or more embodiments of this specification also provide a video stream recognition device. Figure 8 This is a schematic diagram of the module composition of a video stream recognition device provided in one or more embodiments of this specification, the device being used to perform... Figures 1 to 7 The described video stream recognition method, such as Figure 8 As shown, the device includes:

[0123] The optical flow change information determination module 801 determines the optical flow change information corresponding to the target video stream, wherein the theoretical acquisition time period of the target video stream is a preset time period and the theoretical acquisition device is the target user terminal; and,

[0124] The pose change information determination module 802 determines the pose change information of the target user terminal, wherein the pose change information is determined based on the sensor detection information of the target user terminal within the preset time period.

[0125] The video stream credibility identification module 803 determines the credibility identification result for the target video stream based on the comparison information between the optical flow change information and the pose change information.

[0126] In one or more embodiments of this specification, by comparing and analyzing the optical flow change information determined based on the target video stream with the pose change information of the target user terminal, it is possible to identify whether the target video stream is video stream information captured and uploaded in real time for the target object. This enables rapid identification of malicious video stream attacks that use pre-stored non-real-time acquired video streams to replace real-time acquired video streams, so as to promptly intercept the non-real-time acquired video stream injected by the malicious video stream attack and improve the accuracy of subsequent business processing.

[0127] Optionally, the preset time period includes multiple specified time nodes; the optical flow change information determination module 801, wherein:

[0128] Obtain the target video stream to be identified, wherein the target video stream includes: multiple object image frames containing the target object corresponding to the multiple specified time nodes respectively;

[0129] Based on the object image frames at the specified time points, determine the optical flow change information corresponding to the target video stream.

[0130] Optionally, the optical flow change information determination module 801 includes:

[0131] Two adjacent object image frames corresponding to the specified time nodes are determined as an image frame combination;

[0132] For each image frame combination, the image optical flow information of the image frame combination is identified using a preset optical flow method to obtain the corresponding optical flow spatial motion matrix;

[0133] Based on the optical flow spatial motion matrix corresponding to each of the image frame combinations, the optical flow change information corresponding to the target video stream is determined.

[0134] Optionally, the optical flow change information determination module 801 includes:

[0135] For each optical flow spatial motion matrix, a coordinate system transformation is performed on the optical flow spatial motion matrix in a preset spatial coordinate system to obtain the transformed optical flow spatial motion matrix.

[0136] Based on the transformed optical flow spatial motion matrices, the optical flow change information corresponding to the target video stream is determined.

[0137] Optionally, the preset time period includes multiple specified time nodes; the pose change information determination module 802, wherein:

[0138] Obtain sensor detection information collected by at least one preset sensor of the target user terminal at each of the specified time nodes;

[0139] Based on the sensor detection information at the specified time points, the pose change information of the target user terminal is determined.

[0140] Optionally, the pose change information determination module 802 includes:

[0141] Based on the sensor detection information corresponding to each pair of adjacent specified time nodes, determine the corresponding sensor spatial motion matrix;

[0142] Based on the spatial motion matrices of each sensor, the pose change information of the target user terminal is determined.

[0143] Optionally, the pose change information determination module 802 includes:

[0144] For each of the sensor spatial motion matrices, a coordinate system transformation is performed on the sensor spatial motion matrix under a preset spatial coordinate system to obtain the transformed sensor spatial motion matrix;

[0145] Based on the transformed sensor spatial motion matrices, the pose change information of the target user terminal is determined.

[0146] Optionally, the video stream credibility identification module 803 includes:

[0147] The optical flow change information is compared with the pose change information to obtain the corresponding optical flow pose comparison result;

[0148] Determine whether the optical flow pose comparison result meets the preset change consistency condition;

[0149] If the judgment result is yes, then the target video stream is determined to be a reliable video stream information acquired in real time;

[0150] If the judgment result is negative, then the target video stream is determined to be an unreliable video stream information that was not acquired in real time.

[0151] Optionally, the optical flow change information includes: multiple optical flow spatial motion matrices within the preset time period, and the pose change information includes: multiple sensor spatial motion matrices within the preset time period;

[0152] The video stream credibility identification module 803, wherein:

[0153] The multiple optical flow spatial motion matrices are compared with the multiple sensor spatial motion matrices to obtain the corresponding optical flow pose comparison results.

[0154] Optionally, the video stream credibility identification module 803 includes:

[0155] The plurality of optical flow spatial motion matrices and the plurality of sensor spatial motion matrices are preprocessed to obtain preprocessed plurality of optical flow spatial motion matrices and preprocessed plurality of sensor spatial motion matrices, wherein the preprocessing includes at least one of: smoothing filtering, noise reduction processing, and comparison start point alignment;

[0156] The preprocessed optical flow spatial motion matrices and the preprocessed sensor spatial motion matrices are compared to obtain the corresponding optical flow pose comparison results.

[0157] Optionally, the video stream credibility identification module 803 includes:

[0158] For each pair of adjacent specified time nodes within the preset time period, the preprocessed optical flow spatial motion matrix corresponding to each pair of adjacent specified time nodes is subtracted from the preprocessed sensor spatial motion matrix to obtain the corresponding optical flow pose comparison result.

[0159] or,

[0160] The first change trend of the preprocessed multiple optical flow spatial motion matrices and the second change trend of the preprocessed multiple sensor spatial motion matrices are compared to obtain the corresponding optical flow pose comparison results.

[0161] The video stream identification device in one or more embodiments of this specification determines optical flow change information corresponding to a target video stream, wherein the theoretical acquisition time period of the target video stream is a preset time period and the theoretical acquisition device is a target user terminal; and determines the pose change information of the target user terminal, wherein the pose change information is determined based on sensor detection information of the target user terminal within the preset time period; and determines the credibility identification result of the target video stream based on the comparison information between the determined optical flow change information and the pose change information. By comparing and analyzing the optical flow change information determined based on the target video stream with the pose change information of the target user terminal, it identifies whether the target video stream is video stream information captured and uploaded in real time for the target object, thereby enabling rapid identification of malicious video stream attacks that replace real-time video streams with pre-stored non-real-time acquired video streams, so as to promptly intercept the non-real-time acquired video stream injected by the malicious video stream attack and improve the accuracy of subsequent business processing.

[0162] It should be noted that the embodiments of the video stream recognition device and the embodiments of the video stream recognition method in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding video stream recognition method mentioned above, and the repeated parts will not be described again.

[0163] Furthermore, corresponding to the above Figures 1 to 7 Based on the same technical concept, one or more embodiments of this specification also provide a video stream recognition device for performing the above-described video stream recognition method, such as... Figure 9 As shown.

[0164] Video stream recognition devices can vary significantly due to differences in configuration and performance. They may include one or more processors 901 and memory 902, with memory 902 storing one or more application programs or data. Memory 902 can be temporary or persistent storage. The application programs stored in memory 902 may include one or more modules (not shown in the figures), each module including a series of computer-executable instructions for the video stream recognition device. Furthermore, processor 901 may be configured to communicate with memory 902, executing the series of computer-executable instructions stored in memory 902 on the video stream recognition device. The video stream recognition device may also include one or more power supplies 903, one or more wired or wireless network interfaces 904, one or more input / output interfaces 905, one or more keyboards 906, etc.

[0165] In one specific embodiment, the video stream recognition device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for the video stream recognition device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0166] Determine the optical flow variation information corresponding to the target video stream, wherein the theoretical acquisition time period of the target video stream is a preset time period and the theoretical acquisition device is the target user terminal; and,

[0167] Determine the pose change information of the target user terminal, wherein the pose change information is determined based on the sensor detection information of the target user terminal within the preset time period;

[0168] Based on the comparison information between the optical flow change information and the pose change information, the credibility identification result for the target video stream is determined.

[0169] In one or more embodiments of this specification, by comparing and analyzing the optical flow change information determined based on the target video stream with the pose change information of the target user terminal, it is possible to identify whether the target video stream is video stream information captured and uploaded in real time for the target object. This enables rapid identification of malicious video stream attacks that use pre-stored non-real-time acquired video streams to replace real-time acquired video streams, so as to promptly intercept the non-real-time acquired video stream injected by the malicious video stream attack and improve the accuracy of subsequent business processing.

[0170] Optionally, when a computer-executable instruction is executed, the preset time period includes multiple specified time nodes;

[0171] The determination of optical flow change information corresponding to the target video stream includes:

[0172] Obtain the target video stream to be identified, wherein the target video stream includes: multiple object image frames containing the target object corresponding to the multiple specified time nodes respectively;

[0173] Based on the object image frames at the specified time points, determine the optical flow change information corresponding to the target video stream.

[0174] Optionally, when the computer-executable instructions are executed, determining the optical flow change information corresponding to the target video stream based on the object image frames at the plurality of specified time points includes:

[0175] Two adjacent object image frames corresponding to the specified time nodes are determined as an image frame combination;

[0176] For each image frame combination, the image optical flow information of the image frame combination is identified using a preset optical flow method to obtain the corresponding optical flow spatial motion matrix;

[0177] Based on the optical flow spatial motion matrix corresponding to each of the image frame combinations, the optical flow change information corresponding to the target video stream is determined.

[0178] Optionally, when the computer-executable instructions are executed, determining the optical flow change information corresponding to the target video stream based on the optical flow spatial motion matrix corresponding to each of the image frame combinations includes:

[0179] For each optical flow spatial motion matrix, a coordinate system transformation is performed on the optical flow spatial motion matrix in a preset spatial coordinate system to obtain the transformed optical flow spatial motion matrix.

[0180] Based on the transformed optical flow spatial motion matrices, the optical flow change information corresponding to the target video stream is determined.

[0181] Optionally, when a computer-executable instruction is executed, the preset time period includes multiple specified time nodes;

[0182] The determination of the pose change information of the target user terminal includes:

[0183] Obtain sensor detection information collected by at least one preset sensor of the target user terminal at each of the specified time nodes;

[0184] Based on the sensor detection information at the specified time points, the pose change information of the target user terminal is determined.

[0185] Optionally, when the computer-executable instructions are executed, determining the pose change information of the target user terminal based on the sensor detection information at the plurality of specified time points includes:

[0186] Based on the sensor detection information corresponding to each pair of adjacent specified time nodes, determine the corresponding sensor spatial motion matrix;

[0187] Based on the spatial motion matrices of each sensor, the pose change information of the target user terminal is determined.

[0188] Optionally, when the computer-executable instructions are executed, determining the pose change information of the target user terminal based on the spatial motion matrices of each of the sensors includes:

[0189] For each of the sensor spatial motion matrices, a coordinate system transformation is performed on the sensor spatial motion matrix under a preset spatial coordinate system to obtain the transformed sensor spatial motion matrix;

[0190] Based on the transformed sensor spatial motion matrices, the pose change information of the target user terminal is determined.

[0191] Optionally, when the computer-executable instructions are executed, determining the credibility identification result for the target video stream based on the comparison information between the optical flow change information and the pose change information includes:

[0192] The optical flow change information is compared with the pose change information to obtain the corresponding optical flow pose comparison result;

[0193] Determine whether the optical flow pose comparison result meets the preset change consistency condition;

[0194] If the judgment result is yes, then the target video stream is determined to be a reliable video stream information acquired in real time;

[0195] If the judgment result is negative, then the target video stream is determined to be an unreliable video stream information that was not acquired in real time.

[0196] Optionally, when the computer-executable instructions are executed, the optical flow change information includes: multiple optical flow spatial motion matrices within the preset time period, and the pose change information includes: multiple sensor spatial motion matrices within the preset time period;

[0197] The step of comparing the optical flow change information with the pose change information to obtain the corresponding optical flow pose comparison result includes:

[0198] The multiple optical flow spatial motion matrices are compared with the multiple sensor spatial motion matrices to obtain the corresponding optical flow pose comparison results.

[0199] Optionally, when the computer-executable instructions are executed, comparing the plurality of optical flow spatial motion matrices with the plurality of sensor spatial motion matrices to obtain corresponding optical flow pose comparison results includes:

[0200] The plurality of optical flow spatial motion matrices and the plurality of sensor spatial motion matrices are preprocessed to obtain preprocessed plurality of optical flow spatial motion matrices and preprocessed plurality of sensor spatial motion matrices, wherein the preprocessing includes at least one of: smoothing filtering, noise reduction processing, and comparison start point alignment;

[0201] The preprocessed optical flow spatial motion matrices and the preprocessed sensor spatial motion matrices are compared to obtain the corresponding optical flow pose comparison results.

[0202] Optionally, when the computer-executable instructions are executed, the step of comparing the preprocessed plurality of optical flow spatial motion matrices with the preprocessed plurality of sensor spatial motion matrices to obtain corresponding optical flow pose comparison results includes:

[0203] For each pair of adjacent specified time nodes within the preset time period, the preprocessed optical flow spatial motion matrix corresponding to each pair of adjacent specified time nodes is subtracted from the preprocessed sensor spatial motion matrix to obtain the corresponding optical flow pose comparison result.

[0204] or,

[0205] The first change trend of the preprocessed multiple optical flow spatial motion matrices and the second change trend of the preprocessed multiple sensor spatial motion matrices are compared to obtain the corresponding optical flow pose comparison results.

[0206] The video stream recognition device in one or more embodiments of this specification determines optical flow change information corresponding to a target video stream, wherein the theoretical acquisition time period of the target video stream is a preset time period and the theoretical acquisition device is a target user terminal; and determines the pose change information of the target user terminal, wherein the pose change information is determined based on sensor detection information of the target user terminal within the preset time period; and determines the credibility recognition result of the target video stream based on the comparison information between the determined optical flow change information and the pose change information. By comparing and analyzing the optical flow change information determined based on the target video stream with the pose change information of the target user terminal, it identifies whether the target video stream is video stream information captured and uploaded in real time for the target object, thereby enabling rapid identification of malicious video stream attacks that replace real-time video streams with pre-stored non-real-time acquired video streams, so as to promptly intercept the non-real-time acquired video stream injected by the malicious video stream attack and improve the accuracy of subsequent business processing.

[0207] It should be noted that the embodiments of the video stream recognition device and the embodiments of the video stream recognition method in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding video stream recognition method mentioned above, and the repeated parts will not be described again.

[0208] Furthermore, corresponding to the above Figures 1 to 7 Based on the same technical concept, one or more embodiments of this specification also provide a storage medium for storing computer-executable instructions. In one specific embodiment, the storage medium can be a USB flash drive, optical disc, hard disk, etc. When the computer-executable instructions stored in the storage medium are executed by a processor, they can achieve the following process:

[0209] Determine the optical flow variation information corresponding to the target video stream, wherein the theoretical acquisition time period of the target video stream is a preset time period and the theoretical acquisition device is the target user terminal; and,

[0210] Determine the pose change information of the target user terminal, wherein the pose change information is determined based on the sensor detection information of the target user terminal within the preset time period;

[0211] Based on the comparison information between the optical flow change information and the pose change information, the credibility identification result for the target video stream is determined.

[0212] In one or more embodiments of this specification, by comparing and analyzing the optical flow change information determined based on the target video stream with the pose change information of the target user terminal, it is possible to identify whether the target video stream is video stream information captured and uploaded in real time for the target object. This enables rapid identification of malicious video stream attacks that use pre-stored non-real-time acquired video streams to replace real-time acquired video streams, so as to promptly intercept the non-real-time acquired video stream injected by the malicious video stream attack and improve the accuracy of subsequent business processing.

[0213] Optionally, when the computer-executable instructions stored in the storage medium are executed by the processor, the preset time period includes multiple specified time nodes;

[0214] The determination of optical flow change information corresponding to the target video stream includes:

[0215] Obtain the target video stream to be identified, wherein the target video stream includes: multiple object image frames containing the target object corresponding to the multiple specified time nodes respectively;

[0216] Based on the object image frames at the specified time points, determine the optical flow change information corresponding to the target video stream.

[0217] Optionally, when the computer-executable instructions stored in the storage medium are executed by a processor, determining the optical flow change information corresponding to the target video stream based on the object image frames at the plurality of specified time points includes:

[0218] Two adjacent object image frames corresponding to the specified time nodes are determined as an image frame combination;

[0219] For each image frame combination, the image optical flow information of the image frame combination is identified using a preset optical flow method to obtain the corresponding optical flow spatial motion matrix;

[0220] Based on the optical flow spatial motion matrix corresponding to each of the image frame combinations, the optical flow change information corresponding to the target video stream is determined.

[0221] Optionally, when the computer-executable instructions stored in the storage medium are executed by a processor, determining the optical flow change information corresponding to the target video stream based on the optical flow spatial motion matrix corresponding to each of the image frame combinations includes:

[0222] For each optical flow spatial motion matrix, a coordinate system transformation is performed on the optical flow spatial motion matrix in a preset spatial coordinate system to obtain the transformed optical flow spatial motion matrix.

[0223] Based on the transformed optical flow spatial motion matrices, the optical flow change information corresponding to the target video stream is determined.

[0224] Optionally, when the computer-executable instructions stored in the storage medium are executed by the processor, the preset time period includes multiple specified time nodes;

[0225] The determination of the pose change information of the target user terminal includes:

[0226] Obtain sensor detection information collected by at least one preset sensor of the target user terminal at each of the specified time nodes;

[0227] Based on the sensor detection information at the specified time points, the pose change information of the target user terminal is determined.

[0228] Optionally, when the computer-executable instructions stored in the storage medium are executed by a processor, determining the pose change information of the target user terminal based on the sensor detection information at the plurality of specified time points includes:

[0229] Based on the sensor detection information corresponding to each pair of adjacent specified time nodes, determine the corresponding sensor spatial motion matrix;

[0230] Based on the spatial motion matrices of each sensor, the pose change information of the target user terminal is determined.

[0231] Optionally, when the computer-executable instructions stored in the storage medium are executed by a processor, determining the pose change information of the target user terminal based on the spatial motion matrices of each of the sensors includes:

[0232] For each of the sensor spatial motion matrices, a coordinate system transformation is performed on the sensor spatial motion matrix under a preset spatial coordinate system to obtain the transformed sensor spatial motion matrix;

[0233] Based on the transformed sensor spatial motion matrices, the pose change information of the target user terminal is determined.

[0234] Optionally, when the computer-executable instructions stored in the storage medium are executed by a processor, determining the credibility identification result for the target video stream based on the comparison information between the optical flow change information and the pose change information includes:

[0235] The optical flow change information is compared with the pose change information to obtain the corresponding optical flow pose comparison result;

[0236] Determine whether the optical flow pose comparison result meets the preset change consistency condition;

[0237] If the judgment result is yes, then the target video stream is determined to be a reliable video stream information acquired in real time;

[0238] If the judgment result is negative, then the target video stream is determined to be an unreliable video stream information that was not acquired in real time.

[0239] Optionally, when the computer-executable instructions stored in the storage medium are executed by the processor, the optical flow change information includes: multiple optical flow spatial motion matrices within the preset time period, and the pose change information includes: multiple sensor spatial motion matrices within the preset time period;

[0240] The step of comparing the optical flow change information with the pose change information to obtain the corresponding optical flow pose comparison result includes:

[0241] The multiple optical flow spatial motion matrices are compared with the multiple sensor spatial motion matrices to obtain the corresponding optical flow pose comparison results.

[0242] Optionally, when the computer-executable instructions stored in the storage medium are executed by the processor, the step of comparing the plurality of optical flow spatial motion matrices with the plurality of sensor spatial motion matrices to obtain the corresponding optical flow pose comparison results includes:

[0243] The plurality of optical flow spatial motion matrices and the plurality of sensor spatial motion matrices are preprocessed to obtain preprocessed plurality of optical flow spatial motion matrices and preprocessed plurality of sensor spatial motion matrices, wherein the preprocessing includes at least one of: smoothing filtering, noise reduction processing, and comparison start point alignment;

[0244] The preprocessed optical flow spatial motion matrices and the preprocessed sensor spatial motion matrices are compared to obtain the corresponding optical flow pose comparison results.

[0245] Optionally, when the computer-executable instructions stored in the storage medium are executed by the processor, the step of comparing the preprocessed plurality of optical flow spatial motion matrices with the preprocessed plurality of sensor spatial motion matrices to obtain the corresponding optical flow pose comparison result includes:

[0246] For each pair of adjacent specified time nodes within the preset time period, the preprocessed optical flow spatial motion matrix corresponding to each pair of adjacent specified time nodes is subtracted from the preprocessed sensor spatial motion matrix to obtain the corresponding optical flow pose comparison result.

[0247] or,

[0248] The first change trend of the preprocessed multiple optical flow spatial motion matrices and the second change trend of the preprocessed multiple sensor spatial motion matrices are compared to obtain the corresponding optical flow pose comparison results.

[0249] When the computer-executable instructions stored in the storage medium in one or more embodiments of this specification are executed by a processor, they determine the optical flow change information corresponding to the target video stream, wherein the theoretical acquisition time period of the target video stream is a preset time period and the theoretical acquisition device is the target user terminal; and determine the pose change information of the target user terminal, wherein the pose change information is determined based on the sensor detection information of the target user terminal within the preset time period; and determine the credibility identification result of the target video stream based on the comparison information between the determined optical flow change information and the pose change information. By comparing and analyzing the optical flow change information determined based on the target video stream with the pose change information of the target user terminal, it is possible to identify whether the target video stream is video stream information captured and uploaded in real time for the target object, thereby enabling rapid identification of malicious video stream attacks that use pre-stored non-real-time acquired video streams to replace real-time acquired video streams, so as to promptly intercept the non-real-time acquired video stream injected by the malicious video stream attack and improve the accuracy of subsequent business processing.

[0250] It should be noted that the embodiments concerning storage media in this specification and the embodiments concerning video stream recognition methods in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can be referred to the implementation of the corresponding video stream recognition method described above, and the repeated parts will not be described again.

[0251] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0252] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using a hardware physical module. For example, a Programmable Logic Device (PLD) (e.g., a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program a digital system themselves to "integrate" it onto a PLD, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HD Cal, JHDL (Java Hardware Description Language), Lava, Lola, My HDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0253] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0254] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0255] For ease of description, the above devices are described in terms of function, divided into various units. Of course, when implementing one or more of these specifications, the functions of each unit can be implemented in one or more software and / or hardware.

[0256] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more of this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0257] This specification, one or more, is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to one or more embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0258] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0259] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0260] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0261] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0262] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0263] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0264] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more of this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0265] This specification, one or more, can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification, one or more, can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0266] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0267] The above description is merely an embodiment of one or more embodiments of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A video stream identification method, comprising: determining optical flow change information corresponding to a target video stream, wherein the optical flow change information is obtained based on optical flow direction analysis of a plurality of object image frames in the target video stream; and determining pose change information of a target user terminal, wherein the pose change information is determined based on sensor spatial motion information corresponding to sensor detection information of the target user terminal at a plurality of specified time nodes within a preset time period; comparing the optical flow change information and the pose change information to obtain a corresponding optical flow-pose comparison result; judging whether the optical flow-pose comparison result satisfies a preset change consistency condition; if the judgment result is yes, determining that the target video stream is trusted video stream information collected in real time by the target user terminal within the preset time period; if the judgment result is no, determining that the target video stream is untrusted video stream information collected non-in real time by the target user terminal within the preset time period.

2. The method of claim 1, wherein, The determination of the optical flow change information corresponding to the target video stream comprises: obtaining a target video stream to be identified, wherein the target video stream includes a plurality of object image frames containing target objects corresponding to the plurality of specified time nodes respectively; determining the optical flow change information corresponding to the target video stream according to the object image frames at the plurality of specified time nodes.

3. The method of claim 2, wherein, The determination of the optical flow change information corresponding to the target video stream according to the object image frames at the plurality of specified time nodes comprises: determining two object image frames corresponding to two adjacent specified time nodes as an image frame combination; for each image frame combination, performing image optical flow information identification on the image frame combination using a preset optical flow method to obtain a corresponding optical flow spatial motion matrix; determining the optical flow change information corresponding to the target video stream according to the optical flow spatial motion matrix corresponding to each image frame combination.

4. The method of claim 3, wherein, The determination of the optical flow change information corresponding to the target video stream according to the optical flow spatial motion matrix corresponding to each image frame combination comprises: for each optical flow spatial motion matrix, performing coordinate system transformation on the optical flow spatial motion matrix in a preset spatial coordinate system to obtain a transformed optical flow spatial motion matrix; determining the optical flow change information corresponding to the target video stream according to the transformed optical flow spatial motion matrix.

5. The method of claim 1, wherein, The determination of the pose change information of the target user terminal comprises: obtaining sensor detection information collected by at least one preset sensor of the target user terminal at each specified time node; determining the pose change information of the target user terminal according to the sensor detection information at the plurality of specified time nodes.

6. The method of claim 5, wherein, The determination of the pose change information of the target user terminal according to the sensor detection information at the plurality of specified time nodes comprises: determining a corresponding sensor spatial motion matrix according to the sensor detection information corresponding to two adjacent specified time nodes; determining the pose change information of the target user terminal according to each sensor spatial motion matrix.

7. The method of claim 6, wherein, The pose change information of the target user terminal is determined according to each sensor space motion matrix. For each sensor space motion matrix, a coordinate system transformation is performed on the sensor space motion matrix in a preset space coordinate system to obtain a transformed sensor space motion matrix. The pose change information of the target user terminal is determined according to each transformed sensor space motion matrix.

8. The method of claim 1, wherein, The optical flow change information includes a plurality of optical flow space motion matrices in the preset time period, and the pose change information includes a plurality of sensor space motion matrices in the preset time period. The optical flow change information and the pose change information are compared to obtain a corresponding optical flow pose comparison result. The plurality of optical flow space motion matrices and the plurality of sensor space motion matrices are compared to obtain a corresponding optical flow pose comparison result.

9. The method of claim 8, wherein, The plurality of optical flow space motion matrices and the plurality of sensor space motion matrices are compared to obtain a corresponding optical flow pose comparison result. The plurality of optical flow space motion matrices and the plurality of sensor space motion matrices are preprocessed to obtain preprocessed plurality of optical flow space motion matrices and preprocessed plurality of sensor space motion matrices, wherein the preprocessing includes at least one of smoothing filtering, de-noising processing, and alignment of comparison starting points. The preprocessed plurality of optical flow space motion matrices and the preprocessed plurality of sensor space motion matrices are compared to obtain a corresponding optical flow pose comparison result.

10. The method of claim 9, wherein, The preprocessed plurality of optical flow space motion matrices and the preprocessed plurality of sensor space motion matrices are compared to obtain a corresponding optical flow pose comparison result. For each pair of adjacent designated time nodes in the preset time period, the preprocessed optical flow space motion matrix corresponding to the pair of adjacent designated time nodes is subtracted from the preprocessed sensor space motion matrix to obtain a corresponding optical flow pose comparison result. Alternatively, The first change trend of the preprocessed plurality of optical flow space motion matrices is compared with the second change trend of the preprocessed plurality of sensor space motion matrices to obtain a corresponding optical flow pose comparison result.

11. A video stream recognition device, comprising: an optical flow change information determination module that determines optical flow change information corresponding to a target video stream, wherein the optical flow change information is obtained based on optical flow direction analysis of a plurality of object image frames in the target video stream; and a pose change information determination module that determines pose change information of a target user terminal, wherein the pose change information is determined based on sensor space motion information corresponding to sensor detection information of the target user terminal at a plurality of designated time nodes in a preset time period. The video stream credibility identification module compares the optical flow change information with the pose change information to obtain a corresponding optical flow-pose comparison result, judges whether the optical flow-pose comparison result satisfies a preset change consistency condition, and if the result of the judgment is yes, determines that the target video stream is credible video stream information that is collected in real time by the target user terminal within the preset time period, and if the result of the judgment is no, determines that the target video stream is uncredible video stream information that is not collected in real time by the target user terminal within the preset time period.

12. The apparatus of claim 11, wherein, The optical flow change information determination module comprises: acquiring a target video stream to be identified, wherein the target video stream comprises a plurality of object image frames corresponding to a plurality of specified time nodes respectively and containing a target object; determining optical flow change information corresponding to the target video stream according to the object image frames under the plurality of specified time nodes.

13. The apparatus of claim 12, wherein, The optical flow change information determination module comprises: determining two object image frames corresponding to two adjacent specified time nodes as an image frame combination; for each image frame combination, performing image optical flow information identification on the image frame combination by using a preset optical flow method to obtain a corresponding optical flow space motion matrix; determining optical flow change information corresponding to the target video stream according to the optical flow space motion matrices corresponding to the image frame combinations respectively.

14. The apparatus of claim 13, wherein, The optical flow change information determination module comprises: for each optical flow space motion matrix, performing coordinate system transformation on the optical flow space motion matrix in a preset space coordinate system to obtain a transformed optical flow space motion matrix; determining optical flow change information corresponding to the target video stream according to the transformed optical flow space motion matrices.

15. The apparatus of claim 11, wherein, The pose change information determination module comprises: acquiring sensor detection information collected by at least one preset sensor of the target user terminal at each specified time node; determining pose change information of the target user terminal according to the sensor detection information under the plurality of specified time nodes.

16. The apparatus of claim 15, wherein, The pose change information determination module comprises: determining a corresponding sensor space motion matrix according to the sensor detection information corresponding to two adjacent specified time nodes; determining pose change information of the target user terminal according to the sensor space motion matrices.

17. The apparatus of claim 16, wherein, The pose change information determination module comprises: for each sensor space motion matrix, performing coordinate system transformation on the sensor space motion matrix in a preset space coordinate system to obtain a transformed sensor space motion matrix; determining pose change information of the target user terminal according to the transformed sensor space motion matrices.

18. The apparatus of claim 11, wherein, The optical flow change information comprises a plurality of optical flow space motion matrices within the preset time period, and the pose change information comprises a plurality of sensor space motion matrices within the preset time period; The video stream credibility identification module comprises: comparing the plurality of optical flow space motion matrices with the plurality of sensor space motion matrices to obtain a corresponding optical flow-pose comparison result.

19. The apparatus of claim 18, wherein, The video stream credibility identification module comprises: The multiple optical flow space motion matrices and the multiple sensor space motion matrices are preprocessed to obtain preprocessed multiple optical flow space motion matrices and preprocessed multiple sensor space motion matrices, wherein the preprocessing includes at least one of smoothing filtering, de-noising processing, and alignment of starting points. The preprocessed multiple optical flow space motion matrices and the preprocessed multiple sensor space motion matrices are compared to obtain corresponding optical flow pose comparison results.

20. The apparatus of claim 19, wherein, The video stream credibility identification module: For every two adjacent designated time nodes in the preset time period, the preprocessed optical flow space motion matrix corresponding to the two adjacent designated time nodes is subtracted from the preprocessed sensor space motion matrix to obtain a corresponding optical flow pose comparison result. Or, The first change trend of the preprocessed multiple optical flow space motion matrices is compared with the second change trend of the preprocessed multiple sensor space motion matrices to obtain a corresponding optical flow pose comparison result.

21. A video stream identification device, comprising: a processor; and a memory arranged to store computer executable instructions that, when executed, cause the processor to: determine optical flow change information corresponding to a target video stream, wherein the optical flow change information is obtained based on optical flow direction analysis of multiple object image frames in the target video stream; and determine pose change information of a target user terminal, wherein the pose change information is determined based on sensor space motion information corresponding to sensor detection information of the target user terminal at multiple designated time nodes in a preset time period; compare the optical flow change information and the pose change information to obtain a corresponding optical flow pose comparison result; determine whether the optical flow pose comparison result satisfies a preset change consistency condition; if the determination result is yes, determine that the target video stream is credible video stream information that is collected in real time by the target user terminal in the preset time period; if the determination result is no, determine that the target video stream is uncredible video stream information that is not collected in real time by the target user terminal in the preset time period.

22. A storage medium for storing computer executable instructions that, when executed by a processor, implement the following method: determining light flow change information corresponding to the target video stream, wherein the optical flow change information is obtained based on optical flow direction analysis of multiple object image frames in the target video stream; and determine pose change information of a target user terminal, wherein the pose change information is determined based on sensor space motion information corresponding to sensor detection information of the target user terminal at multiple designated time nodes in a preset time period; compare the optical flow change information and the pose change information to obtain a corresponding optical flow pose comparison result; determine whether the optical flow pose comparison result satisfies a preset change consistency condition; if the determination result is yes, determine that the target video stream is credible video stream information that is collected in real time by the target user terminal in the preset time period; ​ If the determination result is no, it is determined that the target video stream is untrusted video stream information that is not collected in real time by the target user terminal within the preset time period.

Images