A method and system for identifying and analyzing audience behavior to prevent voyeuristic filming of movies

Abstract

The present invention relates to the field of computer vision and behavior recognition technology, and specifically discloses a method and system for audience behavior recognition and analysis to prevent movie voyeurism. The method and system collect multi-view video data, process and extract features of the video data according to the characteristics of different viewpoints, input the extracted feature vectors into a pre-trained target recognition model to identify the corresponding target, and implement early warning for suspected voyeurism according to the target recognition results. The present invention establishes three artificial intelligence target recognition models according to the characteristics of videos with different viewpoints, recognizes and analyzes videos from two aspects: shooting equipment detection and audience posture recognition, thereby improving the intelligence, convenience and comprehensiveness of theater voyeurism recognition.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and behavior recognition, and in particular relates to an audience behavior recognition and analysis method and system for preventing voyeuristic filming of movies. Background Art

[0002] Movie voyeurism refers to the unauthorized use of video equipment to illegally record movies in a cinema. This behavior not only infringes on the copyright of filmmakers, but may also cause huge economic losses to the film industry. Existing anti-voyeurism technologies mainly rely on manual monitoring, simple camera monitoring, and digital watermarking technologies, which have problems such as low efficiency and high false alarm rate. Therefore, it is particularly important to develop an efficient and accurate audience behavior recognition and analysis method and system.

[0003] With the continuous development of artificial intelligence and big data technology, the recognition and analysis of target behavior based on computer vision has also been continuously improved. How to use artificial intelligence and big data technology to collect multi-source audience behavior data, realize multi-angle comprehensive recognition and analysis, and obtain more real-time and accurate audience behavior recognition results is the technical problem to be solved by the present invention. Therefore, the present invention aims to provide a method and system for audience behavior recognition and analysis to prevent movie voyeurism. Summary of the invention

[0004] The purpose of the present invention is to overcome the above-mentioned problems existing in the prior art, and to provide a method and system for identifying and analyzing audience behavior to prevent voyeurism of movies. The method and system realize the collection of multi-view video data of the audience during the movie screening, the classification and processing of the multi-view video data and feature extraction, and the construction of a target recognition model. The audience behavior is identified and analyzed through the target recognition model, and an early warning is issued when suspected voyeurism is found. The method and system use artificial intelligence technology to efficiently identify and analyze the audience behavior during the movie screening, and lock in the audience who may have voyeurism in advance. On this basis, manual verification and intervention are carried out, which reduces the workload of manual inspections, improves the efficiency of copyright protection, and is conducive to the further standardization of movie-watching behavior.

[0005] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

[0006] A method for identifying and analyzing audience behavior to prevent voyeuristic filming of movies comprises the following steps:

[0007] Camera setup: 3 video camera positions are pre-set up, with a camera placed on each position. The first camera position is set above the movie screen to collect the overall front-view video data of all moviegoers; the second camera position is set behind the last row of seats in the cinema to collect the overall rear-view video data of all moviegoers; the third camera position is set on the side of the cinema to collect the overall side-view video data of all moviegoers;

[0008] Collecting multi-view video data: during the movie screening process, collecting first video data, the first video data is the overall front-view video data of all the audience members collected by the first camera, and the overall front-view video data records the front area of ​​the head, the front area of ​​the torso, and the front area of ​​the upper limbs of all the audience members during the movie watching process; collecting second video data, the second video data is the overall rear-view video data of all the audience members collected by the second camera, and the overall rear-view video data records the back area of ​​the head, the back area of ​​the torso, and the back area of ​​the upper limbs of all the audience members during the movie watching process; collecting third video data, the third video data is the overall side-view video data of all the audience members collected by the third camera, and the overall side-view video data records the side area of ​​the head, the side area of ​​the torso, and the side area of ​​the upper limbs of all the audience members during the movie watching process;

[0009] Training target recognition model: extracting a shooting device feature vector set from training image set A, inputting it into a pre-built artificial neural network A for training to obtain a shooting device recognition model, wherein the artificial neural network A is based on a multi-layer perceptron, and the model output is a probability value vector of whether the image contains or does not contain a shooting device; extracting a screen recording image feature vector set from training image set B, inputting it into a pre-built artificial neural network B for training to obtain a screen recording image recognition model; wherein the artificial neural network B is based on a multi-layer perceptron, and the model output is a probability value vector of whether the image contains or does not contain a screen recording image; extracting a viewer posture feature atlas from training image set C, inputting it into a pre-built artificial neural network C for training to obtain a holding behavior recognition model, wherein the artificial neural network C is based on CNN, and the model output is a probability value vector of whether holding behavior is detected or not detected in the image;

[0010] Processing multi-view video data: extracting key frames from the first, second and third video data at the same time interval, and obtaining the first, second and third video key frame time sequence image data sets keyframe_i=((I i1 ,t1),(I i2 ,t2),...,(I ij ,t j ), where i represents the video sequence number, i takes the value of 1, 2 or 3, j represents the number of key frame images, t jrepresents the timestamp corresponding to the jth key frame image; the first and second video key frame time sequence image sets are processed to obtain a real-time shooting device feature vector set Vob including timestamp information j ,t j )); Process the second video key frame time sequence image set to obtain a real-time screen recording image feature vector set Vima=((vima1,t1),(vima2,t2),...,(vima j ,t j )); Process the first, second and third video key frame time sequence image sets to obtain a real-time audience posture feature atlas Vpos=((vpos1,t1),(vpos2,t2),...,(vpos j ,t j ));

[0011] Movie viewing video data target recognition: input the Vob into the trained shooting device recognition model to obtain the shooting device recognition result A; input the Vima into the trained screen recording image recognition model to obtain the screen recording image recognition result B; input the Vpos into the trained holding behavior recognition model to obtain the holding behavior recognition result C;

[0012] Data analysis and warning: Comprehensively analyze the recognition results A, B and C, determine the warning information and notify the staff according to the corresponding relationship between the preset recognition results and the warning level.

[0013] Furthermore, the real-time shooting device feature vector set including timestamp information is obtained, specifically including: the image data set keyframe_1 is processed as follows: first, the human body area in the image is located by using the target detection algorithm based on Faster R-CNN, the human hand area is located by using the skeleton key point detection algorithm in the human body area, and the segmentation area is set by using the hand area to obtain the segmented hand area image; the segmented hand area image is mapped to the HSV color space, and the pixel mean and standard deviation of the three color channels of H, S, and V are calculated respectively to obtain 6 color feature values, which are spliced ​​to form a color feature vector; contour detection is performed on the segmented hand area image, and the rectangular area in the segmented hand area image is located as the suspected shooting device area through the contour detection result, and the texture feature of the suspected shooting device area is extracted to obtain a texture feature vector; contour detection is performed on the inside of the suspected shooting device area, and the area value and perimeter value of the contour inside the area are calculated to form a shape feature vector; the image data set keyframe_2 is processed as follows: first, the image data set keyframe_2 is processed as follows: first, the image data set keyframe_3 is processed as follows: The target detection algorithm of R-CNN locates the human body area in the image, segments the human body area to obtain a segmented human body area image, converts the human body area image into a grayscale image, calculates the pixel mean and variance of the grayscale image, and forms a brightness feature vector; splices and fuses the color feature vector, texture feature vector, shape feature vector and brightness feature vector to obtain a real-time shooting device feature vector; and summarizes the timestamp and real-time shooting device feature vector corresponding to each key frame image to form the real-time shooting device feature vector set including the timestamp information.

[0014] Furthermore, the real-time screen recording image feature vector set including timestamp information is obtained, specifically including: processing the image data set keyframe_2 as follows: identifying the movie screen area in the image, dividing the image into a movie screen area image and other area images, converting the movie screen area image into a grayscale image and calculating the brightness mean b1; converting the other area images into grayscale images, binarizing the other area images using the brightness mean b1 as a threshold, obtaining an area with brightness greater than or equal to the brightness mean b1 as a suspected shooting device screen area, extracting the brightness feature vector, color feature vector and texture feature vector of the suspected shooting device screen area, and merging and splicing them into a real-time screen recording image feature vector; aggregating the timestamp and real-time screen recording image feature vector corresponding to each key frame image to form the real-time screen recording image feature vector set including timestamp information.

[0015] Furthermore, the real-time audience posture feature atlas including timestamp information is obtained, specifically including: performing the following processing on the image data sets keyframe_1, keyframe_2 and keyframe_3 respectively: firstly, using the target detection algorithm based on Faster R-CNN to locate the human body area in the image and perform segmentation to obtain the segmented human body area image, performing key point detection on the segmented human body area image to obtain the key points of the head, neck, shoulder, elbow and wrist and their corresponding coordinates, connecting adjacent key points to obtain five upper limb vectors representing the trunk, left upper arm, left forearm, right upper arm and right forearm respectively, using the five upper limb vectors to calculate the angle value between adjacent vectors to obtain the joint angle value vector; respectively calculating the three joint angle value vectors of the key frame images in keyframe_1, keyframe_2 and keyframe_3 at the same timestamp, splicing the three joint angle value vectors to obtain the real-time audience posture feature map; and summarizing the timestamp corresponding to each key frame image and the real-time audience posture feature map to form the real-time audience posture feature atlas including the timestamp information.

[0016] Furthermore, the target recognition of the movie viewing video data specifically includes: if among all the recognition results corresponding to the Vob, the proportion of the number of results identifying that include shooting devices exceeds a preset ratio threshold, then the shooting device recognition result A is considered to be the existence of a shooting device; if among all the recognition results corresponding to the Vima, the proportion of the number of results identifying that include screen recording images exceeds a preset ratio threshold, then the screen recording image recognition result B is considered to be the existence of a screen recording image; if among all the recognition results corresponding to the Vpos, the proportion of the number of results detecting holding behavior exceeds a preset ratio threshold, then the holding behavior recognition result C is considered to be the existence of a holding behavior.

[0017] The present invention also provides the following technical solutions:

[0018] A system for identifying and analyzing audience behavior to prevent voyeuristic filming of movies, comprising the following functional modules:

[0019] Shooting camera setup module: 3 video shooting cameras are pre-set up, with a shooting device placed on each camera. The first camera is set above the movie screen to collect the overall front-view video data of all moviegoers; the second camera is set behind the last row of seats in the cinema to collect the overall rear-view video data of all moviegoers; the third camera is set on the side of the cinema to collect the overall side-view video data of all moviegoers;

[0020] Multi-view video data acquisition module: during the movie screening process, first video data is collected, the first video data is the overall front-view video data of all the audience members collected by the first camera, and the overall front-view video data records the front area of ​​the head, the front area of ​​the torso and the front area of ​​the upper limbs of all the audience members during the movie watching process; second video data is collected, the second video data is the overall rear-view video data of all the audience members collected by the second camera, and the overall rear-view video data records the back area of ​​the head, the back area of ​​the torso and the back area of ​​the upper limbs of all the audience members during the movie watching process; third video data is collected, the third video data is the overall side-view video data of all the audience members collected by the third camera, and the overall side-view video data records the side area of ​​the head, the side area of ​​the torso and the side area of ​​the upper limbs of all the audience members during the movie watching process;

[0021] Target recognition model training module: extract the shooting device feature vector set of the training image set A, input it into the pre-built artificial neural network A for training to obtain the shooting device recognition model, the artificial neural network A is based on a multi-layer perceptron, and the model output is the probability value vector of whether the image contains or does not contain the shooting device; extract the screen recording image feature vector set of the training image set B, input it into the pre-built artificial neural network B for training to obtain the screen recording image recognition model; the artificial neural network B is based on a multi-layer perceptron, and the model output is the probability value vector of whether the image contains or does not contain the screen recording image; extract the audience posture feature atlas set of the training image set C, input it into the pre-built artificial neural network C for training to obtain the holding behavior recognition model, the artificial neural network C is based on CNN, and the model output is the probability value vector of whether the holding behavior is detected or not detected in the image;

[0022] Multi-view video data processing module: extract key frames from the first, second and third video data at the same time interval to obtain the first, second and third video key frame time sequence image data sets keyframe_i=((I i1 ,t1),(I i2 ,t2),...,(I ij ,t j ), where i represents the video sequence number, i takes the value of 1, 2 or 3, j represents the number of key frame images, t j represents the timestamp corresponding to the jth key frame image; the first and second video key frame time sequence image sets are processed to obtain a real-time shooting device feature vector set Vob including timestamp information j ,t j)); Process the second video key frame time sequence image set to obtain a real-time screen recording image feature vector set Vima=((vima1,t1),(vima2,t2),...,(vima j ,t j )); Process the first, second and third video key frame time sequence image sets to obtain a real-time audience posture feature atlas Vpos=((vpos1,t1),(vpos2,t2),...,(vpos j ,t j ));

[0023] Movie viewing video data target recognition module: input the Vob into the trained shooting device recognition model to obtain the shooting device recognition result A; input the Vima into the trained screen recording image recognition model to obtain the screen recording image recognition result B; input the Vpos into the trained holding behavior recognition model to obtain the holding behavior recognition result C;

[0024] Data analysis and warning module: comprehensively analyze the recognition results A, B and C, determine the warning information and notify the staff according to the corresponding relationship between the preset recognition results and the warning level.

[0025] Furthermore, the multi-view video data processing module specifically includes: performing the following processing on the image data set keyframe_1: first, using the target detection algorithm based on Faster R-CNN to locate the human body area in the image, using the skeleton key point detection algorithm to locate the human hand area in the human body area, using the hand area to set the segmentation area, and obtaining the segmented hand area image; mapping the segmented hand area image to the HSV color space, respectively calculating the pixel mean and standard deviation of the three color channels of H, S, and V, obtaining 6 color feature values, and splicing them to form a color feature vector; performing contour detection on the segmented hand area image, locating the rectangular area in the segmented hand area image as the suspected shooting device area through the contour detection result, and extracting the texture feature of the suspected shooting device area to obtain a texture feature vector; performing contour detection on the inside of the suspected shooting device area, calculating the area value and perimeter value of the contour inside the area, and forming a shape feature vector; performing the following processing on the image data set keyframe_2: first, using the target detection algorithm based on Faster R-CNN to locate the human body area in the image, and calculating the area value and perimeter value of the contour inside the area, and forming a shape feature vector. The target detection algorithm of R-CNN locates the human body area in the image, segments the human body area to obtain a segmented human body area image, converts the human body area image into a grayscale image, calculates the pixel mean and variance of the grayscale image, and forms a brightness feature vector; splices and fuses the color feature vector, texture feature vector, shape feature vector and brightness feature vector to obtain a real-time shooting device feature vector; and summarizes the timestamp and real-time shooting device feature vector corresponding to each key frame image to form the real-time shooting device feature vector set including the timestamp information.

[0026] Furthermore, the multi-view video data processing module specifically includes: processing the image data set keyframe_2 as follows: identifying the movie screen area in the image, dividing the image into a movie screen area image and other area images, converting the movie screen area image into a grayscale image and calculating the brightness mean b1; converting the other area images into a grayscale image, using the brightness mean b1 as a threshold to binarize the other area images, and obtaining an area with a brightness greater than or equal to the brightness mean b1 as a suspected shooting device screen area, extracting the brightness feature vector, color feature vector and texture feature vector of the suspected shooting device screen area, and fusing and splicing them into a real-time screen recording image feature vector; summarizing the timestamp and real-time screen recording image feature vector corresponding to each key frame image to form the real-time screen recording image feature vector set including timestamp information.

[0027] Furthermore, the multi-view video data processing module specifically includes: performing the following processing on the image data sets keyframe_1, keyframe_2 and keyframe_3 respectively: first, using the target detection algorithm based on Faster R-CNN to locate the human body area in the image and perform segmentation to obtain the segmented human body area image, performing key point detection on the segmented human body area image to obtain the key points of the head, neck, shoulder, elbow and wrist and their corresponding coordinates, connecting adjacent key points to obtain five upper limb vectors representing the torso, left upper arm, left forearm, right upper arm and right forearm respectively, using the five upper limb vectors to calculate the angle value between adjacent vectors to obtain the joint angle value vector; respectively calculating the three joint angle value vectors of the key frame images in keyframe_1, keyframe_2 and keyframe_3 at the same timestamp, splicing the three joint angle value vectors to obtain the real-time audience posture feature map; summarizing the timestamp corresponding to each key frame image and the real-time audience posture feature map to form the real-time audience posture feature map set including the timestamp information.

[0028] Furthermore, the data analysis and early warning module specifically includes: if among all the recognition results corresponding to the Vob, the proportion of the number of results identifying that include shooting devices exceeds a preset ratio threshold, then the shooting device recognition result A is considered to be the existence of a shooting device; if among all the recognition results corresponding to the Vima, the proportion of the number of results identifying that include screen recording images exceeds a preset ratio threshold, then the screen recording image recognition result B is considered to be the existence of a screen recording image; if among all the recognition results corresponding to the Vpos, the proportion of the number of results detecting holding behavior exceeds a preset ratio threshold, then the holding behavior recognition result C is considered to be the existence of a holding behavior.

[0029] By adopting the above technical solution, compared with the prior art, the beneficial effects of the present invention are:

[0030] The present invention utilizes different camera positions to monitor and shoot the audience seats during the movie screening process, thereby realizing the collection of multi-viewing angle video data.

[0031] When processing multi-view video data, the present invention performs different processing on the multi-view data according to the characteristics of the target to be identified, performs a segmentation operation on the image, and concentrates the feature extraction area on the area of ​​the object to be identified, thereby reducing the amount of data processed and improving the effectiveness of feature extraction.

[0032] When identifying candid photography, the present invention establishes three different target recognition models from the two aspects of shooting equipment recognition and audience posture detection, and performs targeted feature extraction operations on multi-view video data. Video data of the same target from different perspectives can complement each other, enrich feature details, and improve the accuracy of target recognition.

[0033] The present invention improves the intelligence and convenience of identifying candid photography behavior and further standardizes movie-watching behavior by collecting, processing, identifying and analyzing multi-viewing video data. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 This is a flow chart of a method for identifying and analyzing audience behavior to prevent voyeuristic filming of movies according to the present invention;

[0036] Figure 2 A flowchart of the camera setup and acquisition of multi-view video data of the present invention;

[0037] Figure 3 A flow chart of the target recognition model for training of the present invention;

[0038] Figure 4 A flowchart of processing multi-view video data according to the present invention;

[0039] Figure 5 A flow chart of target identification, data analysis and early warning of the present invention;

[0040] Figure 6 This is a functional module diagram of a system for identifying and analyzing audience behavior for preventing voyeuristic filming of movies according to the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Example 1

[0043] like Figure 1 As shown, this embodiment provides a method for identifying and analyzing audience behavior to prevent voyeuristic filming of movies, comprising the following steps:

[0044] Camera setup: 3 video camera positions are pre-set up, with a camera placed on each position. The first camera position is set above the movie screen to collect the overall front-view video data of all moviegoers; the second camera position is set behind the last row of seats in the cinema to collect the overall rear-view video data of all moviegoers; the third camera position is set on the side of the cinema to collect the overall side-view video data of all moviegoers;

[0045] Collecting multi-view video data: during the movie screening process, collecting first video data, the first video data is the overall front-view video data of all the audience members collected by the first camera, and the overall front-view video data records the front area of ​​the head, the front area of ​​the torso, and the front area of ​​the upper limbs of all the audience members during the movie watching process; collecting second video data, the second video data is the overall rear-view video data of all the audience members collected by the second camera, and the overall rear-view video data records the back area of ​​the head, the back area of ​​the torso, and the back area of ​​the upper limbs of all the audience members during the movie watching process; collecting third video data, the third video data is the overall side-view video data of all the audience members collected by the third camera, and the overall side-view video data records the side area of ​​the head, the side area of ​​the torso, and the side area of ​​the upper limbs of all the audience members during the movie watching process;

[0046] Training target recognition model: extracting a shooting device feature vector set from training image set A, inputting it into a pre-built artificial neural network A for training to obtain a shooting device recognition model, wherein the artificial neural network A is based on a multi-layer perceptron, and the model output is a probability value vector of whether the image contains or does not contain a shooting device; extracting a screen recording image feature vector set from training image set B, inputting it into a pre-built artificial neural network B for training to obtain a screen recording image recognition model; wherein the artificial neural network B is based on a multi-layer perceptron, and the model output is a probability value vector of whether the image contains or does not contain a screen recording image; extracting a viewer posture feature atlas from training image set C, inputting it into a pre-built artificial neural network C for training to obtain a holding behavior recognition model, wherein the artificial neural network C is based on CNN, and the model output is a probability value vector of whether holding behavior is detected or not detected in the image;

[0047] Processing multi-view video data: extracting key frames from the first, second and third video data at the same time interval, and obtaining the first, second and third video key frame time sequence image data sets keyframe_i=((I i1 ,t1),(Ii2 ,t2),...,(I ij ,t j ), where i represents the video sequence number, i takes the value of 1, 2 or 3, j represents the number of key frame images, t j represents the timestamp corresponding to the jth key frame image; the first and second video key frame time sequence image sets are processed to obtain a real-time shooting device feature vector set Vob including timestamp information j ,t j )); Process the second video key frame time sequence image set to obtain a real-time screen recording image feature vector set Vima=((vima1,t1),(vima2,t2),...,(vima j ,t j )); Process the first, second and third video key frame time sequence image sets to obtain a real-time audience posture feature atlas Vpos=((vpos1,t1),(vpos2,t2),...,(vpos j ,t j ));

[0048] Movie viewing video data target recognition: input the Vob into the trained shooting device recognition model to obtain the shooting device recognition result A; input the Vima into the trained screen recording image recognition model to obtain the screen recording image recognition result B; input the Vpos into the trained holding behavior recognition model to obtain the holding behavior recognition result C;

[0049] Data analysis and warning: Comprehensively analyze the recognition results A, B and C, determine the warning information and notify the staff according to the corresponding relationship between the preset recognition results and the warning level.

[0050] Example 2

[0051] like Figure 2 As shown, this embodiment provides a method for identifying and analyzing audience behavior to prevent voyeuristic filming of movies, wherein when the camera is set up:

[0052] The present invention assumes that the cinema is a regular rectangular structure with four walls in front, back, left and right. The present invention has a total of 3 camera positions, wherein the first camera position is set above the movie screen to collect the audience's behavior during the viewing process from the front perspective, the second camera position is set behind the last row of the auditorium in the cinema to collect the audience's behavior during the viewing process from the back perspective; the third camera position is set on the side of the cinema, either on the left or right side, so as to collect the audience's behavior during the viewing process from the side perspective. All camera positions can be fixed on the wall of the cinema or fixed by a bracket. The specific installation position and angle adjustment are determined according to the actual situation, and the present invention does not make specific restrictions on this. It is worth noting that in order to ensure the privacy of the audience watching the movie, first of all, the monitoring area will give a clear prompt to remind the audience that they have entered the monitoring area; secondly, the monitoring video is only used to identify whether there is any voyeurism during the audience's viewing process, and after a preset time, the monitoring video will be deleted.

[0053] When collecting multi-view video data: three camera positions are set up mainly to solve the problems of target occlusion and misidentification under a single view. For multi-view video data of the same target, video data from different viewpoints can reflect richer and more comprehensive features of the target, which is conducive to improving the accuracy of subsequent target recognition.

[0054] The method of the present invention can effectively identify the following "secret photography" behaviors: using a mobile phone, camera or other shooting equipment to shoot, and the mobile phone or camera is held in hand or fixed on a bracket to shoot directly at the movie screen. Since the purpose of the present invention is to identify the audience's "secret photography" behavior, considering the specific scene characteristics during the movie watching process, the audience all sits and watches the screen, and the collected video data mainly records the audience's head area, torso area and upper limb area. The multi-view video data can record the shooting equipment and the audience's posture in an all-round way, reducing the probability of misjudgment.

[0055] When collecting multi-view video data, in order to ensure the validity of the sample data, the shooting equipment of different camera positions needs to be adjusted in advance to ensure that the overall front-view video data collected by the first camera position can record the front area of ​​the head, the front area of ​​the torso, and the front area of ​​the upper limbs of all the audience members during the viewing process; the overall rear-view video data collected by the second camera position can record the back area of ​​the head, the back area of ​​the torso, and the back area of ​​the upper limbs of all the audience members during the viewing process; the overall side-view video data collected by the third camera position can record the side area of ​​the head, the side area of ​​the torso, and the side area of ​​the upper limbs of all the audience members during the viewing process.

[0056] Example 3

[0057] like Figure 3As shown, this embodiment provides a method for identifying and analyzing audience behavior to prevent voyeuristic filming of movies, wherein training a target recognition model specifically includes:

[0058] Shooting equipment recognition model: Set a training image set A. The positive samples in the training set A are a group of images with shooting equipment in the movie watching scene, and the negative samples are images without shooting equipment in the movie watching scene. They are manually labeled in advance. Both positive and negative sample data include images collected from the front and rear perspectives of the same target. Multilayer Perceptron (MLP) is used as the basis of the model, including an input layer, an output layer, and 2 hidden layers. Relu is selected as the activation function. During training, the original training set A needs to be processed first to extract the shooting equipment feature vector to reduce data redundancy. The specific extraction method of the shooting equipment feature vector is consistent with the feature extraction method corresponding to the multi-view video data, which will be introduced in subsequent embodiments and will not be repeated here.

[0059] Screen recording image recognition model: Set a training image set B. The positive samples of the training set B are a group of images with screen recording images in the movie watching scene, and the negative samples are images without screen recording images in the movie watching scene. They are manually annotated in advance. The positive and negative sample data are only images collected from the rear view of the same target. Multilayer Perceptron (MLP) is used as the basis of the model, including an input layer, an output layer, and 2 hidden layers. Relu is selected as the activation function. During training, the original training set B needs to be processed first to extract the feature vector of the screen recording image to reduce data redundancy. The specific extraction method of the feature vector of the shooting device is consistent with the feature extraction method corresponding to the multi-view video data, which will be introduced in the subsequent embodiments and will not be repeated here.

[0060] Holding behavior recognition model: Set a training image set C. The positive samples of the training set C are a group of images in which the audience has holding behavior in the movie watching scene, and the negative samples are images in which the audience does not have holding behavior in the movie watching scene. They are manually annotated in advance. The positive and negative sample data are all images collected from the front view, rear view and side view of the same target. Convolutional Neural Networks (CNN) is used as the basis of the model, including an input layer, an output layer, 2 convolutional layers, 1 activation layer, 1 pooling layer, and 1 fully connected layer. Relu is selected as the activation function. During training, the original training set C needs to be processed first to extract the feature vector of the audience's posture to reduce data redundancy. The specific extraction method of the feature vector of the shooting device is consistent with the feature extraction method corresponding to the multi-view video data, which will be introduced in the subsequent embodiments and will not be repeated here.

[0061] The output of the camera recognition model is a probability value vector of whether the image contains or does not contain a camera. For example, the camera feature vector is input into the camera recognition model, and the result is a probability value vector (a, b), where a represents the probability that the image represented by the camera feature vector contains the camera, and b represents the probability that the image represented by the camera feature vector does not contain the camera. Based on the same principle, the output results of the screen recording image recognition model and the holding behavior recognition model are also in the same form, which will not be repeated here.

[0062] Example 4

[0063] like Figure 4 As shown, this embodiment provides an audience behavior recognition and analysis method for preventing unauthorized filming of movies, wherein a real-time shooting device feature vector set including timestamp information is obtained, specifically including:

[0064] The real-time shooting device feature vector is calculated as follows:

[0065] The image dataset keyframe_1 is processed as follows: first, the human body area in the image is located using the target detection algorithm based on Faster R-CNN, the human hand area is located in the human body area using the skeleton key point detection algorithm, the hand area is used to set the segmentation area, and the segmented hand area image is obtained.

[0066] The movie-watching scene in a cinema is a multi-target scene. For the "shooting device", it will be affected by many other interference targets. In order to reduce this interference, the original image data is first detected and segmented. Considering that in most cases, the "shooting device" is either held or operated by a human hand when shooting, the appearance area of ​​the "shooting device" is located near the human hand area. Therefore, the human hand area is first located through the target detection algorithm.

[0067] This embodiment adopts a coarse-to-fine regional positioning and segmentation method, using Faster R-CNN to detect and locate the human body area in the image. Faster R-CNN is a mature algorithm in the prior art and will not be described in detail. After locating the human body area, the human hand area is located using the skeleton key point detection algorithm. The skeleton key point detection algorithm is also a mature algorithm in the prior art and will not be described in detail. After the position of the human hand area is detected, the minimum bounding rectangle that can surround the hand area is calculated, and the center point of the minimum bounding rectangle is used as the origin. After the bounding rectangle is enlarged by n times according to a preset size, the segmented area is obtained. This operation is to enable the segmented image to include the "shooting equipment" that exists near the "hand area".

[0068] The segmented hand area image is mapped to the HSV color space, and the pixel means and standard deviations of the three color channels H, S, and V are calculated respectively to obtain 6 color feature values, which are concatenated to form a color feature vector.

[0069] For a shooting device under a normal viewing angle, its color is usually different from the colors of surrounding objects, so the color feature vector is a better feature expression. The pixel mean and standard deviation of the color channel can reflect the overall color distribution. Therefore, this embodiment selects the pixel mean and standard deviation as the color feature vector.

[0070] Perform contour detection on the segmented hand area image, locate a rectangular area in the segmented hand area image as a suspected shooting device area based on the contour detection result, perform texture feature extraction on the suspected shooting device area, and obtain a texture feature vector.

[0071] For a shooting device under a normal viewing angle, its texture is usually quite different from the texture of surrounding objects, so the texture feature vector is also a better feature expression. This embodiment uses a gray level co-occurrence matrix to extract the texture feature vector.

[0072] The contour of the suspected shooting device area is detected, and the area value and perimeter value of the contour inside the area are calculated to form a shape feature vector.

[0073] For shooting devices under the front angle of view, no matter it is a mobile phone or a camera, it will have an "optical camera", and the most obvious feature of the "optical camera" is its circular shape. Therefore, the shape feature vector is also a better feature expression. This embodiment uses the area value and perimeter value of the contour as the geometric parameters of the shape to form a shape feature vector.

[0074] The image dataset keyframe_2 is processed as follows: first, the human body area in the image is located using the target detection algorithm based on Faster R-CNN, the human body area is segmented to obtain a segmented human body area image, the human body area image is converted into a grayscale image, and the pixel mean and variance of the grayscale image are calculated to form a brightness feature vector.

[0075] For the shooting device under the rear view angle, its outstanding feature is the "lit screen", so the brightness feature can better express this feature. In order to reduce the amount of calculation, this embodiment first locates and segments the human body area image, discards other irrelevant areas, and realizes the extraction of brightness feature vector on this basis.

[0076] The color feature vector, texture feature vector, shape feature vector and brightness feature vector are spliced ​​and fused to obtain a real-time shooting device feature vector. Since the subsequent recognition model is built based on a multi-layer perceptron, each feature vector can be directly spliced ​​horizontally to form a vector with more dimensions.

[0077] The timestamp and real-time shooting device feature vector corresponding to each key frame image are aggregated to form the real-time shooting device feature vector set including the timestamp information. The timestamp information can be regarded as a marker, which is used to reversely locate the time point corresponding to the image in the video. It does not belong to the feature value of the image itself, nor does it participate in the input and calculation process of the subsequent target recognition model.

[0078] The real-time screen recording image feature vector is calculated as follows:

[0079] The image data set keyframe_2 is processed as follows: the movie screen area in the image is identified, the image is divided into a movie screen area image and other area images, the movie screen area image is converted into a grayscale image and the brightness mean b1 is calculated; the other area images are converted into grayscale images, the other area images are binarized using the brightness mean b1 as a threshold, and an area with brightness greater than or equal to the brightness mean b1 is obtained as the suspected shooting device screen area, the brightness feature vector, color feature vector and texture feature vector of the suspected shooting device screen area are extracted, and the fusion and splicing are formed into a real-time screen recording image feature vector; the timestamp and real-time screen recording image feature vector corresponding to each key frame image are aggregated to form the real-time screen recording image feature vector set including the timestamp information.

[0080] The screen recording image will only appear in the rear-view image, so only the rear-view video data is selected for processing. For the rear-view shooting device, another obvious feature is "the existence of a screen recording image that is the same as the movie screen". Based on this feature, the suspected shooting device screen area in the rear-view image can be segmented out first to reduce the overall amount of calculation. For the segmented camera screen area, the brightness feature vector, color feature vector and texture feature vector are extracted to express the characteristics of the "shooting device". The same brightness feature vector, color feature vector and texture feature vector extraction method as the front-view image can be used, which will not be repeated here.

[0081] The real-time audience posture feature map is calculated as follows:

[0082] The image data sets keyframe_1, keyframe_2 and keyframe_3 are processed as follows: first, the human body region in the image is located and segmented using the target detection algorithm based on Faster R-CNN to obtain a segmented human body region image; key point detection is performed on the segmented human body region image to obtain key points of the head, neck, shoulder, elbow and wrist and their corresponding coordinates; adjacent key points are connected to obtain five upper limb vectors representing the trunk, left upper arm, left forearm, right upper arm and right forearm respectively; the angle values ​​between adjacent vectors are calculated using the five upper limb vectors to obtain joint angle value vectors; three joint angle value vectors of the key frame images in keyframe_1, keyframe_2 and keyframe_3 at the same timestamp are calculated respectively, and the three joint angle value vectors are spliced ​​to obtain a real-time audience posture feature map; the timestamp corresponding to each key frame image and the real-time audience posture feature map are aggregated to form the real-time audience posture feature atlas including the timestamp information.

[0083] For the same timestamp, the three perspectives correspond to three images respectively. The joint angle value vector of each image is calculated according to the above method, and the three joint angle value vectors are spliced ​​to form a matrix. Each row of the matrix corresponds to a joint angle value vector. In order to ensure the dimension alignment of the matrix, 0 is used to fill the matrix rows and columns with the same number of eigenvalues. The filled square matrix is ​​the real-time audience posture feature map.

[0084] The timestamp and real-time audience posture feature map corresponding to each key frame image are aggregated to form the real-time audience posture feature image set including the timestamp information. The timestamp information can be regarded as a marker. In order to reversely locate the time point corresponding to the image in the video, it does not belong to the feature value of the image itself, nor does it participate in the input and calculation process of the subsequent target recognition model. Since the subsequent recognition model is built based on CNN, the input of the model is an image, so it is necessary to vertically splice each feature vector to form a matrix so that it is consistent with the expression of the image.

[0085] Example 5

[0086] like Figure 5 As shown, this embodiment provides a method for identifying and analyzing audience behavior to prevent voyeuristic filming of movies, wherein target identification specifically includes:

[0087] If among all the recognition results corresponding to the Vob, the number of results identifying that include shooting devices accounts for more than the preset ratio threshold, then the shooting device recognition result A is considered to be the existence of a shooting device; if among all the recognition results corresponding to the Vima, the number of results identifying that include screen recording images accounts for more than the preset ratio threshold, then the screen recording image recognition result B is considered to be the existence of a screen recording image; if among all the recognition results corresponding to the Vpos, the number of results detecting holding behaviors accounts for more than the preset ratio threshold, then the holding behavior recognition result C is considered to be the existence of a holding behavior.

[0088] The preset ratio threshold is set according to actual needs. For example, the three model results can be set to the same, or different ratio thresholds can be set for each model. For example, the model input is the real-time shooting device feature vector set Vob=((vob1,t1),(vob2,t2),...,(vob j ,t j )), there are j feature vectors in the vector set. Inputting each vector into the camera recognition model one by one for recognition will get j recognition results. The preset ratio threshold is set to 20%. If the number of results containing camera devices exceeds 20%, it is considered that the camera recognition result A means that there is a camera. Similarly, the screen recording image recognition result B and the holding behavior recognition result C can be analyzed.

[0089] The data analysis and early warning specifically include: comprehensively analyzing the recognition results A, B and C, determining the early warning information and notifying the staff according to the correspondence between the preset recognition results and the early warning levels.

[0090] The corresponding relationship between the recognition results and the warning level can be set in advance, and the recognition results can be matched to low risk level, medium risk level and high risk level. For example, if the camera recognition result A, the screen recording image recognition result B and the holding behavior recognition result C are all yes (that is, the camera recognition result A is that the camera exists, the screen recording image recognition result B is that the screen recording image exists, and the holding behavior recognition result C is that the holding behavior exists), it corresponds to the high risk level; if any two of the camera recognition result A, the screen recording image recognition result B and the holding behavior recognition result C are yes, it corresponds to the medium risk level; if only one of the camera recognition result A, the screen recording image recognition result B and the holding behavior recognition result C is yes or all are no, it corresponds to the low risk level. Set different response states according to different risk levels to improve work efficiency. For example, at a high risk level, it means that there is a high possibility of voyeurism, and the staff needs to verify immediately, fix evidence and take intervention measures; at a medium risk level, it means that the possibility of voyeurism is general or the circumstances are relatively minor, and it is only necessary to verify the surveillance video before the end of the movie; at a low risk level, it means that the possibility of voyeurism is very low or there is no need to verify. The specific warning method is set according to the actual needs of the theater, and the present invention does not make specific limitations.

[0091] Example 6

[0092] like Figure 6 As shown, this embodiment provides an audience behavior recognition and analysis system for preventing unauthorized filming of movies, including the following functional modules:

[0093] Shooting camera setup module: 3 video shooting cameras are pre-set up, with a shooting device placed on each camera. The first camera is set above the movie screen to collect the overall front-view video data of all moviegoers; the second camera is set behind the last row of seats in the cinema to collect the overall rear-view video data of all moviegoers; the third camera is set on the side of the cinema to collect the overall side-view video data of all moviegoers;

[0094] Multi-view video data acquisition module: during the movie screening process, first video data is collected, the first video data is the overall front-view video data of all the audience members collected by the first camera, and the overall front-view video data records the front area of ​​the head, the front area of ​​the torso and the front area of ​​the upper limbs of all the audience members during the movie watching process; second video data is collected, the second video data is the overall rear-view video data of all the audience members collected by the second camera, and the overall rear-view video data records the back area of ​​the head, the back area of ​​the torso and the back area of ​​the upper limbs of all the audience members during the movie watching process; third video data is collected, the third video data is the overall side-view video data of all the audience members collected by the third camera, and the overall side-view video data records the side area of ​​the head, the side area of ​​the torso and the side area of ​​the upper limbs of all the audience members during the movie watching process;

[0095] Target recognition model training module: extract the shooting device feature vector set of the training image set A, input it into the pre-built artificial neural network A for training to obtain the shooting device recognition model, the artificial neural network A is based on a multi-layer perceptron, and the model output is the probability value vector of whether the image contains or does not contain the shooting device; extract the screen recording image feature vector set of the training image set B, input it into the pre-built artificial neural network B for training to obtain the screen recording image recognition model; the artificial neural network B is based on a multi-layer perceptron, and the model output is the probability value vector of whether the image contains or does not contain the screen recording image; extract the audience posture feature atlas set of the training image set C, input it into the pre-built artificial neural network C for training to obtain the holding behavior recognition model, the artificial neural network C is based on CNN, and the model output is the probability value vector of whether the holding behavior is detected or not detected in the image;

[0096] Multi-view video data processing module: extract key frames from the first, second and third video data at the same time interval to obtain the first, second and third video key frame time sequence image data sets keyframe_i=((I i1 ,t1),(I i2 ,t2),...,(I ij ,t j ), where i represents the video sequence number, i takes the value of 1, 2 or 3, j represents the number of key frame images, t j represents the timestamp corresponding to the jth key frame image; the first and second video key frame time sequence image sets are processed to obtain a real-time shooting device feature vector set Vob including timestamp information j ,t j)); Process the second video key frame time sequence image set to obtain a real-time screen recording image feature vector set Vima=((vima1,t1),(vima2,t2),...,(vima j ,t j )); Process the first, second and third video key frame time sequence image sets to obtain a real-time audience posture feature atlas Vpos=((vpos1,t1),(vpos2,t2),...,(vpos j ,t j ));

[0097] Movie viewing video data target recognition module: input the Vob into the trained shooting device recognition model to obtain the shooting device recognition result A; input the Vima into the trained screen recording image recognition model to obtain the screen recording image recognition result B; input the Vpos into the trained holding behavior recognition model to obtain the holding behavior recognition result C;

[0098] Data analysis and warning: Comprehensively analyze the recognition results A, B and C, determine the warning information and notify the staff according to the corresponding relationship between the preset recognition results and the warning level.

[0099] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, computer devices, or computer-readable storage media. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware.

[0100] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0101] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. According to the technical essence of the present invention, within the spirit and principles of the present invention, any simple modification, equivalent replacement and improvement made to the above embodiment still falls within the protection scope of the technical solution of the present invention.

Claims

1. A method for identifying and analyzing audience behavior to prevent voyeuristic filming of movies, characterized in that: The method comprises the following steps: setting up shooting positions: setting up three video shooting positions in advance, placing a shooting device on each position, wherein the first position is set above the movie screen, and is used to collect the overall front-view video data of all the moviegoers; the second position is set behind the last row of seats in the movie hall, and is used to collect the overall rear-view video data of all the moviegoers; the third position is set on the side of the movie hall, and is used to collect the overall side-view video data of all the moviegoers; Collecting multi-view video data: during the movie screening process, collecting first video data, the first video data is the overall front-view video data of all the audience members collected by the first camera, and the overall front-view video data records the front area of ​​the head, the front area of ​​the torso, and the front area of ​​the upper limbs of all the audience members during the movie watching process; collecting second video data, the second video data is the overall rear-view video data of all the audience members collected by the second camera, and the overall rear-view video data records the back area of ​​the head, the back area of ​​the torso, and the back area of ​​the upper limbs of all the audience members during the movie watching process; collecting third video data, the third video data is the overall side-view video data of all the audience members collected by the third camera, and the overall side-view video data records the side area of ​​the head, the side area of ​​the torso, and the side area of ​​the upper limbs of all the audience members during the movie watching process; Training target recognition model: extracting a shooting device feature vector set from training image set A, inputting it into a pre-built artificial neural network A for training to obtain a shooting device recognition model, wherein the artificial neural network A is based on a multi-layer perceptron, and the model output is a probability value vector of whether the image contains or does not contain a shooting device; extracting a screen recording image feature vector set from training image set B, inputting it into a pre-built artificial neural network B for training to obtain a screen recording image recognition model; wherein the artificial neural network B is based on a multi-layer perceptron, and the model output is a probability value vector of whether the image contains or does not contain a screen recording image; extracting a viewer posture feature atlas from training image set C, inputting it into a pre-built artificial neural network C for training to obtain a holding behavior recognition model, wherein the artificial neural network C is based on CNN, and the model output is a probability value vector of whether holding behavior is detected or not detected in the image; Processing multi-view video data: extracting key frames from the first, second and third video data at the same time interval, and obtaining the first, second and third video key frame time sequence image data sets keyframe_i=((I i1 ,t1),(I i2 ,t2),...,(I ij ,t j ), where i represents the video sequence number, i takes the value of 1, 2 or 3, j represents the number of key frame images, t j represents the timestamp corresponding to the jth key frame image; the first and second video key frame time sequence image sets are processed to obtain a real-time shooting device feature vector set Vob including timestamp information j ,t j )); The real-time shooting device feature vector set including timestamp information is obtained, specifically including: the image data set keyframe_1 is processed as follows: first, the human body area in the image is located by using the target detection algorithm based on Faster R-CNN, the human hand area is located by using the skeleton key point detection algorithm in the human body area, and the segmentation area is set by using the hand area to obtain the segmented hand area image; the segmented hand area image is mapped to the HSV color space, and the pixel mean and standard deviation of the three color channels of H, S, and V are calculated respectively to obtain 6 color feature values, which are spliced ​​to form a color feature vector; contour detection is performed on the segmented hand area image, and the rectangular area in the segmented hand area image is located as the suspected shooting device area through the contour detection result, and the texture feature of the suspected shooting device area is extracted to obtain a texture feature vector; contour detection is performed on the inside of the suspected shooting device area, and the area value and perimeter value of the contour inside the area are calculated to form a shape feature vector; the image data set keyframe_2 is processed as follows: first, the image data set keyframe_2 is processed as follows: The target detection algorithm of R-CNN locates the human body area in the image, segments the human body area to obtain a segmented human body area image, converts the human body area image into a grayscale image, calculates the pixel mean and variance of the grayscale image, and forms a brightness feature vector; splices and fuses the color feature vector, texture feature vector, shape feature vector and brightness feature vector to obtain a real-time shooting device feature vector; summarizes the timestamp and real-time shooting device feature vector corresponding to each key frame image to form the real-time shooting device feature vector set including the timestamp information; The second video key frame time sequence image set is processed to obtain a real-time screen recording image feature vector set Vima=((vima1,t1),(vima2,t2),...,(vima j ,t j )); Process the first, second and third video key frame time sequence image sets to obtain a real-time audience posture feature atlas Vpos=((vpos1,t1),(vpos2,t2),...,(vpos j ,t j )); Movie viewing video data target recognition: input the Vob into the trained shooting device recognition model to obtain the shooting device recognition result A; input the Vima into the trained screen recording image recognition model to obtain the screen recording image recognition result B; input the Vpos into the trained holding behavior recognition model to obtain the holding behavior recognition result C; Data analysis and warning: Comprehensively analyze the recognition results A, B and C, determine the warning information and notify the staff according to the corresponding relationship between the preset recognition results and the warning level.

2. The method for identifying and analyzing audience behavior to prevent voyeuristic filming of movies as claimed in claim 1, characterized in that: The method of obtaining a real-time screen recording image feature vector set including timestamp information specifically includes: processing the image data set keyframe_2 as follows: identifying the movie screen area in the image, dividing the image into a movie screen area image and other area images, converting the movie screen area image into a grayscale image and calculating the brightness mean b1; converting the other area images into grayscale images, binarizing the other area images using the brightness mean b1 as a threshold, obtaining an area with brightness greater than or equal to the brightness mean b1 as a suspected shooting device screen area, extracting the brightness feature vector, color feature vector and texture feature vector of the suspected shooting device screen area, and fusing and splicing them into a real-time screen recording image feature vector; aggregating the timestamp and real-time screen recording image feature vector corresponding to each key frame image to form the real-time screen recording image feature vector set including timestamp information.

3. The method for identifying and analyzing audience behavior to prevent voyeuristic filming of movies as claimed in claim 1, characterized in that: The method of obtaining a real-time audience posture feature atlas including timestamp information specifically includes: performing the following processing on image data sets keyframe_1, keyframe_2 and keyframe_3 respectively: firstly locating the human body region in the image and segmenting it using a target detection algorithm based on Faster R-CNN to obtain a segmented human body region image, performing key point detection on the segmented human body region image to obtain key points of the head, neck, shoulder, elbow and wrist and their corresponding coordinates, connecting adjacent key points to obtain five upper limb vectors representing the trunk, left upper arm, left forearm, right upper arm and right forearm respectively, using the five upper limb vectors to calculate the angle values ​​between adjacent vectors to obtain joint angle value vectors; respectively calculating three joint angle value vectors of key frame images in keyframe_1, keyframe_2 and keyframe_3 at the same timestamp, splicing the three joint angle value vectors to obtain a real-time audience posture feature map; and aggregating the timestamp corresponding to each key frame image and the real-time audience posture feature map to form the real-time audience posture feature atlas including timestamp information.

4. The method for identifying and analyzing audience behavior to prevent voyeuristic filming of movies as claimed in claim 1, characterized in that: The target recognition of the movie viewing video data specifically includes: if among all the recognition results corresponding to the Vob, the proportion of the number of results identifying that include shooting devices exceeds the preset ratio threshold, then the shooting device recognition result A is considered to be the existence of a shooting device; if among all the recognition results corresponding to the Vima, the proportion of the number of results identifying that include screen recording images exceeds the preset ratio threshold, then the screen recording image recognition result B is considered to be the existence of a screen recording image; if among all the recognition results corresponding to the Vpos, the proportion of the number of results detecting holding behavior exceeds the preset ratio threshold, then the holding behavior recognition result C is considered to be the existence of a holding behavior.

5. A system for identifying and analyzing audience behavior to prevent voyeuristic filming of movies, characterized in that: The system includes the following functional modules: a camera setting module: three video camera positions are set up in advance, and a camera device is placed on each position, wherein the first camera position is set above the movie screen to collect the overall front-view video data of all the audience members; the second camera position is set behind the last row of seats in the cinema to collect the overall rear-view video data of all the audience members; the third camera position is set on the side of the cinema to collect the overall side-view video data of all the audience members; a multi-view video data acquisition module: during the movie screening, the first video data is collected, which is the overall front-view video data of all the audience members collected by the first camera position, and the overall front-view video data records the front area of ​​the head, the front area of ​​the trunk and the front area of ​​the upper limbs of all the audience members during the movie watching process; the second video data is collected, which is the overall rear-view video data of all the audience members collected by the second camera position, and the overall rear-view video data records the back area of ​​the head, the back area of ​​the trunk and the back area of ​​the upper limbs of all the audience members during the movie watching process; the third video data is collected, which is the overall side-view video data of all the audience members collected by the third camera position, and the overall side-view video data is collected. The angle video data records the side areas of the head, torso and upper limbs of all the audience during the viewing process; the target recognition model training module: extract the shooting device feature vector set of the training image set A, input it into the pre-built artificial neural network A to train to obtain the shooting device recognition model, the artificial neural network A is based on a multi-layer perceptron, and the model output is the probability value vector of the image containing and not containing the shooting device; extract the screen recording image feature vector set of the training image set B, input it into the pre-built artificial neural network B to train to obtain the screen recording image recognition model; the artificial neural network B is based on a multi-layer perceptron, and the model output is the probability value vector of the image containing and not containing the screen recording image; extract the audience posture feature atlas of the training image set C, input it into the pre-built artificial neural network C to train to obtain the holding behavior recognition model, the artificial neural network C is based on CNN, and the model output is the probability value vector of the holding behavior detected and not detected in the image; multi-view video data processing module: extract key frames from the first, second and third video data at the same time interval to obtain the first, second and third video key frame time series image data sets keyframe_i=((I i1 ,t1),(I i2 ,t2),...,(I ij ,t j ), where i represents the video sequence number, i takes the value of 1, 2 or 3, j represents the number of key frame images, t j represents the timestamp corresponding to the jth key frame image; the first and second video key frame time sequence image sets are processed to obtain a real-time shooting device feature vector set Vob including timestamp information j ,t j )); The multi-view video data processing module specifically includes: performing the following processing on the image data set keyframe_1: first, using the target detection algorithm based on Faster R-CNN to locate the human body area in the image, using the skeleton key point detection algorithm to locate the human hand area in the human body area, using the hand area to set the segmentation area, and obtaining the segmented hand area image; mapping the segmented hand area image to the HSV color space, respectively calculating the pixel mean and standard deviation of the three color channels of H, S, and V, obtaining 6 color feature values, and splicing them to form a color feature vector; performing contour detection on the segmented hand area image, locating the rectangular area in the segmented hand area image as the suspected shooting device area through the contour detection result, and extracting the texture feature of the suspected shooting device area to obtain a texture feature vector; performing contour detection on the inside of the suspected shooting device area, calculating the area value and perimeter value of the contour inside the area, and forming a shape feature vector; performing the following processing on the image data set keyframe_2: first, using the target detection algorithm based on Faster R-CNN to locate the human body area in the image, and using the skeleton key point detection algorithm to locate the human hand area in the image, and obtaining the texture feature vector; The target detection algorithm of R-CNN locates the human body area in the image, segments the human body area to obtain a segmented human body area image, converts the human body area image into a grayscale image, calculates the pixel mean and variance of the grayscale image, and forms a brightness feature vector; splices and fuses the color feature vector, texture feature vector, shape feature vector and brightness feature vector to obtain a real-time shooting device feature vector; summarizes the timestamp and real-time shooting device feature vector corresponding to each key frame image to form the real-time shooting device feature vector set including the timestamp information; The second video key frame time sequence image set is processed to obtain a real-time screen recording image feature vector set Vima=((vima1,t1),(vima2,t2),...,(vima j ,t j )); Process the first, second and third video key frame time sequence image sets to obtain a real-time audience posture feature atlas Vpos=((vpos1,t1),(vpos2,t2),...,(vpos j ,t j )); Movie viewing video data target recognition module: input the Vob into the trained shooting device recognition model to obtain the shooting device recognition result A; input the Vima into the trained screen recording image recognition model to obtain the screen recording image recognition result B; input the Vpos into the trained holding behavior recognition model to obtain the holding behavior recognition result C; Data analysis and warning module: comprehensively analyze the recognition results A, B and C, determine the warning information and notify the staff according to the corresponding relationship between the preset recognition results and the warning level.

6. The audience behavior recognition and analysis system for preventing unauthorized filming of movies as claimed in claim 5, characterized in that: The multi-view video data processing module specifically includes: processing the image data set keyframe_2 as follows: identifying the movie screen area in the image, dividing the image into a movie screen area image and other area images, converting the movie screen area image into a grayscale image and calculating the brightness mean b1; converting the other area images into a grayscale image, using the brightness mean b1 as a threshold to binarize the other area images, obtaining an area with a brightness greater than or equal to the brightness mean b1 as a suspected shooting device screen area, extracting the brightness feature vector, color feature vector and texture feature vector of the suspected shooting device screen area, and fusing and splicing them into a real-time screen recording image feature vector; summarizing the timestamp and real-time screen recording image feature vector corresponding to each key frame image to form the real-time screen recording image feature vector set including timestamp information.

7. The audience behavior recognition and analysis system for preventing unauthorized filming of movies as claimed in claim 5, characterized in that: The multi-view video data processing module specifically includes: Further, the multi-view video data processing module specifically includes: performing the following processing on the image data sets keyframe_1, keyframe_2 and keyframe_3 respectively: first, using the target detection algorithm based on FasterR-CNN to locate the human body area in the image and perform segmentation to obtain the segmented human body area image, performing key point detection on the segmented human body area image to obtain the key points of the head, neck, shoulder, elbow and wrist and their corresponding coordinates, connecting adjacent key points to obtain 5 upper limb vectors representing the torso, left upper arm, left forearm, right upper arm and right forearm respectively, using the 5 upper limb vectors to calculate the angle value between adjacent vectors to obtain the joint angle value vector; respectively calculate the 3 joint angle value vectors of the key frame images in keyframe_1, keyframe_2 and keyframe_3 at the same timestamp, splice the 3 joint angle value vectors to obtain the real-time audience posture feature map; the timestamp corresponding to each key frame image and the real-time audience posture feature map are aggregated to form the real-time audience posture feature map set including the timestamp information.

8. The audience behavior recognition and analysis system for preventing unauthorized filming of movies as claimed in claim 5, characterized in that: The data analysis and early warning module specifically includes: if among all the recognition results corresponding to the Vob, the proportion of the number of results identifying that include shooting devices exceeds the preset ratio threshold, then the shooting device recognition result A is considered to be the existence of a shooting device; if among all the recognition results corresponding to the Vima, the proportion of the number of results identifying that include screen recording images exceeds the preset ratio threshold, then the screen recording image recognition result B is considered to be the existence of a screen recording image; if among all the recognition results corresponding to the Vpos, the proportion of the number of results detecting holding behavior exceeds the preset ratio threshold, then the holding behavior recognition result C is considered to be the existence of a holding behavior.

Images