Shooting behavior detection method and device

Abstract

The invention discloses a shooting behavior detection method and device, and belongs to the technical field of image recognition. The method comprises the following steps: acquiring a to-be-detected video of a target detection scene, wherein the target detection scene comprises a shooting target area and a shooting object area; based on the to-be-detected video, determining relative position information between a shooting target in the shooting target area and a shooting object in the shooting object area and gesture information of the shooting object; based on the relative position information and the gesture information, a behavior detection result of the shooting object is determined, and the behavior detection result is used for representing whether the shooting object has a shooting behavior on the shooting target or not. According to the method, the probability that other gestures of the shooting object are misjudged as the shooting behavior for the shooting target can be reduced, and the obtained shooting behavior detection result is more accurate.

Description

Technical Field

[0001] The present application belongs to the field of image recognition technology, and in particular, relates to a method and device for detecting shooting behavior. Background Art

[0002] In daily life, photography is usually prohibited in places involving privacy or security, such as museums, exhibition halls, special meetings, office R&D bases, etc. However, with the popularization of smart devices and the enhancement of photography functions, people can take photos at any time, which increases the risk of privacy leakage and security risks. Therefore, a method that can accurately and quickly identify shooting behavior is needed to effectively prevent information leakage and reduce security risks.

[0003] Currently, image recognition technology is usually used to identify whether someone is using a shooting device to perform a shooting action and to detect whether there is any unauthorized shooting behavior in the corresponding area. The accuracy of the shooting behavior detection results is low. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a shooting behavior detection method and device, which can reduce the probability of misjudging other gestures of the shooting object as shooting behaviors directed at the shooting target, and the obtained shooting behavior detection results are more accurate.

[0005] In a first aspect, the present application provides a method for detecting shooting behavior, the method comprising:

[0006] Obtain a video to be detected of a target detection scene, where the target detection scene includes a shooting target area and a shooting object area;

[0007] Based on the video to be detected, determining relative position information between the shooting target in the shooting target area and the shooting object in the shooting object area and gesture information of the shooting object;

[0008] Based on the relative position information and the gesture information, a behavior detection result of the shooting subject is determined, where the behavior detection result is used to characterize whether the shooting subject has a shooting behavior towards the shooting target.

[0009] According to the shooting behavior detection method of the present application, by analyzing the gesture information of the shooting object and the relative position information between the shooting target and the shooting object, it can be determined whether the shooting object is using a shooting device to shoot, and whether the lens of the shooting device is facing the direction corresponding to the shooting target. This can reduce the probability of misjudging other gestures of the shooting object as shooting behaviors directed at the shooting target, and the obtained shooting behavior detection result is more accurate.

[0010] According to an embodiment of the present application, after determining the relative position information between the shooting target in the shooting target area and the shooting object in the shooting object area and the gesture information of the shooting object, and before determining the behavior detection result of the shooting object, the method further includes:

[0011] Determine, based on the relative position information and the gesture information, a first behavior detection result of the photographed subject, where the first behavior detection result is used to represent a first probability that the photographed subject has a photographing behavior toward the photographed target;

[0012] When the first probability is greater than a first preset probability threshold, determining, based on the video to be detected, a target image corresponding to the photographed target and a photographed image corresponding to the photographing terminal of the photographed object;

[0013] Determine, based on the image features of the target image and the image features of the captured image, a second behavior detection result of the captured object, wherein the second behavior detection result is used to represent a second probability that the captured object has a capturing behavior of the captured object;

[0014] A behavior detection result of the photographed object is determined based on the first behavior detection result and the second behavior detection result.

[0015] According to one embodiment of the present application, the captured image includes a first captured image displayed by the capturing terminal and a second captured image corresponding to a capturing field of view of the capturing terminal, and the method includes:

[0016] A second behavior detection result of the photographed object is determined based on the image features of the target image, the image features of the first photographed image, and the image features of the second photographed image.

[0017] According to an embodiment of the present application, determining the second behavior detection result of the photographed object based on the image features of the target image, the image features of the first photographed image, and the image features of the second photographed image includes:

[0018] determining a first similarity between the target image and the first captured image based on an image feature of the target image and an image feature of the first captured image;

[0019] determining a second similarity between the target image and the second captured image based on an image feature of the target image and an image feature of the second captured image;

[0020] determining a third similarity between the first photographed image and the second photographed image based on an image feature of the first photographed image and an image feature of the second photographed image;

[0021] The second behavior detection result is determined based on the first similarity, the second similarity, and the third similarity.

[0022] According to an embodiment of the present application, determining the second behavior detection result based on the first similarity, the second similarity, and the third similarity includes:

[0023] When at least one of the first similarity and the third similarity is greater than a similarity threshold, determining the second behavior detection result, wherein the second behavior detection result indicates that the second probability that the photographing subject has a photographing behavior toward the photographing target is greater than a second preset probability threshold;

[0024] When it is determined that the second probability is greater than the second preset probability threshold based on the second behavior detection result, it is determined that the shooting subject has a shooting behavior of shooting the shooting target.

[0025] According to an embodiment of the present application, determining the second behavior detection result based on the first similarity, the second similarity, and the third similarity includes:

[0026] When both the first similarity and the third similarity are smaller than a similarity threshold, and the second similarity is greater than the similarity threshold, determining the second behavior detection result, wherein the second behavior detection result indicates that the second probability that the photographing subject has a photographing behavior toward the photographing target is smaller than a second preset probability threshold;

[0027] When it is determined based on the second behavior detection result that the second probability is less than the second preset probability threshold, the first probability and the second probability are calculated, and the behavior detection result is determined based on the calculation result.

[0028] According to an embodiment of the present application, determining, based on the video to be detected, relative position information between the shooting target in the shooting target area and the shooting object in the shooting object area and gesture information of the shooting object includes:

[0029] Inputting the video to be detected into a shooting behavior detection model, and determining the relative position information and the gesture information through the shooting behavior detection model;

[0030] Based on the relative position information and the gesture information, a behavior detection result of the photographed object output by the photographing behavior detection model is obtained.

[0031] According to one embodiment of the present application, the shooting behavior detection model is trained by the following steps:

[0032] Obtaining a sample detection video of the target detection scene;

[0033] Parsing the sample detection video to obtain a plurality of photographed behavior images and a plurality of non-photographed behavior images;

[0034] Changing the relative position between the area corresponding to the sample shooting target and the area corresponding to the sample shooting object in the shooting behavior image multiple times to obtain multiple sample shooting behavior images corresponding to the shooting behavior image;

[0035] Changing the relative positions of the first area and the second area in the non-photographing behavior image multiple times to obtain a plurality of sample non-photographing behavior images corresponding to the non-photographing behavior image, wherein the first area and the second area are any areas in the non-photographing behavior image, and the first area and the second area are different areas;

[0036] The shooting behavior detection model is trained based on the multiple sample shooting behavior images corresponding to each shooting behavior image and the multiple sample non-shooting behavior images corresponding to each non-shooting behavior image.

[0037] According to an embodiment of the present application, determining, based on the video to be detected, relative position information between the shooting target in the shooting target area and the shooting object in the shooting object area and gesture information of the shooting object includes:

[0038] Based on the video to be detected, obtaining a plurality of images to be detected corresponding to the video to be detected;

[0039] Based on the video to be detected, determining first relative position information between the shooting target and the shooting object and first gesture information of the shooting object;

[0040] Based on the plurality of images to be detected, determining second relative position information between the shooting target and the shooting object and second gesture information of the shooting object;

[0041] A behavior detection result of the photographed object is determined based on the first relative position information, the second relative position information, the first gesture information, and the second gesture information.

[0042] In a second aspect, the present application provides a shooting behavior detection device, the device comprising:

[0043] An acquisition module is used to acquire a video to be detected of a target detection scene, where the target detection scene includes a shooting target area and a shooting object area;

[0044] A first processing module, configured to determine, based on the video to be detected, relative position information between the shooting target in the shooting target area and the shooting object in the shooting object area and gesture information of the shooting object;

[0045] The second processing module is used to determine the behavior detection result of the shooting object based on the relative position information and the gesture information, and the behavior detection result is used to characterize whether the shooting object has a shooting behavior of the shooting target.

[0046] According to the shooting behavior detection device of the present application, by analyzing the gesture information of the shooting object and the relative position information between the shooting target and the shooting object, it can be determined whether the shooting object is using the shooting device to shoot, and whether the lens of the shooting device is facing the direction corresponding to the shooting target. This can reduce the probability of misjudging other gestures of the shooting object as shooting behaviors directed at the shooting target, and the obtained shooting behavior detection result is more accurate.

[0047] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the shooting behavior detection method as described in the first aspect above is implemented.

[0048] In a fourth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the shooting behavior detection method as described in the first aspect above is implemented.

[0049] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the shooting behavior detection method as described in the first aspect above.

[0050] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0052] Figure 1 is a flow chart of a shooting behavior detection method provided in an embodiment of the present application;

[0053] Figure 2 is a schematic diagram of the shooting field of view of the shooting terminal provided in an embodiment of the present application;

[0054] Figure 3It is a schematic diagram of the process of training a shooting behavior detection model provided in an embodiment of the present application;

[0055] Figure 4 is a structural schematic diagram of a shooting behavior detection device provided in an embodiment of the present application;

[0056] Figure 5 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0058] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0059] The following, in conjunction with the accompanying drawings, describes in detail the shooting behavior detection method, shooting behavior detection device, electronic device and readable storage medium provided in the embodiments of the present application through specific embodiments and their application scenarios.

[0060] The shooting behavior detection method may be applied to a terminal, and may be specifically executed by hardware or software in the terminal.

[0061] The terminal includes, but is not limited to, a portable communication device such as a mobile phone or tablet computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad). It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., a touch screen display and / or a touch pad).

[0062] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse and a joystick.

[0063] The shooting behavior detection method provided in the embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the shooting behavior detection method. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The shooting behavior detection method provided in the embodiment of the present application is described below using an electronic device as an example of the execution subject.

[0064] like Figure 1 As shown, the shooting behavior detection method includes: step 110, step 120 and step 130.

[0065] Step 110: Obtain a to-be-detected video of a target detection scene, where the target detection scene includes a shooting target area and a shooting object area.

[0066] Among them, the target detection scene is the scene where shooting behavior detection is to be performed. For example, the target detection scene can be a scene where shooting is prohibited, such as a museum, exhibition hall, special meeting, office R&D base, etc. Shooting can include taking photos and recording videos.

[0067] The target detection scene may be provided with a video acquisition device, and the video acquisition device is used to monitor the target detection scene and acquire the video to be detected of the target detection scene.

[0068] It can be understood that the video to be detected is a video of the target detection scene captured in real time by a video capture device.

[0069] In this embodiment, the shooting target area is the area where the shooting target is located, and the shooting target is a target that is prohibited from being photographed in the target detection scene, for example, it can be a display screen for a meeting, a cultural relic in a museum, and R&D equipment in a R&D base.

[0070] In this embodiment, the shooting object area is the area where the shooting object is located, and the shooting object may be a person in the target detection scene.

[0071] Step 120: Based on the video to be detected, determine the relative position information between the shooting target in the shooting target area and the shooting object in the shooting object area and the gesture information of the shooting object.

[0072] The relative position information is used to represent the relative position between the shooting target and the shooting object, and the gesture information can represent the gesture action corresponding to the hand of the shooting object.

[0073] The relative position between the shooting target and the shooting object may include the distance and orientation of each key point of the shooting object such as the hand, face and torso relative to the shooting target.

[0074] The subject may hold the photographing device, and the gestures of the subject may include gestures when holding the photographing device.

[0075] In this embodiment, target detection and recognition may be performed on the video to be detected to obtain a target detection frame corresponding to the target in the video frame of the video to be detected and a subject detection frame corresponding to the subject.

[0076] The relative position between the target detection frame and the object detection frame can be determined based on the coordinates of the target detection frame and the object detection frame, and the positions of key points of the object can be determined through posture estimation technology, thereby inferring the distance and orientation of each key point such as the hand, face and torso of the object relative to the target to obtain relative position information.

[0077] In actual implementation, the target detection model can be used to perform target detection and recognition on the video to be detected, and the key point detection model can be used to detect the key points of the photographed object.

[0078] In this embodiment, the hand key points of the photographed subject can be detected and tracked through the gesture recognition model, and the changes and movement trajectories of the gestures can be analyzed according to the positions and relative relationships of the hand key points to obtain gesture information.

[0079] In this embodiment, the video to be detected can be input into a trained deep learning model to obtain relative position information and gesture information output by the deep learning model.

[0080] Step 130: Determine a behavior detection result of the photographed object based on the relative position information and the gesture information.

[0081] The behavior detection result is used to characterize whether the subject has a shooting behavior towards the shooting target.

[0082] In this embodiment, it can be determined based on gesture information whether the subject has taken an action of photographing using the photographing device, and it can be determined based on relative position information whether the subject's action of photographing using the photographing device is directed at a photographing target.

[0083] When it is determined based on gesture information that the subject uses a photographing device to take a photo, and based on relative position information that the subject directs the lens of the photographing device toward a direction corresponding to the photographing target, a behavior detection result that the subject has a photographing behavior of taking a photo of the photographing target can be obtained.

[0084] When it is determined that the subject is not using a photographing device, or that the subject is using a photographing device but does not point the lens of the photographing device toward a direction corresponding to the photographing target, a behavior detection result indicating that the subject does not have a photographing behavior of the photographing target can be obtained.

[0085] In the related art, image recognition technology is usually used to identify whether someone is using a shooting device to perform a shooting action, and to detect whether there is any unauthorized shooting behavior in the corresponding area.

[0086] However, the corresponding gestures when people use mobile phones, tablets and other devices to send messages and browse the mobile phone interface may be similar to the gestures when taking pictures, and may be recognized as people using mobile phones or tablets to take pictures. In addition, when people use mobile phones, tablets and other devices to take pictures, they may not take pictures of the area corresponding to the target where shooting is prohibited. It is easy to cause false detection by only detecting whether someone uses the shooting device to perform the shooting action to determine whether there is any prohibited shooting behavior in the detection area, and the accuracy of the shooting behavior detection results is low.

[0087] In an embodiment of the present application, the relative position information between the shooting target in the shooting target area and the shooting object in the shooting object area and the gesture information of the shooting object are determined through the video to be detected in the target detection scene. Based on the relative position information and the gesture information, the behavior detection result of the shooting object is determined. The relative position information between the shooting target and the shooting object is added to determine whether the shooting object has directed the lens of the shooting device toward the direction corresponding to the shooting target, so that the shooting behavior detection result obtained is more accurate.

[0088] According to the shooting behavior detection method provided in the embodiment of the present application, by analyzing the gesture information of the shooting object and the relative position information between the shooting target and the shooting object, it can be determined whether the shooting object is using a shooting device to shoot, and whether the lens of the shooting device is facing the direction corresponding to the shooting target. This can reduce the probability of misjudging other gestures of the shooting object as shooting behaviors directed at the shooting target, and the obtained shooting behavior detection result is more accurate.

[0089] In some embodiments, after determining the relative position information between the shooting target in the shooting target area and the shooting object in the shooting object area and the gesture information of the shooting object, and before determining the behavior detection result of the shooting object, the method further includes:

[0090] Based on the relative position information and the gesture information, determining a first behavior detection result of the photographed subject, where the first behavior detection result is used to represent a first probability that the photographed subject has a photographing behavior toward the photographed target;

[0091] When the first probability is greater than a first preset probability threshold, determining, based on the video to be detected, a target image corresponding to the photographed target and a photographed image corresponding to the photographing terminal of the photographed object;

[0092] Based on the image features of the target image and the image features of the captured image, a second behavior detection result of the captured object is determined, and the second behavior detection result is used to characterize a second probability that the captured object has a capturing behavior toward the captured object.

[0093] In this embodiment, a labeled data set can be used to train a machine learning model or a deep learning model, so that the model can learn to predict the probability of shooting behavior based on relative position information and gesture information, and the relative position information and gesture information are input into the trained model to obtain the first probability of the presence of shooting behavior output by the model as the first behavior detection result.

[0094] Among them, the first preset probability threshold is a preset value. When the first probability is greater than the first preset probability threshold, it means that the probability that the shooting subject has a shooting behavior of shooting the shooting target is relatively high, and further judgment can be made based on the target image corresponding to the shooting target and the shooting image corresponding to the shooting terminal of the shooting subject.

[0095] The target image is an image including a shooting target. For example, the shooting target may be a projection screen, the target image may be an image including the projection screen, and the shot image is an image shot by a shooting terminal of the shooting object.

[0096] In this embodiment, target recognition and detection may be performed on the image frames of the video to be detected to obtain the photographed object and the photographed image.

[0097] In this embodiment, feature extraction is performed on the target image to obtain image features of the target image, and feature extraction is performed on the captured image to obtain image features of the captured image.

[0098] The image features of the target image and the image features of the captured image are analyzed and compared, and a second probability is determined based on the similarity between the image features of the target image and the image features of the captured image as the second behavior detection result. The greater the similarity between the image features of the target image and the image features of the captured image, the larger the corresponding second probability.

[0099] In this embodiment, determining the behavior detection result of the photographed object includes:

[0100] Based on the first behavior detection result and the second behavior detection result, a behavior detection result is determined.

[0101] In this embodiment, based on the first behavior detection result and the second behavior detection result, corresponding mathematical operations may be performed on the first probability and the second probability, and the behavior detection result may be determined according to the operation result.

[0102] For example, a weighted sum operation may be performed on the first probability and the second probability to obtain a third probability value. When the third probability value is greater than a set probability threshold, it is determined that the subject has taken a photographing action on the target.

[0103] In this embodiment, when the recognition confidence and the shooting behavior statistical frequency of the first behavior detection result and the second behavior detection result both reach a set value, it can be determined that there is a detection result of the shooting behavior.

[0104] In this embodiment, the suspected shooting behavior can be quickly and effectively identified through the acquired video to be detected, and the shooting target image and the display image of the shooting device can be further compared and judged for consistency, so that the existence of shooting behavior can be more accurately and fully identified, so as to prevent information leakage and eliminate safety hazards in the future. Effective management and control and traceability evidence can be carried out.

[0105] In some embodiments, the captured image includes a first captured image displayed by the capturing terminal and a second captured image corresponding to the capturing field of view of the capturing terminal, and based on image features of the target image and image features of the captured image, determining a second behavior detection result of the captured object includes:

[0106] A second behavior detection result is determined based on the image features of the target image, the image features of the first captured image, and the image features of the second captured image.

[0107] The first captured image is an image displayed on a display screen of the shooting terminal, and the second captured image is an image that can be captured by the shooting terminal within a corresponding shooting field of view.

[0108] In this embodiment, an image rectangular area R2 where the subject is holding a mobile phone, camera, camcorder, tablet computer or other shooting device to perform shooting action is detected in the image frame of the video to be detected, and an image rectangular area R3 where the mobile phone, camera, camcorder, tablet computer or other shooting device is held is further detected and segmented.

[0109] Then, through the feature point detection and extraction methods such as the RetinaNet model and the Warped PlanarObject Detection Network (WPOD-Net), the four feature vertices (p1, p2, p3 and p4) of the shooting device screen are obtained, and affine transformation is performed according to the four feature vertices to correct the shooting device display screen image R3', which is the first captured image.

[0110] In this embodiment, the depth information of the image frame of the video to be detected can be estimated. The depth information of the image frame of the video to be detected can be directly acquired by using a binocular, laser or other depth camera device, or the depth information of the image frame of the video to be detected can be obtained by using a monocular vision depth estimation method based on deep learning. Therefore, for each pixel point p(i, j) in the image frame, the corresponding 3D coordinate value (x ij ,y ij , zij ).

[0111] According to the image depth information estimated above and the four feature vertices (p1, p2, p3 and p4) extracted from the above analysis, the corresponding 3D coordinate value p1 (x p1 ,y p1 , z p1 )、p2(x p2 ,y p2 , z p2 )、p3(x p3 ,y p3 , z p3 ) and p4(x p4 ,y p4 , z p4 ).

[0112] By connecting points p1, p2, p3, and p4 in sequence, the display screen image of the camera device can be obtained. Based on the prior information such as the screen size ratio, orientation, and installation position of the camera on the camera device, the image position point C of the camera can be estimated, and the 3D coordinate value C (x c ,y c , z c ).

[0113] Then, based on the spatial stereoscopic geometric relationship, the normal CO passing through the camera position point C and perpendicular to the plane composed of the four points p1, p2, p3 and p4 (i.e., the screen plane of the shooting device) can be calculated to complete the pose estimation of the shooting device.

[0114] According to the camera position point C and normal line CO obtained above, a spatial rectangular coordinate system Cxyz is established with C as the origin and the straight line CO as the x-axis, and the spatial rectangular coordinate system Cxyz is used to perform coordinate transformation of the spatial rectangular coordinate system on the estimated image depth information.

[0115] According to the prior information such as the field of view of most existing cameras and the ratio of the image resolution, it can be estimated that the maximum field of view of the camera of the camera in the vertical and horizontal directions is a and b, and a = 90 degrees and b = 120 degrees can be taken. Then, according to the spatial stereoscopic geometric relationship of each point, a four-sided pyramid CP1P2P3P4 with C as the vertex and the plane rectangle P1P2P3P4 can be extended along the straight line CO (x-axis) and gradually increase, as shown in Figure 2 As shown, the field of view angle a is the field of view angle corresponding to the point ACB, and the field of view angle b is the field of view angle corresponding to the point ECF.

[0116] According to the above-mentioned estimated image depth information, the spatial position of each image point in the image frame relative to the tetrahedron CP1P2P3P4 is calculated. When the image point falls within the tetrahedron CP1P2P3P4, it is determined that the image point can be imaged by the camera of the shooting device, the image value of the image point is retained, and the points q1, q2, q3 and q4 whose spatial positions fall on the edges CP1, CP2, CP3 and CP4 of the tetrahedron CP1P2P3P4 are recorded. Conversely, when the image point falls outside the tetrahedron CP1P2P3P4, it is impossible for the image point to be imaged by the camera of the shooting device, and the image value of the image point is set to zero.

[0117] Then, affine transformation is performed on the above-recorded image points q1, q2, q3 and q4 to correct and obtain an estimated screen image R4 of the object photographed by the photographing device, where R4 is a second photographed image corresponding to the photographing field of view of the photographing terminal.

[0118] In this embodiment, feature extraction is performed on the target image to obtain image features of the target image, feature extraction is performed on the first captured image to obtain image features of the first captured image, and feature extraction is performed on the second captured image to obtain image features of the second captured image.

[0119] The image features of the target image, the image features of the first captured image, and the image features of the second captured image are analyzed and compared, and a second probability is determined as a second behavior detection result based on the similarity between the image features of the target image, the image features of the first captured image, and the image features of the second captured image.

[0120] In this embodiment, the posture of the shooting device in the video image is first estimated to obtain the three-dimensional position coordinates and angle of the shooting device, as well as the depth estimation between the shooting device and the shooting target, so as to accurately estimate the image area of ​​the field of view that can be captured by the camera of the shooting device, and obtain the second shot image. Then, based on the segmented shooting device display screen as the first shot image, the similarity and consistency of features of the first shot image and the second shot image are estimated, thereby reducing the computational complexity of subsequent feature analysis and extraction processes, and avoiding noise interference in image areas that do not correspond to the shooting target.

[0121] In some embodiments, determining the second behavior detection result based on the image feature of the target image, the image feature of the first captured image, and the image feature of the second captured image includes:

[0122] determining a first similarity between the target image and the first captured image based on an image feature of the target image and an image feature of the first captured image;

[0123] determining a second similarity between the target image and the second captured image based on an image feature of the target image and an image feature of the second captured image;

[0124] determining a third similarity between the first captured image and the second captured image based on an image feature of the first captured image and an image feature of the second captured image;

[0125] Based on the first similarity, the second similarity, and the third similarity, a second behavior detection result is determined.

[0126] The first similarity may represent the similarity between the target image and the first captured image, the second similarity may represent the similarity between the target image and the second captured image, and the third similarity may represent the similarity between the first captured image and the second captured image.

[0127] In this embodiment, the first similarity, the second similarity, and the third similarity may be expressed as Euclidean distance, cosine similarity, Manhattan distance, Pearson correlation coefficient, etc. between features.

[0128] In this embodiment, the first similarity may be the distance between the image features of the target image and the image features of the first captured image, the second similarity may be the distance between the image features of the target image and the image features of the second captured image, and the third similarity may be the distance between the image features of the first captured image and the image features of the second captured image.

[0129] The degree of similarity between the target image and the first captured image can be determined based on the distance between the image features of the target image and the image features of the first captured image, the degree of similarity between the target image and the second captured image can be determined based on the distance between the image features of the target image and the image features of the second captured image, and the degree of similarity between the first captured image and the second captured image can be determined based on the image features of the first captured image and the image features of the second captured image.

[0130] In this embodiment, the second behavior detection result can be determined by analyzing whether the subject has already photographed the target or is likely to prepare to photograph the target through the similarity between the target image and the first captured image, the similarity between the target image and the second captured image, and the similarity between the first captured image and the second captured image.

[0131] In some embodiments, determining a second behavior detection result based on the first similarity, the second similarity, and the third similarity includes:

[0132] When at least one of the first similarity and the third similarity is greater than the similarity threshold, determining a second behavior detection result, the second behavior detection result indicating that a second probability that the photographing subject has a photographing behavior toward the photographing target is greater than a second preset probability threshold;

[0133] Determining a behavior detection result based on the first behavior detection result and the second behavior detection result includes:

[0134] When it is determined based on the second behavior detection result that the second probability is greater than the second preset probability threshold, it is determined that the photographing subject has a photographing behavior of the photographing target.

[0135] The similarity threshold is a preset value, and the second preset probability threshold is a preset value.

[0136] In this embodiment, when the first similarity is greater than the similarity threshold, or the third similarity is greater than the similarity threshold, it is determined that the second probability that the shooting subject has a shooting behavior of shooting the shooting target is greater than the second preset probability threshold. When the first probability is greater than the first preset probability threshold and the second probability is greater than the second preset probability threshold, it can be determined that the shooting subject has a shooting behavior of shooting the shooting target.

[0137] It can be understood that when the first similarity and the third similarity are expressed as distances, the first similarity and the third similarity being greater than the similarity threshold may mean that the distance corresponding to the first similarity and the distance corresponding to the third similarity are less than a set distance threshold.

[0138] In this embodiment, according to the target image R1 and the first captured image R3', the feature vector F1 of the target image R1 and the feature vector F3' of the first captured image R3' are calculated, and then the distance d between the feature vector F1 and the feature vector F3' is calculated. 13' .

[0139] The feature vector F4 is calculated for the second captured image R4, and then the distance d between the feature vector F1 and the feature vector F4 is calculated. 14 , the distance d between eigenvectors F3' and F4 3'4 .

[0140] When the distance d between eigenvector F1 and eigenvector F3' 13' If the distance is less than the set distance threshold, it can be determined that images R1 and R3' have similarities and consistency in features, that is, the shooting device has captured the image of the shooting target and there is shooting behavior; otherwise, it cannot be determined whether there is shooting behavior.

[0141] When the distance d between eigenvector F3' and eigenvector F4 3'4If the distance is less than the set distance threshold, it can be determined that there are similarities and consistency in features between images R3' and R4, that is, the image captured by the shooting device is consistent with the estimated screen image, the first shot image is consistent with the second shot image, the camera of the shooting device captures the shooting target in the image position, and there is shooting behavior. Otherwise, it cannot be determined whether there is shooting behavior.

[0142] It can be understood that when the feature distance d 13' and d 3'4 When one of them is smaller than the set distance threshold, it can be determined that there is a shooting behavior.

[0143] In some embodiments, determining a second behavior detection result based on the first similarity, the second similarity, and the third similarity includes:

[0144] When the first similarity and the third similarity are both less than the similarity threshold, and the second similarity is greater than the similarity threshold, determining a second behavior detection result, wherein the second behavior detection result indicates that a second probability that the photographing subject has a photographing behavior of the photographing target is less than a second preset probability threshold;

[0145] Determining a behavior detection result based on the first behavior detection result and the second behavior detection result includes:

[0146] When it is determined based on the second behavior detection result that the second probability is less than the second preset probability threshold, the first probability and the second probability are calculated, and the behavior detection result is determined based on the calculation result.

[0147] In this embodiment, when the distance d between the feature vector F1 and the feature vector F4 is 14 If the distance is less than the set distance threshold, it can be judged that there are similarities and consistency in features between images R1 and R4, that is, the captured target image is consistent with the estimated screen image, the camera of the shooting device can capture the target in the image, and there is suspected shooting behavior, otherwise, it cannot.

[0148] The second probability that the photographing subject has a photographing behavior toward the photographing target is less than a second preset probability threshold, indicating that the presence of the photographing behavior cannot be directly determined based on the second probability.

[0149] When the characteristic distance d 14 Less than the set distance threshold, d 13' and d 3'4 When neither of the two conditions is less than the set distance threshold, there is a suspected shooting behavior, and the first probability and the second probability are further calculated, and the behavior detection result is determined based on the calculation result.

[0150] In actual implementation, the first probability and the second probability may be weightedly calculated, and the calculated result may be compared with a set probability threshold. When the weighted probability is greater than the probability threshold, it is determined that there is a shooting behavior.

[0151] In some embodiments, based on the video to be detected, determining the relative position information between the shooting target in the shooting target area and the shooting object in the shooting object area and the gesture information of the shooting object includes:

[0152] The video to be detected is input into the shooting behavior detection model, and the relative position information and gesture information are determined by the shooting behavior detection model;

[0153] Based on the relative position information and gesture information, the behavior detection result of the photographed object is determined, including:

[0154] Based on the relative position information and gesture information, a behavior detection result output by the shooting behavior detection model is obtained.

[0155] Among them, the shooting behavior detection model can be a trained machine learning model or a deep learning model.

[0156] In this embodiment, the video to be detected can be input into the shooting behavior detection model, the shooting behavior detection model can determine the relative position information and gesture information, and based on the relative position information and gesture information, obtain the behavior detection result output by the shooting behavior detection model.

[0157] In some embodiments, Figure 3 As shown, the shooting behavior detection model is trained through the following steps:

[0158] Step 310: Obtain a sample detection video of the target detection scene.

[0159] Step 320: parse the sample detection video to obtain a plurality of captured behavior images and a plurality of non-captured behavior images.

[0160] Step 330 : Changing the relative position between the area corresponding to the sample shooting target and the area corresponding to the sample shooting object in the shooting behavior image multiple times to obtain multiple sample shooting behavior images corresponding to the shooting behavior image.

[0161] Step 340 : Changing the relative positions of the first area and the second area in the non-photographing behavior image multiple times to obtain multiple sample non-photographing behavior images corresponding to the non-photographing behavior image.

[0162] The first area and the second area are any areas in the non-photographing behavior image, and the first area and the second area are different areas.

[0163] Step 350: training a shooting behavior detection model based on a plurality of sample shooting behavior images corresponding to each shooting behavior image and a plurality of sample non-shooting behavior images corresponding to each non-shooting behavior image.

[0164] The sample detection video is a video of a target detection scene collected by a video acquisition device within a specific period of time, and the sample detection video is used to train a shooting behavior detection model.

[0165] By converting the sample detection video into an image sequence and parsing it, we can artificially divide the positive and negative sample sets with and without shooting behavior, that is, obtain multiple shooting behavior images and multiple non-shooting behavior images. The positive sample set includes shooting behavior images, and the negative sample set includes non-shooting behavior images.

[0166] The shooting behavior image is an image in which the sample shooting subject has a shooting behavior toward the sample shooting target, and the non-shooting behavior image is an image in which the sample shooting subject has no shooting behavior toward the sample shooting target.

[0167] For the single-shot behavioral image in the positive sample set, the 3D Photography using Context-aware Layered Depth Inpainting method is used to convert the 2D static image into a 3D video image, generating a dynamic image sequence with effects similar to lens shaking and zooming.

[0168] For the generated dynamic image sequence, the image rectangular area of ​​the shooting target, i.e., the sample shooting target corresponding area R1 and the image rectangular area of ​​the person holding a mobile phone, camera, camcorder, tablet computer or other shooting device to perform shooting actions, i.e., the sample shooting object corresponding area R2 are detected through image instance segmentation and target detection methods, and two rectangular area images containing the shooting target and the shooting posture and action of the shooting subject holding the shooting device are obtained.

[0169] According to the two rectangular areas R1 and R2 obtained above, the corresponding center points C1 and C2 of the two rectangular areas are calculated, and then the two rectangular areas respectively use their respective center points as reference points, randomly select the scaling factor p of the width and height of any rectangle, and the value range of p can be 0.7-1.3, and scale the two rectangular area images, and then translate any rectangular area image along the direction of the line connecting C1 and C2 toward the other rectangular area image until the nearest edges of the two rectangles overlap.

[0170] Take C1 or C2 as the rotation center point, the line connecting C1 and C2 as the baseline, randomly select any clockwise or counterclockwise rotation angle α, -θ≤α≤θ, θ can be 30 degrees, and then according to the triangular geometry relationship, the position C1' of the other rectangle center point can be calculated, and then translate the rectangular area R1 image until the rectangle center point C1 coincides with C1', and the stitched sample image is obtained, that is, the sample shooting behavior image.

[0171] By changing the relative position between the area corresponding to the sample shooting target and the area corresponding to the sample shooting object in the captured behavior image multiple times, and then normalizing the size, brightness, mean, etc. of the multiple sample shooting behavior images obtained, a large number of rich positive sample image classification data sets can be obtained.

[0172] Two regions are randomly selected in the non-photographed behavior image as the first region and the second region, and the relative positions of the first region and the second region in the non-photographed behavior image are changed multiple times. A negative sample image classification dataset is obtained by the same method as that of a positive sample image classification dataset.

[0173] In this embodiment, the shooting behavior detection model can be trained by using a plurality of sample shooting behavior images in the positive sample image classification data set and a plurality of sample non-shooting behavior images in the negative sample image classification data set.

[0174] In this embodiment, the relative positions between the areas corresponding to the sample shooting targets and the areas corresponding to the sample shooting objects in the shooting behavior images are changed multiple times to obtain multiple sample shooting behavior images corresponding to the shooting behavior images. By training the shooting behavior detection model with multiple sample shooting behavior images, it is possible to solve the problem that the shooting position of a person holding a shooting device and the shooting target position have different direction angles and distances, and there are too many interfering image areas in the middle and they are inconsistent.

[0175] In some embodiments, based on the video to be detected, determining the relative position information between the shooting target in the shooting target area and the shooting object in the shooting object area and the gesture information of the shooting object includes:

[0176] Based on the video to be detected, a plurality of images to be detected corresponding to the video to be detected are obtained;

[0177] Based on the video to be detected, determining first relative position information between the shooting target and the shooting object and first gesture information of the shooting object;

[0178] Determining second relative position information between the shooting target and the shooting object and second gesture information of the shooting object based on the multiple images to be detected;

[0179] Based on the relative position information and gesture information, the behavior detection result of the photographed object is determined, including:

[0180] A behavior detection result is determined based on the first relative position information, the second relative position information, the first gesture information, and the second gesture information.

[0181] In this embodiment, the video to be detected includes multiple frames, and each frame may correspond to an image to be detected.

[0182] In this embodiment, the first relative position information between the shooting target and the shooting object and the first gesture information of the shooting object can be determined through a single frame of the image to be detected according to the spatial feature information of the image to be detected, and the corresponding behavior detection result can be determined based on the first relative position information between the shooting target and the shooting object and the first gesture information of the shooting object.

[0183] A multi-frame image sequence corresponding to the video to be detected determines the second relative position information between the shooting target and the shooting object and the second gesture information of the shooting object according to the spatial feature information and time series information of the multi-frame image sequence, and determines the corresponding behavior detection result based on the second relative position information between the shooting target and the shooting object and the second gesture information of the shooting object.

[0184] In this embodiment, the behavior detection result of the photographed object can be determined according to the behavior detection result corresponding to the single-frame image to be detected and the behavior detection results corresponding to the multi-frame image sequence.

[0185] In this embodiment, the shooting behavior detection model may include a first shooting behavior detection model for detecting a single image and a second shooting behavior detection model for detecting an image sequence corresponding to a video.

[0186] In this embodiment, the first shooting behavior detection model can be trained by using a plurality of sample shooting behavior images and a plurality of sample non-shooting behavior images, with a single image as a unit.

[0187] It is also possible to randomly sample from multiple sample shot behavior images in chronological order to obtain a sample shot behavior image sequence, and to randomly sample from multiple sample non-shot behavior images in chronological order to obtain a sample non-shot behavior image sequence, and to train a second shooting behavior detection model using the sample shot behavior image sequence and the sample non-shot behavior image sequence.

[0188] In this embodiment, for the training and recognition of the first shooting behavior detection model, a plurality of sample shooting behavior images and a plurality of sample non-shooting behavior images are used, and a residual neural network (ResNet) or a self-attention transformer (Tramsformer) network model method based on deep learning is adopted to train the first shooting behavior detection model.

[0189] In this embodiment, for the training and recognition of the second shooting behavior detection model, the second shooting behavior detection model is trained by using a sample shooting behavior image sequence and a sample non-shooting behavior image sequence, using a method based on a time shift module (TSM) or a slow-fast network (SlowFast) network model.

[0190] The first shooting behavior detection model is used to infer whether there is a shooting action or shooting posture, and the continuous frequency of the shooting action and shooting posture is counted. When the continuous frequency reaches a set number of times, the result information of the existence of the shooting behavior can be output.

[0191] The second shooting behavior detection model is used to infer whether there is a shooting behavior, and the frequency of the shooting behavior is counted. When the frequency reaches the set number of times, the shooting behavior result information is output.

[0192] In this embodiment, the results output by the first shooting behavior detection model and the results output by the second shooting behavior detection model are fused, which can solve the problem of insufficient accuracy and easy misidentification of shooting behavior by using only image classification or video classification model, and obtain more accurate behavior detection results.

[0193] The shooting behavior detection method provided in the embodiment of the present application can be executed by a shooting behavior detection device. In the embodiment of the present application, the shooting behavior detection device provided in the embodiment of the present application is described by taking the shooting behavior detection device executing the shooting behavior detection method as an example.

[0194] The embodiment of the present application also provides a shooting behavior detection device.

[0195] like Figure 4 As shown, the shooting behavior detection device includes:

[0196] An acquisition module 410 is used to acquire a to-be-detected video of a target detection scene, where the target detection scene includes a shooting target area and a shooting object area;

[0197] A first processing module 420 is used to determine the relative position information between the shooting target in the shooting target area and the shooting object in the shooting object area and the gesture information of the shooting object based on the video to be detected;

[0198] The second processing module 430 is used to determine a behavior detection result of the shooting object based on the relative position information and the gesture information, wherein the behavior detection result is used to indicate whether the shooting object has a shooting behavior of shooting the shooting target.

[0199] According to the shooting behavior detection device provided in the embodiment of the present application, by analyzing the gesture information of the shooting object and the relative position information between the shooting target and the shooting object, it can be determined whether the shooting object is using a shooting device to shoot, and whether the lens of the shooting device is facing the direction corresponding to the shooting target. This can reduce the probability of misjudging other gestures of the shooting object as shooting behaviors directed at the shooting target, and the obtained shooting behavior detection result is more accurate.

[0200] In some embodiments, the second processing module 430 is used to determine a first behavior detection result of the photographed subject based on the relative position information and the gesture information, wherein the first behavior detection result is used to represent a first probability that the photographed subject has a photographing behavior of the photographed target;

[0201] When the first probability is greater than a first preset probability threshold, determining, based on the video to be detected, a target image corresponding to the photographed target and a photographed image corresponding to the photographing terminal of the photographed object;

[0202] Determine, based on the image features of the target image and the image features of the captured image, a second behavior detection result of the captured object, wherein the second behavior detection result is used to represent a second probability that the captured object has a capturing behavior of the captured object;

[0203] The behavior detection result is determined based on the first behavior detection result and the second behavior detection result.

[0204] In some embodiments, the captured image includes a first captured image displayed by the capturing terminal and a second captured image corresponding to a capturing field of view of the capturing terminal, and a second processing module 430 is used to determine the second behavior detection result based on image features of the target image, image features of the first captured image, and image features of the second captured image.

[0205] In some embodiments, the second processing module 430 is used to determine a first similarity between the target image and the first captured image based on an image feature of the target image and an image feature of the first captured image;

[0206] determining a second similarity between the target image and the second captured image based on an image feature of the target image and an image feature of the second captured image;

[0207] determining a third similarity between the first photographed image and the second photographed image based on an image feature of the first photographed image and an image feature of the second photographed image;

[0208] The second behavior detection result is determined based on the first similarity, the second similarity, and the third similarity.

[0209] In some embodiments, the second processing module 430 is used to determine the second behavior detection result when at least one of the first similarity and the third similarity is greater than a similarity threshold, wherein the second behavior detection result indicates that the second probability that the photographing subject has a photographing behavior toward the photographing target is greater than a second preset probability threshold;

[0210] When it is determined that the second probability is greater than the second preset probability threshold based on the second behavior detection result, it is determined that the shooting subject has a shooting behavior of shooting the shooting target.

[0211] In some embodiments, the second processing module 430 is configured to determine the second behavior detection result when both the first similarity and the third similarity are less than a similarity threshold and the second similarity is greater than the similarity threshold, wherein the second behavior detection result indicates that the second probability that the photographing subject has a photographing behavior toward the photographing target is less than a second preset probability threshold;

[0212] When it is determined based on the second behavior detection result that the second probability is less than the second preset probability threshold, the first probability and the second probability are calculated, and the behavior detection result is determined based on the calculation result.

[0213] In some embodiments, the first processing module 420 is used to input the video to be detected into a shooting behavior detection model, and determine the relative position information and the gesture information through the shooting behavior detection model;

[0214] Based on the relative position information and the gesture information, the behavior detection result output by the shooting behavior detection model is obtained.

[0215] In some embodiments, the shooting behavior detection model is trained by the following steps:

[0216] Obtaining a sample detection video of the target detection scene;

[0217] Parsing the sample detection video to obtain a plurality of photographed behavior images and a plurality of non-photographed behavior images;

[0218] Changing the relative position between the area corresponding to the sample shooting target and the area corresponding to the sample shooting object in the shooting behavior image multiple times to obtain multiple sample shooting behavior images corresponding to the shooting behavior image;

[0219] Changing the relative positions of the first area and the second area in the non-photographing behavior image multiple times to obtain a plurality of sample non-photographing behavior images corresponding to the non-photographing behavior image, wherein the first area and the second area are any areas in the non-photographing behavior image, and the first area and the second area are different areas;

[0220] The shooting behavior detection model is trained based on the multiple sample shooting behavior images corresponding to each shooting behavior image and the multiple sample non-shooting behavior images corresponding to each non-shooting behavior image.

[0221] In some embodiments, the first processing module 420 is used to obtain a plurality of images to be detected corresponding to the video to be detected based on the video to be detected;

[0222] Based on the video to be detected, determining first relative position information between the shooting target and the shooting object and first gesture information of the shooting object;

[0223] Based on the plurality of images to be detected, determining second relative position information between the shooting target and the shooting object and second gesture information of the shooting object;

[0224] The behavior detection result is determined based on the first relative position information, the second relative position information, the first gesture information, and the second gesture information.

[0225] The shooting behavior detection device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than the terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0226] The shooting behavior detection device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0227] The shooting behavior detection device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0228] In some embodiments, Figure 5 As shown, an embodiment of the present application also provides an electronic device 500, including a processor 501, a memory 502, and a computer program stored in the memory 502 and executable on the processor 501. When the program is executed by the processor 501, each process of the above-mentioned shooting behavior detection method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0229] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0230] An embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned shooting behavior detection method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0231] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0232] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned shooting behavior detection method when executed by a processor.

[0233] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0234] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned shooting behavior detection method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0235] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0236] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0237] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0238] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0239] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0240] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A shooting behavior detection method, characterized in that: include: Obtain a video to be detected of a target detection scene, where the target detection scene includes a shooting target area and a shooting object area; Based on the video to be detected, determining relative position information between the shooting target in the shooting target area and the shooting object in the shooting object area and gesture information of the shooting object; Based on the relative position information and the gesture information, a behavior detection result of the shooting subject is determined, where the behavior detection result is used to characterize whether the shooting subject has a shooting behavior towards the shooting target.

2. The shooting behavior detection method according to claim 1, characterized in that: After determining the relative position information between the shooting target in the shooting target area and the shooting object in the shooting object area and the gesture information of the shooting object, and before determining the behavior detection result of the shooting object, the method further includes: Determine, based on the relative position information and the gesture information, a first behavior detection result of the photographed subject, where the first behavior detection result is used to represent a first probability that the photographed subject has a photographing behavior toward the photographed target; When the first probability is greater than a first preset probability threshold, determining, based on the video to be detected, a target image corresponding to the photographed target and a photographed image corresponding to the photographing terminal of the photographed object; Determine, based on the image features of the target image and the image features of the captured image, a second behavior detection result of the captured object, wherein the second behavior detection result is used to represent a second probability that the captured object has a capturing behavior of the captured object; A behavior detection result of the photographed object is determined based on the first behavior detection result and the second behavior detection result.

3. The shooting behavior detection method according to claim 2, characterized in that: The captured image includes a first captured image displayed by the capturing terminal and a second captured image corresponding to a capturing field of view of the capturing terminal, and the method includes: A second behavior detection result of the photographed object is determined based on the image features of the target image, the image features of the first photographed image, and the image features of the second photographed image.

4. The shooting behavior detection method according to claim 3, characterized in that: The determining, based on the image features of the target image, the image features of the first captured image, and the image features of the second captured image, a second behavior detection result of the captured object includes: determining a first similarity between the target image and the first captured image based on an image feature of the target image and an image feature of the first captured image; determining a second similarity between the target image and the second captured image based on an image feature of the target image and an image feature of the second captured image; determining a third similarity between the first photographed image and the second photographed image based on an image feature of the first photographed image and an image feature of the second photographed image; The second behavior detection result is determined based on the first similarity, the second similarity, and the third similarity.

5. The shooting behavior detection method according to claim 4, characterized in that: The determining the second behavior detection result based on the first similarity, the second similarity, and the third similarity includes: When at least one of the first similarity and the third similarity is greater than a similarity threshold, determining the second behavior detection result, wherein the second behavior detection result indicates that the second probability that the photographing subject has a photographing behavior toward the photographing target is greater than a second preset probability threshold; When it is determined that the second probability is greater than the second preset probability threshold based on the second behavior detection result, it is determined that the shooting subject has a shooting behavior of shooting the shooting target.

6. The shooting behavior detection method according to claim 4, characterized in that: The determining the second behavior detection result based on the first similarity, the second similarity, and the third similarity includes: When both the first similarity and the third similarity are smaller than a similarity threshold, and the second similarity is greater than the similarity threshold, determining the second behavior detection result, wherein the second behavior detection result indicates that the second probability that the photographing subject has a photographing behavior toward the photographing target is smaller than a second preset probability threshold; When it is determined based on the second behavior detection result that the second probability is less than the second preset probability threshold, the first probability and the second probability are calculated, and the behavior detection result is determined based on the calculation result.

7. The shooting behavior detection method according to any one of claims 1 to 6, characterized in that: The determining, based on the video to be detected, the relative position information between the shooting target in the shooting target area and the shooting object in the shooting object area and the gesture information of the shooting object comprises: Inputting the video to be detected into a shooting behavior detection model, and determining the relative position information and the gesture information through the shooting behavior detection model; Based on the relative position information and the gesture information, a behavior detection result of the photographed object output by the photographing behavior detection model is obtained.

8. The shooting behavior detection method according to claim 7, characterized in that: The shooting behavior detection model is trained by the following steps: Obtaining a sample detection video of the target detection scene; Parsing the sample detection video to obtain a plurality of photographed behavior images and a plurality of non-photographed behavior images; Changing the relative position between the area corresponding to the sample shooting target and the area corresponding to the sample shooting object in the shooting behavior image multiple times to obtain multiple sample shooting behavior images corresponding to the shooting behavior image; Changing the relative positions of the first area and the second area in the non-photographing behavior image multiple times to obtain a plurality of sample non-photographing behavior images corresponding to the non-photographing behavior image, wherein the first area and the second area are any areas in the non-photographing behavior image, and the first area and the second area are different areas; The shooting behavior detection model is trained based on the multiple sample shooting behavior images corresponding to each shooting behavior image and the multiple sample non-shooting behavior images corresponding to each non-shooting behavior image.

9. The shooting behavior detection method according to any one of claims 1 to 6, characterized in that: The determining, based on the video to be detected, the relative position information between the shooting target in the shooting target area and the shooting object in the shooting object area and the gesture information of the shooting object comprises: Based on the video to be detected, obtaining a plurality of images to be detected corresponding to the video to be detected; Based on the video to be detected, determining first relative position information between the shooting target and the shooting object and first gesture information of the shooting object; Based on the plurality of images to be detected, determining second relative position information between the shooting target and the shooting object and second gesture information of the shooting object; A behavior detection result of the photographed object is determined based on the first relative position information, the second relative position information, the first gesture information, and the second gesture information.

10. A shooting behavior detection device, characterized in that: include: An acquisition module is used to acquire a video to be detected of a target detection scene, where the target detection scene includes a shooting target area and a shooting object area; A first processing module, configured to determine, based on the video to be detected, relative position information between the shooting target in the shooting target area and the shooting object in the shooting object area and gesture information of the shooting object; The second processing module is used to determine the behavior detection result of the shooting object based on the relative position information and the gesture information, and the behavior detection result is used to characterize whether the shooting object has a shooting behavior of the shooting target.

Images