A large-screen anti-mis-touch method, device, storage medium and equipment

Abstract

The present application discloses a method, apparatus, storage medium and device for preventing accidental touches on a large screen, which obtains image frames captured by a camera device pre-installed on the large screen. The human body posture is recognized for the character to obtain the plane coordinates of the key points of the face. The plane coordinates of the key points of the face are used as feature variables of the image frame and input into the classification model to obtain the classification results output by the classification model. When it is detected that in a plurality of image frames captured by the camera device within a preset time period, the body orientation of the character is all facing away from the large screen, the operating system is controlled to generate a transparent mask. When it is detected that the operating system has generated a mask, and in a plurality of image frames captured by the camera device within a preset time period, the body orientation of the character is all facing the large screen, the operating system is controlled to close the mask. It can be seen that the solution described in the present application can effectively prevent the operating system from executing useless control instructions due to accidental touches.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction, and particularly to a method, device, storage medium, and equipment for preventing accidental touch on a large screen. Background Art

[0002] The current interaction with a large screen (usually a touch screen) is mainly limited to: the capacitance or infrared signals generated when a user touches the large screen. In practical applications, some contact behaviors of the user do not necessarily expect the operating system to execute control instructions. For example, when a teacher is giving a lecture on the stage, he needs to turn around to face the students and use his finger to touch a certain area (possibly a formula) on the large screen to explain. The intention of the contact behavior is not to edit the content shown on the large screen, but to provide visual guidance to the audience. Therefore, at this time, the contact behavior should not cause the operating system to react and is regarded as an accidental touch behavior. However, the existing large-screen recognition solutions based on capacitance or infrared can only recognize the contact behavior, but cannot recognize the intention of the user when touching, and it is easy to cause the operating system to execute useless control instructions due to accidental touch behaviors.

[0003] Therefore, how to avoid the situation where the operating system executes useless control instructions due to accidental touch behaviors has become an urgent problem to be solved in this field. Summary of the Invention

[0004] This application provides a method, device, storage medium, and equipment for preventing accidental touch on a large screen, aiming to avoid the situation where the operating system executes useless control instructions due to accidental touch behaviors.

[0005] To achieve the above object, this application provides the following technical solutions:

[0006] A method for preventing accidental touch on a large screen includes:

[0007] Obtaining an image frame collected by a camera device pre-installed on the large screen; the image frame is used to indicate a person;

[0008] Performing human pose recognition on the person to obtain the planar coordinates of the facial key points;

[0009] Using the planar coordinates of the facial key points as the feature variables of the image frame and inputting them into a classification model to obtain the classification result output by the classification model; the classification model is trained by using the feature variables of the sample image frame as the input and the body orientation of the person shown in the pre-annotated sample image frame as the training target; the classification result is used to indicate the body orientation of the person shown in the image frame;

[0010] In the case where the body orientations of the person in multiple image frames collected by the imaging device within a preset time period are all facing away from the large screen, control the operating system to generate a transparent overlay layer.

[0011] In the case where it is detected that the operating system has generated the overlay layer and the body orientations of the person in multiple image frames collected by the imaging device within the preset time period are all facing the large screen, control the operating system to close the overlay layer.

[0012] Optionally, the controlling the operating system to generate a transparent overlay layer includes:

[0013] Control the operating system to create a window representing the overlay layer and set the window style of the window to null;

[0014] Control the operating system to adjust the size of the window so that the window covers the large screen;

[0015] Control the operating system to set the background color of the window to transparent;

[0016] Control the operating system to set the Topmost property of the window to true;

[0017] Control the operating system to bind a touch callback event to the window; wherein, the touch callback event is used to intercept the triggering operation of the user on the large screen.

[0018] Optionally, it further includes:

[0019] In the case where the number of persons shown in the image frame is multiple, calculate the position of each person relative to the large screen and the distance between two adjacent persons based on the planar coordinates of the facial key points of each person;

[0020] In the case where it is detected that there is a person satisfying a preset condition among the persons, control the operating system to generate a transparent overlay layer, and adjust the position and area of the overlay layer based on the position of each person relative to the large screen and the distance between two adjacent persons;

[0021] Wherein, the preset condition is that in multiple image frames collected by the imaging device within the preset time period, the body orientations of the person are all facing away from the large screen.

[0022] Optionally, after controlling the operating system to generate a transparent overlay layer, it further includes:

[0023] Record the correspondence between the overlay layer and the person;

[0024] The controlling the operating system to close the mask layer includes:

[0025] The operating system is controlled to close the mask corresponding to the character.

[0026] Optionally, performing body posture recognition on the person to obtain plane coordinates of facial key points includes:

[0027] The image frame is input into a preset human posture recognition model to obtain feature variables output by the human posture recognition model; the feature variables are used to indicate the plane coordinates of each facial key point of the person shown in the image frame.

[0028] Optionally, the classification result includes a first value and a second value, the first value being used to indicate a probability that the person shown in the image frame is facing the large screen, and the second value being used to indicate a probability that the person shown in the image frame is facing away from the large screen;

[0029] The classification result is used to indicate the body orientation of the person shown in the image frame, including:

[0030] When the first value is greater than the second value, determining that the body orientation of the person shown in the image frame is facing the large screen;

[0031] When the first value is not greater than the second value, it is determined that the body orientation of the person shown in the image frame is facing away from the large screen.

[0032] A large screen accidental touch prevention device, comprising:

[0033] An acquisition unit, configured to acquire image frames captured by a camera device pre-installed on a large screen; the image frames are used to indicate a person;

[0034] A recognition unit, configured to perform human posture recognition on the person and obtain plane coordinates of key points on the face;

[0035] a classification unit, configured to input the plane coordinates of the facial key points as feature variables of the image frame into a classification model, and obtain a classification result output by the classification model; the classification model is trained based on the feature variables of the sample image frame as input and the body orientation of the person shown in the pre-labeled sample image frame as a training target; the classification result is used to indicate the body orientation of the person shown in the image frame;

[0036] a first control unit configured to control an operating system to generate a transparent mask when detecting that, in a plurality of image frames captured by the camera device within a preset time period, the person's body is facing away from the large screen;

[0037] A second control unit, configured to control the operating system to close the matte layer when it is detected that the operating system has generated the matte layer and the body orientations of the person in a plurality of the image frames captured within a preset time period of the imaging device are all facing the large screen.

[0038] Optionally, it further includes:

[0039] A third control unit;

[0040] The third control unit is configured to:

[0041] When it is detected that the number of persons shown in the image frame is multiple, calculate the position of each person relative to the large screen and the distance between two adjacent persons based on the planar coordinates of the facial key points of each person;

[0042] When it is detected that there is a person satisfying a preset condition among all the persons, control the operating system to generate a matte layer with a transparent color, and adjust the position and area of the matte layer based on the position of each person relative to the large screen and the distance between two adjacent persons;

[0043] Wherein, the preset condition is that the body orientations of the person in a plurality of the image frames captured within a preset time period of the imaging device are all facing away from the large screen.

[0044] A computer-readable storage medium, the computer-readable storage medium includes a stored program, wherein the program executes the large-screen anti-mis-touch method described above.

[0045] A large-screen anti-mis-touch device, including: a processor, a memory, and a bus; the processor is connected to the memory through the bus;

[0046] The memory is used to store a program, and the processor is used to run the program, wherein the program executes the large-screen anti-mis-touch method when running.

[0047] The technical solution provided by this application obtains an image frame collected by a camera device preset on a large screen, and the image frame is used to indicate a person. Perform human pose recognition on the person to obtain the planar coordinates of the facial key points. Use the planar coordinates of the facial key points as the feature variables of the image frame and input them into a classification model to obtain the classification result output by the classification model. The classification model is trained by using the feature variables of the sample image frame as the input and the body orientation of the person shown in the pre-annotated sample image frame as the training target. The classification result is used to indicate the body orientation of the person shown in the image frame. When it is detected that in multiple image frames collected by the camera device within a preset time period, the body orientation of the person is all facing away from the large screen, control the operating system to generate a transparent overlay. When it is detected that the operating system has generated the overlay and in multiple image frames collected by the camera device within the preset time period, the body orientation of the person is all facing the large screen, control the operating system to close the overlay. When the body orientation of the person is facing away from the large screen, control the operating system to generate a transparent overlay. Since the color of the overlay is transparent, it will not affect the display of the original content on the large screen, and it can also use the overlay to prevent the user from triggering operations on the large screen. It can be seen that by using the solution described in this application, it is possible to effectively avoid the situation where the operating system executes useless control instructions due to accidental touch behavior. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0049] Figure 1a Schematic diagram of a large screen anti-accidental touch method provided by an embodiment of the present application;

[0050] Figure 1b Schematic diagram of a large screen provided by an embodiment of the present application;

[0051] Figure 1c Another schematic diagram of a large screen provided by an embodiment of the present application;

[0052] Figure 1d Schematic diagram of a person's body orientation provided by an embodiment of the present application;

[0053] Figure 1e Another schematic diagram of a person's body orientation provided by an embodiment of the present application;

[0054] Figure 1f Schematic diagram of a sample image frame provided by an embodiment of the present application;

[0055] Figure 1g Another schematic diagram of a sample image frame provided by an embodiment of the present application;

[0056] Figure 1h Another schematic diagram of a sample image frame provided by an embodiment of the present application;

[0057] Figure 1i Another schematic diagram of a sample image frame provided by an embodiment of the present application;

[0058] Figure 2 Another schematic diagram of a large screen anti-mis-touch method provided by an embodiment of the present application;

[0059] Figure 3 Another schematic diagram of the architecture of a large screen anti-mis-touch device provided by an embodiment of the present application. Detailed implementation manners

[0060] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0061] As Figure 1a shown, it is a schematic diagram of a large screen anti-mis-touch method provided by an embodiment of the present application, including the following steps:

[0062] S101: Obtain a sample image frame collected by a camera device pre-set on the large screen.

[0063] Among them, the camera device includes but is not limited to being installed directly above the large screen, and the camera device can capture environmental information in front of the large screen, specifically, for the installation position of the camera device and the environmental information that can be captured, reference can be made to Figure 1b and Figure 1c shown (in Figure 1b and Figure 1c ,"fisheye lens camera" represents the camera device, "large screen" represents the large screen, and "Interactive space" represents the environment). For example, during the entire process from when the user raises their hand to touches the large screen, the field of view angle of the camera device is 180 degrees, and images are captured at a frame rate of 30 frames per second.

[0064] It should be noted that the camera device includes but is not limited to an RGB fisheye camera. In the embodiments of the present application, the sample image frame is used to indicate a person.

[0065] S102: Label the sample image frame with a type identifier.

[0066] Among them, the type identifier is used to indicate the body orientation of the person shown in the sample image frame.

[0067] In the embodiments of the present application, the types of body orientations include facing the large screen directly and facing away from the large screen. Specifically, facing the large screen directly, as Figure 1d shown, facing away from the large screen, as Figure 1e shown.

[0068] S103: Input the sample image frame into a preset human pose recognition model to obtain the feature variables output by the human pose recognition model.

[0069] Among them, the preset human pose recognition model includes but is not limited to the human pose recognition neural network model in the Mediapipe framework (this model has been pre-trained on a wide range of human pose image datasets, so the model parameters provided by the official can be directly used without re-training the model). The human pose recognition neural network model is used to perform human pose recognition on the person shown in the sample image frame and label each facial key point of the person in the sample image frame, that is, output the planar coordinates (including the abscissa and ordinate) of each facial key point of the person.

[0070] In the embodiments of the present application, the number of feature variables can be twenty-two, that is, representing the planar coordinates of eleven facial key points. In other words, the planar coordinates of the facial key points of the person shown in the sample image frame can be regarded as the feature variables of the sample image frame.

[0071] Specifically, if the person shown in the sample image frame faces the large screen directly, as Figure 1f shown, then the recognition result of the facial key points of the person is as Figure 1g shown. If the person shown in the sample image frame faces away from the large screen, as Figure 1h shown, then the recognition result of the facial key points of the person is as Figure 1i shown.

[0072] It should be noted that the above specific implementation process is only for illustrative purposes.

[0073] S104: Use the feature variables of the sample image frame as the input and the body orientation of the person shown in the sample image frame as the training target to train a classification model.

[0074] Among them, the network structure of the classification model includes but is not limited to: VGG19 (usually used as the backbone structure), 16 convolutional layers, and 3 fully connected layers. Specifically, when training the classification model, the Adam optimizer can be used to reduce the cross-entropy of the loss function so as to improve the training efficiency of the classification model.

[0075] S105: When it is detected that the operating system is turned on, obtain the image frames collected by the imaging device at a preset time interval.

[0076] Among them, when the operating system is turned on, it is determined that the large screen has entered the human-computer interaction state. The preset time interval can also be understood as the preset frame rate. In the field of imaging, the frame rate is usually used to represent the time interval.

[0077] In the embodiment of the present application, the image frames are used to indicate a person. The person shown in the image frame can also be understood as the user.

[0078] S106: Input the image frames into the human pose recognition model to obtain the feature variables output by the human pose recognition model.

[0079] S107: Input the feature variables of the image frames into the classification model to obtain the classification results output by the classification model.

[0080] Among them, the classification results include a first value and a second value. The first value is used to indicate the probability that the person shown in the image frame is facing the large screen, and the second value is used to indicate the probability that the person shown in the image frame is facing away from the large screen.

[0081] In the embodiment of the present application, the classification results output by the classification model can be specifically represented by a group of vectors. The vector contains two elements, one element is used to represent the first value, and the other element is used to represent the second value.

[0082] S108: When the first value is greater than the second value, determine that the body orientation of the person shown in the image frame is facing the large screen.

[0083] S109: When the first value is not greater than the second value, determine that the body orientation of the person shown in the image frame is facing away from the large screen.

[0084] Among them, the body orientation types of each image frame can also be stored in a preset queue in the order of the time stamps of the image frames from early to late.

[0085] S110: When it is detected that the number of people shown in the image frame is one, and among the multiple image frames collected by the imaging device within the preset time period, the body orientations of the people are all facing away from the large screen, control the operating system to generate a mask layer with a transparent color and globally topmost, and record the corresponding relationship between the mask layer and the person.

[0086] Among them, the mask layer is used to capture the trigger operation of the user on the large screen. The transparent color of the mask layer will not affect the display of the content in the original window on the large screen. The so-called globally topmost mask layer means that the mask layer can cover the original window to prevent the user from performing control operations on the original window on the large screen.

[0087] It should be noted that the specific implementation steps for controlling the operating system (such as Windows system and Android system, etc.) to set the color of the mask to be transparent and globally set to the top include:

[0088] 1. Control the operating system to create a window for representing the mask and set the window style of the window to empty.

[0089] 2. Control the operating system to adjust the size of the window so that the window covers the large screen.

[0090] 3. Control the operating system to set the background color of the window to transparent.

[0091] 4. Control the operating system to set the Topmost property of the window to true (i.e., to globally place the window on top).

[0092] 5. Control the operating system to bind touch callback events on the window.

[0093] Among them, the touch callback event is used to intercept the user's trigger operation on the large screen.

[0094] It should be emphasized that when the user's back is facing the large screen, the user's behavior of touching the large screen can be understood as a false touch. That is, when the user's back is facing the large screen and touches the large screen, the operating system should not respond. When the user is facing the large screen and touches the large screen, the operating system can execute the corresponding control instructions.

[0095] S111: When it is detected that there are multiple persons in the image frame, the position of each person relative to the large screen and the distance between two adjacent persons are calculated based on the plane coordinates of the facial key points of each person.

[0096] After executing S111 , continue executing S112 .

[0097] S112: When it is detected that there is a character that meets the preset conditions among the various characters, the operating system is controlled to generate a transparent mask, and based on the position of each character relative to the large screen and the distance between two adjacent characters, the position and area of the mask are adjusted, and the correspondence between the mask and the character is recorded.

[0098] The preset condition is that in a plurality of image frames captured by the camera device within a preset time period, the person's body is facing away from the large screen.

[0099] Specifically, assuming that user A and user B are located on the left and right sides of the large screen respectively, and both user A and user B are facing away from the large screen, the operating system is controlled to generate two mask layers, and the area of each mask layer is half of the area of the large screen.

[0100] Furthermore, the correspondence between the mask and the person is independent, and the mask and the person do not interfere with each other. Specifically, suppose user A and user B are located on the left and right sides of the large screen, respectively, with user A facing the screen and user B facing away. The control operating system generates a mask corresponding to user B. This mask is located on the side of the screen that user B is facing, and the area of the mask is half of the large screen area. In this case, user B cannot trigger operations on the side of the screen that they are facing, but user A can still trigger operations on the side of the screen that they are facing.

[0101] It should be noted that the above specific implementation process is only for illustration.

[0102] S113: When it is detected that the operating system has generated a mask layer and that the person's body is facing the large screen in multiple image frames captured by the camera device within a preset time period, the operating system is controlled to close the mask layer corresponding to the person.

[0103] Among them, after controlling the operating system to close the mask corresponding to the character, the area on the large screen not covered by the mask can be restored to normal use, that is, the user can touch the area to make the operating system execute the corresponding control instruction.

[0104] In summary, a camera pre-installed on the large screen captures image frames, performs body posture recognition on the person depicted in the image frames, and uses a classification model to identify the person's body orientation. If the person's body orientation is facing away from the large screen, the operating system is controlled to generate a transparent overlay. Because the overlay is transparent, it does not affect the display of the original content on the large screen and can also prevent the user from triggering the large screen. This demonstrates that the solution described in this embodiment effectively prevents the operating system from executing unnecessary control commands due to accidental touches.

[0105] It should be noted that S101 mentioned in the above embodiment is an optional implementation of the large screen anti-mistouch method described in this application. In addition, S102 mentioned in the above embodiment is also an optional implementation of the large screen anti-mistouch method described in this application. To this end, the process mentioned in the above embodiment can be summarized as follows: Figure 2 The method shown.

[0106] S201: Acquire image frames captured by a camera device preset on a large screen.

[0107] The image frame is used to indicate a person.

[0108] S202: Performing body posture recognition on the person to obtain the plane coordinates of key points on the face.

[0109] S203: The plane coordinates of the facial key points are used as feature variables of the image frame and input into the classification model to obtain the classification result output by the classification model.

[0110] The classification model is trained based on the feature variables of the sample image frames as input and the body orientation of the person shown in the pre-labeled sample image frames as training targets. The classification result is used to indicate the body orientation of the person shown in the image frame.

[0111] S204: When it is detected that the body of the person in the plurality of image frames captured by the camera device within a preset time period is all facing away from the large screen, the operating system is controlled to generate a transparent mask.

[0112] S205: When it is detected that the operating system has generated a mask layer and that the body of the person in a plurality of image frames captured by the camera device within a preset time period is facing the large screen, the operating system is controlled to close the mask layer.

[0113] In summary, a camera pre-installed on the large screen captures image frames, performs body posture recognition on the person depicted in the image frames, and uses a classification model to identify the person's body orientation. If the person's body orientation is facing away from the large screen, the operating system is controlled to generate a transparent overlay. Because the overlay is transparent, it does not affect the display of the original content on the large screen and can also prevent the user from triggering the large screen. This demonstrates that the solution described in this embodiment effectively prevents the operating system from executing unnecessary control commands due to accidental touches.

[0114] Corresponding to the large-screen anti-accidental-touch method provided in the above-mentioned embodiment of the present application, the embodiment of the present application also provides a large-screen anti-accidental-touch device.

[0115] like Figure 3 FIG. 1 is a schematic diagram of the architecture of a large-screen anti-mistouch device provided in an embodiment of the present application, including:

[0116] The acquisition unit 100 is used to acquire image frames captured by a camera device preset on a large screen; the image frames are used to indicate people.

[0117] The recognition unit 200 is used to perform human body posture recognition on a person and obtain the plane coordinates of key points of the face.

[0118] Among them, the recognition unit 200 is specifically used to: input the image frame into a preset human posture recognition model to obtain feature variables output by the human posture recognition model; the feature variables are used to indicate the plane coordinates of each facial key point of the person shown in the image frame.

[0119] Classification unit 300 is used to input the planar coordinates of facial key points as feature variables of an image frame into a classification model to obtain a classification result output by the classification model. The classification model is trained by using the feature variables of sample image frames as inputs and the body orientation of the person shown in the pre-annotated sample image frames as the training target. The classification result is used to indicate the body orientation of the person in the image frame. Among them, the classification result includes a first value and a second value. The first value is used to indicate the probability that the person shown in the image frame is facing the large screen directly, and the second value is used to indicate the probability that the person shown in the image frame has their back to the large screen. In the case where the first value is greater than the second value, it is determined that the body orientation of the person shown in the image frame is facing the large screen directly; in the case where the first value is not greater than the second value, it is determined that the body orientation of the person shown in the image frame is with their back to the large screen.

[0120] The first control unit 400 is used to control the operating system to generate a transparent overlay when it is detected that in multiple image frames captured by the camera device within a preset time period, the body orientation of the person is with their back to the large screen.

[0121] Among them, the process by which the first control unit 400 is used to control the operating system to generate a transparent overlay includes: controlling the operating system to create a window for representing the overlay and setting the window style of the window to be empty; controlling the operating system to adjust the size of the window so that the window covers the large screen; controlling the operating system to set the background color of the window to be transparent; controlling the operating system to set the Topmost attribute of the window to true; controlling the operating system to bind a touch callback event to the window; where the touch callback event is used to intercept the triggering operation of the user on the large screen.

[0122] The second control unit 500 is used to control the operating system to close the overlay when it is detected that the operating system has generated the overlay and in multiple image frames captured by the camera device within a preset time period, the body orientation of the person is facing the large screen directly.

[0123] The third control unit 600 is used to calculate the position of each person relative to the large screen and the distance between adjacent two people based on the planar coordinates of the facial key points of each person when it is detected that the number of people shown in the image frame is multiple; when it is detected that there is a person satisfying a preset condition among each person, control the operating system to generate a transparent overlay and adjust the position and area of the overlay based on the position of each person relative to the large screen and the distance between adjacent two people; where the preset condition is that in multiple image frames captured by the camera device within a preset time period, the body orientation of the person is with their back to the large screen.

[0124] A recording unit 700 for recording the correspondence between the mask layer and the person.

[0125] Correspondingly, the process of the second control unit 500 for controlling the operating system to close the mask layer includes: controlling the operating system to close the mask layer corresponding to the person.

[0126] In summary, an image frame is collected by a camera device pre-set on the large screen, human pose recognition is performed on the person shown in the image frame, and the body orientation of the person shown in the image frame is recognized by using a classification model. When the body orientation of the person is facing away from the large screen, the operating system is controlled to generate a mask layer with a transparent color. Since the color of the mask layer is transparent, it will not affect the display of the original content on the large screen, and it can also use the mask layer to prevent the user from triggering operations on the large screen. It can be seen that by using the solution described in this embodiment, the situation where the operating system executes useless control instructions due to accidental touch behavior can be effectively avoided.

[0127] The present application also provides a computer-readable storage medium. The computer-readable storage medium includes a stored program, wherein the program executes the large-screen anti-accidental touch method provided by the present application.

[0128] The present application also provides a large-screen anti-accidental touch device, including: a processor, a memory, and a bus. The processor is connected to the memory through the bus. The memory is used to store the program, and the processor is used to run the program. When the program runs, it executes the large-screen anti-accidental touch method provided by the present application, including the following steps:

[0129] Obtain the image frame collected by the camera device pre-set on the large screen; the image frame is used to indicate a person;

[0130] Perform human pose recognition on the person to obtain the planar coordinates of the facial key points;

[0131] Use the planar coordinates of the facial key points as the feature variables of the image frame and input them into the classification model to obtain the classification result output by the classification model; the classification model is trained based on using the feature variables of the sample image frame as the input and using the pre-annotated body orientation of the person shown in the sample image frame as the training target; the classification result is used to indicate the body orientation of the person shown in the image frame;

[0132] When it is detected that the body orientation of the person in multiple image frames collected by the camera device within a preset time period is facing away from the large screen, control the operating system to generate a mask layer with a transparent color;

[0133] When it is detected that the operating system has generated the overlay, and in multiple image frames captured by the imaging device within a preset time period, the body orientation of the person is facing the large screen, control the operating system to close the overlay.

[0134] Optionally, controlling the operating system to generate an overlay with a transparent color includes:

[0135] Control the operating system to create a window for representing the overlay, and set the window style of the window to null;

[0136] Control the operating system to adjust the size of the window so that the window covers the large screen;

[0137] Control the operating system to set the background color of the window to transparent;

[0138] Control the operating system to set the Topmost property of the window to true;

[0139] Control the operating system to bind a touch callback event to the window; wherein, the touch callback event is used to intercept the triggering operation of the user on the large screen.

[0140] Optionally, it further includes:

[0141] When it is detected that the number of people shown in the image frame is multiple, based on the planar coordinates of the facial key points of each person, calculate the position of each person relative to the large screen and the distance between two adjacent people;

[0142] When it is detected that there is a person satisfying a preset condition among each person, control the operating system to generate an overlay with a transparent color, and adjust the position and area of the overlay based on the position of each person relative to the large screen and the distance between two adjacent people;

[0143] Wherein, the preset condition is: in multiple image frames captured by the imaging device within a preset time period, the body orientation of the person is facing away from the large screen.

[0144] Optionally, after controlling the operating system to generate an overlay with a transparent color, it further includes:

[0145] Record the correspondence between the overlay and the person;

[0146] Controlling the operating system to close the overlay includes:

[0147] Control the operating system to close the overlay corresponding to the person.

[0148] Optionally, performing human pose recognition on the person to obtain the planar coordinates of the facial key points includes:

[0149] Inputting the image frame into a preset human pose recognition model to obtain the feature variables output by the human pose recognition model; the feature variables are used to indicate the planar coordinates of each facial key point of the person shown in the image frame.

[0150] Optionally, the classification result includes a first value and a second value. The first value is used to indicate the probability that the person shown in the image frame is facing the large screen, and the second value is used to indicate the probability that the person shown in the image frame is facing away from the large screen;

[0151] The classification result is used to indicate the body orientation of the person shown in the image frame, including:

[0152] In the case where the first value is greater than the second value, determining that the body orientation of the person shown in the image frame is facing the large screen;

[0153] In the case where the first value is not greater than the second value, determining that the body orientation of the person shown in the image frame is facing away from the large screen.

[0154] If the functions described in the method embodiments of this application are implemented in the form of software function units and sold or used as independent products, they can be stored in a storage medium readable by a computing device. Based on this understanding, the part that contributes to the prior art or part of the technical solution in the embodiments of this application can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions for causing a computing device (which may be a personal computer, a server, a mobile computing device, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

[0155] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other.

[0156] The foregoing description of the disclosed embodiments enables those skilled in the art to practice or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A large-screen anti-mis-touch method, characterized in that, Including: Obtaining an image frame collected by a camera device preset on a large screen; The image frame is used to indicate a person; Performing human pose recognition on the person to obtain the planar coordinates of facial key points; Using the planar coordinates of the facial key points as feature variables of the image frame and inputting them into a classification model to obtain a classification result output by the classification model; The classification model is trained by using the feature variables of the sample image frame as input and the body orientation of the person shown in the pre-annotated sample image frame as the training target; The classification result is used to indicate the body orientation of the person shown in the image frame; When it is detected that in multiple image frames collected by the camera device within a preset time period, the body orientation of the person is always facing away from the large screen, controlling the operating system to generate a transparent mask layer; When it is detected that the operating system has generated the mask layer and in multiple image frames collected by the camera device within the preset time period, the body orientation of the person is always facing the large screen, controlling the operating system to close the mask layer, where the mask layer is used to capture the trigger operation of the user on the large screen; When it is detected that the number of persons shown in the image frame is multiple, calculating the position of each person relative to the large screen and the distance between two adjacent persons based on the planar coordinates of the facial key points of each person; When it is detected that there is a person satisfying a preset condition among all the persons, controlling the operating system to generate a transparent mask layer and adjusting the position and area of the mask layer based on the position of each person relative to the large screen and the distance between two adjacent persons; Wherein, the preset condition is that in multiple image frames collected by the camera device within the preset time period, the body orientation of the person is always facing away from the large screen.

2. The method according to claim 1, wherein The controlling the operating system to generate a transparent mask layer includes: Controlling the operating system to create a window for representing the mask layer and setting the window style of the window to be empty; Controlling the operating system to adjust the size of the window so that the window covers the large screen; Controlling the operating system to set the background color of the window to be transparent; Controlling the operating system to set the Topmost attribute of the window to true; Controlling the operating system to bind a touch callback event to the window; wherein, the touch callback event is used to intercept the trigger operation of the user on the large screen.

3. The method according to claim 1, characterized in that, After controlling the operating system to generate a transparent mask layer, it further includes: Recording the correspondence between the mask layer and the person; The controlling the operating system to close the mask layer includes: Controlling the operating system to close the mask layer corresponding to the person.

4. The method according to claim 1, wherein The performing human pose recognition on the person to obtain the planar coordinates of facial key points includes: The image frame is input into a preset human posture recognition model to obtain feature variables output by the human posture recognition model; the feature variables are used to indicate the plane coordinates of each facial key point of the person shown in the image frame.

5. The method according to claim 1, wherein The classification result includes a first value and a second value, the first value is used to indicate the probability that the person shown in the image frame is facing the large screen, and the second value is used to indicate the probability that the person shown in the image frame is facing away from the large screen; The classification result is used to indicate the body orientation of the person shown in the image frame, including: When the first value is greater than the second value, determining that the body orientation of the person shown in the image frame is facing the large screen; When the first value is not greater than the second value, it is determined that the body orientation of the person shown in the image frame is facing away from the large screen.

6. A large-screen anti-mis-touch device, characterized in that, include: An acquisition unit, used for acquiring image frames captured by a camera device preset on the large screen; The image frame is used to indicate a person; A recognition unit, configured to perform body posture recognition on the person to obtain the plane coordinates of key points on the face; a classification unit, configured to input the plane coordinates of the facial key points as feature variables of the image frame into a classification model, and obtain a classification result output by the classification model; The classification model is trained based on taking the feature variables of the sample image frames as input and the body orientation of the person shown in the pre-labeled sample image frames as training targets; The classification result is used to indicate the body orientation of the person shown in the image frame; a first control unit configured to control an operating system to generate a transparent mask when detecting that, in a plurality of image frames captured by the camera device within a preset time period, the person's body is facing away from the large screen; a second control unit, configured to control the operating system to close the mask layer when it is detected that the operating system has generated the mask layer and that, in a plurality of image frames captured by the camera device within a preset time period, the body orientation of the person is facing the large screen, wherein the mask layer is used to capture a user's triggering operation on the large screen; The third control unit is used to: when it is detected that there are multiple people shown in the image frame, calculate the position of each person relative to the large screen and the distance between two adjacent people based on the plane coordinates of the facial key points of each person; when it is detected that there is a person who meets the preset conditions among the people, control the operating system to generate a transparent mask, and adjust the position and area of the mask based on the position of each person relative to the large screen and the distance between two adjacent people; wherein the preset condition is: in the multiple image frames captured within the preset time period of the camera device, the body direction of the person is all facing away from the large screen.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program executes the large-screen accidental touch prevention method according to any one of claims 1 to 5.

8. A large-screen anti-mis-touch device, characterized in that, include: processor, memory, and bus; The processor is connected to the memory via the bus; The memory is used to store a program, and the processor is used to run the program. When the program runs, it executes the large-screen anti-mis-touch method according to any one of claims 1-5.

Images