Video recognition-based device control method, apparatus and system

Abstract

The present invention relates to technical fields such as video processing, intelligent control, artificial intelligence, digital culture, and human-machine interaction, and provides a video recognition-based device control method, apparatus and system. The method comprises: performing character behavior analysis on video content currently viewed by a user, so as to obtain different behavior types of video characters in the current video content; and automatically controlling the working mode of a controlled device on the basis of the different behavior types of the video characters in the current video content, so that the controlled device acts on the user in different working modes, thereby establishing the degree of association of video character behaviors, the controlled device, and controlled device use experience. Thus, the degree of intelligence of intelligent device control is improved, and the user experience of the intelligent device is improved.

Description

Device control method, device and system based on video recognition TECHNICAL FIELD

[0001] The present application relates to the technical fields of video processing, intelligent control, artificial intelligence, digital culture, human-computer interaction, and the like, and in particular to a device control method, device and system based on video recognition. BACKGROUND

[0002] With the development of information technology, many service-type intelligent devices can act on the user's body through certain control methods to bring different use experiences to the user. Generally, these control methods mainly include key control, touch screen control, remote control, mobile phone APP control, etc. For example, a massager can control its vibration frequency through a key, can control its vibration frequency through a touch screen, can control its vibration frequency through a remote controller, and can control its vibration frequency through a mobile phone APP. Overall, these existing control methods require user operation interaction, and the controlled device can only trigger the corresponding state and function. For example, when the user needs the device to work in a certain state to achieve a certain function, the user needs to input a control instruction through a key or an APP interface. Therefore, the control method of the existing service-type intelligent device has a high degree of dependence on user operation, and the intelligent degree is low. When the user uses the device, the use experience needs to be improved.

[0003] In summary, in the control technology of the existing service-type intelligent device, the intelligent degree needs to be improved, and the user use experience needs to be improved. SUMMARY

[0004] In view of the above problems in the prior art, the present application provides a device control method, device and system based on video recognition to improve the intelligent degree of intelligent device control and improve the user use experience of the intelligent device.

[0005] In a first aspect, the present application provides a device control method based on video recognition, comprising the following steps:

[0006] Performing character behavior analysis on the video content currently watched by the user to obtain different behavior types of the video character in the current video content;

[0007] According to the different behavior types of the video character in the current video content, automatically controlling the working mode of the controlled device to make the controlled device act on the user in different working modes, and establishing the correlation degree of the video character behavior, the controlled device and the use experience of the controlled device.

[0008] In a second aspect, the present application provides a device control method based on video recognition, comprising the following steps:

[0009] According to the video content currently selected by the user, playing and respectively communicating with the server and the controlled device; the server analyzes the character behavior of the video content currently watched by the user to obtain different behavior types of the video character in the current video content; the server transmits a control signal for controlling different working modes of the controlled device to the user terminal according to the different behavior types of the video character in the current video content.

[0010] Receiving the control signal transmitted by the server, and sending the received control signal to the controlled device to make the controlled device act on the user in different working modes, and establishing the correlation degree of the video character behavior, the controlled device and the use experience of the controlled device.

[0011] In a third aspect, the application provides a device control apparatus based on video recognition, comprising:

[0012] A video character behavior analysis module is configured to analyze the character behavior of the video content currently watched by the user to obtain different behavior types of the video character in the current video content;

[0013] A device mode automatic control module is configured to automatically control the working mode of the controlled device according to the different behavior types of the video character in the current video content, so that the controlled device acts on the user in different working modes, and the correlation degree of the video character behavior, the controlled device and the use experience of the controlled device is established.

[0014] In a fourth aspect, the application provides a device control system based on video recognition, comprising:

[0015] A user terminal is configured to allow the user to select video content for watching;

[0016] A server is in communication connection with the user terminal, and runs a computer program to realize the device control method based on video recognition as described above, and transmits a control signal for controlling the working mode of the controlled device to the user terminal;

[0017] A controlled device is in communication connection with the user terminal, and receives the control signal obtained by the user terminal to work in the corresponding working mode.

[0018] Compared with the prior art, the application has the following beneficial effects:

[0019] The application provides a device control method, device and system based on video recognition, which analyzes the behavior of a character in a video content currently watched by a user to obtain different behavior types of the character in the video content, automatically controls the working mode of a controlled device according to the different behavior types of the character in the video content, and makes the controlled device work in different working modes to affect the user, so as to improve the intelligent degree of the intelligent device control and the user experience of the intelligent device. BRIEF DESCRIPTION OF DRAWINGS

[0020] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings, which are not necessarily drawn to scale, like reference numerals describe similar components throughout the several views. It should be appreciated that the illustrative embodiments provide examples of the various embodiments of the application and do not limit the scope of the application. There can be many variations made to the embodiments described herein without departing from the spirit and scope of the application.

[0021] Fig. 1 is a flowchart of a device control method based on video recognition according to the application;

[0022] Fig. 2 is another flowchart of a device control method based on video recognition according to the application;

[0023] Fig. 3 is a schematic diagram of an architecture of a device control method based on video recognition according to the application;

[0024] Fig. 4 is a schematic diagram of an architecture of a server according to the application;

[0025] Fig. 5 is a schematic diagram of an architecture of a user terminal according to the application;

[0026] Fig. 6 is a schematic diagram of an architecture of a device control system based on video recognition according to the application. DETAILED DESCRIPTION

[0027] In order to make the technical personnel in the art better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor should be within the scope of protection of the present application.

[0028] Embodiment one

[0029] Referring to FIG. 1 and FIG. 4, the embodiment provides a device control method based on video recognition, comprising steps S101 and S102. Wherein, the steps S101 and S102 can be run on a server, by analyzing the character behavior of the current video content watched by the user, to obtain different behavior types of the video character in the current video content, automatically controlling the working mode of the controlled device according to the different behavior types of the video character in the current video content, so that the controlled device acts on the user in different working modes, establishing the correlation degree of the video character behavior, the controlled device and the use experience of the controlled device, thereby improving the intelligent degree of intelligent device control and improving the user use experience of the intelligent device.

[0030] It should be noted that the server includes a memory, a processor and a network interface which are connected to each other through a system bus. Wherein, the server herein is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions, which hardware includes but is not limited to microprocessor, application specific integrated circuit (ASIC), field-programmable gate array (FPGA), digital signal processor (DSP), embedded device, etc. The server can interact with the user through keyboard, mouse, remote controller, touchpad or voice control device, etc. The memory includes at least one type of readable storage medium, which includes flash memory, hard disk, multimedia card, card type memory (such as SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory can be an internal storage unit of the server, such as the hard disk or memory of the server. In other embodiments, the memory can also be an external storage device of the server, such as the plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the server. Of course, the memory can include both the internal storage unit and the external storage device of the server.

[0031] Step S101, performing character behavior analysis on the video content currently watched by the user to obtain different behavior types of the video character in the current video content. Wherein, the character behavior analysis on the video content currently watched by the user to obtain different behavior types of the video character in the current video content can include: obtaining the video content currently watched by the user, and performing image frame extraction on the video content; performing character behavior recognition on the image content of each extracted frame to obtain different behavior types of the video character in the current video content. Further, the character behavior recognition on the image content of each extracted frame to obtain different behavior types of the video character in the current video content can include: inputting the image content of each extracted frame into an AI character behavior recognition model; performing character behavior recognition on the image content of each extracted frame by the AI character behavior recognition model to obtain different behavior types of the video character in the current video content. Further, the AI character behavior recognition model includes any one of convolutional neural network, recurrent neural network, long short-term memory network, graph convolutional network and spatio-temporal graph convolutional network.

[0032] In some preferred embodiments, the character behavior analysis on the video content currently watched by the user to obtain different behavior types of the video character in the current video content can include: the user end user selects the video content for watching, and the server synchronously performs character behavior analysis on the video content currently watched by the user to obtain different behavior types of the video character in the current video content. Further, the user end user can search the video content provided by the video service platform for watching through the video recognition module running on the user end, and the server of the video recognition module synchronously performs character behavior analysis on the video content currently watched by the user to obtain different behavior types of the video character in the current video content.

[0033] In further some preferred embodiments, the video recognition module is embedded in a browser, the user end user searches the video content provided by the video service platform for watching through the browser, and the video recognition module synchronously uploads the video content currently watched to the server of the video recognition module for character behavior analysis of the video content after recognizing the video content currently watched by the user. Further, the video recognition module supports multiple ways of video search and recognition; it can search and recognize the video content provided by the video service platform, and also can search and recognize the local video of the user. It can be understood that the video recognition module can support multiple video search and video upload of the user, for example, the video recognition module embeds a search engine to support online video search and local uploaded video search.

[0034] It should be noted that in the embodiment, the user can directly browse and watch the video in the video recognition module, without switching to other platforms or applications, thereby improving user experience and operation convenience. By real-time recognition of the video content watched by the video recognition module and sending the information to the server, the behavior of the characters in the video content can be analyzed and processed in real time, and different behavior types of the video characters in the current video content are obtained, which are used for automatically controlling the working mode of the controlled device.

[0035] In step S102, according to the different behavior types of the video characters in the current video content, the working mode of the controlled device is automatically controlled to make the controlled device act on the user in different working modes, and the correlation between the behavior of the video characters, the controlled device and the use experience of the controlled device is established.

[0036] It should be noted that the controlled device acting on the user in different working modes can include direct physical contact with the user's body, and can also include physical contact with the user's body, and can also include non-physical contact with the user's body. In some preferred embodiments, the controlled device includes a massager; the working mode of the massager includes reciprocating motion mode and / or rotating motion mode. In some other preferred embodiments, the controlled device can include a massager; the working mode of the massager can include different vibration frequency modes and / or different vibration intensity modes. It should be noted that the working mode of the massager can include multi-frequency vibration mode, single-frequency vibration mode, pulse mode, kneading mode, air pressure mode and heating mode, etc.

[0037] In further some preferred embodiments, according to the different behavior types of the video characters in the current video content, the working mode of the controlled device is automatically controlled, comprising:

[0038] Creating a database of different working modes of the controlled device and different behaviors of the video characters; the different behaviors of the video characters are correspondingly associated with the different working modes of the controlled device;

[0039] After analyzing and identifying the different behavior types of the video characters in the current video content, the different working modes of the controlled device corresponding to the different behaviors of the video characters are queried, and the corresponding control signals are sent to automatically control the working mode of the controlled device according to the different working modes queried.

[0040] It should be noted that in the embodiment, the video content is synchronized with the device behavior in the actual environment of the user, thereby enhancing the immersion and interactive experience of the user. The video content is automatically identified and the working mode of the device is adjusted, reducing the manual operation of the user and improving the intelligent level of the device control.

[0041] In some preferred embodiments, the character behavior analysis on the video content currently watched by the user to obtain different behavior types of the video character in the current video content can comprise: obtaining the video content currently watched by the user, and performing video segment extraction on the video content; and performing character behavior recognition on the content of each video segment extracted to obtain different behavior types of the video character in the current video content.

[0042] It should be noted that the character behavior in the video content can be persistent or repetitive, and the use of a single video frame extraction can cause data processing to be tedious and not conducive to the conservation of data processing resources and transmission resources. In the present embodiment, the video content is extracted into video segments, and the content of each video segment extracted is subjected to character behavior recognition to obtain different behavior types of the video character in the current video content, thereby saving data processing resources and data transmission resources.

[0043] In some preferred embodiments, the character behavior analysis on the video content currently watched by the user to obtain different behavior types of the video character in the current video content can comprise extracting audio in the video for analysis to obtain different behavior types of the video character in the current video content. Preferably, the analysis of the audio in the video can comprise analysis of parameters such as the volume, frequency, bit rate, sampling rate, amplitude, waveform, audio format, duration, dynamic range, signal-to-noise ratio (SNR), total harmonic distortion (THD), and number of channels of the audio. Further, the character behavior analysis on the video content currently watched by the user to obtain different behavior types of the video character in the current video content can comprise extracting audio in the video for analysis to obtain different behavior types of the video character in the current video content while performing image frame extraction analysis on the video content currently watched by the user. It should be noted that in the present embodiment, image and audio analysis is performed on the video content currently watched by the user, so that more accurate different behavior types of the video character in the current video content can be obtained.

[0044] Embodiment Two

[0045] Referring to FIG. 2 and FIG. 5, the embodiment provides a device control method based on video recognition, comprising steps S201 and S202. Wherein, the steps S201 and S202 can be run on the user side, for example, on the user's smart phone side, by playing according to the video content currently selected by the user, and being respectively connected with the server and the controlled device, the server analyzes the character behavior of the video content currently watched by the user to obtain different behavior types of the video character in the current video content, the server transmits the control signal of different working modes of the controlled device to the user side according to the different behavior types of the video character in the current video content, and then sends the received control signal to the controlled device through the control signal transmitted by the server, so that the controlled device acts on the user in different working modes, establishes the correlation degree of the video character behavior, the controlled device and the use experience of the controlled device, thereby improving the intelligent degree of intelligent device control and improving the user use experience of the intelligent device.

[0046] It should be noted that the user side includes a memory, a processor and a network interface which are connected with each other through a system bus. Wherein, the user side is a device capable of automatically performing numerical calculation and / or information processing according to pre-set or stored instructions, which hardware includes but is not limited to microprocessor, application specific integrated circuit (ASIC), field-programmable gate array (FPGA), digital signal processor (DSP), embedded device, etc. The user side can interact with the user through keyboard, mouse, remote controller, touchpad or voice control device, etc. The memory includes at least one type of readable storage medium, which includes flash memory, hard disk, multimedia card, card type memory (such as SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), programmable read only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory can be the internal storage unit of the user side, such as the hard disk or memory of the user side. In other embodiments, the memory can also be the external storage device of the user side, such as the plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on the user side. Of course, the memory can include both the internal storage unit and the external storage device of the user side.

[0047] It should be noted that the character behavior analysis of the video content currently watched by the user to obtain different behavior types of the video character in the current video content can include: obtaining the video content currently watched by the user, and performing image frame extraction on the video content; performing character behavior recognition on the image content of each extracted frame to obtain different behavior types of the video character in the current video content. Further, the character behavior recognition on the image content of each extracted frame to obtain different behavior types of the video character in the current video content can include: inputting the image content of each extracted frame into an AI character behavior recognition model; performing character behavior recognition on the image content of each extracted frame by the AI character behavior recognition model to obtain different behavior types of the video character in the current video content. Further, the AI character behavior recognition model includes any one of a convolutional neural network, a recurrent neural network, a long short-term memory network, a graph convolutional network, and a spatio-temporal graph convolutional network.

[0048] It should be noted that the user end user can search for video content provided by the video service platform for viewing through the video recognition module running on the user end, and the server of the video recognition module synchronously analyzes the character behavior of the video content currently watched by the user to obtain different behavior types of the video character in the current video content.

[0049] It should be noted that the controlled device acting on the user in different working modes can include direct physical contact with the user's body, and can also include physical contact with the user's body, and can also include non-physical contact with the user's body. In some preferred embodiments, the controlled device includes a massager; the working mode of the massager includes a reciprocating motion mode and / or a rotating motion mode. In some other preferred embodiments, the controlled device can include a massager; the working mode of the massager can include different vibration frequency modes and / or different vibration intensity modes.

[0050] In further preferred embodiments, the video recognition module is embedded in a browser and a video recognition module, and the user end user searches for video content provided by the video service platform for viewing through the browser, and the video recognition module synchronously synchronizes the video content currently watched by the user to the server of the video recognition module after recognizing the video content currently watched by the user, to perform character behavior analysis on the video content.

[0051] It should be noted that in the embodiment, the user can directly browse and watch the video in the video recognition module, without switching to other platforms or applications, thereby improving user experience and operation convenience. By identifying the video content in real time through the video recognition module and sending the information to the server, the character behavior of the video content can be analyzed and processed in real time, and different behavior types of the video character in the current video content are obtained, which can be used to automatically control the working mode of the controlled device.

[0052] In further preferred embodiments, the character behavior analysis on the video content currently watched by the user to obtain different behavior types of the video character in the current video content can include: obtaining the video content currently watched by the user, and extracting video clips from the video content; and performing character behavior recognition on the content of each extracted video clip to obtain different behavior types of the video character in the current video content.

[0053] It should be noted that the character behavior in the video content can be persistent or repetitive, and the use of a single video frame extraction will cause data processing to be tedious and not conducive to the conservation of data processing resources and transmission resources. In the embodiment, the video content is extracted into video clips, and the content of each extracted video clip is subjected to character behavior recognition to obtain different behavior types of the video character in the current video content, thereby saving data processing resources and data transmission resources.

[0054] It should be noted that the user has multiple video content selection methods in the user terminal. For example, the user terminal video recognition module is embedded with a video player to support local files and network streaming media videos. For example, the user terminal video recognition module is embedded with a browser to support searching for HTML5 video playing web pages, such as YouTube, Vimeo, and various videos. For example, the user terminal video recognition module can integrate third-party video SDKs to provide rich functions such as on-demand, live broadcast, and advertisement preview. For example, the video recognition module can also support live streaming media videos. For example, the video recognition module can also support picture-in-picture mode when playing a video. In addition, the video recognition module can support video playing methods such as screen projection playing, DLNA projection, and video file sharing.

[0055] Embodiment Three

[0056] Referring to FIG. 3, the embodiment provides a device control apparatus based on video recognition, comprising:

[0057] a video character behavior analysis module configured to analyze the character behavior of the video content currently watched by the user to obtain different behavior types of the video character in the current video content;

[0058] The device mode automatic control module is configured to automatically control the working mode of the controlled device according to different behavior types of the video character in the current video content, so that the controlled device acts on the user in different working modes, and establishes the correlation degree of the video character behavior, the controlled device, and the controlled device use experience.

[0059] It should be noted that in the present embodiment, the character behavior analysis is performed on the video content currently watched by the user to obtain different behavior types of the video character in the current video content, and the working mode of the controlled device is automatically controlled according to the different behavior types of the video character in the current video content, so that the controlled device acts on the user in different working modes, and the correlation degree of the video character behavior, the controlled device, and the controlled device use experience is established, thereby improving the intelligent degree of intelligent device control and improving the user use experience of the intelligent device.

[0060] It should be noted that the character behavior analysis is performed on the video content currently watched by the user to obtain different behavior types of the video character in the current video content, which can include: obtaining the video content currently watched by the user, and performing image frame extraction on the video content; performing character behavior recognition on the image content of each frame extracted to obtain different behavior types of the video character in the current video content. Further, the character behavior recognition on the image content of each frame extracted to obtain different behavior types of the video character in the current video content can include: inputting the image content of each frame extracted into an AI character behavior recognition model; performing character behavior recognition on the image content of each frame extracted by the AI character behavior recognition model to obtain different behavior types of the video character in the current video content. Further, the AI character behavior recognition model includes any one of a convolutional neural network, a recurrent neural network, a long short-term memory network, a graph convolution network, and a spatio-temporal graph convolution network.

[0061] It should be noted that for the video content currently watched by the user, the user can perform video search through a video recognition module running on the user terminal. The server of the video recognition module synchronously performs character behavior analysis on the video content currently watched by the user to obtain different behavior types of the video character in the current video content.

[0062] It should be noted that the controlled device acting on the user in different working modes can include the action of force in direct physical contact with the user's body, can also include the action of heat in physical contact with the user's body, and can also include the action of sound waves that are not in physical contact with the user's body. In some preferred embodiments, the controlled device includes a massager; the working mode of the massager includes a reciprocating motion mode and / or a rotating motion mode. In other preferred embodiments, the controlled device can include a massager; the working mode of the massager can include different vibration frequency modes and / or different vibration intensity modes.

[0063] Embodiment Four

[0064] Referring to FIG. 6, the present embodiment provides a video recognition-based device control system, comprising:

[0065] a user terminal; a user selects video content for viewing through the user terminal;

[0066] a server, in communication connection with the user terminal, running a computer program to implement the video recognition-based device control method of any one of Embodiment One, and transmitting a control signal for controlling the working mode of the controlled device to the user terminal;

[0067] a controlled device, in communication connection with the user terminal, receiving the control signal obtained by the user terminal to work in the corresponding working mode.

[0068] It should be noted that in the present embodiment, the server running a computer program to implement the video recognition-based device control method of any one of Embodiment One can transmit a control signal for controlling the working mode of the controlled device to the user terminal. Among them, the server can analyze the behavior of the characters in the video content currently watched by the user to obtain different behavior types of the video characters in the current video content, automatically control the working mode of the controlled device according to the different behavior types of the video characters in the current video content, so that the controlled device acts on the user in different working modes, establishes the correlation degree of video character behavior, controlled device and controlled device use experience, thereby improving the intelligent degree of intelligent device control and improving the user experience of intelligent device.

[0069] It should be noted that the character behavior analysis of the video content currently watched by the user to obtain different behavior types of the video character in the current video content can include: obtaining the video content currently watched by the user, and performing image frame extraction on the video content; performing character behavior recognition on the image content of each extracted frame to obtain different behavior types of the video character in the current video content. Further, the character behavior recognition on the image content of each extracted frame to obtain different behavior types of the video character in the current video content can include: inputting the image content of each extracted frame into an AI character behavior recognition model; performing character behavior recognition on the image content of each extracted frame by the AI character behavior recognition model to obtain different behavior types of the video character in the current video content. Further, the AI character behavior recognition model includes any one of a convolutional neural network, a recurrent neural network, a long short-term memory network, a graph convolutional network, and a spatio-temporal graph convolutional network.

[0070] It should be noted that for the video content currently watched by the user, the user can obtain the video content through the video recognition module running on the user terminal. The server of the video recognition module synchronously performs character behavior analysis on the video content currently watched by the user to obtain different behavior types of the video character in the current video content.

[0071] It should be noted that the controlled device acting on the user in different working modes can include direct physical contact with the user's body, and can also include physical contact with the user's body, and can also include non-physical contact with the user's body. In some preferred embodiments, the controlled device includes a massager; the working mode of the massager includes a reciprocating motion mode and / or a rotating motion mode. In some other preferred embodiments, the controlled device can include a massager; the working mode of the massager can include different vibration frequency modes and / or different vibration intensity modes.

[0072] It should be noted that the user has multiple video content selection methods in the user terminal. For example, the user terminal video recognition module embeds a video player to support local files and network streaming videos. For example, the user terminal video recognition module embeds a browser to support searching for HTML5 video playing web pages such as YouTube, Vimeo, and various videos. For example, the user terminal video recognition module can integrate third-party video SDKs to provide rich functions such as on-demand, live broadcast, and advertisement preview. For example, the video recognition module can also support live streaming video. For example, the video recognition module can also support picture-in-picture mode when playing a video.

[0073] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A device control method based on video recognition, characterized in that: The following steps are involved: Performing character behavior analysis on the video content currently watched by the user to obtain different behavior types of the video characters in the current video content; According to different behavior types of the video characters in the current video content, the working mode of the controlled device is automatically controlled so that the controlled device acts on the user in different working modes, and the correlation between the video character behavior, the controlled device and the user experience of the controlled device is established.

2. The device control method based on video recognition according to claim 1, characterized in that: Perform character behavior analysis on the video content currently being watched by the user to obtain different behavior types of the characters in the current video content, including: Obtain the video content currently being watched by the user, and extract image frames from the video content; The extracted image content of each frame is used to perform character behavior recognition to obtain different behavior types of the video characters in the current video content.

3. The device control method based on video recognition according to claim 2, characterized in that: Perform character behavior recognition on the extracted image content of each frame to obtain different behavior types of the video characters in the current video content, including: Input the extracted image content of each frame into the AI ​​character behavior recognition model; The AI ​​character behavior recognition model is used to perform character behavior recognition on the extracted image content of each frame to obtain different behavior types of the video characters in the current video content.

4. The device control method based on video recognition according to claim 3, characterized in that: The AI ​​character behavior recognition model includes any one of a convolutional neural network, a recurrent neural network, a long short-term memory network, a graph convolutional network, and a spatiotemporal graph convolutional network.

5. The device control method based on video recognition according to claim 1, characterized in that: The controlled device includes a massager; the working mode of the massager includes a reciprocating motion mode and / or a rotating motion mode.

6. The device control method based on video recognition according to claim 1, characterized in that: The controlled device includes a massager; the working modes of the massager include different vibration frequency modes and / or different vibration intensity modes.

7. The device control method based on video recognition according to any one of claims 1 to 6, characterized in that: Perform character behavior analysis on the video content currently being watched by the user to obtain different behavior types of the characters in the current video content, including: The user at the client end selects video content to watch, and the server synchronously performs character behavior analysis on the video content currently watched by the user to obtain different behavior types of the video characters in the current video content.

8. A device control method based on video recognition, characterized in that: include: Play the video content currently selected by the user and communicate with the server and the controlled device respectively; The server performs character behavior analysis on the video content currently watched by the user to obtain different behavior types of the video characters in the current video content; The server transmits a control signal for controlling different working modes of the controlled device to the user terminal according to different behavior types of the video characters in the current video content; The control signal transmitted by the server is received, and the received control signal is sent to the controlled device, so that the controlled device acts on the user in different working modes, and the correlation between the video character behavior, the controlled device and the user experience of the controlled device is established.

9. A device control device based on video recognition, characterized in that: include: The video character behavior analysis module is used to analyze the character behavior of the video content currently watched by the user to obtain different behavior types of the video characters in the current video content; The device mode automatic control module is used to automatically control the working mode of the controlled device according to the different behavior types of the video characters in the current video content, so that the controlled device acts on the user in different working modes and establishes the correlation between the video character behavior, the controlled device and the user experience of the controlled device.

10. A device control system based on video recognition, characterized in that: include: User terminal; a user selects video content to watch through the user terminal; A server, communicatively connected to the user terminal, running a computer program to implement the device control method based on video recognition according to any one of claims 1 to 7, and transmitting a control signal for controlling the working mode of the controlled device to the user terminal; The controlled device is connected to the user terminal for communication and receives the control signal obtained by the user terminal to perform work in the corresponding working mode.

Images