Method, device, electronic device and storage medium for controlling application program operation

Abstract

An embodiment of the present disclosure discloses a method for controlling the operation of an application program, which is applied to a first electronic device. The method includes: when in a connected state with a second electronic device, casting the running screen of the application program running on the first electronic device to the second electronic device; detecting the motion state of the first electronic device to obtain motion parameters; determining an operation instruction for controlling the operation of the application program based on the detected motion parameters; and controlling the operation of the application program based on the operation instruction. In the embodiment of the present disclosure, the operation is more convenient, and the application program running is the application program of the first electronic device itself, which improves the user experience.

Description

Technical Field

[0001] The present disclosure relates to the field of communication technologies, and in particular, to a method, an apparatus, an electronic device, and a storage medium for controlling the operation of an application program. Background Art

[0002] When a user plays a game, the way of playing the game is implemented based on an input device. Here, the input device may be a keyboard, a mouse, a gamepad, etc. However, when the user plays a game in the above manner, a fixed sitting posture is generally adopted, which is likely to have an adverse impact on the health of the human body.

[0003] With the rapid development of mobile terminal technologies, the quality and quantity of sensors carried by mobile terminals are constantly improving, and rich and diverse data can be collected. The rich and diverse data is beneficial to improving the ability of motion recognition. Therefore, the research on using mobile terminals for preliminary human motion recognition has become more and more mature. Body-sensing games based on human motion recognition have become the mainstream in the game field. A body-sensing game operates the game based on the body movements of a human body. During the game process, the amount of exercise and entertainment of the human body are significantly increased, significantly improving the user experience. Summary of the Invention

[0004] The present disclosure provides a method, an apparatus, an electronic device, and a storage medium for controlling the operation of an application program.

[0005] According to a first aspect of the present disclosure, there is provided a method for controlling the operation of an application program, which is applied to a first electronic device. The method includes:

[0006] When in a connected state with a second electronic device, cast the running screen of the application program running on the first electronic device to the second electronic device;

[0007] Detect the motion state of the first electronic device to obtain motion parameters;

[0008] Based on the detected motion parameters, determine an operation instruction for controlling the operation of the application program;

[0009] Based on the operation instruction, control the operation of the application program.

[0010] In one embodiment, the motion parameters include at least one of the following: motion direction, motion speed, and motion displacement.

[0011] In one embodiment, the determining an operation instruction for controlling the operation of the application program based on the detected motion parameters includes:

[0012] Generate a body-sensing instruction according to the motion parameters;

[0013] Convert the somatosensory instruction to obtain the operation instruction.

[0014] In one embodiment, the method further includes:

[0015] Select the second electronic device from multiple alternative electronic devices according to the detected selection operation acting on the first electronic device.

[0016] In one embodiment, determining the operation instruction for controlling the running of the application based on the detected motion parameters includes:

[0017] Determine the operation instruction for controlling the running of the application based on the detected motion parameters and the mapping relationship between the motion parameters and the operation instruction.

[0018] In one embodiment, the application is a game application.

[0019] According to a second aspect of the present disclosure, there is provided a device for controlling the running of an application, which is applied to a first electronic device. The device includes a screen mirroring module, a detection module, a determination module, and a control module; wherein,

[0020] The screen mirroring module is configured to: when in a connected state with a second electronic device, mirror the running screen of the application running on the first electronic device to the second electronic device;

[0021] The detection module is configured to: detect the motion state of the first electronic device to obtain motion parameters;

[0022] The determination module is configured to: based on the detected motion parameters, determine the operation instruction for controlling the running of the application;

[0023] The control module is configured to: based on the operation instruction, control the running of the application.

[0024] In one embodiment, the detection module is further configured to: the motion parameters include at least one of the following: motion direction, motion speed, and motion displacement.

[0025] In one embodiment, the device further includes a generation module and a conversion module, wherein,

[0026] The generation module is configured to: generate a somatosensory instruction according to the motion parameters;

[0027] The conversion module is configured to: convert the somatosensory instruction to obtain the operation instruction.

[0028] In one embodiment, the device includes a selection module, wherein,

[0029] The selection module is configured to: select the second electronic device from multiple alternative electronic devices according to a detected selection operation acting on the first electronic device.

[0030] In one embodiment, the determination module is further configured to: determine an operation instruction for controlling the operation of the application program based on the detected motion parameter and the mapping relationship between the motion parameter and the operation instruction.

[0031] In one embodiment, the screen mirroring module is further configured to: the application program is a game application program.

[0032] According to a third aspect of the embodiments of the present disclosure, there is also provided a communication device, including:

[0033] An antenna;

[0034] A memory;

[0035] A processor, connected to the antenna and the memory respectively, for controlling the antenna to transmit and receive wireless signals by executing an executable program stored on the memory, and capable of executing the steps of the method provided by any of the foregoing technical solutions.

[0036] According to a fourth aspect of the embodiments of the present disclosure, there is also provided a non-transitory computer-readable storage medium storing an executable program, wherein when the executable program is executed by a processor, the steps of the method provided by any of the foregoing technical solutions are implemented.

[0037] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

[0038] In an embodiment of the present disclosure, when the first electronic device is in a connected state with the second electronic device, the running screen of the application program running on the first electronic device is projected onto the second electronic device. In this way, the running screen of the application program running on the first electronic device can be displayed on the second electronic device independent of the first electronic device. While the user operates the application program on the first electronic device, the running screen of the application program can be viewed on the second electronic device, facilitating information interaction with the user. Detect the motion state of the first electronic device to obtain motion parameters. In this way, while the first electronic device running the application program is running the application program, it can also detect the first electronic device and obtain motion parameters, without additionally adding a device for obtaining the motion parameters. Based on the detected motion parameters, determine an operation instruction for controlling the running of the application program. In this way, the first electronic device can determine an operation instruction for controlling the running of the application program based on the motion parameters, and control the running of the application program based on the operation instruction. Thus, in the embodiment of the present disclosure, it is realized that while the first electronic device is running its own application program, it can control the running of the application program with the operation instruction determined by the motion parameters detected by itself, and at the same time, the screen of the application program can be directly projected onto the second electronic device for display. Compared with the method that the first electronic device can only send operation instructions acting on the first electronic device and needs to start the application program on the second electronic device, and use the operation instructions to control the application program running on the second electronic device, the operation is more convenient, and the application program running is the own application program of the first electronic device, improving the user experience.

[0039] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention.

[0041] Figure 1 is a schematic diagram of a game system shown according to an exemplary embodiment.

[0042] Figure 2 is a schematic diagram of a game system shown according to an exemplary embodiment.

[0043] Figure 3 is a schematic diagram of a game system shown according to an exemplary embodiment.

[0044] Figure 4It is a flowchart of a method for controlling the operation of an application program shown according to an exemplary embodiment.

[0045] Figure 5 It is a flowchart of a method for controlling the operation of an application program shown according to an exemplary embodiment.

[0046] Figure 6 It is a flowchart of a method for controlling the operation of an application program shown according to an exemplary embodiment.

[0047] Figure 7 It is a flowchart of a method for controlling the operation of an application program shown according to an exemplary embodiment.

[0048] Figure 8 It is a flowchart of a method for controlling the operation of an application program shown according to an exemplary embodiment.

[0049] Figure 9 It is a flowchart of a method for controlling the operation of an application program shown according to an exemplary embodiment.

[0050] Figure 10 It is a block diagram of a device for controlling the operation of an application program shown according to an exemplary embodiment.

[0051] Figure 11 It is a block diagram of an electronic device shown according to an exemplary embodiment. Detailed implementation

[0052] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0053] To facilitate the understanding of any embodiment of the present disclosure, first, the application scenario of the game system will be described.

[0054] In one embodiment, according to the different input devices of the game system, the game system can be at least divided into: a general game system and a motion-sensing game system.

[0055] Here, the general game system can be a game system that uses general hardware devices as input devices. Here, the general hardware devices can refer to input devices that do not have a large number, high precision, and high-quality motion data acquisition sensors. For example, keyboards, mice, and mechanical game controllers, etc.

[0056] Here, the somatosensory game system may be a game system that uses a hardware device equipped with a motion data acquisition sensor as an input device. Here, the data acquisition sensor may be a geomagnetic sensor, a gravity sensor, and / or a gyroscope sensor, etc. Here, the data acquisition sensor is used to acquire the motion speed, motion displacement, and / or motion direction of the input device's movement, etc.

[0057] In one embodiment, please refer to Figure 1 , Figure 1 The one shown in is the ordinary game system. The ordinary game system includes a mouse and a computer. The game application runs in the computer. The mouse can send control instructions for controlling the game application to the computer to control the operation of the game application. The running screen of the game application can be displayed on the display screen of the computer.

[0058] Figure 1 In the implementation shown, the motion parameters of the mouse itself cannot be parsed by the mouse to obtain the human motion posture, so as to control the operation of the game application using the operation instructions determined by the motion posture. For example, the operation instructions determined by the motion parameters cannot be used to control the movement of the hand of the character object in the game application scenario.

[0059] In one embodiment, please refer to Figure 2 , Figure 2 The one shown in is the somatosensory game system. The somatosensory game system includes a somatosensory input device, a processing device, and a display device. The game application can run in the processing device or the display device. The somatosensory input device can control the operation of the game application according to the operation instructions determined by the running parameters of the somatosensory input device. The running screen of the game application is displayed in the display device.

[0060] Figure 2 In the implementation shown, the motion parameters of the somatosensory input device itself can be obtained, and an additional processing device is required to parse the human motion posture, so as to control the operation of the game application using the operation instructions determined by the motion posture. For example, the operation instructions determined by the motion parameters can be used to control the movement of the hand of the character object in the game application scenario. In this implementation, a relatively large number of hardware devices are required. And the game application is a fixed game application running in the processing device or the display device, and the types of the game application are relatively single.

[0061] In one embodiment, please refer to Figure 3 , Figure 3Shown is a somatosensory game system, which includes a somatosensory input device and a processing device. A game application runs in the processing device, and the somatosensory input device can control the operation of the game application according to the operation instructions determined based on the running parameters of the somatosensory input device. The running screen of the game application is displayed in the processing device.

[0062] Figure 3 In the shown implementation, the motion parameters of the somatosensory input device itself can be obtained. It is required that the processing device analyzes the human motion posture based on the motion parameters, so as to control the operation of the game application by using the operation instructions determined by the motion posture. For example, the operation instructions determined by the motion parameters can be used to control the actions of the feet of the character object in the game application scenario. And in this implementation, the game application is a single game application running in the processing device, and the types of the game application are relatively single.

[0063] It should be noted that those skilled in the art can understand that the method provided in the embodiments of the present disclosure can be executed alone or together with some methods in the embodiments of the present disclosure or some methods in the related art.

[0064] As Figure 4 shown, the embodiments of the present disclosure provide a method for controlling the operation of an application program, which is applied to a first electronic device. The method includes:

[0065] Step 41, when in a connected state with a second electronic device, cast the running screen of the application program running in the first electronic device to the second electronic device;

[0066] Step 42, detect the motion state of the first electronic device to obtain motion parameters;

[0067] Step 43, based on the detected motion parameters, determine the operation instructions for controlling the operation of the application program;

[0068] Step 44, based on the operation instructions, control the operation of the application program.

[0069] In one embodiment, the first electronic device can be various types of mobile devices. For example, the first electronic device can be a mobile terminal such as a mobile phone, a game device, a sports device, a smart wearable terminal, and a smart home terminal. Here, the game device can be a game controller. The sports device can be a spinning bike, a dance machine, a sports skateboard, etc.

[0070] Here, the first electronic device is provided with a display screen. The display screen can be used to display the running screen of the application program running in the first electronic device.

[0071] In one embodiment, the first electronic device is provided with a motion data acquisition sensor.

[0072] In one embodiment, the data acquisition sensor is used to acquire the motion speed, motion displacement, and / or motion direction of the first electronic device, etc.

[0073] In one embodiment, the data acquisition sensor can be a geomagnetic sensor, a gravity sensor, and / or a gyroscope sensor, etc.

[0074] In one embodiment, the geomagnetic sensor is used to detect the orientation of the first electronic device to obtain the orientation data of the first electronic device.

[0075] In one embodiment, the gravity sensor is used to detect the motion direction of the first electronic device to obtain the motion direction data of the first electronic device.

[0076] In one embodiment, the gyroscope sensor is used to detect the motion speed of the first electronic device to obtain the motion speed data of the first electronic device.

[0077] In one embodiment, the second electronic device can be a display device. For example, the display device is a display, a digital TV, a projection device, etc.

[0078] In one embodiment, the first electronic device and the second electronic device can establish a communication connection.

[0079] In one embodiment, the first electronic device and the second electronic device access the communication network based on the same access point.

[0080] In one embodiment, the first electronic device can establish a wireless communication connection between the first electronic device and the second electronic device based on a wireless communication protocol. For example, the first electronic device establishes a wireless communication connection between the first electronic device and the second electronic device based on the wireless fidelity (WiFi, Wireless Fidelity) end-to-end communication protocol.

[0081] Exemplarily, the first electronic device and the second electronic device turn on their respective wireless fidelity (WiFi) end-to-end connection services.

[0082] In one embodiment, the first electronic device can scan at least one of the second electronic devices that have turned on the wireless fidelity (WiFi) end-to-end connection service.

[0083] In one embodiment, the first electronic device may display the identifier of the second electronic device on the display screen.

[0084] In one embodiment, in response to an operation on the first electronic device for the identifier of the second electronic device, the first electronic device may select the second electronic device corresponding to the identifier and establish a Wi-Fi peer-to-peer connection with the second electronic device.

[0085] In one embodiment, the first electronic device may establish a wired communication connection between the first electronic device and the second electronic device based on a wired communication protocol.

[0086] In one embodiment, in response to the establishment of the connection between the first electronic device and the second electronic device, the second electronic device is in a connected state.

[0087] In one embodiment, casting the running screen of an application program to the second electronic device may be to cast the screen content that the application program needs to display to the second electronic device for display based on the communication connection.

[0088] In one embodiment, the screen content cast to the second electronic device may be updated in real time according to an update period.

[0089] In one embodiment, in response to the required latency for playing the screen content being less than the latency threshold, set the update period to be less than the period threshold; in response to the required latency for playing the screen content being greater than the latency threshold, set the update period to be greater than the period threshold.

[0090] In this way, the update period can adapt to the required latency.

[0091] In one embodiment, in response to the establishment of the connection between the first electronic device and the second electronic device, cast the running screen of the application program running on the first electronic device to the second electronic device.

[0092] In one embodiment, in response to the establishment of the connection between the first electronic device and the second electronic device and detecting a screen mirroring operation on the first electronic device, cast the running screen of the application program running on the first electronic device to the second electronic device.

[0093] In one embodiment, the screen mirroring operation may be a touch operation on any position or a specific position of the display screen of the first electronic device, or may be a double-tap operation on the display screen of the first electronic device, or may be a touch sliding operation on the display screen of the first electronic device, etc.

[0094] In one embodiment, in response to the connection between the first electronic device and the second electronic device being established and the distance between the first electronic device and the second electronic device being less than a distance threshold, the running screen of the application running on the first electronic device is cast to the second electronic device.

[0095] In one embodiment, in response to the connection between the first electronic device and the second electronic device being established and the distance between the first electronic device and the second electronic device being greater than a distance threshold, the running screen of the application running on the first electronic device is cast to the second electronic device.

[0096] In one embodiment, the first electronic device may be a mobile phone and the second electronic device may be a digital TV.

[0097] In one embodiment, the display area of the display screen of the second electronic device is larger than the display area of the display screen of the first electronic device.

[0098] In one embodiment, the first electronic device and the second electronic device may simultaneously display the screen of the application.

[0099] In some embodiments, the application may be a game application or a multimedia application, etc.

[0100] Here, the multimedia application may be an e-reader, a video player, an audio player, etc.

[0101] In some embodiments, the application may be an application running on a web page or an application installed separately.

[0102] In one embodiment, it may be to periodically detect the motion state of the first electronic device.

[0103] In one embodiment, the period for detecting the motion state of the first electronic device may be determined according to the sensitivity requirement for controlling the operation of the application.

[0104] In one embodiment, in response to the required sensitivity for controlling the operation of the application being greater than or equal to a sensitivity threshold, the period for detecting the motion state of the first electronic device is less than a period threshold.

[0105] In one embodiment, in response to the required sensitivity for controlling the operation of the application being less than or equal to a sensitivity threshold, the period for detecting the motion state of the first electronic device is greater than a period threshold.

[0106] In one embodiment, the sensitivity threshold may be selected from a predetermined set of sensitivity thresholds and / or the period threshold may be selected from a predetermined set of sensitivity thresholds.

[0107] In one embodiment, it may be to detect the motion state of the first electronic device in a predetermined direction.

[0108] In one embodiment, detecting the motion state of the first electronic device may be to detect the motion state of a reference object in the first electronic device. For example, it may be to detect the motion state of a camera in the first electronic device.

[0109] In one embodiment, in response to the distance between the first electronic device and the second electronic device being less than a distance threshold, the motion state of the first electronic device is detected; in response to the distance between the first electronic device and the second electronic device being greater than the distance threshold, the motion state of the first electronic device is not detected.

[0110] In this way, when the first electronic device is outside the distance threshold range of the second electronic device, the motion state of the first electronic device is not detected, so that power consumption can be saved.

[0111] In one embodiment, in response to the distance between the first electronic device and the second electronic device being greater than a distance threshold, the motion state of the first electronic device is detected. In response to the distance between the first electronic device and the second electronic device being less than the distance threshold, the motion state of the first electronic device is not detected.

[0112] In this way, when the first electronic device is within the distance threshold range of the second electronic device, the motion state of the first electronic device is not detected, so that the user's eyesight can be protected.

[0113] In one embodiment, the motion parameters include at least one of the following: motion direction, motion speed, and motion displacement.

[0114] In one embodiment, the motion parameters may be parameters directly collected by a sensor. For example, motion direction and motion speed.

[0115] In one embodiment, the motion direction of the first electronic device may be obtained by using a geomagnetic sensor, and the motion speed of the first electronic device may be obtained by using a gyroscope sensor.

[0116] In one embodiment, the motion parameters may be determined through calculation based on the parameters directly collected by the sensor. For example, based on time parameters, motion direction, and motion speed, the motion displacement is determined.

[0117] In one embodiment, the information on the motion state can be used to determine the moving posture of the first electronic device. In one embodiment, the motion parameters may be input into a trained neural network model for identifying the moving posture of the first electronic device to obtain the type of the moving posture of the first electronic device, and an operation instruction may be determined based on the type of the moving posture.

[0118] In one embodiment, the operation instruction may be determined periodically based on the motion parameters.

[0119] In one embodiment, the first electronic device may determine the motion trajectory of the first electronic device based on the change of the motion parameters within a set period. An operation instruction is generated based on the motion trajectory. The action of the target object in the application program is controlled according to the change of the motion trajectory based on the operation instruction.

[0120] In one embodiment, when the first electronic device moves a unit distance along the motion trajectory, the operation instruction will control the target object in the application program to move the unit distance along the motion trajectory in real time.

[0121] For example, if the application program is a game application program and the target object is the fist of a character in the game program scene, when the first electronic device moves a meters along the motion trajectory, the operation instruction will control the fist to move a meters along the motion trajectory.

[0122] In one embodiment, the mapping relationship between the type of the moving posture and the operation instruction is pre-stored in the first electronic device. Here, the first electronic device may determine the operation instruction based on the mapping relationship and the type of the moving posture.

[0123] In one embodiment, the operation instruction may be an instruction for controlling the action of an object in the running screen. For example, if the application program is a game application program, the operation instruction may be an instruction for controlling the limb movement of the task object in the running screen of the game application program; if the application program is a video player, the operation instruction may be an instruction for controlling the video player to pause the playback.

[0124] In one embodiment, the screen in the second electronic device is updated synchronously according to the operation of the application program.

[0125] In an embodiment of the present disclosure, when in a connected state with a second electronic device, the running screen of an application running on the first electronic device is projected onto the second electronic device. In this way, the running screen of the application running on the first electronic device can be displayed on the second electronic device independent of the first electronic device. While the user operates the application on the first electronic device, the running screen of the application can be viewed on the second electronic device, facilitating information interaction with the user.

[0126] Here, the motion state of the first electronic device is detected to obtain motion parameters. The first electronic device running the application can also detect the motion state of the first electronic device and obtain motion parameters while running the application, without the need to additionally add a device for obtaining the motion parameters.

[0127] Here, based on the detected motion parameters, an operation instruction for controlling the running of the application is determined. In this way, the first electronic device can determine an operation instruction for controlling the running of the application based on the motion parameters, and based on the operation instruction, control the running of the application.

[0128] In this way, in an embodiment of the present disclosure, it is realized that while the first electronic device is running its own application, it can control the running of the application using an operation instruction determined based on the motion parameters detected by itself, and at the same time, the screen of the application can be directly projected onto the second electronic device for display. Compared with the method in which the first electronic device can only send operation instructions acting on the first electronic device and needs to start the application on the second electronic device and use the operation instructions to control the application running on the second electronic device, the operation is more convenient, and it is the application of the first electronic device itself that is running, improving the user experience.

[0129] It should be noted that those skilled in the art can understand that the method provided in the embodiment of the present disclosure can be executed alone or together with some methods in the embodiment of the present disclosure or some methods in related technologies.

[0130] As Figure 5 shown, an embodiment of the present disclosure provides a method for controlling the running of an application, which is applied to a first electronic device. The method includes:

[0131] Step 51, generate a somatosensory instruction according to the motion parameters;

[0132] Step 52, perform instruction conversion on the somatosensory instruction to obtain the operation instruction.

[0133] In one embodiment, the motion parameters include at least one of the following: motion direction, motion speed, and motion displacement.

[0134] In one embodiment, the somatosensory instruction may be an instruction for the first electronic device to determine the motion characteristics when the first electronic device moves.

[0135] For example, the somatosensory instruction may be an instruction for the first electronic device to determine motion characteristics such as jumping, waving, and translation when the first electronic device moves.

[0136] In one embodiment, the somatosensory instruction can be converted into the operation instruction that can directly control the application program.

[0137] In one embodiment, based on the motion characteristics determined by the somatosensory instruction, the somatosensory instruction can be converted into the corresponding operation instruction that directly controls the application program.

[0138] For example, if the application program is a game application program, and it is determined based on the somatosensory instruction that the first electronic device makes actions of jumping, waving, and translation respectively, then the corresponding somatosensory instructions can be respectively converted into operation instructions for controlling the character in the game application program to swipe up, swipe down, and swipe horizontally.

[0139] In one embodiment, it may be to input the motion parameters into a trained neural network model for identifying the moving posture of the first electronic device to obtain the moving posture of the first electronic device. Based on the mapping relationship between the moving posture, the moving posture, and the somatosensory instruction, the somatosensory instruction is generated.

[0140] In one embodiment, the operation instruction may be an instruction for controlling the operation of the application program.

[0141] For example, if the application program is a game application program, the operation instruction may be an instruction for controlling the character in the game scene to jump, flip, and punch.

[0142] In one embodiment, different somatosensory instructions correspond to different operation instructions.

[0143] For example, if the application program is a game application program, the somatosensory instruction for the first electronic device to swipe up corresponds to the operation instruction for controlling the character in the game scene to jump; the somatosensory instruction for the first electronic device to swipe down corresponds to the operation instruction for controlling the character in the game scene to flip; the somatosensory instruction for the first electronic device to swipe horizontally corresponds to the operation instruction for the character in the game scene to punch.

[0144] In one embodiment, the conversion relationship between the somatosensory instruction and the operation instruction may be pre-stored in the first electronic device. In this way, the first electronic device can obtain the operation instruction based on the somatosensory instruction and the conversion relationship between the somatosensory instruction and the operation instruction.

[0145] It should be noted that those skilled in the art can understand that the method provided by the embodiments of the present disclosure can be executed alone or together with some methods in the embodiments of the present disclosure or some methods in related technologies.

[0146] As Figure 6 shown, the embodiments of the present disclosure provide a method for controlling the operation of an application program, which is applied to a first electronic device. The method includes:

[0147] Step 61, select the second electronic device from multiple alternative electronic devices according to the detected selection operation acting on the first electronic device.

[0148] In one embodiment, the identifiers of multiple alternative electronic devices connected to the first electronic device are displayed on the display interface of the first electronic device.

[0149] In one embodiment, the selection operation detected by the first electronic device acting on the first electronic device may be a touch operation detected by the first electronic device on the identifier of the alternative device acting on the first electronic device.

[0150] In one embodiment, the touch operation may be a point touch operation on any position or a specific position in the area corresponding to the identifier displayed on the display screen of the mobile terminal, or may be a double click operation on the area corresponding to the identifier displayed on the display screen, or may be a touch sliding operation on the area corresponding to the identifier displayed on the display screen, etc.

[0151] In one embodiment, the multiple alternative electronic devices are electronic devices within a distance range from the first electronic device.

[0152] In one embodiment, the multiple alternative devices are electronic devices that have established a communication connection with the first electronic device.

[0153] In one embodiment, the multiple alternative electronic devices are electronic devices with wireless communication quality within a quality threshold range from the first electronic device.

[0154] It should be noted that those skilled in the art can understand that the method provided by the embodiments of the present disclosure can be executed alone or together with some methods in the embodiments of the present disclosure or some methods in related technologies.

[0155] As shown Figure 7 in the figure, an embodiment of the present disclosure provides a method for controlling the operation of an application, which is applied to a first electronic device. The method includes:

[0156] Step 71: Based on the detected motion parameters and the mapping relationship between the motion parameters and the operation instructions, determine the operation instructions for controlling the operation of the application.

[0157] In one embodiment, the motion parameters include at least one of the following: motion direction, motion speed, and motion displacement.

[0158] In one embodiment, different motion parameters may represent different operation instructions.

[0159] In one embodiment, the motion parameters may be input into a trained neural network model for determining operation instructions to obtain the operation instructions of the first electronic device.

[0160] In one embodiment, the operation instructions may be instructions for controlling the operation of the application. For example, if the application is a game application, the operation instructions may be instructions for controlling a character object in the game scene to jump, flip, and punch.

[0161] In one embodiment, different motion parameters may determine different operation instructions.

[0162] For example, if the application is a game application, the motion parameter of the first electronic device swiping upward corresponds to the operation instruction for controlling the character object in the game scene to jump; the motion parameter of the first electronic device swiping downward corresponds to the operation instruction for controlling the character object in the game scene to flip; and the motion parameter of the first electronic device swiping horizontally corresponds to the operation instruction for the character to punch in the game scene.

[0163] In one embodiment, the mapping relationship between the motion parameters and the operation instructions may be pre-stored in the first electronic device. In this way, the first electronic device can obtain the operation instructions based on the motion parameters and the conversion relationship between the motion parameters and the operation instructions.

[0164] It should be noted that those skilled in the art can understand that the method provided by the embodiments of the present disclosure can be executed alone or together with some methods in the embodiments of the present disclosure or some methods in related technologies.

[0165] To further understand the embodiments of the present disclosure, the following further illustrates the implementation of the present disclosure through an exemplary embodiment:

[0166] Example 1: The first electronic device is a mobile phone, the second electronic device is a digital TV, and the application program is a game application program. Among them, the mobile phone and the digital TV have established a communication connection in advance based on Wi-Fi communication technology.

[0167] As Figure 8 and Figure 9 shown, this example provides a method for controlling the operation of an application program, which is applied to a mobile phone. The method includes:

[0168] Step 81, start the game application program in the mobile phone;

[0169] Step 82, when the mobile phone and the digital TV are in a connected state, project the running screen of the game application program running on the mobile phone to the digital TV;

[0170] Step 83, detect the motion state of the mobile phone to obtain motion parameters; here, the motion parameters include at least one of the following: motion direction, motion speed, and motion displacement;

[0171] Step 84, use a neural network algorithm to generate a somatosensory instruction according to the motion parameters;

[0172] Step 85, perform instruction conversion on the somatosensory instruction to obtain the operation instruction.

[0173] Step 86, control the operation of the game application program based on the operation instruction.

[0174] It should be noted that those skilled in the art can understand that the method provided by the embodiments of the present disclosure can be executed alone or together with some methods in the embodiments of the present disclosure or some methods in related technologies.

[0175] Figure 10 There is provided a device for controlling the operation of an application program shown in an exemplary embodiment, which is applied to the first electronic device. The device includes a screen projection module 101, a detection module 102, a determination module 103, and a control module 104; among them,

[0176] The screen projection module 101 is used to project the running screen of the application program running on the first electronic device to the second electronic device when it is in a connected state with the second electronic device;

[0177] The detection module 102 is used to detect the motion state of the first electronic device to obtain motion parameters;

[0178] The determination module 103 is used to determine an operation instruction for controlling the operation of the application program based on the detected motion parameters;

[0179] The control module 104 is configured to: control the operation of the application program based on the operation instruction.

[0180] In one embodiment, the detection module 102 is further configured to: the motion parameters include at least one of the following: motion direction, motion speed, and motion displacement.

[0181] In one embodiment, the device further includes a generation module 105 and a conversion module 106, where

[0182] The generation module 105 is configured to: generate a somatosensory instruction according to the motion parameters;

[0183] The conversion module 106 is configured to: perform instruction conversion on the somatosensory instruction to obtain the operation instruction.

[0184] In one embodiment, the device includes a selection module 107, where

[0185] The selection module 107 is configured to: select the second electronic device from a plurality of alternative electronic devices according to a detected selection operation acting on the first electronic device.

[0186] In one embodiment, the determination module 103 is further configured to: determine the operation instruction for controlling the operation of the application program based on the detected motion parameters and the mapping relationship between the motion parameters and the operation instruction.

[0187] In one embodiment, the screen mirroring module 101 is further configured to: the application program is a game application program.

[0188] It should be noted that those skilled in the art can understand that the method provided by the embodiments of the present disclosure can be executed alone, or can be executed together with some methods in the embodiments of the present disclosure or some methods in related technologies.

[0189] Embodiments of the present disclosure further provide an electronic device, including:

[0190] A processor;

[0191] A memory for storing processor-executable instructions;

[0192] Wherein, the processor is configured to: when running the executable instructions, implement the method described in any embodiment of the present disclosure.

[0193] The memory may include various types of storage media, and the storage media is a non-temporary computer storage medium that can continue to remember and store the information thereon after the communication device loses power.

[0194] The processor can be connected to the memory via a bus or the like, and is used to read the executable program stored on the memory.

[0195] Embodiments of the present disclosure also provide a computer-readable storage medium storing an executable program, wherein when the executable program is executed by a processor, the methods described in any embodiment of the present disclosure are implemented.

[0196] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0197] Figure 11 is a block diagram of an electronic device 600 shown according to an exemplary embodiment. For example, the electronic device 600 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0198] Referring to Figure 11 , the electronic device 600 may include one or more of the following components: a processing component 602, a memory 604, a power supply component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0199] The processing component 602 generally controls the overall operation of the electronic device 600, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 602 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above methods. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.

[0200] The memory 604 is configured to store various types of data to support the operation of the device 600. Examples of these data include instructions for any application or method operating on the electronic device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0201] The power supply component 606 provides power for various components of the electronic device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 600.

[0202] The multimedia component 608 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of the touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operations. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0203] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 600 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 604 or transmitted via the communication component 616. In some embodiments, the audio component 610 further includes a speaker for outputting audio signals.

[0204] The I / O interface 612 provides an interface between the processing component 602 and a peripheral interface module, which can be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to: a home button, a volume button, a power button, and a lock button.

[0205] The sensor assembly 614 includes one or more sensors for providing an assessment of various aspects of the electronic device 600. For example, the sensor assembly 614 can detect the on / off state of the device 600, the relative positioning of components, such as the display and keypad of the electronic device 600. The sensor assembly 614 can also detect a change in the position of the electronic device 600 or a component of the electronic device 600, the presence or absence of user contact with the electronic device 600, the orientation or acceleration / deceleration of the electronic device 600, and a change in the temperature of the electronic device 600. The sensor assembly 614 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0206] The communication component 616 is configured to facilitate communication between the electronic device 600 and other devices in a wired or wireless manner. The electronic device 600 can access a wireless network based on communication standards, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0207] In an exemplary embodiment, the electronic device 600 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described methods.

[0208] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 604 including instructions, is also provided. The above instructions can be executed by the processor 820 of the electronic device 600 to complete the above-described methods. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, among others.

[0209] Other embodiments of the present invention will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following the general principles of the invention and including known common general knowledge or conventional technical means in the technical field not disclosed in this disclosure. The specification and examples are only regarded as exemplary, and the true scope and spirit of the invention are pointed out by the following claims.

[0210] It should be understood that the present invention is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A method for controlling the operation of an application program, characterized in that, applied to a first electronic device, the method includes: When the first electronic device and the second electronic device are in a connected state and the distance between the first electronic device and the second electronic device is greater than a distance threshold, projecting the running screen of the application program running on the first electronic device to the second electronic device; Detect the motion state of the first electronic device to obtain motion parameters; wherein, the motion parameters include at least one of the following: motion direction, motion speed, and motion displacement; wherein, the period for detecting the motion state of the first electronic device is determined according to the sensitivity requirement of the operation for controlling the application program; when the sensitivity required for the operation for controlling the application program is greater than or equal to a sensitivity threshold, the period is less than a period threshold; when the sensitivity is less than the sensitivity threshold, the period is greater than the period threshold; the sensitivity threshold is a value selected from a predetermined set of sensitivity thresholds, and / or, the period threshold is a threshold selected from a predetermined set of period thresholds; Based on the detected motion parameters, determine an operation instruction for controlling the operation of the application program; The determining, based on the detected motion parameters, an operation instruction for controlling the operation of the application program includes: Generating a somatosensory instruction according to the motion parameters; wherein, the somatosensory instruction is an instruction for the first electronic device to determine the motion characteristics of jumping, waving, and translation when the first electronic device moves; Performing instruction conversion on the somatosensory instruction to obtain the operation instruction; Based on the operation instruction, control the operation of the application program.

2. The method according to claim 1, characterized in that, the method further includes: Selecting the second electronic device from multiple alternative electronic devices according to the detected selection operation acting on the first electronic device.

3. The method according to claim 1, characterized in that, the determining, based on the detected motion parameters, an operation instruction for controlling the operation of the application program includes: Based on the detected motion parameters and the mapping relationship between the motion parameters and the operation instruction, determine the operation instruction for controlling the operation of the application program.

4. The method according to claim 1, characterized in that, the application program is a game application program.

5. A device for controlling the operation of an application program, characterized in that, applied to a first electronic device, the device includes a screen projection module, a detection module, a determination module, a generation module, a conversion module, and a control module; wherein, the screen projection module is configured to: when the first electronic device and the second electronic device are in a connected state and the distance between the first electronic device and the second electronic device is greater than a distance threshold, project the running screen of the application program running on the first electronic device to the second electronic device; The detection module is configured to: detect the motion state of the first electronic device to obtain motion parameters; wherein, the motion parameters include at least one of the following: motion direction, motion speed, and motion displacement; wherein, the period for detecting the motion state of the first electronic device is determined according to the sensitivity requirement for controlling the operation of the application program; when the sensitivity required for controlling the operation of the application program is greater than or equal to the sensitivity threshold, the period is less than the period threshold; when the sensitivity is less than the sensitivity threshold, the period is greater than the period threshold; the sensitivity threshold is a value selected from a predetermined set of sensitivity thresholds, and / or, the period threshold is a threshold selected from a predetermined set of period thresholds; The determination module is configured to: determine an operation instruction for controlling the operation of the application program based on the detected motion parameters; The generation module is configured to: generate a somatosensory instruction according to the motion parameters; wherein, the somatosensory instruction is an instruction for the first electronic device to determine the motion characteristics of jumping, waving, and translation when the first electronic device moves; The conversion module is configured to: perform instruction conversion on the somatosensory instruction to obtain the operation instruction; The control module is configured to: control the operation of the application program based on the operation instruction.

6. The apparatus according to claim 5, wherein, the apparatus includes a selection module, wherein, the selection module is configured to: select the second electronic device from multiple alternative electronic devices according to the detected selection operation acting on the first electronic device.

7. The apparatus according to claim 5, wherein, the determination module is further configured to: determine the operation instruction for controlling the operation of the application program based on the detected motion parameters and the mapping relationship between the motion parameters and the operation instruction.

8. The apparatus according to claim 5, wherein, the screen mirroring module is further configured to be: the application program is a game application program.

9. An electronic device, wherein, comprises: an antenna; a memory; a processor, connected to the antenna and the memory respectively, and configured to control the transceiver of the antenna by executing computer-executable instructions stored on the memory, and be capable of implementing the method provided in any one of claims 1 to 4.

10. A computer storage medium storing computer-executable instructions, which can implement the method provided in any one of claims 1 to 4 after being executed by a processor.

Images