Interaction control method and electronic equipment

Abstract

The embodiment of the invention discloses an interaction control method and electronic equipment, and the method comprises the steps: obtaining first interaction data of an interaction operation in a first interaction region in response to the obtained interaction operation acting on an interaction control region of the electronic equipment, the first interaction area is an area connected with at least one edge of a second interaction area in the interaction control area; determining first behavior data corresponding to the interaction operation in the first interaction area based on the first interaction data so as to control the electronic equipment to at least execute a first control action responding to the first behavior data; wherein the type of the first behavior data is the same as that of second interaction data of the interaction operation in the second interaction area, or the type of the first behavior data is the same as that of second behavior data obtained by the electronic equipment in response to the second interaction data.

Description

Technical Field

[0001] The present application relates to but is not limited to the field of information technology, and in particular to an interactive control method and electronic equipment. Background Art

[0002] In the related art, when the user's interactive operation spans across multiple interactive areas and the types of interactive data that can be responded to between the interactive areas are different, there is a problem that the user's interactive operation cannot be responded to in a timely and accurate manner. Summary of the invention

[0003] In view of this, the embodiments of the present application at least provide an interactive control method and an electronic device.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides an interactive control method, including:

[0006] In response to obtaining an interactive operation acting on an interactive control area of ​​the electronic device, first interactive data of the interactive operation in a first interactive area is obtained, where the first interactive area is an area connected to at least one edge of a second interactive area in the interactive control area, and the types of interactive data obtained by the second interactive area and the first interactive area in response to the interactive operation are different;

[0007] Determine, based on the first interaction data, first behavior data corresponding to the interaction operation in the first interaction area, so as to control the electronic device to at least perform a first control action in response to the first behavior data;

[0008] The first behavior data is of the same type as the second interaction data of the interaction operation in the second interaction area, or,

[0009] The first behavior data is of the same type as the second behavior data obtained by the electronic device in response to the second interaction data.

[0010] The present application provides an interactive control method, including:

[0011] In response to obtaining a touch operation acting on a touch area of ​​the electronic device, obtaining at least one frame image of the touch operation in a first area, where the first area is an area connected to at least one edge of a second area in the touch area, the second area is an area supporting touch control, and the first area is an area not supporting touch control;

[0012] The first touch data corresponding to the touch operation in the first area is determined based on at least one frame of image, so as to control the electronic device to execute a third control action in response to the first touch data.

[0013] An embodiment of the present application provides an electronic device, comprising: a first body;

[0014] A touch panel disposed on the first body, the touch panel comprising a second area and a first area connected to at least one edge of the second area, the second area being provided with a touch module, and the first area being provided with a first recognition module different from the touch module;

[0015] At least one controller can process at least one frame of image collected by the first recognition module into first touch data, and can control the electronic device to execute a third control action in response to the first touch data.

[0016] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program can be executed by a processor to implement the above-mentioned interactive control method.

[0017] An embodiment of the present application provides a computer program, including a computer-readable code. When the computer-readable code is executed in a computer device, a processor in the computer device executes some or all of the steps for implementing the above-mentioned interactive control method.

[0018] An embodiment of the present application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements some or all of the steps in the above-mentioned interactive control method.

[0019] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram of an implementation process of an interactive control method provided in an embodiment of the present application Figure 1 ;

[0021] Figure 2 A schematic diagram of an implementation process of an interactive control method provided in an embodiment of the present application Figure 2 ;

[0022] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application Figure 1 ;

[0023] Figure 4A A schematic diagram of an integrated structure of a fingerprint module and a touch module in the related art Figure 1 ;

[0024] Figure 4B A schematic diagram of an integrated structure of a fingerprint module and a touch module in the related art Figure 2 ;

[0025] Figure 4CA schematic diagram of a touch panel including an area where the touch function cannot be realized in the related art;

[0026] Figure 4D A schematic diagram of the principle of fingerprint recognition based on ultrasound in the related art;

[0027] Figure 5A A schematic diagram of the structure of an electronic device provided in an embodiment of the present application Figure 2 ;

[0028] Figure 5B A schematic diagram of an implementation process of an interactive control method provided in an embodiment of the present application Figure 3 ;

[0029] Figure 5C A schematic diagram of an implementation process for calculating a finger movement trajectory provided in an embodiment of the present application Figure 1 ;

[0030] Figure 5D A schematic diagram of an implementation process for calculating a finger movement trajectory provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0031] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further elaborated in detail below in conjunction with the drawings and embodiments. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0032] In the following description, the terms "first / second / third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first / second / third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field to which this application belongs. The terms used herein are only for the purpose of describing this application and are not intended to limit this application. It should also be noted that, for ease of description, only the parts related to the relevant application are shown in the accompanying drawings.

[0033] The present application provides an interactive control method, which can be executed by a processor of an electronic device. The electronic device may refer to a server, a laptop, a tablet computer, a desktop computer, a smart TV, a set-top box, a mobile device (such as a mobile phone, a portable video player, a personal digital assistant, a dedicated messaging device, a portable gaming device), or other device with data processing capabilities. Figure 1As shown, the interactive control method includes the following steps S101 to S102:

[0034] Step S101, in response to obtaining an interactive operation acting on an interactive control area of ​​an electronic device, obtaining first interactive data of the interactive operation in a first interactive area, the first interactive area being an area connected to at least one edge of a second interactive area in the interactive control area, and the types of interactive data obtained by the second interactive area and the first interactive area in response to the interactive operation are different.

[0035] Here, the interactive control area may be a control area in the electronic device for responding to user interactive operations, and may include a plane detection area or a three-dimensional sensing area.

[0036] For example, the interactive control area may include, but is not limited to, a capacitive touch pad area, a force pad area, a touch screen area, a touch bar area, and other planar detection areas, and / or a three-dimensional sensing area of ​​a sensor such as a camera, a time of flight technology (TOF), a radar, etc. Among them, the camera may include, but is not limited to, an RGB camera, an infrared camera, a depth camera, etc.

[0037] For different types of interactive control areas, the interactive operations may be different.

[0038] For example, for a touch-type interactive control area, the interactive operation may include a touch operation, a click operation, a double-click operation, a press operation, a slide operation, and the like.

[0039] For another example, for a three-dimensional sensing type interactive control area, the interactive operation may include, but is not limited to, a three-dimensional space gesture operation (including dynamic gestures and static gestures), an operating body movement operation, and the like.

[0040] In some embodiments, the first interaction area may be an area in the interaction control area or an area not in the interaction control area, and may include a plane detection area or a three-dimensional sensing area. For example, the first interaction area may include but is not limited to at least one of a fingerprint recognition area, an under-screen camera area, a force pad area, a TOF sensing area, etc.; wherein the fingerprint recognition area may include a fingerprint sensor area, an ultrasonic fingerprint recognition area, or a camera fingerprint recognition area.

[0041] For different types of first interaction areas, the first interaction data may be different.

[0042] For example, the first interaction data may include fingerprint image data of a fingerprint recognition area.

[0043] For another example, the first interaction data may include imaging data of the under-screen camera area.

[0044] For another example, the first interaction data may include pressure data and / or touch data of the pressure sensing area.

[0045] For another example, the first interaction data may include operating body contour data and / or operating body movement data in the TOF sensing area.

[0046] The second interaction area may include a portion or the entire area of ​​the interaction control area of ​​the electronic device.

[0047] It is understandable that the first interaction area and the second interaction area are connected at least at one edge, that is, the first interaction area and the second interaction area can be crossed in one interaction operation, and the types of interaction data obtained by the second interaction area and the first interaction area in response to the interaction operation are different.

[0048] For example, when the first interaction area is a fingerprint recognition area, the interaction control area can be a touchpad area, and the second interaction area can be a capacitive touch area or a pressure-sensitive touch area; wherein the fingerprint recognition area can be located in the upper right corner area, the upper left corner area or a middle area of ​​the second interaction area.

[0049] In this case, in response to the user's touch operation, the first interaction data corresponding to the first interaction area may be fingerprint image data and / or ultrasonic fingerprint data, and the second interaction data corresponding to the second interaction area may be induced capacitance data corresponding to the touch operation, coordinate data of the touch point and / or pressure value data.

[0050] For another example, when the first interaction area is a pressure-sensitive touch area, the second interaction area may be a capacitive touch area; wherein the pressure-sensitive touch area may be located in the upper right corner area, the upper left corner area, the lower right corner area, the upper left corner area or a middle area of ​​the capacitive touch area.

[0051] In this case, in response to the user's touch operation, the first interaction data corresponding to the first interaction area can be the pressure value data of the touch point corresponding to the touch operation, and the second interaction data corresponding to the second interaction area can be the coordinate data of the touch point corresponding to the touch operation or the induced capacitance data obtained by sensing the touch operation.

[0052] For another example, when the first interaction area is the under-screen camera area, the interaction control area may be the touch screen area, and the second interaction area may be the touch display area; wherein, the under-screen camera area may be located in the upper right corner area, the upper left corner area, the lower right corner area, the upper left corner area or a middle area of ​​the touch screen area.

[0053] In this case, in response to the user's touch operation, the first interaction data corresponding to the first interaction area can be fingerprint image data, and the second interaction data corresponding to the second interaction area can be coordinate data of the touch point corresponding to the touch operation or the induced capacitance data obtained by sensing the touch operation.

[0054] For example, the first interaction area and the second interaction area are respectively one of the pressure-sensitive touch area and the capacitive touch area of ​​the touchpad area; wherein the first interaction area can be located in the upper right corner area, the upper left corner area, the lower right corner area, the upper left corner area or a middle area of ​​the second interaction area.

[0055] In this case, in response to the user's touch operation, the first interaction data corresponding to the first interaction area and the second interaction data corresponding to the second interaction area can be respectively the pressure value data and one of the coordinate data and the induced capacitance data of the touch point corresponding to the touch operation.

[0056] For another example, when the first interaction area is a TOF sensing area and / or a radar sensing area, the interaction control area may be a field of view area of ​​a wide-angle camera, and the second interaction area may be a camera monitoring area.

[0057] In some embodiments, a first response strategy to first interaction data in response to an interaction operation in a first interaction area is different from a second response strategy to second interaction data in response to an interaction operation in a second interaction area.

[0058] For example, when the first interactive area is a fingerprint recognition area and the second interactive area is a force pad, for the user's touch operation, the first interactive data may be fingerprint image data, and the second interactive data may be pressure value data corresponding to the touch point. The fingerprint image data may be processed based on the first response strategy to obtain the fingerprint features of the user, and further complete the user identification or determine the user's action area or operation position in the fingerprint recognition area based on the obtained fingerprint image data; the pressure value data corresponding to the touch point may be processed based on the second response strategy to obtain the touch track corresponding to the touch operation, and further complete the triggering of the function corresponding to the touch track.

[0059] In this case, in response to the user's movement operation, the first interaction data corresponding to the first interaction area may be TOF point cloud data and / or radar point cloud data, and the second interaction data corresponding to the second interaction area may be RGB camera data.

[0060] Step S102: determining first behavior data corresponding to the interaction operation in the first interaction area based on the first interaction data, so as to control the electronic device to at least perform a first control action in response to the first behavior data;

[0061] The first behavior data is of the same type as the second interaction data of the interaction operation in the second interaction area, or,

[0062] The first behavior data is of the same type as the second behavior data obtained by the electronic device in response to the second interaction data.

[0063] Here, the first behavior data may be data representing user behavior characteristics obtained by processing the first interaction data.

[0064] For example, when the first behavior data is a touch trajectory corresponding to a touch operation obtained by processing a fingerprint image and / or an under-screen camera image, or a pressure value corresponding to a touch point, the display content or response action corresponding to the touch trajectory or touch pressure value can be displayed in the display interface of the electronic device based on the touch trajectory or the pressure value corresponding to the touch point, such as controlling the display status of the current application or switching the running status of the current application.

[0065] For another example, when the first behavior data is touch coordinate data converted based on the pressure value corresponding to the touch point, a control at a corresponding position in an application interface currently running on the electronic device can be triggered based on the touch coordinate data.

[0066] For another example, when the first behavior data is user gesture data and / or behavior posture data obtained by converting TOF point cloud data and / or radar point cloud data, the parameters or operating status of the electronic device can be adjusted based on the user gesture data and / or behavior posture data.

[0067] The second behavior data may be data characterizing user behavior characteristics obtained by processing the second interaction data, and may be the same as or different from the first behavior data.

[0068] In some implementations, the first interaction data may be converted into first behavior data of the same type as the second interaction data or the second behavior data, so that the data types corresponding to the first interaction area and the second interaction area are the same. Further, based on a response strategy to the second interaction data or the second behavior data, a response may be made to the interaction operation of the first interaction area.

[0069] For example, the fingerprint recognition controller in the fingerprint recognition area converts the fingerprint image into capacitance value data or pressure value data in the fingerprint recognition area, which is the same data type as the capacitance value data sensed in the capacitance sensing area or the pressure value data sensed in the pressure sensing area. Further, in the area jointly composed of the fingerprint recognition area and the capacitance sensing area or the pressure sensing area, the control instructions for the interactive operation in the fingerprint recognition area can be determined based on the capacitance value data or pressure value data of the fingerprint recognition area.

[0070] For another example, the fingerprint recognition controller in the fingerprint recognition area converts the fingerprint image into coordinate data in the fingerprint recognition area, which is the same data type as the capacitance value data in the capacitive sensing area through the capacitive touch module, or the pressure value data in the pressure sensing area through the pressure sensing touch module. Furthermore, in the area jointly composed of the fingerprint recognition area and the capacitive sensing area or the pressure sensing area, the touch trajectory of the interactive operation in the fingerprint recognition area can be determined based on the coordinate data of the fingerprint recognition area.

[0071] In the embodiment of the present application, the first behavior data corresponding to the interactive operation in the first interactive area is determined based on the first interactive data, and the first behavior data is of the same type as the second interactive data or the second behavior data of the second interactive area of ​​the electronic device. In this way, on the one hand, the interaction data can be unified when the same interactive operation is completed across interactive areas; on the other hand, it is convenient to respond to the first interactive data based on the response strategy of the second interactive data or the second behavior data, that is, in the combined interactive area composed of the first interactive area and the second interactive area, respond to multiple types of interactive data corresponding to the interactive operation across interactive areas, make full use of the device resources of different interactive areas, and improve the utilization rate of each interactive area.

[0072] In some embodiments, the above interactive control method may further include at least one of the following steps S111 to S112:

[0073] Step S111: acquiring second interaction data of the interaction operation in the second interaction area, and fusing the first behavior data and the second interaction data into target interaction data to control the electronic device to execute a target control action in response to the target interaction data.

[0074] Here, for different types of second interaction areas, the second interaction data may be different.

[0075] For example, the second interaction data may include imaging data of the under-screen camera area.

[0076] For another example, the second interaction data may include pressure data and / or touch data of the pressure sensing area.

[0077] For another example, the second interaction data may include point cloud data corresponding to the TOF sensing area or the radar sensing area.

[0078] In some implementations, the data type of the target interaction data may be the same as the second interaction data or the second behavior data.

[0079] The target control action may be a complete response to the interaction operation in the first interaction area and the second interaction area corresponding to the target interaction data.

[0080] In some embodiments, when the first behavior data and the second interaction data are of the same type, the first behavior data and the second interaction data can be processed based on a response strategy to the second interaction data to obtain complete behavior data corresponding to the interaction operations in the first interaction area and the second interaction area; and the complete behavior data is determined as target interaction data to control the electronic device to perform the target control action.

[0081] For example, when the data type of the target interaction data is the same as that of the second interaction data, the target interaction data may be original sensing data, such as sensing capacitance data.

[0082] In some implementations, the original sensing data may be processed by a controller corresponding to the second interactive area to determine a target control action.

[0083] For example, the image data collected by the camera area under the screen can be converted into touch coordinates; the touch coordinates of the camera area under the screen and the touch coordinates of the capacitive sensing area are fused into a complete set of touch coordinates through the capacitive sensing controller of the capacitive sensing area; based on the touch trajectory corresponding to the complete touch coordinates, the electronic device is controlled to execute the target control action in response to the target interaction data.

[0084] Step S112: obtaining second behavior data obtained by the second controller corresponding to the second interaction area in response to the second interaction data, and fusing the first behavior data and the second behavior data into target behavior data to control the electronic device to perform a target control action in response to the target behavior data.

[0085] In some implementations, when the first behavior data and the second behavior data are of the same type, the electronic device can be controlled to respond to the interactive operation in the first interaction area and the second interaction area based on a response strategy to the second behavior data.

[0086] For example, the radar point cloud of the operating body contour collected by the radar sensor in the radar sensing area can be converted into a first motion trajectory of the operating body; through the controller of the TOF sensing area, the TOF point cloud data of the operating body contour can be converted into a second motion trajectory of the operating body; through the controller of the TOF sensing area, based on the first motion trajectory and the second motion trajectory, the electronic device is controlled to perform the target control action.

[0087] For example, the controller of the capacitive touch area can determine the first touch coordinate data and / or the first touch trajectory data corresponding to the capacitive data based on the collected capacitive data; obtain image data corresponding to the under-screen camera area, and convert the image data into second touch coordinate data and / or second touch trajectory data; determine the touch trajectory or gesture action of the operating body based on the first touch coordinate data and the second touch coordinate data, and / or the first touch trajectory data and the second touch trajectory data, so as to control the electronic device to execute the target control action corresponding to the touch trajectory or gesture action.

[0088] In some embodiments, a microcontroller unit (MCU) or a central processing unit (CPU) of the electronic device can be used to obtain second behavior data obtained by a second controller corresponding to the second interaction area in response to second interaction data, and the first behavior data and the second behavior data can be fused and processed into target behavior data to control the electronic device to perform a target control action in response to the target behavior data.

[0089] It can be understood that when the first behavior data and the second behavior data are the same type, there is no need for the controller of the first interaction area and the controller of the second interaction area to respond to the control actions corresponding to the data of the first interaction area and the second interaction area respectively. Therefore, by processing the same type of behavior data through the microprocessing unit or central processing unit of the electronic device, the processing efficiency of the interactive operation can be improved, the response efficiency of the electronic device to the interactive operation can be further improved, and the user experience can be improved.

[0090] In an embodiment of the present application, the first behavior data and the second behavior data are fused and processed into target interaction data to control the electronic device to execute a target control action in response to the target interaction data. In this way, based on a response strategy for interaction data, it is possible to respond to interaction data in different interaction areas, thereby achieving unified processing of cross-area interaction operations; the first behavior data and the second behavior data are fused and processed into target behavior data to control the electronic device to execute a target control action in response to the target behavior data. In this way, based on a response strategy for behavior data, it is possible to respond to behavior data in different interaction areas, thereby achieving a complete response to cross-area interaction operations.

[0091] In some embodiments, the above interactive control method may further include the following steps S121 to S122:

[0092] Step S121: Obtain the time interval between when the interactive operation acts on the first interactive area and when it acts on the second interactive area.

[0093] In some implementations, a first timestamp and a second timestamp of the interaction operation acting on the first interaction area and the second interaction area respectively may be obtained; based on the first timestamp and the second timestamp, a time interval between the interaction operation acting on the first interaction area and the interaction operation acting on the second interaction area may be determined.

[0094] For example, the time interval between the interaction operation acting on the first interaction area and the interaction operation acting on the second interaction area may be determined based on the difference between the start time when the interaction operation acts on the second interaction area and the end time when the interaction operation acts on the first interaction area.

[0095] Step S122: If the time interval is not greater than the first threshold, then executing a processing operation of fusing the first behavior data and the second interaction data, or executing a processing operation of fusing the first behavior data and the second behavior data; or,

[0096] If the time interval is greater than the first threshold, the electronic device is controlled to perform a first control action or a second control action in response to the second interaction data based on the layout information between the first interaction area and the second interaction area.

[0097] Here, the first threshold may include a minimum threshold of the time interval, which is used to determine whether the interactive operation is continuous in the first interactive area and the second interactive area. If the time interval is not greater than the first threshold, it can be determined that the interactive operation is continuous between the first interactive area and the second interactive area, and the processing operation is further performed based on the fusion of the first behavior data and the second interaction data, or the complete data after the fusion of the first behavior data and the second behavior data.

[0098] For example, when the time interval is no more than 0.1ms, the movement operation of the operating body in the TOF sensing area and the radar sensing area can be determined as a continuous operation; the end point of the movement trajectory corresponding to the TOF point cloud data can be connected with the starting point of the movement trajectory corresponding to the radar point cloud data to obtain the complete movement trajectory of the operating body across the TOF sensing area and the radar sensing area, and based on the complete movement trajectory, the electronic device can be controlled to perform the control operation.

[0099] In some embodiments, when the time interval between an interactive operation acting on a first interaction area and an interactive operation acting on a second interaction area is greater than a first threshold, the first interaction data or the second interaction data can be determined as valid interaction data based on the layout information between the first interaction area and the second interaction area; when the first interaction data is valid interaction data, the electronic device is controlled to perform a first control action, or when the second interaction data is valid interaction data, the electronic device is controlled to perform a second control action; wherein the layout information between the first interaction area and the second interaction area may include but is not limited to at least one of the area size, sensing range, position relationship, etc. of the first interaction area and the second interaction area.

[0100] For example, when the interaction area of ​​the first interaction data in the first interaction area is smaller than the effective area threshold and the interaction area of ​​the second interaction data in the second interaction area is larger than the effective area threshold, the first interaction data can be determined to be erroneous operation data; and then the electronic device can be controlled to perform the second control action.

[0101] It can be understood that, based on the layout information between the first interaction area and the second interaction area, it can be judged that the interactive operation acting on the first interaction area or the second interaction area may be an erroneous operation. Therefore, by only responding to valid operations after judging the erroneous operation, the accuracy of the response to the interactive operation can be improved, thereby further improving the user's operating experience.

[0102] In the embodiment of the present application, the control operation performed by the control electronic device is determined based on the time interval between the interaction operation acting on the first interaction area and the interaction operation acting on the second interaction area. In this way, the accuracy of judging whether the interaction operation is continuous between the first interaction area and the second interaction area can be improved, thereby improving the accuracy of matching the control operation performed by the control electronic device with the interaction operation, further improving the possibility of meeting user needs, and improving the user's interaction experience.

[0103] In some embodiments, the above interactive control method may further include at least one of the following steps S131 to S132:

[0104] Step S131: When the interactive operation includes a first interactive operation acting on a first interactive area and a second interactive operation acting on a second interactive area, obtain layout information between the first interactive area and the second interactive area, and based on the layout information, process first interaction data corresponding to the first interactive operation into first behavior data or process second interaction data corresponding to the second interactive operation into third behavior data; wherein the third behavior data is of the same type as the first interaction data or the third behavior data is of the same type as fourth behavior data obtained by the electronic device in response to the first interaction data.

[0105] Here, in the case where the interactive operation spans the first interactive area and the second interactive area, the first interactive area or the second interactive area can be determined as the main area for responding to the interactive data based on the layout information between the first interactive area and the second interactive area, and the data type of the interactive data corresponding to the interactive operation of another interactive area other than the main area can be converted into a data type that can be recognized or processed by the main area.

[0106] In some embodiments, when the layout information between the first interaction area and the second interaction area represents that the first interaction area is the main area, the second interaction data corresponding to the second interaction operation can be processed into third behavior data; and the electronic device can be controlled to execute a target control action in response to the first behavior data and the third behavior data.

[0107] In some implementations, the controller of the first interactive area may be used to control the electronic device to execute a target control action in response to the first behavior data and the third behavior data.

[0108] In some embodiments, when the layout information between the first interaction area and the second interaction area represents that the second interaction area is the main area, the first interaction data corresponding to the first interaction operation can be processed into first behavior data; and the electronic device can be controlled to execute a target control action in response to the first behavior data and the second behavior data.

[0109] In some implementations, the controller of the second interactive area may be used to control the electronic device to execute a target control action in response to the first behavior data and the second behavior data.

[0110] Step S132: When the interactive operation includes a first interactive operation acting on a first interactive area and a second interactive operation acting on a second interactive area, obtain target reference information and layout information between the first interactive area and the second interactive area, and based on the layout information and the target reference information, process first interaction data corresponding to the first interactive operation into first behavior data or process second interaction data corresponding to the second interactive operation into third behavior data.

[0111] Here, the target reference information may include but is not limited to at least one of user habits, device configuration, behavior type corresponding to the interactive operation, usage scenario of the electronic device and / or applications running in the electronic device.

[0112] For example, when the user's frequently used interaction area is the first interaction area and the sensing range of the second interaction area is larger than the sensing range of the first interaction area, the user's interaction habits can be given priority, the second interaction data corresponding to the second interaction operation can be processed into third behavior data, and the interaction operation can be responded to based on the first interaction data and the third behavior data.

[0113] For another example, when the electronic device includes a sensor in a first interaction area, and a sensor and a controller in a second interaction area, the controller in the second interaction area may process first interaction data collected by the sensor in the first interaction area into first behavior data, and respond to the interaction operation based on the first behavior data and the second interaction data.

[0114] In the embodiment of the present application, when the interactive operation includes a first interactive operation acting on a first interactive area and a second interactive operation acting on a second interactive area, the first interactive data corresponding to the first interactive operation can be processed into first behavior data or the second interactive data corresponding to the second interactive operation can be processed into third behavior data based on the layout information between the first interactive area and the second interactive area, or the target reference information and the layout information between the first interactive area and the second interactive area. In this way, the overall data type that responds to the interactive operation can be flexibly selected in different scenarios based on the layout information between the first interactive area and the second interactive area, or the target reference information and the layout information between the first interactive area and the second interactive area.

[0115] The present application embodiment provides an interactive control method, which can be executed by a processor or controller of an electronic device. Figure 2 As shown, the interactive control method includes the following steps S201 to S202:

[0116] Step S201, in response to obtaining a touch operation acting on a touch area of ​​an electronic device, obtaining at least one frame image of the touch operation in a first area, where the first area is an area connected to at least one edge of a second area in the touch area, the second area is an area that supports touch, and the first area is an area that does not support touch.

[0117] Here, the touch area can be an area in the electronic device that responds to the user's touch operation, and can include a plane detection area. For example, the touch area can include but is not limited to a capacitive touch pad area, a force pad area, a touch screen area, a touch bar area, and other plane detection areas.

[0118] In some embodiments, the first area may be an area in the touch area or an area not in the touch area, and may include a plane detection area. For example, the first interaction area may include but is not limited to at least one of a fingerprint recognition area, an under-screen camera area, a force pad area, a TOF sensing area, etc.; wherein the fingerprint recognition area may include a fingerprint sensor area, an ultrasonic fingerprint recognition area, or a camera fingerprint recognition area.

[0119] For different types of first regions, the content of at least one frame of image of the first region under the touch operation may be different.

[0120] For example, the image collected by the fingerprint recognition area may be a fingerprint image of the user.

[0121] For another example, the image captured by the under-screen camera may include an image of the location where the user touches the screen;

[0122] The second area may include a portion or the entire area of ​​the touch area.

[0123] It is understandable that the first area and the second area are connected at least at one edge, that is, the first area and the second area can be crossed in one touch operation, and the second area supports touch control for the touch operation, while the first area does not support touch control for the touch operation.

[0124] In some embodiments, the first area may be located in the upper right corner area, the upper left corner area, the lower right corner area, the upper left corner area or a middle area of ​​the second area, such as the first area is a fingerprint recognition area.

[0125] Step S202: determining first touch data corresponding to the touch operation in the first area based on at least one frame of image, so as to control the electronic device to execute a third control action in response to the first touch data.

[0126] Here, the first touch data may be data representing touch operation characteristics obtained by processing at least one frame of image, for example, may include touch track, touch coordinates or point cloud data corresponding to the touch point of the touch operation.

[0127] For example, the touch point coordinates of the user's touch operation in the under-screen camera area can be determined based on the position of the user's finger in a frame of image captured by the under-screen camera.

[0128] For another example, the touch trajectory of the user's touch operation in the fingerprint recognition area may be determined based on the user's fingerprint images collected by the multi-frame fingerprint recognition module.

[0129] In an embodiment of the present application, by controlling the electronic device to execute a third control action in response to the first touch data determined based on at least one frame of an image of the first area, a touch response to the first area that does not support touch can be achieved, thereby fully utilizing the device resources of the first area and improving the utilization rate of the first area.

[0130] In some embodiments, the above interactive control method may further include the following steps S211:

[0131] Step S211: obtaining second touch data of the touch operation on the second area, and fusing the first touch data and the second touch data into target touch data to control the electronic device to execute a target control action in response to the target touch data.

[0132] Here, the second touch data can be the interaction data of the touch operation in the second area, or it can be the behavior data determined based on the interaction data of the touch operation in the second area. For example, it can include the capacitance value, pressure value, touch trajectory, touch coordinates and / or point cloud data corresponding to the touch point of the touch operation.

[0133] In some embodiments, the first touch data can be processed into first target touch data by a processor or a control chip in the first interaction area; the second touch data can be processed into second target touch data by a processor or a control chip in the second interaction area; the first target touch data and the second target touch data can be fused and processed into target touch data to control the electronic device to perform a target control action in response to the target touch data.

[0134] For example, the first touch data may be a pressure value of a touch point of a touch operation, and a first touch trajectory corresponding to the first touch data may be determined by a pressure sensing controller based on the pressure value of the touch point. When the second area is a capacitive sensing area, a second touch trajectory corresponding to the second touch data may be determined by a capacitive sensing controller based on the capacitance value of the touch point of the touch operation. The first touch trajectory is superimposed on the second touch trajectory to obtain a target touch trajectory as the target touch data, so as to control the electronic device to perform a target control action in response to the target touch trajectory.

[0135] In the embodiment of the present application, the first touch data and the second touch data are fused and processed into target touch data to control the electronic device to perform a target control action in response to the target touch data. In this way, on the basis of realizing touch response to the first area that does not support touch, a complete response to touch operations corresponding to multiple touch data across regions can be realized based on a response strategy for a target touch data.

[0136] In some embodiments, the above interactive control method may further include the following steps S221 to S222:

[0137] Step S221: Obtain the time interval between when the touch operation acts on the first area and when the touch operation acts on the second area.

[0138] In some implementations, a first touch timestamp and a second touch timestamp of a touch operation acting on the first area and the second area respectively may be acquired; based on the first touch timestamp and the second touch timestamp, a time interval between the touch operation acting on the first area and the touch operation acting on the second area may be determined.

[0139] For example, the time interval between the touch operation acting on the first area and the touch operation acting on the second area may be determined based on the difference between the start time when the touch operation acts on the second area and the end time when the touch operation acts on the first area.

[0140] Step S222: If the time interval is not greater than the second threshold, performing a processing operation of fusing the first touch data and the second touch data; or,

[0141] If the time interval is greater than the second threshold, the electronic device is controlled to perform a third control action based on the target reference information, the electronic device is controlled to perform a fourth control action to match the recognition result of at least one frame of image, or the electronic device is controlled to perform a fifth control action in response to the second touch data.

[0142] Here, the second threshold may include a minimum threshold of the time interval, which is used to determine whether the touch operation is continuous in the first area and the second area. If the time interval is not greater than the second threshold, it can be determined that the touch operation is continuous between the first area and the second interactive area, and further based on the complete data after the first touch data and the second touch data are fused, a processing operation is performed.

[0143] For example, when the time interval is no more than 0.2ms, it can be determined that the touch trajectory of the user in the TOF sensing area and the capacitive touch area is a continuous trajectory; the trajectory end point of the moving trajectory corresponding to the TOF point cloud data can be connected with the trajectory starting point of the moving trajectory determined based on the capacitance value of the touch point in the capacitive touch area to obtain the complete touch trajectory of the user across the TOF sensing area and the capacitive sensing area, and based on the complete touch trajectory, the electronic device can be controlled to perform control operations.

[0144] The target reference information may include at least one of a time interval value, user sight data, a camera monitoring image, an operation mode of the electronic device, a usage scenario of the electronic device, and / or an application running in the electronic device.

[0145] In some embodiments, when the time interval between the touch acting on the first area and the touch acting on the second area is greater than a second threshold, the first touch data or the second touch data may be determined as valid touch data based on the target reference information;

[0146] When the first touch data is valid touch data, controlling the electronic device to perform a third control action, or,

[0147] Based on the recognition result of at least one frame of image, control the electronic device to perform a fourth control action, or,

[0148] When the second touch data is valid touch data, the electronic device is controlled to execute a fifth control action.

[0149] Exemplarily, for the recognition result based on at least one frame of image, the electronic device is controlled to perform the fourth control action. When the first area is a fingerprint recognition area, the fingerprint image can be recognized to realize operations such as calling up the application, switching the operating status of the electronic device, and performing data sharing verification.

[0150] For example, in response to the user's long press operation, the fingerprint recognition area can obtain at least one frame of the user's fingerprint image, identify the user corresponding to the at least one frame of fingerprint image as the holder of the electronic device, and the touch coordinates corresponding to the second touch operation represent that the user has performed a selection operation on the data in the interface. Further, when the time interval between the long press operation and the second touch operation is greater than a second threshold, the data at the position corresponding to the second touch operation in the current display interface of the electronic device is uploaded to the target address.

[0151] Exemplarily, controlling the electronic device to perform a fifth control action in response to the second touch data may include determining the fifth control action corresponding to the second touch data based on an application running in the electronic device or a current usage scenario of the electronic device, thereby further controlling the electronic device.

[0152] For example, when the application running in the electronic device is a document application, and the fingerprint recognition area responds to the user's sliding operation, and after a time interval greater than a second threshold, the operation corresponding to the second touch data is also a sliding operation; multiple frames of the user's fingerprint images are obtained, and based on the moving trajectories corresponding to the multiple frames of fingerprint images and the moving trajectory of the second touch data, it is determined that the sliding trajectory of the sliding operation is similar to the sliding trajectory of the second touch data, and it is judged that the user is flipping through the document, and the display interface of the electronic device is further focused on the position corresponding to the sliding trajectory of the second touch data.

[0153] It can be understood that based on the target reference information, it is possible to determine whether the touch operation is a false touch operation in the first area or the second area. Therefore, by only responding to valid touch operations after determining the false touch operation, the accuracy of the response to the touch operation can be improved, further improving the user's touch experience.

[0154] In the embodiment of the present application, the control operation performed by the control electronic device is determined based on the time interval between the touch operation acting on the first area and the touch operation acting on the second area, or the time interval and the target reference information. In this way, on the one hand, the accuracy of judging whether the touch operation is continuous between the first area and the second area can be improved based on the time interval; on the other hand, whether the touch operation contains a false touch operation can be judged based on the target reference information, thereby improving the accuracy of matching the control operation performed by the control electronic device with the touch operation, further improving the possibility of meeting user needs, and improving the user's touch experience.

[0155] In some embodiments, the above step S202 may include the following steps S231 to S232:

[0156] Step S231: Determine a target reference point in the first area and a target feature point in the image.

[0157] Here, the target reference point may include a reference point for determining the position of an arbitrary point in the first area.

[0158] For example, the target reference point may be at least one of the intersection of the left edge and the upper edge, the intersection of the right edge and the upper edge, the intersection of the left edge and the lower edge, and the intersection of the right edge and the lower edge of the first area.

[0159] The target feature point may be a point in the image that represents a feature of the touch operation.

[0160] For example, the image captured by the under-screen camera represents the touch point corresponding to the contact surface between the user and the first area.

[0161] Another example is the point in the fingerprint image that represents the user's fingerprint features.

[0162] Step S232: determine first touch data based on relative position information or relative position change information of the target feature point and the target reference point in at least one frame of image.

[0163] In some implementations, the relative position change information of the target feature point and the target reference point in the previous frame image of the frame may be determined based on the relative position information of the target feature point and the target reference point in each frame image.

[0164] Here, for each frame image, based on the relative position information of the target feature point and the target reference point in the image, the position of the target feature point in the frame image in the first area can be determined; based on the position of the target feature point in the first area in each frame image, the acquisition moments corresponding to different frame images and the characteristics of the touch operation can be determined.

[0165] For each frame of image, based on the relative position change information of the target feature point and the target reference point, the change characteristics of the touch operation between the acquisition time of the frame and the acquisition time of the historical image frame corresponding to the relative position change information can be determined.

[0166] For example, the intersection of the left edge and the upper edge in the first area can be determined as the target reference point. Based on the distance between at least one starting feature point of the contact surface between the user and the first area in the starting frame image and the target reference point in multiple frame images of the touch operation in the first area, the moving trajectory of the touch operation in the first area can be determined.

[0167] In the embodiment of the present application, the first touch data is determined based on the relative position information or relative position change information of the target feature point in the image and the target reference point in the first area in at least one frame of the image. In this way, the accuracy of obtaining the first touch data feature can be improved, and the accuracy of determining the first touch data can be further improved.

[0168] An embodiment of the present application provides an electronic device, including a first body, a touch panel disposed on the first body, and at least one controller.

[0169] like Figure 3 As shown, the electronic device 300 may include a first body 310;

[0170] A touch panel 311 disposed on the first body 310, the touch panel 311 includes a second area and a first area connected to at least one edge of the second area, the second area is provided with a touch module 312, and the first area is provided with a first recognition module 313 different from the touch module;

[0171] At least one controller 320 can process at least one frame of image collected by the first recognition module into first touch data, and can control the electronic device to execute a third control action in response to the first touch data.

[0172] In some embodiments, the touch module in the second area may include a capacitive touch module and / or a pressure sensing touch module.

[0173] In some embodiments, the first recognition module may be an ultrasonic recognition module or an under-screen camera recognition module.

[0174] Here, the touch panel 311 can correspond to the touch area in the above-mentioned interactive control method. In response to obtaining a touch operation acting on the touch panel 311, at least one frame image of the touch operation in the first area can be obtained through the first recognition module 313; at least one controller 320 determines the first touch data corresponding to the touch operation in the first area based on at least one frame image, and controls the electronic device to perform a third control action in response to the first touch data.

[0175] In some embodiments, the first area is located on the first display screen of the first body, and the first body of the electronic device may also include a second display screen connected to at least one edge of the second area; the second display screen may include modules that can respond to interactive data, such as a camera, a TOF sensor, and / or a radar sensor. The response area corresponding to the module that can respond to interactive data in the second display screen may correspond to the first interactive area in the above-mentioned interactive control method; based on the first interactive data of the first interactive area in the second display screen, the first behavior data corresponding to the interactive operation in the first interactive area of ​​the second display surface may be determined to control the electronic device to at least perform the first control action in response to the first behavior data corresponding to the second display surface.

[0176] In some embodiments, the electronic device may further include a second body connected to at least one edge of the second area; the second body may be a body for placing a display screen, for example, the second body may include a body integrating a camera, a TOF sensor, and / or a radar sensor, etc., which are modules that can respond to interactive data. The response area corresponding to the module that can respond to interactive data in the second body may correspond to the first interactive area in the above-mentioned interactive control method; based on the first interactive data of the first interactive area in the second body, the first behavior data corresponding to the interactive operation in the first interactive area of ​​the second body may be determined to control the electronic device to at least perform the first control action in response to the first behavior data corresponding to the second body.

[0177] In some embodiments, the first recognition module is a fingerprint recognition module, and at least one controller includes a first controller connected to the fingerprint recognition module signal and a second controller connected to the first controller signal; the second controller can, when the touch module obtains the second touch data, fuse the second touch data and the first touch data obtained by the first controller processing at least one frame of image into target touch data, so as to control the electronic device to perform a target control action in response to the target touch data.

[0178] In some embodiments, at least one controller can also be used to: obtain the time interval between the touch operation acting on the first area and the touch operation acting on the second area; if the time interval is not greater than a second threshold, perform a processing operation that fuses the first touch data and the second touch data; or, if the time interval is greater than the second threshold, control the electronic device to perform a third control action based on the target reference information, control the electronic device to perform a fourth control action to match the recognition result of at least one frame of image, or control the electronic device to perform a fifth control action in response to the second touch data.

[0179] In some embodiments, at least one controller can also be used to: determine a target reference point in the first area and a target feature point in the image; and determine first touch data based on relative position information or relative position change information of the target feature point and the target reference point in at least one frame of image.

[0180] The fingerprint module is an important biometric identification component for computer device user login. There are different solutions in the relevant technology regarding the placement and shape of the fingerprint module.

[0181] like Figure 4A As shown, in the related art, the fingerprint module 21 and the touch module 11 of the touch panel area 10 are integrated into the touch panel.

[0182] This form brings a simple display effect, but in the related technology, the touch panel realizes touch through a printed circuit board (PCB), and there is a layer and distance between the sensing layer of the touch module and the finger; when touch is based on fingerprints, the fingerprint module needs to be in close contact with the finger, for example, the distance between the fingerprint module and the finger cannot be greater than 0.175mm.

[0183] In this case, a separate area needs to be reserved for the fingerprint module on the touchpad, such as Figure 4B As shown, the PCB layer 12 of the touch module is covered with a cover plate 13, and a hole corresponding to the identification area of ​​the fingerprint module 21 in the cover plate 13 needs to be dug for fingerprint identification, resulting in a height difference between the fingerprint module area and the touch module area of ​​the cover plate 13 perpendicular to the horizontal plane. This results in an obvious plane difference when the user uses the touch panel.

[0184] And if Figure 4C As shown, due to the presence of the fingerprint module, the area 31 where the fingerprint module is located in the touch panel 30 cannot realize the touch function, which will cause a waste of device resources and affect the user's touch operation experience.

[0185] In the related technology, there is also a fingerprint module based on ultrasonic fingerprint recognition. The principle of ultrasonic fingerprint collection is to use the ability of ultrasonic waves to penetrate materials and produce reflected waves of different sizes depending on the material. That is, when ultrasonic waves reach the surfaces of different materials, they are absorbed, penetrated and reflected to different degrees.

[0186] Therefore, if Figure 4D As shown, by utilizing the difference in acoustic impedance when ultrasonic waves propagate through fingerprint ridges 41, i.e., raised lines on the surface of the finger, and fingerprint valleys 42, i.e., concave portions on the surface of the finger, the positions of the fingerprint ridges 41 and the fingerprint valleys 42 in the fingerprint image 40 can be distinguished.

[0187] The fingerprint module based on ultrasonic recognition has the characteristics of strong penetration and can penetrate glass with a thickness of more than 1.2mm. However, since ultrasound cannot penetrate more than two media at the same time, this solution still requires opening a hole in the touch panel cover, which means that there is still a problem of not being able to achieve touch in the area corresponding to the fingerprint module.

[0188] On this basis, an embodiment of the present application provides an electronic device, such as Figure 5A As shown, the electronic device includes: a cover plate 501, an ultrasonic fingerprint module 502, a PCB503 of a PCB touch module, and a supporting structure 504; wherein, the cover plate 501 can be made of glass, the ultrasonic fingerprint module 502 is located below the cover plate 501, and the ultrasonic fingerprint module 502 can penetrate the cover plate 501; the PCB503 is located below the cover plate 501, and an opening corresponding to the position of the ultrasonic fingerprint module 502 is reserved in the PCB503; the supporting structure 504 is located below the PCB503.

[0189] In this way, on the one hand, retaining a complete cover plate 501 as the upper surface of the touch pad of the electronic device 500 can bring a simple and clean display effect; on the other hand, reserving an opening position in PCB503 according to the ultrasonic fingerprint module 502 can avoid mutual influence between the PCB touch module and the ultrasonic fingerprint module 502, and at the same time can reduce the thickness of the entire touch pad area.

[0190] In some embodiments, support structure 504 may be a metal structure.

[0191] Based on the above electronic device, the embodiment of the present application provides an interactive control method, which can be executed by a processor of a computer. Figure 5B As shown, the method includes the following steps S501 to S504:

[0192] Step S501: In response to a touch operation by a user, any point in the fingerprint recognition area is set as a reference point, and distance calculation is performed through multiple feature points of the fingerprint image to obtain a movement trajectory of the finger in the fingerprint recognition area.

[0193] Here, the fingerprint recognition area may correspond to the first interactive area and the first area in the above interactive control method. The fingerprint module corresponding to the fingerprint recognition area may correspond to the ultrasonic fingerprint module in the above interactive control method.

[0194] The reference point corresponding to any point in the fingerprint recognition area can correspond to the target reference point in the above-mentioned interactive control method.

[0195] The fingerprint image may correspond to the first interactive data in the above interactive control method, or may correspond to at least one image frame of the first area. The feature points of the fingerprint image may correspond to the target feature points in the above interactive control method.

[0196] The movement trajectory of the finger in the fingerprint recognition area may correspond to the first behavior data in the above-mentioned interactive control method, or may correspond to the first touch data in the above-mentioned interactive control method.

[0197] like Figure 5C As shown, based on the relative position information between the target reference point 511 and the target feature points 521, 522, 523, 524, 525, 526, and 527, the movement trajectory 530 of the touch operation in the fingerprint recognition area can be determined.

[0198] During implementation, the characteristic of ultrasound waves that they can collect images at ultra-high frequencies can be utilized, and multiple fingerprint images can be collected during the movement of the finger.

[0199] Step S502: quantify the movement trajectory information and report it to the host end.

[0200] During implementation, the controller in the fingerprint module corresponding to the fingerprint recognition area can quantify the movement trajectory information and report it to the host after determining the movement trajectory of the finger in the fingerprint recognition area; or the fingerprint module corresponding to the fingerprint recognition area can collect the fingerprint image and report it to the host, which will calculate the movement trajectory.

[0201] In some implementations, a high-performance MCU may be used to calculate the movement trajectory of target feature points of multiple images.

[0202] Step S503: The host side performs a fusion calculation of the movement trajectory based on the reported movement trajectory of the finger in the fingerprint recognition area and the movement trajectory of the finger in the touch area corresponding to the touch module of the touchpad, obtains a combined movement trajectory from the touch area to the fingerprint recognition area, and reports the combined movement trajectory to the system layer.

[0203] Here, based on the fusion calculation of the moving trajectory on the host side, compared with the system layer, there is no need for the system to judge whether the trajectory is located in the touchpad area or the fingerprint recognition area based on the trajectory data. The touch operation can be responded to directly based on the fused trajectory calculation result, thereby improving the processing efficiency and response efficiency. Compared with the hardware layer, the processing speed of the host side is also faster and the calculation efficiency is higher.

[0204] The touch area corresponding to the touch module may correspond to the second interaction area and the second area in the above-mentioned interaction control method.

[0205] During implementation, the host side can determine whether the two trajectories are continuous based on the timestamps of the two trajectories, and if a continuous trajectory is obtained, the continuous trajectory is reported to the system layer.

[0206] Step S504: The system layer controls the electronic device to execute a control action in response to the touch operation based on the combined movement trajectory.

[0207] Here, for the system layer, the movement trajectory of the fingerprint recognition area reported by the host is the same as the movement trajectory of the touchpad, and touch operations under the operating system such as cursor control and multi-finger manipulation can also be performed as control actions.

[0208] like Figure 5D As shown, after the touch module 531 obtains the electrical signal of the touch operation in the touch area 530 corresponding to the touch module 531, the processor 532 of the touch module 531 determines the movement trajectory 533 of the touch operation in the touch area 530 based on the capacitance value corresponding to the electrical signal, and reports it to the host end 550; after the fingerprint module 541 obtains the fingerprint image of the touch operation in the fingerprint recognition area 540 corresponding to the fingerprint module 541, the processor 542 of the fingerprint module 541 calculates the movement trajectory 543 of the touch operation in the touch area 540 based on the fingerprint image, and reports it to the host end 550; the host end 550 determines that the movement trajectory 533 is continuous with the movement trajectory 543 based on the time interval between the received movement trajectory 533 and the movement trajectory 543, merges the movement trajectory 533 and the movement trajectory 543 into a complete and continuous movement trajectory 551, and reports the movement trajectory 551 to the operating system 560, which responds.

[0209] In the embodiments of the present application, on the one hand, it is possible to realize touch response for the fingerprint recognition area where the fingerprint module that does not support touch is located, thereby making full use of device resources; on the other hand, by unifying the fingerprint image and the electrical signal into data of the movement trajectory type, it is convenient to realize a complete response to touch operations corresponding to multiple touch data across regions based on a response strategy of one data.

[0210] An embodiment of the present application provides a computer program, including a computer-readable code. When the computer-readable code is executed in a computer device, a processor in the computer device executes some or all of the steps for implementing the above-mentioned interactive control method.

[0211] An embodiment of the present application provides a computer program product, including a computer program or instructions, which, when executed by a processor, implements some or all of the steps in the above-mentioned interactive control method.

[0212] An embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program can be executed by a processor to implement the above-mentioned interactive control method.

[0213] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatuses and devices according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0214] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0215] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0216] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed in the present application should be included in the protection scope of the present application.

[0217] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0218] It should be understood that "one embodiment" or "an embodiment" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution, and the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are only for description and do not represent the advantages and disadvantages of the embodiments.

[0219] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0220] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0221] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0222] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0223] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0224] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application can be essentially or partly embodied in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0225] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. An interactive control method, comprising: In response to obtaining an interactive operation acting on an interactive control area of ​​an electronic device, first interactive data of the interactive operation in a first interactive area is obtained, where the first interactive area is an area connected to at least one edge of a second interactive area in the interactive control area, and the types of interactive data obtained by the second interactive area and the first interactive area in response to the interactive operation are different; Determining first behavior data corresponding to the interaction operation in the first interaction area based on the first interaction data, so as to control the electronic device to at least perform a first control action in response to the first behavior data; The first behavior data is of the same type as the second interaction data of the interaction operation in the second interaction area, or, The first behavior data is of the same type as the second behavior data obtained by the electronic device in response to the second interaction data.

2. The method according to claim 1, further comprising at least one of the following: Acquire second interaction data of the interaction operation in the second interaction area, and fuse the first behavior data and the second interaction data into target interaction data to control the electronic device to perform a target control action in response to the target interaction data; Acquire second behavior data obtained by the second controller corresponding to the second interaction area in response to the second interaction data, and fuse the first behavior data and the second behavior data into target behavior data to control the electronic device to perform a target control action in response to the target behavior data.

3. The method according to claim 2, further comprising: Acquire a time interval between when the interactive operation acts on the first interactive area and when the interactive operation acts on the second interactive area; If the time interval is not greater than a first threshold, performing a processing operation of fusing the first behavior data and the second interaction data, or performing a processing operation of fusing the first behavior data and the second behavior data; or, If the time interval is greater than a first threshold, the electronic device is controlled to perform the first control action or to perform a second control action in response to the second interaction data based on layout information between the first interaction area and the second interaction area.

4. The method according to claim 1, wherein: Also includes at least one of the following: In a case where the interaction operation includes a first interaction operation acting on the first interaction area and a second interaction operation acting on the second interaction area, obtaining layout information between the first interaction area and the second interaction area, and processing first interaction data corresponding to the first interaction operation into first behavior data or processing second interaction data corresponding to the second interaction operation into third behavior data based on the layout information; In a case where the interaction operation includes a first interaction operation acting on the first interaction area and a second interaction operation acting on the second interaction area, acquiring target reference information and layout information between the first interaction area and the second interaction area, and processing first interaction data corresponding to the first interaction operation into first behavior data or processing second interaction data corresponding to the second interaction operation into third behavior data based on the layout information and the target reference information; The third behavior data is of the same type as the first interaction data or the third behavior data is of the same type as fourth behavior data obtained by the electronic device in response to the first interaction data.

5. An interactive control method, comprising: In response to obtaining a touch operation acting on a touch area of ​​the electronic device, obtaining at least one frame image of the touch operation in a first area, the first area being an area connected to at least one edge of a second area in the touch area, the second area being an area supporting touch control, and the first area being an area not supporting touch control; The first touch data corresponding to the touch operation in the first area is determined based on the at least one frame of image, so as to control the electronic device to execute a third control action in response to the first touch data.

6. The method according to claim 5, wherein: Also includes: Second touch data of the touch operation in the second area is obtained, and the first touch data and the second touch data are fused and processed into target touch data to control the electronic device to perform a target control action in response to the target touch data.

7. The method according to claim 6, further comprising: Acquire a time interval between when the touch operation acts on the first area and when the touch operation acts on the second area; If the time interval is not greater than a second threshold, performing a processing operation of fusing the first touch data and the second touch data; or, If the time interval is greater than a second threshold, the electronic device is controlled to perform the third control action based on the target reference information, the electronic device is controlled to perform the fourth control action matching the recognition result of the at least one frame of image, or the electronic device is controlled to perform the fifth control action in response to the second touch data.

8. The method according to claim 5, wherein: Determining first touch data corresponding to the touch operation in the first area based on the at least one frame of image data includes: Determining a target reference point in the first area and a target feature point in the image; The first touch data is determined based on relative position information or relative position change information of the target feature point and the target reference point in the at least one frame of image.

9. An electronic device, comprising: first ontology; A touch panel disposed on the first body, the touch panel comprising a second area and a first area connected to at least one edge of the second area, the second area being provided with a touch module, and the first area being provided with a first identification module different from the touch module; At least one controller is capable of processing at least one frame of image collected by the first recognition module into first touch data, and is capable of controlling the electronic device to execute a third control action in response to the first touch data.

10. The electronic device according to claim 9, wherein: The first recognition module is a fingerprint recognition module, and the at least one controller includes a first controller connected to the fingerprint recognition module signal and a second controller connected to the first controller signal; the second controller can, when the touch module obtains second touch data, fuse the second touch data and the first touch data obtained by the first controller processing the at least one frame of image into target touch data, so as to control the electronic device to perform a target control action in response to the target touch data.

Images