Vehicle-mounted terminal control method and device based on gesture recognition, equipment and medium

Abstract

The invention discloses a vehicle-mounted terminal control method and device based on gesture recognition, equipment and a medium, and the method comprises the steps: extracting corresponding operation feature information from received touch operation information according to an operation feature extraction rule, carrying out the gesture recognition of the operation feature information according to a gesture recognition model, and obtaining corresponding gesture information, and obtaining an operation instruction corresponding to the gesture information from a gesture instruction library according to the current active application window, and executing the operation instruction in the current active application window to control the application. According to the vehicle-mounted terminal control method, the touch operation information can be recognized through gesture recognition to obtain the corresponding gesture information, so that the application is controlled through the gesture information; the driver does not need to carefully identify the key function and the key type, so that the driver is prevented from controlling the vehicle-mounted terminal by shifting the sight line in the driving process, and the driver is prevented from being distracted in the vehicle driving process to influence the driving safety.

Description

Technical Field

[0001] The present invention relates to the field of intelligent control technology, and in particular to a vehicle-mounted terminal control method, device, equipment and medium based on gesture recognition. Background Art

[0002] With the development of automobile intelligence, more and more cars are equipped with multimedia terminals, on which applications can be run, and the multimedia terminals can obtain control information input by users to control automobile equipment or multimedia playback. However, the multimedia terminals in the prior art are usually equipped with multiple physical buttons and / or touch screens, and users can input relevant control information through physical buttons; multiple virtual buttons can be configured on the touch screen, and users can also input relevant control information through virtual buttons. However, when the driver is concentrating on driving the vehicle, the driver must also check the virtual buttons or physical buttons and distinguish the button functions and button types. The control method in the prior art makes it impossible for the driver to control the vehicle terminal without diverting his sight, which easily leads to the driver's distraction when operating and controlling the vehicle terminal, thereby affecting the safety of vehicle driving. Therefore, the control method for the vehicle terminal in the prior art method has the problem of affecting driving safety. Summary of the invention

[0003] The embodiments of the present invention provide a vehicle-mounted terminal control method, device, equipment and medium based on gesture recognition, aiming to solve the problem of affecting driving safety in the control method for the vehicle-mounted terminal in the prior art method.

[0004] In a first aspect, an embodiment of the present invention provides a vehicle terminal control method based on gesture recognition, the method is applied to a controller of the vehicle terminal, the controller establishes a communication connection with a touch screen configured in the vehicle terminal to realize data information transmission, wherein the method includes:

[0005] If touch operation information is received from the touch screen, corresponding operation feature information is extracted from the touch operation information according to a preset operation feature extraction rule;

[0006] Performing gesture recognition on the operation feature information according to a preset gesture recognition model to obtain corresponding gesture information;

[0007] Acquire an operation instruction corresponding to the gesture information from a pre-stored gesture instruction library according to the current active application window;

[0008] The operation instruction is executed in the currently active application window to control the application.

[0009] In a second aspect, an embodiment of the present invention further provides a vehicle-mounted terminal control device based on gesture recognition, wherein the device is used to execute the vehicle-mounted terminal control method based on gesture recognition as described in the first aspect above, the device is configured in a controller of the vehicle-mounted terminal, the controller establishes a communication connection with a touch screen configured in the vehicle-mounted terminal to realize the transmission of data information, and the device includes:

[0010] an operation feature information acquisition unit, configured to extract corresponding operation feature information from the touch operation information according to a preset operation feature extraction rule upon receiving touch operation information from the touch screen;

[0011] A gesture information acquisition unit, used to perform gesture recognition on the operation feature information according to a preset gesture recognition model to obtain corresponding gesture information;

[0012] An operation instruction acquisition unit, used for acquiring an operation instruction corresponding to the gesture information from a pre-stored gesture instruction library according to the current active application window;

[0013] An operation instruction execution unit is used to execute the operation instruction in the currently active application window to control the application.

[0014] In a third aspect, an embodiment of the present invention further provides a computer device, wherein the device includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0015] Memory, used to store computer programs;

[0016] The processor is used to implement the steps of the vehicle-mounted terminal control method based on gesture recognition described in the first aspect when executing the program stored in the memory.

[0017] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps of the vehicle-mounted terminal control method based on gesture recognition as described in the first aspect above are implemented.

[0018] The embodiment of the present invention provides a vehicle terminal control method, device, equipment and medium based on gesture recognition, the method comprising: extracting corresponding operation feature information from received touch operation information according to operation feature extraction rules, performing gesture recognition on the operation feature information according to a gesture recognition model to obtain corresponding gesture information, obtaining operation instructions corresponding to the gesture information from a gesture instruction library according to the current active application window, and executing the operation instructions in the current active application window to control the application. The above vehicle terminal control method can identify the touch operation information through gesture recognition to obtain corresponding gesture information, thereby controlling the application through gesture information; there is no need for the driver to carefully identify the button function and button type, thereby avoiding the driver's need to divert his or her sight to control the vehicle terminal during driving, and avoiding the driver's distraction during driving and affecting driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0020] Figure 1 A method flow chart of a vehicle terminal control method based on gesture recognition provided by an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of an application scenario of a vehicle terminal control method based on gesture recognition provided by an embodiment of the present invention;

[0022] Figure 3 A schematic block diagram of a vehicle-mounted terminal control device based on gesture recognition provided by an embodiment of the present invention;

[0023] Figure 4 It is a schematic block diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0026] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0027] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0028] See also Figure 1 and Figure 2 As shown in the figure, the embodiment of the present invention provides a vehicle terminal control method based on gesture recognition, which is applied to the controller 101 of the vehicle terminal 10 and is executed by the application software installed in the controller 101. Figure 2 As shown, the vehicle-mounted terminal 10 is also configured with a touch screen 102, and the controller 101 establishes a communication connection with the touch screen 102 to realize the transmission of data information. The vehicle-mounted terminal 10 is a terminal device installed in the car for display and control. The user can view the content displayed on the touch screen 102, and the user can also input corresponding control information through the touch screen 102 to control the vehicle-mounted terminal 10. The controller 101 can monitor the touch operation of the touch screen 102 to obtain touch operation information. The controller 101 performs gesture recognition on the touch operation information to obtain corresponding operation instructions and control the application running on the vehicle-mounted terminal 10. In order to meet the actual use requirements of the vehicle-mounted terminal 10, other components such as buttons and speakers can be configured in the vehicle-mounted terminal 10. This application is in Figure 2 Other components are not marked. Figure 1 As shown, the method includes steps S110 to S140.

[0029] S110: If touch operation information is received from the touch screen, corresponding operation feature information is extracted from the touch operation information according to a preset operation feature extraction rule.

[0030] If touch operation information is received from the touch screen, the corresponding operation feature information is extracted from the touch operation information according to the preset operation feature extraction rules. The touch screen can send the received touch operation information to the controller, and the controller can extract the features of the touch operation information to obtain the operation feature information. The touch screen can sense the change of voltage / current. In the case of no touch operation (the user's hand does not touch the touch screen), the voltage / current of the touch screen is the initial value; when the user performs a touch operation (the user's hand touches the touch screen), the voltage / current of the touch screen will change and be different from the initial value; when the user completes the touch operation, the hand is separated from the touch screen, and the voltage / current of the touch screen returns to the initial value. Each change in voltage / current corresponds to a touch operation, and a set of touch operation information can be obtained accordingly. The touch IC analyzes the current / voltage changes of the touch screen to obtain the corresponding touch operation information, and the controller can obtain the touch operation information accordingly and perform subsequent analysis and processing.

[0031] In a specific embodiment, step S110 includes sub-steps: splitting according to the touch points contained in the touch operation information to obtain corresponding single-point operation information; obtaining corresponding single-point operation features from each of the single-point operation information according to the extraction items in the operation feature extraction rule; and combining each of the single-point operation features to obtain the operation feature information.

[0032] Specifically, the touch operation information contains at least one touch point's operation information, and the touch points contained in the touch operation information can be split to obtain the single-point operation information of each touch point; each touch point corresponds to a finger or a hand area (such as the palm) operating on the touch screen. According to the extraction items in the operation feature extraction rule, the corresponding single-point operation features are obtained from each single-point operation information, and the feature values ​​included in the single-point operation features can characterize the features included in the single-point operation information of a touch point.

[0033] The extraction items in the operation feature extraction rule include single-point contact duration, single-point drag distance, single-point drag direction, single-point average contact area (when the touch point is dragged on the screen, the contact area may change, and the average value of the single-point contact area collected multiple times during the time period from the touch point contacting the screen to leaving the screen is taken); the single-point contact duration is the time period from the touch point contacting the screen to leaving the screen; the single-point drag distance is the distance between the center point of the touch point contacting the screen and the center point of leaving the screen, and the single-point drag distance is measured in pixels; the single-point drag direction is the normal between the center point of the touch point contacting the screen and the center point of leaving the screen (based on the clock 12 o'clock direction, ranging from 0 to 360°), and the single-point drag direction can be determined by connecting the center point of the touch point contacting the screen as the starting point and the center point of leaving the screen as the end point. Then each extraction item can correspond to a feature value, and the single-point operation feature of the single-point operation information can be obtained by obtaining the feature values ​​corresponding to each feature item from the single-point operation information. By acquiring the single-point operation features of each single-point operation information and combining them, the operation feature information corresponding to the touch operation information can be obtained.

[0034] S120: Perform gesture recognition on the operation feature information according to a preset gesture recognition model to obtain corresponding gesture information.

[0035] The operation feature information is gesture-recognized according to a preset gesture recognition model to obtain corresponding gesture information. Further, the operation feature information can be gesture-recognized by the gesture recognition model to identify the gesture type of the user performing the operation feature information, that is, to obtain the corresponding gesture information.

[0036] In a specific embodiment, step S120 includes sub-steps: obtaining a gesture type in the gesture recognition model that matches the number of valid contacts contained in the operation feature information as a target gesture type; determining whether the single-point operation feature in the operation feature information matches the limiting conditions of any gesture operation in the target gesture type; if the operation feature information matches the limiting conditions of a gesture operation in the target gesture type, determining that the gesture operation is the corresponding gesture information.

[0037] Specifically, the number of valid contact points included in the operation feature information can be obtained. Specifically, the single-point average contact area of ​​each single-point operation feature in the operation feature information can be determined based on the preset minimum contact area to determine whether it is not less than the minimum contact area. If it is not less than the minimum contact area, the contact point corresponding to the single-point average contact area is determined to be a valid contact point; if it is less than the minimum contact area, the contact point corresponding to the single-point average contact area is determined not to be a valid contact point. The number of valid contact points included in the operation feature information can be obtained based on the above judgment results, and the matching gesture type in the gesture recognition model can be obtained as the target gesture type based on the number of valid contacts. For example, the gesture type can be a single-contact type, a double-contact type, a three-contact type, a four-contact type, a five-contact type, etc.

[0038] Furthermore, each gesture type corresponds to multiple gesture operations, and each gesture operation corresponds to a limiting condition. It can be determined whether the single-point operation feature in the operation feature information matches the limiting condition of any gesture operation in the target gesture type.

[0039] For example, two-touch gestures include two-finger tap, two-finger left swipe, two-finger right swipe, two-finger up swipe, two-finger down swipe, etc. The corresponding restrictions for two-finger tap are: single-point contact duration ≤ 0.4S, single-point drag distance ≤ 20Pixels, single-point average contact area ≤ 1200Pixels 2 (pixel square). The corresponding limiting conditions for two-finger swipe up are: single point contact duration > 0.4S, single point drag distance > 25Pixel (pixels), single point average contact area ≤ 1200Pixel 2 (pixel square), 45°<single point drag direction≤135°.

[0040] If the single-point operation characteristics of each valid contact in the operation characteristic information match the limiting conditions of a gesture operation in the target gesture type, then the operation characteristic information is determined to match the limiting conditions of the gesture operation, and the gesture operation can be determined to be the corresponding gesture information; if the single-point operation characteristics of each valid contact in the operation characteristic information do not match the limiting conditions of the gesture operation, then the operation characteristic information is determined not to match the limiting conditions of the gesture operation. For example, if the operation characteristic information contains two sets of single-point operation characteristics corresponding to two valid contacts, it can be determined in turn whether the two sets of single-point operation characteristics both match the limiting conditions in two-finger tapping.

[0041] In a specific embodiment, after determining whether the single-point operation feature in the operation feature information matches the limiting conditions of any gesture operation in the target gesture type, it also includes: if the operation feature information does not match the limiting conditions of any gesture operation in the target gesture type, inputting the operation feature information into the neural network of the gesture recognition model for recognition, so as to obtain the output gesture recognition type as the corresponding gesture information.

[0042] If the operation feature information does not match any gesture operation in the target gesture type, the corresponding operation feature information can be input into the neural network of the gesture recognition model for recognition. The neural network can be composed of input nodes, intermediate nodes and output nodes, and can include 20 input nodes. Each group of 4 nodes can be used to input a group of single-point operation features. The four input nodes included in a group can input four feature values ​​corresponding to the single-point contact duration, single-point drag distance, single-point drag direction, and single-point average contact area. If the operation feature information includes two groups of single-point operation features, the neural network can input two groups of single-point operation features through the corresponding eight input nodes, and the feature values ​​input by other input nodes are all "0". The feature values ​​input by each input node are associated and calculated through the intermediate node to obtain the output node value corresponding to each output node. Each output node corresponds to a gesture recognition type, and the output node value is the matching degree between the operation feature information currently input to the neural network and the corresponding gesture recognition type. The output node value of each output node is obtained, and the gesture recognition type corresponding to the output node with the largest output node value is obtained as the gesture information corresponding to the operation feature information.

[0043] S130: Acquire an operation instruction corresponding to the gesture information from a pre-stored gesture instruction library according to the currently active application window.

[0044] The operation instruction corresponding to the gesture information is obtained from the pre-stored gesture instruction library according to the current active application window. The operation instruction corresponding to the gesture information can be obtained from the pre-stored gesture instruction library according to the current active application window. Different applications correspond to different instruction mapping information in the gesture instruction library. The gesture information is mapped through the instruction mapping information corresponding to the current active application window, so as to obtain the corresponding operation instruction.

[0045] In a specific embodiment, step S130 includes sub-steps: obtaining instruction mapping information corresponding to the currently active application window in the gesture instruction library; and obtaining a target mapping instruction matching the gesture information from the instruction mapping information as a corresponding operation instruction.

[0046] Specifically, the application to which the current active application window belongs can be determined, and the command mapping information matching the application can be obtained from the gesture command library according to the application; wherein the current active application window is the application window that is currently running and at the top layer in the system. Since different application programs may need to execute different operation instructions corresponding to the same gesture information, it is necessary to distinguish a set of corresponding command mapping information for different application programs.

[0047] The gesture information is mapped according to the acquired set of command mapping information to obtain a target mapping instruction, and the mapped target mapping instruction is used as the corresponding operation instruction. For example, a set of command mapping information corresponding to application A maps the target mapping instruction of "two-finger tapping" to "play a song", and a set of command mapping information corresponding to application B maps the target mapping instruction of "two-finger tapping" to "take a photo".

[0048] In a specific embodiment, before obtaining the instruction mapping information corresponding to the currently active application window in the gesture instruction library, it also includes: obtaining the application window information running in the system background; obtaining an application window matching the currently active application in the application running information from the application window information according to the application running information, as the corresponding currently active application window.

[0049] Furthermore, before obtaining the instruction mapping information, the application window information running in the background of the system can also be obtained. The application window information includes the specific information of all application windows running in the system, and the application window information includes information such as the application to which each application window belongs and the system resource ratio. According to the application running information, the corresponding application window can be obtained from the application window information as the current active application window. The system of the vehicle terminal also records the running status of each application program, thereby obtaining the application running information. The application running information in the system can be read, and the current active application can be determined according to the application running information. Specifically, the application running information includes the running status and running status type of the application program. The running status may include running or not running, and the running status type includes foreground running or background running. The application program whose running status is running and whose running status type is foreground running is determined as the current active application. Further, according to the current active application, the application window corresponding to the current active application in the application window information is determined as the current active application window. According to the determined current active application window, the operation instruction corresponding to the gesture information can be obtained from the gesture instruction library.

[0050] S140: Execute the operation instruction in the currently active application window to control the application.

[0051] Execute the operation instruction in the current active application window to control the application. Execute the above operation instruction in the current active application window, and then operate the display content, setting parameters, etc. of the current active application window through the operation instruction, so that the current active application window responds to the operation instruction. Through the above operation instruction, it is possible to control the application running in the vehicle terminal.

[0052] In a specific embodiment, after step S140, the step is also included: monitoring the touch operation of the touch screen to determine whether new touch operation information is received; if touch operation information is received from the touch screen, returning to execute the step of extracting corresponding operation feature information from the touch operation information according to preset operation feature extraction rules.

[0053] After the operation command is executed, the touch operation of the touch screen can be monitored again to determine whether new touch operation information is received; if new touch operation information is received, the process returns to step S110. If the shutdown information input by the user is received, the vehicle terminal is controlled to terminate the application execution and disconnect the power supply.

[0054] In the vehicle terminal control method based on gesture recognition disclosed in the above embodiment, the method includes: extracting corresponding operation feature information from the received touch operation information according to the operation feature extraction rule, performing gesture recognition on the operation feature information according to the gesture recognition model to obtain corresponding gesture information, obtaining the operation instruction corresponding to the gesture information from the gesture instruction library according to the current active application window, and executing the operation instruction in the current active application window to control the application. The above vehicle terminal control method can identify the touch operation information through gesture recognition to obtain the corresponding gesture information, thereby controlling the application through gesture information; there is no need for the driver to carefully identify the button function and button type, thereby avoiding the driver's need to divert his eyes to control the vehicle terminal during driving, and avoiding the driver's distraction during driving and affecting driving safety.

[0055] The embodiment of the present invention also provides a vehicle-mounted terminal control device based on gesture recognition, which can be configured in the controller 101 of the vehicle-mounted terminal 10, and is used to execute any embodiment of the vehicle-mounted terminal control method based on gesture recognition. Figure 3 , Figure 3 A schematic block diagram of a vehicle-mounted terminal control device based on gesture recognition provided by an embodiment of the present invention.

[0056] like Figure 3As shown, the vehicle terminal control device 100 based on gesture recognition includes an operation feature information acquisition unit 110 , a gesture information acquisition unit 120 , an operation instruction acquisition unit 130 and an operation instruction execution unit 140 .

[0057] The operation feature information acquisition unit 110 is configured to extract corresponding operation feature information from the touch operation information according to a preset operation feature extraction rule upon receiving touch operation information from the touch screen.

[0058] The gesture information acquisition unit 120 is used to perform gesture recognition on the operation feature information according to a preset gesture recognition model to obtain corresponding gesture information.

[0059] The operation instruction acquisition unit 130 is used to acquire the operation instruction corresponding to the gesture information from a pre-stored gesture instruction library according to the current active application window.

[0060] The operation instruction execution unit 140 is used to execute the operation instruction in the currently active application window to control the application.

[0061] The vehicle terminal control device based on gesture recognition provided in the embodiment of the present invention applies the vehicle terminal control method based on gesture recognition, extracts corresponding operation feature information from the received touch operation information according to the operation feature extraction rule, performs gesture recognition on the operation feature information according to the gesture recognition model to obtain corresponding gesture information, obtains the operation instruction corresponding to the gesture information from the gesture instruction library according to the current active application window, and executes the operation instruction in the current active application window to control the application. The vehicle terminal control method can identify the touch operation information through gesture recognition to obtain the corresponding gesture information, thereby controlling the application through gesture information; there is no need for the driver to carefully identify the button function and button type, thereby avoiding the driver's need to divert his or her sight to control the vehicle terminal during driving, and avoiding the driver's distraction during driving and affecting driving safety.

[0062] The above-mentioned vehicle-mounted terminal control device based on gesture recognition can be implemented in the form of a computer program, and the computer program can be used in Figure 4 Runs on the computer device shown.

[0063] See also Figure 4 , Figure 4 is a schematic block diagram of a computer device provided by an embodiment of the present invention. The computer device may be a controller for executing a vehicle terminal control method based on gesture recognition to perform gesture recognition and realize control of the vehicle terminal.

[0064] See also Figure 4The computer device 500 includes a processor 502 , a memory and a communication interface 505 connected via a communication bus 501 , wherein the memory may include a storage medium 503 and an internal memory 504 .

[0065] The storage medium 503 may store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 may execute a vehicle terminal control method based on gesture recognition, wherein the storage medium 503 may be a volatile storage medium or a non-volatile storage medium.

[0066] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0067] The internal memory 504 provides an environment for the operation of the computer program 5032 in the storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute the vehicle terminal control method based on gesture recognition.

[0068] The communication interface 505 is used for network communication, such as providing data information transmission, etc. Those skilled in the art will appreciate that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device 500 to which the solution of the present invention is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0069] The processor 502 is used to run a computer program 5032 stored in the memory to implement the corresponding functions in the above-mentioned vehicle terminal control method based on gesture recognition.

[0070] Those skilled in the art will understand that Figure 4 The embodiments of the computer device shown in the figure do not constitute a limitation on the specific composition of the computer device. In other embodiments, the computer device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, in some embodiments, the computer device may only include a memory and a processor. In such embodiments, the structure and function of the memory and the processor are the same as those of the embodiment of the present invention. Figure 4 The embodiments shown are consistent and will not be described again here.

[0071] It should be understood that in the embodiment of the present invention, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0072] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps included in the above-mentioned vehicle-mounted terminal control method based on gesture recognition are implemented.

[0073] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the above-described equipment, devices and units can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. Those of ordinary skill in the art can appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0074] In the several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. Units with the same function may also be combined into one unit. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices or units, or may be an electrical, mechanical or other form of connection.

[0075] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0076] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0077] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a computer-readable storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned computer-readable storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a magnetic disk or an optical disk.

[0078] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A vehicle terminal control method based on gesture recognition, characterized in that: The method is applied to a controller of a vehicle-mounted terminal, wherein the controller establishes a communication connection with a touch screen configured in the vehicle-mounted terminal to realize the transmission of data information, and the method comprises: If touch operation information is received from the touch screen, corresponding operation feature information is extracted from the touch operation information according to a preset operation feature extraction rule; Performing gesture recognition on the operation feature information according to a preset gesture recognition model to obtain corresponding gesture information; Acquire an operation instruction corresponding to the gesture information from a pre-stored gesture instruction library according to the current active application window; The operation instruction is executed in the currently active application window to control the application.

2. The vehicle-mounted terminal control method based on gesture recognition according to claim 1 is characterized in that: The extracting corresponding operation feature information from the touch operation information according to a preset operation feature extraction rule includes: Splitting the touch points contained in the touch operation information to obtain corresponding single-point operation information; Acquire corresponding single-point operation features from each of the single-point operation information according to the extraction items in the operation feature extraction rule; The single-point operation features are combined to obtain the operation feature information.

3. The vehicle-mounted terminal control method based on gesture recognition according to claim 1 or 2, characterized in that: The performing gesture recognition on the operation feature information according to a preset gesture recognition model to obtain corresponding gesture information includes: Acquire a gesture type in the gesture recognition model that matches the number of valid contacts included in the operation feature information as a target gesture type; Determining whether the single-point operation feature in the operation feature information matches a limiting condition of any gesture operation in the target gesture type; If the operation characteristic information matches a limiting condition of a certain gesture operation in the target gesture type, the gesture operation is determined to be corresponding gesture information.

4. The vehicle-mounted terminal control method based on gesture recognition according to claim 3 is characterized in that: After determining whether the single-point operation feature in the operation feature information matches the limiting condition of any gesture operation in the target gesture type, the method further includes: If the operation feature information does not match the limiting condition of any gesture operation in the target gesture type, the operation feature information is input into the neural network of the gesture recognition model for recognition to obtain the output gesture recognition type as the corresponding gesture information.

5. The vehicle-mounted terminal control method based on gesture recognition according to claim 1 is characterized in that: The acquiring the operation instruction corresponding to the gesture information from a pre-stored gesture instruction library according to the current active application window includes: Acquire command mapping information corresponding to the currently active application window in the gesture command library; A target mapping instruction matching the gesture information is acquired from the instruction mapping information as a corresponding operation instruction.

6. The vehicle-mounted terminal control method based on gesture recognition according to claim 1 or 5, characterized in that: Before acquiring the instruction mapping information corresponding to the currently active application window in the gesture instruction library, the method further includes: Get the application window information running in the system background; According to the application running information, an application window matching the currently active application in the application running information is obtained from the application window information as the corresponding currently active application window.

7. The vehicle-mounted terminal control method based on gesture recognition according to claim 1 is characterized in that: After executing the operation instruction in the currently active application window to control the application, the method further includes: Monitoring the touch operation of the touch screen to determine whether new touch operation information is received; If the touch operation information is received from the touch screen, the process returns to the step of extracting corresponding operation feature information from the touch operation information according to the preset operation feature extraction rule.

8. A vehicle-mounted terminal control device based on gesture recognition, characterized in that: The vehicle-mounted terminal control device based on gesture recognition is used to execute the vehicle-mounted terminal control method based on gesture recognition according to any one of claims 1 to 7. The device is configured in a controller of the vehicle-mounted terminal. The controller establishes a communication connection with a touch screen configured in the vehicle-mounted terminal to realize the transmission of data information. The device includes: an operation feature information acquisition unit, configured to extract corresponding operation feature information from the touch operation information according to a preset operation feature extraction rule upon receiving touch operation information from the touch screen; A gesture information acquisition unit, used to perform gesture recognition on the operation feature information according to a preset gesture recognition model to obtain corresponding gesture information; An operation instruction acquisition unit, used for acquiring an operation instruction corresponding to the gesture information from a pre-stored gesture instruction library according to the current active application window; An operation instruction execution unit is used to execute the operation instruction in the currently active application window to control the application.

9. A computer device, characterized in that: The device includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; The processor is used to implement the steps of the vehicle-mounted terminal control method based on gesture recognition described in any one of claims 1 to 7 when executing the program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the vehicle-mounted terminal control method based on gesture recognition as described in any one of claims 1 to 7 are implemented.

Images