Control method of intelligent surface, intelligent surface and computer readable storage medium

Abstract

The invention discloses a control method of an intelligent surface, the intelligent surface and a computer readable storage medium, the control method of the intelligent surface is applied to a vehicle infotainment device, the vehicle infotainment device comprises the intelligent surface, the intelligent surface comprises a plurality of graphic keys, an LED lamp and a vibration motor, the control method of the intelligent surface comprises the following steps: detecting a first action for any graphic key, and generating an electric signal; analyzing the electric signal, and generating a first light control signal and a vibration control signal; the LED lamp receives the first light control signal and carries out light display of graphic keys, and the light display is a preset light and character display style; and detecting a second action or a third action for the smart surface, and generating a corresponding change by the smart surface. Through cooperation of the three interaction modes, the system autonomously judges, selects and analyzes the actions of the user, the man-machine interaction modes of the user and the vehicle-mounted terminal are enriched, interaction becomes simple and rapid, and the experience feeling of the user is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent automobiles, and in particular to a control method of an intelligent surface, an intelligent surface and a computer-readable storage medium. Background Art

[0002] Smart surface is the development trend of automobile interior and exterior decoration. It means embedding tactile function into surfaces made of different materials, textures and geometric shapes to realize the interaction between users and smart surfaces. However, in the existing technology, the human-computer interaction method with smart surfaces is mostly through tactile sensing through fingers, which is relatively simple. Summary of the invention

[0003] In order to overcome the above shortcomings, the purpose of the present invention is to provide a control method for a smart surface to solve the problem that the human-computer interaction mode of the smart surface proposed in the above background technology is relatively single.

[0004] In order to achieve the above purpose, the technical solution adopted by the present invention is a control method of a smart surface, which is applied to a vehicle computer. The vehicle computer includes a smart surface, and the smart surface includes a plurality of graphic buttons, an LED light, and a vibration motor. The control method of the smart surface includes:

[0005] A first action on any graphic key is detected and an electrical signal is generated, wherein the first action at least includes touching or pressing.

[0006] The electrical signal is analyzed and a first light control signal and a vibration control signal are generated.

[0007] The LED lamp receives the first light control signal and performs light display of the graphic key, wherein the light display is a preset light and character display style.

[0008] The vibration motor receives the vibration control signal and performs a vibration response, wherein the vibration response includes changes in the vibration amplitude, vibration frequency and vibration time of the vibration motor.

[0009] A second motion directed to the smart surface is detected and a digital signal is generated, the second motion comprising movement of the hand.

[0010] The digital signal is analyzed and a second lighting control signal is generated.

[0011] The LED lamp receives the second light control signal and performs light changes, wherein the light changes include changes in the direction of the light path, and changes in the light color or frequency.

[0012] A third action directed to the smart surface is detected, and action information is recorded and generated, where the third action includes a movement trajectory of the eyeball.

[0013] Feature extraction is performed on the action information, and a third lighting control signal is generated according to the extracted features.

[0014] The LED lamp receives the third light control signal and performs light changes, wherein the light changes include changes in the direction of the light path, and changes in the light color or frequency.

[0015] detecting that the electrical signal, the digital signal or the motion information disappears, and generating an analog signal;

[0016] Analyzing the analog signal and generating a fourth lighting control signal;

[0017] The LED lamp receives the fourth light control signal and changes from the light change state to an off state.

[0018] The beneficial effects brought by the technical solution provided by the present invention are:

[0019] The smart surface provided in this solution is provided with three different human-computer interaction modes. The first mode is finger operation actions such as finger touch or pressing. For example, when a finger clicks a graphic button on the smart surface, a relevant response signal is generated inside the smart surface, thereby driving the vibration motor to generate vibration, generating real-time vibration feedback for the click action, and the LED light displays the light of the graphic button. The second mode is the movement of the hand, that is, the gesture is recognized by the smart surface, and corresponding light changes are generated. The third mode is the movement trajectory of the eyeball, that is, eye tracking is performed through the smart surface, and corresponding feedback is generated based on the generated action information. In addition, the LED light can be automatically turned off after the first action, the second action or the third action is completed.

[0020] Through the collaboration of three interactive methods, the system independently judges and chooses to analyze the user's actions, enriching the human-computer interaction methods between the user and the car computer, making the interaction simple and fast, and enhancing the user experience.

[0021] In some embodiments, the smart surface is provided with a pressure sensor, the pressure sensor includes a touch film and a pressure sensing chip, the touch film includes a touch layer and a sensing layer; analyzing the electrical signal and generating a first light control signal and a vibration control signal, including:

[0022] The first action is detected through the touch layer, so that the touch layer contacts the sensing layer to form a sampling circuit, and the position information of the first action is determined.

[0023] The pressure sensing chip converts the position information into an electrical signal.

[0024] The electrical signal is converted into a first light control signal and a vibration control signal.

[0025] By adopting the above technical solution, the smart surface adopts resistive touch screen technology, which determines the position, size and shape of the touch by detecting the change of the resistance value of the sampling circuit in the touch membrane. The pressure of these action information is processed internally at the sensing chip and then converted into a first light control signal for controlling the LED light and a vibration control signal for driving the vibration motor.

[0026] In some embodiments, the smart surface is provided with an infrared sensor, and the infrared sensor is provided with a data processing chip; parsing the digital signal and generating the second light control signal comprises:

[0027] The infrared sensor emits an infrared signal, and when the infrared signal detects the second motion, it is reflected and received by the infrared sensor.

[0028] The data processing chip collects and measures the intensity and timing of the reflected signal and converts it into a digital signal.

[0029] The digital signal is converted into a second lighting control signal.

[0030] With the above technical solution, the infrared sensor in the smart surface will emit an infrared signal. When the signal encounters an object, it will be reflected back and received by the infrared sensor. The infrared sensor can determine the distance and position of the object by measuring the intensity and time of the reflected signal, thereby sensing the hand movement and converting this information into a second light control signal.

[0031] In some embodiments, the smart surface is provided with an infrared camera, and the infrared camera includes a feature analysis chip; extracting features from the motion information, and generating a third light control signal according to the extracted features, including:

[0032] The third action is detected by an infrared camera and collected and recorded as eye movement data.

[0033] The feature analysis chip extracts feature information from the eye movement data, and the feature information at least includes the movement trajectory of the eyeball, the position of the gaze point, the moving speed and the acceleration.

[0034] The characteristic information is converted into a third lighting control signal.

[0035] With the above technical solution, when the human eye moves, the smart surface collects and records eye movement data through an infrared camera and extracts useful features from the collected data. Feature extraction usually uses algorithms such as machine learning to facilitate the subsequent automatic identification of eye movement features and speed up the response speed of human-computer interaction.

[0036] In some embodiments, the smart surface includes an MCU main control unit; converting the electrical signal into a first light control signal and a vibration control signal, converting the digital signal into a second light control signal, and converting the characteristic information into a third light control signal, including:

[0037] The MCU main control unit receives the electrical signal, the digital signal and the characteristic information, and parses and generates a first light control signal, a vibration control signal, a second light control signal and a third light control signal.

[0038] By adopting the above technical solution, the smart surface is provided with an MCU main control unit, which is used to control the analysis and construction process of the vibration control signal, the second light control signal and the third light control signal inside the smart surface, and directly transmit the processed control signal to the LED light and the vibration motor, thereby speeding up the signal transmission speed and the response speed of the smart surface.

[0039] The present application also provides a smart surface, which is applied to a vehicle computer. The vehicle computer includes an ECU (Electronic Control Unit, Chinese name is a vehicle computer). The smart surface is connected to the ECU via a CAN bus. The smart surface includes multiple graphic buttons, LED lights and a vibration motor. The smart surface also includes:

[0040] The touch sensing module is used to detect a first action on any graphic button and generate an electrical signal, wherein the first action at least includes touching or pressing.

[0041] The gesture recognition module is used to detect a second action directed to the smart surface and generate a digital signal, where the second action includes movement of a hand.

[0042] The eye tracking module is used to detect a third action directed to the smart surface, record and generate action information, and the third action includes a movement trajectory of an eyeball.

[0043] The MCU main control unit is used to analyze the electrical signal and generate a first light control signal and a vibration control signal, analyze the digital signal and generate a second light control signal, and extract features of the motion information and generate a third light control signal based on the extracted features.

[0044] The LED array display module includes an LED lamp, which is used to receive a first light control signal, a second light control signal and a third light control signal, and perform light display or light change of a graphic key; wherein the light display is a pre-set light and character display style, and the light change includes a change in the direction of the light path, a change in the light color or a change in the frequency.

[0045] The vibration module includes a vibration motor, which is used to receive a vibration control signal and perform a vibration response, wherein the vibration response includes changes in the vibration amplitude, vibration frequency and vibration time of the vibration motor.

[0046] By adopting the above technical solution, the MCU main control unit is used to convert the signals containing relevant information generated by the first action, the second action and the third action into control signals. The first action is sensed by the touch sensing module, and the generated electrical signal is sent to the MCU main control unit. The second action is sensed by the gesture recognition module, and the generated digital signal is sent to the MCU main control unit. The third action is sensed by the eye tracking module, and the extracted feature information is sent to the MCU main control unit. The LED array display module is used to respond to the light control signal sent by the MCU main control unit and make corresponding light changes. The LED light can play a functional role in helping to locate the button position, and can bring a decorative effect of enhancing the atmosphere of the car computer through the change of light effect. Vibration module, the vibration module receives the instruction of the MCU to make corresponding vibrations.

[0047] In some embodiments, the smart surface comprises:

[0048] The shell extends along a first direction, a receiving cavity is arranged in the shell, and an opening is arranged at the top along a second direction.

[0049] The skeleton extends along the first direction and is arranged at the opening; the skeleton is provided with an eye tracking module and a vibration module, the eye tracking module includes an infrared camera, and the vibration module includes a vibration motor.

[0050] The display layer extends along the first direction and is glued to the upper surface of the frame; the display layer includes a character film, a touch film and a light shielding film arranged in sequence along the second direction; the touch film includes a touch sensing module and a plurality of graphic buttons, and the touch sensing module includes a pressure sensor.

[0051] A touch PCB board extends along a first direction and is disposed in the accommodating cavity; an LED array display module is disposed on the upper surface of the touch PCB board, and an MCU main control unit is disposed on the lower surface of the touch PCB board; the touch PCB board includes a gesture recognition module, and the gesture recognition module includes an infrared sensor.

[0052] By adopting the above technical solution, the overall frame of the smart surface is mainly composed of a shell and a frame. The frame is fixed at the opening of the shell, and the display layer is glued to the surface of the frame. The frame plays a supporting and light-transmitting role relative to the display layer.

[0053] In some embodiments, the character film, the touch film, and the light shielding film are combined into a display layer through an IML (In Molding Label, Chinese name is in-mold injection molding) process.

[0054] By adopting the above technical solution, the display layer has the advantages of scratch resistance, long service life, good three-dimensional effect and strong anti-deformation ability through the IML process, and the color and pattern can be changed at will.

[0055] In some embodiments, a skin is further included, and the skin is glued to the upper surface of the display layer.

[0056] With the above technical solution, the surface is used to transmit light and protect the display layer, so that the display layer can display light or patterns. The surface can be made of materials including but not limited to wood, plastic and glass. Surfaces of different materials can bring different user experiences to customers.

[0057] The present application also provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed on a computer, the computer executes the control method of the smart surface described in the above multiple embodiments.

[0058] By adopting the above technical solution, the computer program product includes program codes, and when the program codes are executed on a computer, the above-mentioned smart surface control method is executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work, among which:

[0060] Figure 1 This is a process flow of an embodiment of a method for controlling a smart surface provided by the present invention. Figure 1 ;

[0061] Figure 2 This is a process flow of an embodiment of a method for controlling a smart surface provided by the present invention. Figure 2 ;

[0062] Figure 3 This is a process flow of an embodiment of a method for controlling a smart surface provided by the present invention. Figure 3 ;

[0063] Figure 4 is a structural schematic diagram of an embodiment of a smart surface provided by the present invention;

[0064] Figure 5 This is a process flow of an embodiment of a method for controlling a smart surface provided by the present invention. Figure 4 ;

[0065] Figure 6This is a process flow of an embodiment of a method for controlling a smart surface provided by the present invention. Figure 5 ;

[0066] Figure 7 This is a process flow of an embodiment of a method for controlling a smart surface provided by the present invention. Figure 6 ;

[0067] Figure 8 It is a structural framework diagram of an embodiment of a control method of a smart surface provided by the present invention;

[0068] In the figure:

[0069] Skin-10; display layer-11; character film-110; touch film-111; light shielding film-112; frame-12; LED light-13; touch PCB board-14; MCU main control unit-15; housing-16. DETAILED DESCRIPTION

[0070] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0071] refer to Figure 1 to Figure 4 , Figure 1 This is a process flow of an embodiment of a method for controlling a smart surface provided by the present invention. Figure 1 ; Figure 2 This is a process flow of an embodiment of a method for controlling a smart surface provided by the present invention. Figure 2 ; Figure 3 This is a process flow of an embodiment of a method for controlling a smart surface provided by the present invention. Figure 3 ; Figure 4 It is a structural schematic diagram of an embodiment of a smart surface provided by the present invention.

[0072] In some embodiments, reference Figure 1 to Figure 4 The technical solution adopted by the present invention is a control method of a smart surface, which is applied to a vehicle computer. The vehicle computer includes a smart surface, and the smart surface includes a plurality of graphic buttons, an LED light 13 and a vibration motor. The control method of the smart surface includes:

[0073] like Figure 1 As shown, Figure 1 The flowchart for detecting the first action is as follows:

[0074] Step S1, detecting a first action on any graphic button, where the first action at least includes touching or pressing.

[0075] The user can operate the smart surface by touching or pressing the graphic button. For example, in an application scenario, when the user clicks the graphic button, the smart surface will sense the touched position and transmit the touch information to the MCU main control unit 15 and the ECU. The MCU main control unit 15 processes the touch information, and the ECU performs corresponding operations according to the user's instructions, such as selecting a menu, adjusting the temperature, etc.

[0076] For example, when the user slides the finger on the touch layer, sliding from point A (X1, Y1) to point B (X2, Y2) in sequence, the pressure sensor generates corresponding position information data AB. If the finger slides in the opposite direction, sliding from point B (X2, Y2) to point A (X1, Y1) in sequence, the pressure sensor generates corresponding position information data BA. After processing, these position information data obtain preset operation instructions. The preset operation instructions are, for example, increasing or decreasing the volume. The operation instruction corresponding to AB is increasing the volume, and the operation instruction corresponding to BA is decreasing the volume, that is, AB and BA correspond to different operation instructions.

[0077] Step S2, generating an electrical signal.

[0078] The touch film 111 generates a change in internal resistance value according to the position information of the clicked graphic button, and the pressure sensor monitors the change in resistance value and converts it into a corresponding electrical signal.

[0079] Step S3, analyzing the electrical signal and generating a first light control signal and a vibration control signal.

[0080] The MCU control unit in the smart surface processes the electrical signal, converts it into a digital signal through an analog-to-digital converter, and uses an algorithm to perform calculations and analysis on it. The algorithm includes a touch event processing algorithm, a touch accuracy improvement algorithm, a pressure detection algorithm, a pressure analysis algorithm, a pressure feedback algorithm, and a vibration feedback control algorithm, thereby generating a first light control signal and a vibration control signal.

[0081] In step S40, the LED lamp 13 receives the first light control signal and performs light display of the graphic key, wherein the light display is a preset light and character display style.

[0082] The first light control signal includes control signals for changes in light brightness, color, flashing frequency, etc. The LED light 13 receives the light control signal and flashes according to the information in the control signal. For example, when a control signal is received that the LED light 13 in the touch button area flashes blue light, the LED light 13 in the touch button area starts to flash red light, thereby causing the touch button area to flash red light, thereby allowing the user to get feedback.

[0083] Step S41: The vibration motor receives the vibration control signal and performs a vibration response, wherein the vibration response includes changes in the vibration amplitude, vibration frequency and / or vibration time of the vibration motor.

[0084] The vibration control signal includes a control signal for the vibration amplitude, vibration frequency and vibration time of the vibration motor. The vibration motor receives the vibration control signal and performs vibration feedback according to the information in the control signal. For example, if a control signal for a small amplitude and short vibration is received, the vibration motor starts to vibrate with a small amplitude and short vibration, corresponding to the touch action, to enhance the user's experience when using the device.

[0085] like Figure 2 As shown, Figure 2 The flowchart for detecting the second action is as follows:

[0086] Step S5: a second action on the smart surface is detected, where the second action includes movement of a hand.

[0087] The user can operate the smart surface through gestures. For example, in an application scenario, when the user makes a gesture, the smart surface will recognize the gesture, the MCU main control unit 15 will process the gesture information, and the ECU will convert the gesture into a corresponding instruction, and then perform corresponding operations according to the instruction, such as switching music and adjusting the volume through gestures.

[0088] For example, the infrared sensor emits two infrared signals (f1 and f2) of different frequencies. The user's hand moves from left to right relative to the smart surface and passes through f1 and f2 in sequence. f1 will first touch the hand on the left side of the smart surface to generate reflection, and the infrared receiving unit will receive the reflected signal f1. f2 on the right side of the smart surface will then touch the hand to generate reflection, and the infrared receiving unit will receive the reflected signal f2. By judging the different frequencies and their occurrence time, it can be judged whether the hand is moving from left to right at this time, as well as the movement direction of other gestures. After processing these reflection signal data, preset operation instructions are obtained, such as page turning actions: previous page and next page, gestures from left to right are for the previous page, and gestures from right to left are for the next page.

[0089] Step S6, generating a digital signal.

[0090] The data processing chip obtains the gesture information and generates a digital signal matching the gesture information.

[0091] When the infrared sensor detects an object at an effective distance, the target information extraction array in the infrared receiving unit acquires the characteristic raw data of the detected target, namely the intensity and reflection time of the reflected signal. The data processing chip amplifies and filters the characteristic raw data, and then converts these analog signals into digital signals through an analog-to-digital converter.

[0092] Step S7, analyzing the digital signal and generating a second lighting control signal.

[0093] The MCU control unit in the smart surface analyzes the digital signal, including gesture recognition algorithm, template matching algorithm, and machine learning classification algorithm, matches the characteristic parameters of the gesture to be recognized with the pre-stored template characteristic parameters, completes the recognition task by measuring the similarity between the two, and generates a second light control signal based on the recognized information.

[0094] In step S8, the LED lamp 13 receives the second light control signal and changes the light, wherein the light change includes a change in the direction of the light path, or a change in the light color or frequency.

[0095] The second light control signal includes a control signal for the light path direction, light color or frequency. The LED light 13 receives the light control signal and flashes according to the information in the control signal. For example, when the car computer plays music, if all the LED lights 13 flash and rotate red light, the LED lights 13 start to flash and rotate red light, so that the smart surface flashes at a corresponding frequency with the music, thereby enhancing the atmosphere in the car computer.

[0096] like Figure 3 As shown, Figure 3 The flowchart for detecting the first action is as follows:

[0097] Step S9: Detecting a third action on the smart surface, recording and generating action information, where the third action includes a movement trajectory of the eyeball.

[0098] The infrared camera continuously shoots and detects the user, determines the position of the eye sockets by the rectangular shape of the two eye sockets, and classifies the state of the eyes by the position of the center of the eyeball in the eye sockets. Then, for the continuous video stream, all the states of the eyes within a period of time are stored, and the dynamic movements of the eyes, such as left, right, up and down, are identified according to the state of the eyes within a period of time, so that the user can achieve human-computer interaction through the movement of the eyes. For example, in an application scenario, when the user's eyes move, the corresponding eye movement signal will be detected in the smart surface, the MCU main control unit 15 will process the eye movement information, and the ECU will convert the gesture into the corresponding instruction, and then perform the corresponding operation according to the instruction. For example, when the user's eyes are focused on a menu on the smart surface, the relevant operation options will be automatically displayed.

[0099] Step S10, extracting features from the action information.

[0100] Obtain eye movement information and extract useful feature information based on the eye movement information.

[0101] The feature analysis chip uses the slope of the straight line where the two eyes are located as a constraint, extracts the eye socket rectangle in the specified direction, and after obtaining the result of the separation of the eye structure, it can extract the rectangular area surrounding the eyeball and the white of the eye as the eye socket, and then extract the center of the eyeball as the center of the eye. These two features can be used to determine the direction of eye gaze, the position of the gaze point, the movement speed and acceleration.

[0102] Step S11, generating a third lighting control signal according to the extracted features.

[0103] The MCU control unit in the smart surface analyzes the motion information, artificial neural network algorithm, Kalman filter algorithm, through-hole positioning algorithm and activity judgment algorithm to generate a third lighting control signal.

[0104] In step S12, the LED lamp 13 receives the third light control signal and performs light changes, wherein the light changes include changes in the direction of the light path, and changes in the light color or frequency.

[0105] The third light control signal includes a control signal for the light path direction, light color or frequency. The LED lamp 13 receives the light control signal and flashes according to the information in the control signal. For example, when a control signal is received that all LED lamps 13 emit continuous green light, the LED lamp 13 starts to emit continuous green light.

[0106] Step S13, detecting that the electrical signal, digital signal or motion information disappears, and generating an analog signal;

[0107] Step S14, analyzing the analog signal and generating a fourth lighting control signal;

[0108] Step S15: the LED lamp receives the fourth light control signal and changes from the light change state to the off state.

[0109] Steps S13 to S15 belong to the end process subsequent to the first action, the second action and the third action. The pressure sensors, infrared sensors and infrared cameras used are all human behavior sensors. When a human body approaches the above human behavior sensors, these sensors will sense the human body approaching signal and generate, for example, a change in the charge amount of the smart surface and a change in the shape of the smart surface. Such analog signals are physically very small changes. When the human body sensor senses such signals, the tiny signals need to be amplified by the post-amplifier circuit, and then converted into digital signals through analog signal conversion. The sensed signals are analyzed by the main control unit, and the lights are controlled according to specific logic.

[0110] When the human body leaves the human behavior sensor, the human body approach signal sensed by the sensor disappears, and the signal after the post-amplification circuit returns to the original environmental value. The changed signal is transmitted to the main control unit for calculation, generating the fourth light control signal, which the LED light receives accordingly. And the light is turned off according to the logic. In addition, the LED light can also be turned off by time setting, which can be a fixed time or an adjustable time, to control the turn-off after the light changes.

[0111] refer to Figure 5 , Figure 5 This is a process flow of an embodiment of a method for controlling a smart surface provided by the present invention. Figure 4 .

[0112] In some embodiments, reference Figure 4 and Figure 5 The smart surface is provided with a pressure sensor, the pressure sensor includes a touch film 111 and a pressure sensing chip, the touch film 111 includes a touch layer and a sensing layer; the electrical signal is analyzed, and a first light control signal and a vibration control signal are generated, including:

[0113] Step S20: detecting a first action through the touch layer so that the touch layer contacts the sensing layer to form a sampling circuit, and determining position information of the first action.

[0114] The resistive pressure sensor used in this embodiment utilizes the strain effect of the resistor. When the resistive sensor is subjected to external force, its resistance value will also change accordingly. The change in resistance value can be measured by the circuit and converted into a corresponding electrical signal output.

[0115] The touch layer is composed of two layers of conductive film, the upper layer is the touch layer, and the lower layer is the sensing layer. The sensing layer is made of ITO (indium tin oxide) material and can be regarded as a thin film resistor, whose resistance value is inversely proportional to the applied pressure. For example, the greater the applied pressure, the smaller the resistance value of the sensing layer. When the touch layer is subjected to pressure, the conductive film deforms and contacts the sensing layer, forming a sampling circuit. The resistance value of the sampling circuit is related to the position, size and shape of the touch, and the resistance change of the touch layer is converted into a millivolt signal output through the sampling circuit.

[0116] In this embodiment, the sensing layer is made of a transparent conductive film of ITO material, whose main components are indium oxide and tin oxide, and has the characteristic of high visible light transmittance. However, there is no limitation on the material of the transparent conductive film. Metal grids, nanosilver, carbon nanotubes, and graphene can all be applied to this solution.

[0117] Step S21: the pressure sensing chip converts the position information into an electrical signal.

[0118] The pressure sensing chip determines the corresponding touch position information by sampling the millivolt signal of the circuit, and converts it into an electrical signal and sends it to the MCU.

[0119] Step S22, converting the electrical signal into a first light control signal and a vibration control signal.

[0120] The MCU processes the received signal according to the preset to judge and send out the command signal: send the first light control signal to the LED lamp 13 to generate the light display of the graphic key; send the vibration control signal to the vibration motor to generate vibration feedback.

[0121] refer to Figure 6 , Figure 6 This is a process flow of an embodiment of a method for controlling a smart surface provided by the present invention. Figure 5 .

[0122] In some embodiments, reference Figure 6 Combined with Figure 4 As shown, the smart surface is provided with an infrared sensor, and the infrared sensor is provided with a data processing chip; the digital signal is analyzed and a second light control signal is generated, including:

[0123] Step S60: the infrared sensor transmits an infrared signal. When the infrared signal detects the second action, it is reflected and received by the infrared sensor.

[0124] In this embodiment, an infrared sensor is used to sense hand movements. The infrared sensor includes an infrared light-emitting unit, an infrared receiving unit, and a data processing chip. When the user's hand moves, the intensity and time of the reflected signal of the infrared sensor will change. The infrared light-emitting unit emits infrared signals of different frequencies, and the infrared receiving unit receives the reflected signal generated by the infrared signal. The data processing chip processes and analyzes these reflected signals to determine the trajectory of the gesture and convert the reflected signal into a digital signal containing gesture information. The infrared sensor can be provided with multiple infrared light-emitting units and multiple infrared receiving units to detect complex gestures.

[0125] Step S61: The data processing chip collects and measures the intensity and time of the reflected signal and converts it into a digital signal.

[0126] The data processing chip amplifies and filters the characteristic raw data, and then converts these analog signals into digital signals through an analog-to-digital converter.

[0127] Step S62, converting the digital signal into a second lighting control signal.

[0128] The MCU will judge according to the current working state and send a second light control signal to the LED lamp 13 to make it change the light, such as changing the light path direction of the LED lamp 13, or changing the light color or frequency.

[0129] refer to Figure 7 , Figure 7 This is a process flow of an embodiment of a method for controlling a smart surface provided by the present invention. Figure 6 .

[0130] In some embodiments, reference Figure 7 Combined with Figure 4 As shown, the smart surface is provided with an infrared camera, and the infrared camera includes a feature analysis chip; extracts features of the motion information, and generates a third light control signal according to the extracted features; including:

[0131] Step S100, detecting the third action through the infrared camera, collecting and recording it as eye movement data.

[0132] In this embodiment, the user's eye movement is sensed and the movement trajectory of the eyeball is captured, including parameters such as the position of the gaze point, moving speed and acceleration, to form eye movement data.

[0133] The infrared camera uses the skin color detection method to extract and remove the skin area in the eye socket area, and uses the threshold segmentation method to obtain the black eyeball area. On this basis, the white eye area is detected, and the remaining area is defined as the edge of the eye socket. The length and width of the eye socket rectangle are set as l and w respectively. The coordinates of the center of the eyeball in the first frame taken by the infrared camera are (x1, y1), and the coordinates of the center of the eyeball in the second frame are (x2, y2), and so on. In addition, the user's open and closed eyes state can be judged according to the value of l / w. When the value of l / w is the maximum peak, it is the closed eyes state, so the closed eyes data is filtered.

[0134] Step S101, the feature analysis chip extracts feature information from the eye movement data, and the feature information at least includes the movement trajectory of the eyeball, the position of the gaze point, the movement speed and the acceleration.

[0135] After obtaining the static state of the eye in each frame, the feature analysis chip can calculate the coordinates of the eyeball center in each frame, thereby obtaining the eyeball's movement trajectory, the location of the gaze point, the movement speed and acceleration. After processing, these eye movement information data obtain preset operation instructions, and convert this information into digital signals through an analog-to-digital converter. Feature extraction usually uses algorithms such as machine learning to facilitate the subsequent automatic identification of eye movement characteristics.

[0136] In addition, to prevent misidentification of the user's eye movement, the user needs to maintain this state for more than 5 frames after making an eye movement, otherwise the infrared camera will not respond. For example, the center of the user's eyeball is located at the initial position, with coordinates (x0, y0), that is, at the center of the eye socket rectangle, and then the user's eyeball turns right for more than 5 frames, and the user's eyeball center moves to the right position in the eye socket rectangle, with coordinates (x3, y3). After processing these eye movement coordinate data, preset operation instructions are obtained, such as turning the light on and off. Turning the eyeball right turns the light off, and turning the eyeball left turns the light on.

[0137] Step S102: convert the characteristic information into a third lighting control signal.

[0138] The eye tracking module needs to extract useful features from the collected data. Features may include the position of the eye's gaze point, movement speed, and acceleration. Feature extraction usually uses algorithms such as machine learning to automatically identify the features of eye movement. The processed data will be sent to the MCU, which will determine based on the current working mode to send a third light control signal to the LED lamp 13 to make it change the light, such as changing the light path direction of the LED lamp 13, or changing the light color or frequency.

[0139] refer to Figure 8 , Figure 8It is a structural framework diagram of an embodiment of a smart surface control method provided by the present invention.

[0140] In some embodiments, reference Figure 8 The smart surface includes an MCU main control unit 15; converting an electrical signal into a first light control signal and a vibration control signal, converting a digital signal into a second light control signal, and converting characteristic information into a third light control signal, including:

[0141] The MCU main control unit 15 receives the electrical signal, the digital signal and the characteristic information, and parses and generates a first light control signal, a vibration control signal, a second light control signal and a third light control signal.

[0142] Exemplarily, the smart surface is provided with an MCU main control unit 15, which is used to control the analysis and construction process of the vibration control signal, the second light control signal and the third light control signal within the smart surface, and directly transmit the processed control signal to the LED light 13 and the vibration motor, thereby speeding up the signal transmission speed and the response speed of the smart surface.

[0143] In some embodiments, reference Figure 8 The present application also provides a smart surface, which is applied to a vehicle computer. The vehicle computer includes an ECU. The smart surface is connected to the ECU via a CAN bus. The smart surface includes a plurality of graphic buttons, an LED light 13 and a vibration motor. The smart surface also includes:

[0144] The touch sensing module is used to detect a first action on any graphic button and generate an electrical signal, wherein the first action at least includes touching or pressing.

[0145] The gesture recognition module is used to detect a second action directed to the smart surface and generate a digital signal, where the second action includes movement of a hand.

[0146] The eye tracking module is used to detect a third action directed to the smart surface, record and generate action information, and the third action includes a movement trajectory of an eyeball.

[0147] The MCU main control unit 15 is used to analyze the electrical signal and generate a first light control signal and a vibration control signal, analyze the digital signal and generate a second light control signal, and extract features of the motion information and generate a third light control signal based on the extracted features.

[0148] The LED array display module includes an LED lamp 13, which is used to receive a first light control signal, a second light control signal and a third light control signal, and perform light display or light change of a graphic key; wherein the light display is a pre-set light and character display style, and the light change includes a change in the direction of the light path, a change in the light color or a change in the frequency.

[0149] The vibration module includes a vibration motor, which is used to receive a vibration control signal and perform a vibration response, wherein the vibration response includes changes in the vibration amplitude, vibration frequency and vibration time of the vibration motor.

[0150] The power module is used to connect to the vehicle computer line and adopts DC-DC mode for internal power supply conversion.

[0151] The CAN transmission unit includes a CAN bus, which is used to connect with the MCU main control unit 15 and the ECU to transmit signals.

[0152] Exemplarily, the control module of the smart surface is the MCU main control unit 15, and the analysis process of multiple actions is constructed on the smart surface, which can effectively realize the independent operation of the smart surface. The touch sensing module implements step S1, detects the first action for any graphic button and step S2, and generates an electrical signal. The gesture recognition module implements step S5, detects the second action for the smart surface and step S6, and generates a digital signal. The eye tracking module implements step S9, detects the third action for the smart surface and step S10, and extracts features from the action information.

[0153] The MCU main control unit 15 implements step S3, analyzes the electrical signal, generates a first light control signal and a vibration control signal, step S7, analyzes the digital signal, generates a second light control signal, and step S11, generates a third light control signal based on the extracted features. The LED lamp 13 receives multiple light control signals and changes the light accordingly. The vibration motor receives the vibration control signal and performs vibration feedback.

[0154] In some embodiments, reference Figure 4 , smart surfaces include:

[0155] The shell 16 extends along a first direction (i.e., the length direction of the smart surface), a receiving cavity is provided inside the shell 16, and an opening is provided at the top along a second direction (i.e., the height direction of the smart surface).

[0156] The skeleton 12 extends along the first direction and is disposed at the opening; the skeleton 12 is provided with an eye tracking module and a vibration module, the eye tracking module includes an infrared camera, and the vibration module includes a vibration motor.

[0157] The display layer 11 extends along the first direction and is glued to the upper surface of the skeleton 12; the display layer 11 includes a character film 110, a touch film 111 and a shading film 112 arranged in sequence along the second direction; the touch film 111 includes a touch sensing module and a plurality of graphic buttons, and the touch sensing module includes a pressure sensor.

[0158] The touch PCB board 14 extends along the first direction and is disposed in the accommodating cavity. The upper surface of the touch PCB board 14 is provided with an LED array display module, and the lower surface is provided with an MCU main control unit 15. The touch PCB board 14 includes a gesture recognition module, and the gesture recognition module includes an infrared sensor.

[0159] Exemplarily, the overall framework of the smart surface is mainly composed of a shell 16 and a skeleton 12 . The skeleton 12 is fixed at the opening of the shell 16 , and the display layer 11 is glued to the surface of the skeleton 12 . The skeleton 12 plays a supporting and light-transmitting role relative to the display layer 11 .

[0160] In some embodiments, the character film 110 , the touch film 111 , and the light shielding film 112 are combined into the display layer 11 through an IML process.

[0161] For example, the IML process is a process in which the printed film is processed by a molding machine, and then cut and placed in an injection mold. It is most suitable for film products, especially for the display of color patterns, backlights and multiple colors that require consistency. The IML process makes the display layer 11 have the advantages of scratch resistance, long service life, good three-dimensional effect and strong anti-deformation ability, and the color and pattern can be changed at will.

[0162] In some embodiments, the smart surface further includes a surface skin 10 , which is glued to the upper surface of the display layer 11 .

[0163] Exemplarily, the surface skin 10 is used to transmit light and protect the display layer 11, so that the display layer 11 can display light or patterns. The surface skin 10 can be made of materials including but not limited to wood, plastic and glass. The surface skin 10 of different materials can bring different usage experiences to customers.

[0164] The present application also provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed on a computer, the computer executes the atmosphere light control method described in the above-mentioned multiple embodiments.

[0165] Exemplarily, the computer program product includes program codes, and when the program codes are executed on a computer, the above-mentioned method for controlling the smart surface is executed.

[0166] It should be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0167] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products of the embodiments provided by the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be completed by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0168] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, the instruction device implementing the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0169] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0170] The above description is only an implementation mode of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly used in other related technical fields, should be included in the protection scope of the present invention.

Claims

1. A method for controlling a smart surface, applied to a vehicle computer, wherein the vehicle computer comprises a smart surface, wherein the smart surface comprises a plurality of graphic buttons, an LED light and a vibration motor, wherein: The control method of the smart surface comprises: Detecting a first action on any one of the graphic keys and generating an electrical signal, wherein the first action at least includes touching or pressing; Analyzing the electrical signal and generating a first light control signal and a vibration control signal; The LED lamp receives the first light control signal and performs light display of the graphic key, wherein the light display is a preset light and character display style; The vibration motor receives the vibration control signal and performs a vibration response, wherein the vibration response includes changes in the vibration amplitude, vibration frequency and vibration time of the vibration motor; detecting a second action directed to the smart surface and generating a digital signal, the second action comprising movement of a hand; Analyzing the digital signal and generating a second lighting control signal; The LED lamp receives the second light control signal and performs light changes, wherein the light changes include changes in the direction of the light path, or changes in the light color or frequency; detecting a third action directed to the smart surface, recording and generating action information, wherein the third action includes a movement trajectory of an eyeball; Extracting features from the action information, and generating a third light control signal according to the extracted features; The LED lamp receives the third light control signal and performs light changes, wherein the light changes include changes in the direction of the light path, the color of the light or the frequency of the light; detecting that the electrical signal, the digital signal or the motion information disappears, and generating an analog signal; Analyzing the analog signal and generating a fourth lighting control signal; The LED lamp receives the fourth light control signal and changes from the light change state to an off state.

2. The control method of the smart surface according to claim 1, characterized in that: The smart surface is provided with a pressure sensor and a touch film, the pressure sensor includes a pressure sensing chip, and the touch film includes a touch layer and a sensing layer; the analyzing the electrical signal and generating a first light control signal and a vibration control signal include: The first action is detected by the touch layer, so that the touch layer contacts the sensing layer to form a sampling circuit, and position information of the first action is determined; The pressure sensing chip converts the position information into the electrical signal; The electrical signal is converted into the first light control signal and the vibration control signal.

3. The control method of the smart surface according to claim 1, characterized in that: The smart surface is provided with an infrared sensor, and the infrared sensor is provided with a data processing chip; the step of analyzing the digital signal and generating a second light control signal includes: The infrared sensor emits an infrared signal, and when the infrared signal detects the second action, it is reflected and received by the infrared sensor; The data processing chip collects and measures the intensity and time of the reflected signal and converts it into a digital signal; The digital signal is converted into the second light control signal.

4. The method for controlling a smart surface according to claim 1, characterized in that: The smart surface is provided with an infrared camera, and the infrared camera includes a feature analysis chip; extracting features from the action information and generating a third light control signal according to the extracted features, including: Detecting the third action by the infrared camera, collecting and recording it as eye movement data; The feature analysis chip extracts feature information from the eye movement data, wherein the feature information at least includes the movement trajectory of the eyeball, the position of the gaze point, the movement speed and the acceleration; The characteristic information is converted into the third lighting control signal.

5. The control method of the smart surface according to claims 2 to 4, characterized in that: The smart surface includes an MCU main control unit; converting the electrical signal into the first light control signal and the vibration control signal, converting the digital signal into the second light control signal, and converting the characteristic information into the third light control signal, including: The MCU main control unit receives the electrical signal, the digital signal and the characteristic information, and parses and generates the first light control signal and the vibration control signal, the second light control signal and the third light control signal.

6. A smart surface, applied to a vehicle computer, the vehicle computer includes an ECU, the smart surface is connected to the ECU via a CAN bus, the smart surface includes a plurality of graphic buttons, LED lights and a vibration motor, characterized in that: The smart surface further comprises: A touch sensing module, configured to detect a first action on any one of the graphic keys and generate an electrical signal, wherein the first action at least includes touching or pressing; a gesture recognition module, configured to detect a second action on the smart surface and generate a digital signal, wherein the second action includes a movement of a hand; an eye tracking module, configured to detect a third action directed to the smart surface, record and generate action information, wherein the third action includes a movement trajectory of an eyeball; The MCU main control unit is used to analyze the electrical signal and generate a first light control signal and a vibration control signal, analyze the digital signal and generate a second light control signal, and extract features from the action information and generate a third light control signal according to the extracted features; The LED array display module is used to receive the first light control signal, the second light control signal and the third light control signal, and to perform light display or light change of the graphic key; wherein the light display is a preset light and character display style, and the light change includes a change in light path direction, light color or frequency; The vibration module is used to receive the vibration control signal and perform a vibration response, wherein the vibration response includes changes in the vibration amplitude, vibration frequency and vibration time of the vibration motor.

7. The smart surface according to claim 6, characterized in that include: A shell extending along a first direction, having a receiving cavity in the shell and an opening at the top along a second direction; A frame, the frame extending along the first direction and arranged at the opening; the frame is provided with the eye tracking module and the vibration module, the eye tracking module includes an infrared camera, and the vibration module includes a vibration motor; a display layer, the display layer extending along the first direction and being glued to the upper surface of the frame; the display layer comprising a character film, a touch film and a light shielding film arranged in sequence along the second direction; the touch film comprising the touch sensing module and the plurality of graphic keys, the touch sensing module comprising a pressure sensor; A touch PCB board, the touch PCB board extends along the first direction and is arranged in the accommodating cavity; the LED array display module is arranged on the upper surface of the touch PCB board, and the MCU main control unit is arranged on the lower surface; the touch PCB board includes the gesture recognition module, and the gesture recognition module includes an infrared sensor.

8. The smart surface according to claim 7, characterized in that The character film, the touch film and the light shielding film are combined into the display layer through an IML process.

9. The smart surface according to claim 7, characterized in that: It also includes a skin, which is glued to the upper surface of the display layer.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method for controlling a smart surface according to any one of claims 1 to 5.

Images