Interaction device and control method thereof, electronic equipment, storage medium and vehicle

Abstract

The invention discloses an interaction device and a control method thereof, electronic equipment, a storage medium and a vehicle, the interaction device comprises a movable display module, the control method of the interaction device comprises the following steps: obtaining position information of a target user, the position information being obtained based on voice information and image information of a space where the interaction device is located; and based on the position information of the target user, controlling the display module to move to the position corresponding to the target user. According to the interaction device and the control method, the voice information and the image information are combined, and the relative position between the user and the interaction object is determined more accurately, so that the movement position of the display module is more accurate, and the interaction experience is improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and in particular to a control method for an interactive device, as well as electronic devices, computer-readable storage media, and vehicles. Background Technology

[0002] With the continuous development of technology, screen interaction methods have changed from physical buttons to screen touch, voice interaction, gesture interaction, etc. However, regardless of the interaction method, if the relative position between the user and the interaction object cannot be accurately judged, it will reduce the user's interaction experience. Summary of the Invention

[0003] The present invention aims to at least solve one of the technical problems existing in the prior art. To this end, one object of the present invention is to provide a control method for an interactive device that can more accurately determine the relative position between the user and the interactive object, thereby enabling more accurate movement of the display module and improving the interactive experience.

[0004] The second objective of this invention is to provide an electronic device.

[0005] A third objective of this invention is to provide a computer-readable storage medium.

[0006] The fourth objective of this invention is to provide an interactive device.

[0007] The fifth objective of this invention is to provide a vehicle.

[0008] To achieve the above objectives, the present invention provides a control method for an interactive device according to a first embodiment, wherein the interactive device includes a movable display module, and the control method includes: obtaining location information of the target user, the location information being obtained based on voice information and image information of the space where the interactive device is located; and controlling the display module to move to the location corresponding to the target user based on the location information.

[0009] The control method of the interactive device according to an embodiment of the present invention combines voice information and image information. Voice information can initially determine the user's position, and then the image information is combined to determine the user's spatial position. By combining these two types of information, the user's position can be accurately located, improving the accuracy of the drive module in adjusting the angle of the display module and providing a better user experience.

[0010] In some embodiments, the location information is determined based on the facial image information of the target user within the initial interaction area; wherein the initial interaction area is determined based on the voice information; and the facial image information of the target user is determined based on the image information within the initial interaction area.

[0011] In some embodiments, the image information within the initial interaction area is obtained by preprocessing and cropping the image information within the space where the interaction device is located.

[0012] In some embodiments, the facial image information of the target user is the output of a target image processing neural network model, which takes the image information within the initial interaction area as input.

[0013] In some embodiments, the target image processing neural network model is constructed based on a target loss function, which is obtained based on a face binary classification loss function, a face bounding box loss function, and a face recognition information loss function.

[0014] In some embodiments, the target loss function is obtained by summing a first loss function value, a second loss function value, and a third loss function value; the first loss function value is obtained by multiplying a face binary classification loss function and a first coefficient; the second loss function value is obtained by multiplying a face bounding box loss function and a second coefficient; and the third loss function value is obtained by multiplying a face recognition information loss function and a third coefficient.

[0015] In some embodiments, the target loss function is expressed as follows: ; in, Let the target loss function be... Let the face binary classification loss function be... Let the loss function be the face bounding box. Let the loss function be the facial recognition information. For the true values ​​of the labels in a binary classification, , and These are weighting coefficients, and the first coefficient is... The second coefficient is The third coefficient is .

[0016] In some embodiments, the output of the target image processing neural network model further includes a face bounding box; The control method further includes: when multiple face target boxes are determined based on image information within the initial interaction area, the target user corresponds to the largest face target box among the multiple face target boxes.

[0017] In some embodiments, before controlling the display module to move to the position corresponding to the target user based on the location information, the control method further includes: maintaining the display module at its current position when the position deviation of the display module is less than or equal to a position deviation threshold; or controlling the display module to move based on the location information when the position deviation of the display module is greater than the position deviation threshold; wherein the position deviation is the deviation between the current position of the display module and the position information of the target user.

[0018] In some embodiments, the control method further includes: the interactive device being in a gesture interaction state; based on the relative position information of the target user's interactive gesture for the image displayed on the interactive device in the touch area, controlling the touch feedback module to emit a touch feedback signal to the location of the interactive gesture, so that the user's hand generates tactile sensation.

[0019] In some embodiments, the touch feedback signal is an ultrasonic signal.

[0020] In some embodiments, the relative position information is obtained based on image information of the target user's interactive gestures to the image displayed on the interactive device.

[0021] In some embodiments, the interactive device displays an image as a real image formed by a negative refractive index plate.

[0022] To achieve the above objectives, a second aspect of the present invention provides an electronic device, comprising: at least one processor; a memory communicatively connected to the at least one processor; the memory storing a computer program executable by the at least one processor, wherein the at least one processor executes the computer program to implement a control method for the interactive device.

[0023] According to the electronic device of the present invention, the location information of the target user is determined by combining voice information and image information. Voice information can initially determine the user's location, and then the user's spatial location is determined by combining the image information. By combining these two types of information, the user's location can be accurately located, improving the accuracy of the drive module in adjusting the angle of the display module and providing a better user experience.

[0024] To achieve the above objectives, a third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the control method of the interactive device.

[0025] According to the computer-readable storage medium of the present invention, by executing the control method, the location information of the target user is determined by combining voice information and image information. Voice information can initially determine the user's location, and then the user's spatial location is determined by combining image information. By combining these two types of information, the user's location can be accurately located, improving the accuracy of the drive module in adjusting the angle of the display module and providing a better user experience.

[0026] To achieve the above objectives, a fourth aspect of the present invention provides an interactive device, comprising: a display module for displaying a target image; and a driving module connected to the display module, wherein the driving module is used to drive the display module to move to a position corresponding to the target user based on the location information of the target user; wherein the location information of the target user is obtained based on voice information and image information in the space where the interactive device is located.

[0027] According to the interactive device of the present invention, the location information of the target user is determined by combining voice information and image information. Voice information can initially determine the user's location, and then the user's spatial location is determined by combining the image information. By combining these two types of information, the user's location can be accurately located, improving the accuracy of the driving module in adjusting the angle of the display module and providing a better user experience.

[0028] In some embodiments, the interactive device further includes a tactile feedback module, which is used to emit tactile feedback signals to the touch area where the interactive gesture is located in response to an interactive gesture of a target user to an image displayed on the display module, so as to generate tactile sensation in the hand.

[0029] In some embodiments, the touch module includes an ultrasonic array, which generates a focused ultrasonic signal in a target touch area as a touch feedback signal, wherein the target touch area includes at least the display area of ​​the display module.

[0030] In some embodiments, the interactive device further includes a sensing module for sensing user gestures in response to images displayed by the display module.

[0031] In some embodiments, the display module includes: a display assembly, the display assembly including a display screen for displaying the target image; and a negative refractive index plate, the negative refractive index plate being at a preset angle to the display screen, the negative refractive index plate being used to generate a real image of the target image on an imaging plane in an aerial target area when light containing the target image is incident on it; wherein the imaging plane and the display screen are located on different sides of the negative refractive index plate, and the imaging plane and the display screen are symmetrical about the negative refractive index plate.

[0032] In some embodiments, the display module further includes: a housing having a receiving space, a window on the top of the housing communicating with the receiving space, and a negative refractive index plate disposed at the window; the display assembly being located in the receiving space.

[0033] In some embodiments, the cross-section of the housing parallel to the window is circular on the outside and rectangular with rounded corners on the inside.

[0034] In some embodiments, the display module further includes a connector located at the bottom of the housing for connecting the display module and the driving module.

[0035] In some embodiments, the interactive device further includes a controller connected to the display module, the driving module, the touch feedback module, and the sensing module, for executing a control method for the interactive device.

[0036] To achieve the above objectives, a fifth aspect of the present invention provides a vehicle that includes the aforementioned electronic device; or, the vehicle includes the aforementioned interactive device.

[0037] According to the vehicle of the present invention, by employing the aforementioned electronic device or interactive device, the location information of the target user is determined by combining voice information and image information. Voice information can initially determine the user's location, and then the user's spatial location is determined by combining image information. By combining these two types of information, the user's location can be accurately located, improving the accuracy of the drive module in adjusting the angle of the display module and providing a better user experience.

[0038] In some embodiments, the vehicle further includes: a voice acquisition device for acquiring voice information of the space where the interactive device is located; and an image acquisition device for acquiring image information of the space where the interactive device is located.

[0039] In some embodiments, the center console of the vehicle forms the housing of the display module of the interactive device.

[0040] In some embodiments, the touch module of the interactive device is disposed on the central control armrest box.

[0041] In some embodiments, the sensing module of the interactive device is disposed on the vehicle's in-vehicle display screen.

[0042] In some embodiments, the controller of the interactive device is an in-vehicle domain controller or a vehicle controller.

[0043] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0044] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart of a control method for an interactive device according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating the learning process of a target image processing neural network model according to an embodiment of the present invention; Figure 3 This is a flowchart of the process of controlling the orientation of the display module according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a cropping area of ​​an image inside a vehicle cabin according to an embodiment of the present invention; Figure 5 This is a flowchart of a process for adjusting the orientation of a display module using a rotating mechanism according to an embodiment of the present invention; Figure 6 This is a flowchart of a control method for an interactive device according to an embodiment of the present invention; Figure 7 This is a block diagram of an electronic device according to an embodiment of the present invention; Figure 8 This is a schematic diagram of an interactive device according to an embodiment of the present invention; Figure 9 This is a schematic diagram of the semi-transparent structure of a display module according to an embodiment of the present invention; Figure 10 This is a schematic diagram of the split structure of a display module according to an embodiment of the present invention; Figure 11 This is a schematic diagram of the imaging plane of a display module in an interactive device according to an embodiment of the present invention; Figure 12 This is a schematic diagram of the installation position of an interactive device according to an embodiment of the present invention; Figure 13 This is a schematic diagram of the communication connection between vehicle components and components in an interactive device according to an embodiment of the present invention. Detailed Implementation

[0045] The embodiments of the present invention are described in detail below. The embodiments described with reference to the accompanying drawings are exemplary. The embodiments of the present invention are described in detail below.

[0046] In embodiments of the present invention, the interactive device may include, but is not limited to, in-vehicle interactive devices, touch devices, or aerial imaging devices, as described below. Figures 1-6 A control method for an interactive device according to an embodiment of the present invention is described.

[0047] In embodiments of the present invention, the display module of the interactive device is movable, such as rotatable or movable, and the display module can be adjusted according to the position of the target user in order to better interact with the user.

[0048] Specifically, the interactive device can be equipped with a driving module, such as a rotating mechanism, a sliding mechanism, or a moving mechanism, to drive the rotation and position of the display module, so that the display module can interact better with the target user.

[0049] If a single image is used as input, it becomes difficult to distinguish user intentions when multiple users appear in the image, potentially preventing the selection of a specific rotation direction. If only sound is used as input, the user's orientation can be determined, but their precise location remains uncertain. Therefore, the display module's driver module cannot achieve accurate driving.

[0050] In an embodiment of the present invention, Figure 1 This is a flowchart of a control method for an interactive device according to an embodiment of the present invention, such as... Figure 1 As shown, it includes the following steps S1-S2.

[0051] S1, obtain the target user's location information based on the voice and image information of the space where the interactive device is located.

[0052] S2, based on the target user's location information, controls the display module of the interactive device to move to the corresponding target user's location, so as to facilitate better interaction with the target user.

[0053] Specifically, this solution combines sound and image information. Sound information can initially determine the user's location, and then combined with image information, the user's spatial location can be determined. Combining these two types of information can accurately locate the user's position, improve the accuracy of the drive module in adjusting the angle of the display module, and provide a better user experience.

[0054] In some embodiments, the target user's location is determined based on facial image information within an initial interaction area. This initial interaction area can be determined based on voice information; that is, the target user's location can be initially determined through sound. The voice information can be a specified voice command or other voice information. The target user's facial image information can be determined from image information within the initial interaction area.

[0055] Specifically, voice information can be collected through a microphone array, while image information can be collected through an in-cabin camera. The user's initial position is initially determined based on the user's voice information, and then the user's specific position is identified based on the user's facial image information, such as the position of the user's face, pupils, eyes, or nose. Based on the specific position, the angle of the display module can be adjusted to move to the corresponding target user's position for better interaction.

[0056] In this embodiment, the image information within the initial interaction area can be obtained by preprocessing and cropping the image information within the space where the interaction device is located.

[0057] The initial interaction area can be divided based on requirements; for example, for the vehicle cabin, such as... Figure 3 As shown, the initial interaction area may include, but is not limited to, the driver's seat, the passenger seat, and the rear seats.

[0058] This includes preprocessing the image information within the space where the interactive device is located. Preprocessing may include removing blurred areas or interference information from the image.

[0059] The cropping process may include cropping out portions of the image information within the space where the interactive device is located that do not belong to the preset initial interactive area, while retaining the image information of the initial interactive area.

[0060] Furthermore, in an embodiment, the image information within the obtained initial interaction area can be processed using an image processing algorithm to obtain the facial image information of the target user.

[0061] In some embodiments, a neural network can be used for image processing, that is, the image information in the initial interaction area is input into the target image processing neural network model, and the output of the target image processing neural network model may include the facial image information of the target user.

[0062] In embodiments of the present invention, the target image processing neural network model is a method of deep learning for processing data. It is not limited to a specific deep learning neural network model and can use most image processing neural network models, such as UNet, MobileNet, ResNet-50, etc. The choice of different neural network models mainly depends on the computing power of the hardware and the difficulty of the detection task.

[0063] Furthermore, in some embodiments of the present invention, the output of the target image processing neural network model can be divided into three parts: the first part is the classification task, i.e., whether it belongs to a face; the second part is the face target box; and the third part is the user's specific location, such as the position of the eyes or nose.

[0064] Based on the output of the above target image processing neural network model, the model is constructed using a target loss function. This target loss function can be obtained from the face binary classification loss function, the face bounding box loss function, and the face recognition information loss function. For example, the target loss function is a relational expression between the face binary classification loss function, the face bounding box loss function, and the face recognition information loss function. Specifically, the face binary classification loss function corresponds to the first part of the model output, the face bounding box loss function corresponds to the second part, and the face recognition information loss function corresponds to the third part. This allows for the layer-by-layer acquisition of the target user's specific location information.

[0065] Specifically, in some embodiments, the target loss function is obtained as the sum of a first loss function value, a second loss function value, and a third loss function value; the first loss function value is obtained as the product of a face binary classification loss function and a first coefficient; the second loss function value is obtained as the product of a face bounding box loss function and a second coefficient; and the third loss function value is obtained as the product of a face recognition information loss function and a third coefficient. The first, second, and third coefficients can be the weight coefficients of the three loss functions, the product of other combined values, or other forms of coefficients, and can be configured based on specific circumstances.

[0066] Furthermore, in some embodiments, the target loss function can be specifically expressed as follows: (1) in, Let be the target loss function. For face binary classification loss function, The loss function is the face bounding box. Let be the loss function for face recognition information. For the true values ​​of the labels in a binary classification, , and These are the weighting coefficients. The first coefficient can be... The second coefficient can be The third coefficient can be .

[0067] It is understood that the above formula (1) is only an example of the target loss function in the embodiments of this application, and the target loss function can also be expressed by other applicable expressions.

[0068] In this embodiment, the specific form of the face binary classification loss function can be as follows: (2) in, This represents the binary classification result output by the network. In this scheme, The label is for binary classification, and its value is 0 or 1.

[0069] In this embodiment, for the face bounding box loss function, the deep learning model will output the following in this part: , Where b1 and b2 are the coordinates of the top-left corner of the target bounding box, and b3 and b4 are the length and width of the target bounding box, respectively, in the following form: , (3) in, This is a truth label.

[0070] In this embodiment, the facial recognition information loss function, such as the pupil and nose position information, can be set as two points for the eyes, namely the positions of the left and right eye pupils. The nose position is set as one point. The neural network model outputs a total of six values ​​for this part, represented as follows: , The loss value calculated between the true value and the actual value is: (4) in, The true value is marked.

[0071] When optimizing the model based on the loss functions of the above three parts, it is necessary to introduce three weight parameters to adjust the proportion of each loss function. The specific expression is shown in formula (1) above.

[0072] Figure 2 The learning process of a neural network model for processing target images, such as Figure 2 As shown, it specifically includes: S801, input preprocessed and segmented image.

[0073] S802, the initial neural network processes the input.

[0074] S803, Predicted Output.

[0075] S804, the loss function optimizes the neural network.

[0076] As can be seen from the expression for L, the smaller L is, the better the model results. By adjusting the value of L, gradient descent is performed to optimize the model parameters, which is the optimization process from S804 to S802.

[0077] Specifically, the model optimization direction is the direction of decreasing L. Optimization stops once a certain threshold is reached, and the model parameters are recorded. In the example application, the trained model, i.e., the target image processing neural network model, is deployed on the board. The model output results are obtained from S801, S802, and S803. These results may include whether it is a face, the face bounding box, the pupils of the eyes, and the coordinates of the nose. All coordinate information output by the model is in image pixel coordinates. For example, the top left corner of the image is the origin, the horizontal direction to the right is the positive x-axis, and the vertical direction downwards is the positive y-axis.

[0078] Based on the above description, the control method of the interactive device in this embodiment of the invention accurately locates the user's position by combining sound and image information, and controls the display module to move to the position of the corresponding target user, for example by rotating it through a vehicle-mounted rotating mechanism, to achieve an adaptive angle function.

[0079] Specifically, Figure 3 This is a flowchart of the process for controlling the orientation of a display module according to an embodiment of the present invention, as follows: Figure 3 As shown, it includes the following steps: S601, System Initialization. This includes: Rotary Mechanism Communication, Rotary Mechanism Position Initialization, Camera Initialization, and Microphone Array Initialization.

[0080] S602: After system initialization is complete, the system enters the waiting phase for interactive signal input. If no signal is input, the system maintains its current state, i.e., S609. If an interactive signal is input, the signal is processed, and the system proceeds to S603.

[0081] The S603 confirms user interaction by collecting audio signals. In practice, the audio signal can be a custom voice message. After collecting a specific voice segment, the microphone array signal is used to confirm the location of the interacting user within the cabin. The cabin area includes the driver's seat, front passenger seat, and rear seats.

[0082] S604 acquires image signals and inputs them into image signal preprocessing.

[0083] For example, the image signal is an RGB image, and the image matrix is... The image is preprocessed by subtracting the mean of the statistical image. Divide by the statistical variance of the image : ; in, and All Matrix, each layer is Size, its value or The constant in.

[0084] Specifically, the location of the interacting user is initially located using sound signals, and the image to be processed is then cropped and partitioned. An example of cropping and partitioning is as follows: Figure 4 As shown. The segmented region may vary slightly across different vehicle models. Image segmentation can effectively reduce the computational load of the algorithm and effectively mask user detection outside the target region.

[0085] S605 inputs the preprocessed and segmented image into a deep learning neural network to recognize the user's face and output the image coordinates of the user's eyes.

[0086] Specifically, the preprocessed and segmented images are input into a neural network. The network extracts features from the images, processes them through an encoder and decoder, and constrains the network output using a loss function to obtain the final result. The neural network model is trained with data, an optimization problem is constructed using the loss function, the parameters in the network model are adjusted, and finally, a usable result is output. The process is as follows: Figure 4 As shown.

[0087] S606, position transformation between the image coordinate system and the rotation mechanism coordinate system of the display module, calculates the user's servo coordinate system position, i.e., the position of the drive component, under the current image.

[0088] In this case study, a one-dimensional servo motor is used as an example for detailed explanation. The servo motor rotation is one-dimensional, and the camera pixel position can be converted to the servo motor's rotation angle position. The pixel position is converted between pixels and angles based on the FOV (Field of View) principle of camera imaging, achieving mutual conversion between the camera coordinate system and the servo motor coordinate system. After the conversion, the range of the camera's FOV and the servo motor's rotation range need to be calibrated to align the two coordinate systems. In the specific implementation, the angular position of the pixel is calculated using the camera's horizontal resolution and horizontal FOV. The specific calculation formula is as follows: ; in, The camera's horizontal field of view (FOV) This represents the camera's maximum horizontal resolution. To identify the horizontal pixel coordinates of the target, the formula above shows that the horizontal pixel coordinates of the target can be converted into angular coordinates. Typically, the angular coordinates will be converted to... ,exist At 0, it is centered in the image. The servo's rotation range can also be switched to... Between. and At the same time, the rotation mechanism and camera coordinate system are aligned and corrected.

[0089] S607, Determine position information. After the model outputs position information and transforms it to the coordinate system of the rotating mechanism, it is necessary to determine whether the rotating mechanism needs to rotate. If the movement requirement is met, proceed to step S609; otherwise, proceed to step S608.

[0090] S608, maintain the current status.

[0091] S609, send a movement signal, the rotating mechanism rotates, and the existing status is updated.

[0092] Furthermore, to optimize the user experience, since users cannot remain completely still during use, and to prevent the drive module from swinging back and forth within a small range, a movement position threshold is set for the drive module.

[0093] For example, in one embodiment, before controlling the display module of the interactive device to move to the location of the corresponding target user based on the location information, the control method further includes: maintaining the display module at its current position when the position deviation of the display module is less than or equal to a position deviation threshold; or controlling the display module to move based on the location information when the position deviation of the display module is greater than the position deviation threshold; wherein, the position deviation is the deviation between the current position of the display module and the location information of the target user.

[0094] Specifically, the display module is driven to rotate and move to the target user's position via a rotating mechanism. The rotating mechanism has a set rotation angle threshold. The rotating mechanism records its current position. When the user's position output by the model is within ±3° of the current position, the rotating mechanism will not rotate, indicating that the movement requirement is not met, and the rotating mechanism remains in its current state (step S608). When the model's output position is outside ±3° of the rotating mechanism's current position but within the rotating mechanism's maximum stroke, the rotating mechanism will receive a movement signal and rotate (step S609).

[0095] Furthermore, in some embodiments, the output of the target image processing neural network model also includes face target boxes; the control method further includes: when multiple face target boxes are determined based on image information within the initial interaction area, the target user corresponds to the largest face target box among the multiple face target boxes.

[0096] Specifically, when determining user location based on image information, after image segmentation, multiple users' facial information may be captured. For example, in the cropped driver's area, the faces and pupil recognition information of rear-seat passengers may be present. Similarly, when cropping the middle rear-seat area, the facial position information of the driver or front passenger may also be present. When the sound signal identifies the driver's position, if both driver and rear-seat passenger facial information are present after image capture, the algorithm further determines the face size. This is done by judging the size of the pixel area occupied by the target bounding box returned by the model, for example, by parameters b3 and b4 in the second part of the loss function, i.e., the face recognition information loss function. The target bounding box with the largest pixel area corresponds to the target user.

[0097] Based on the above description, taking the in-vehicle cockpit interactive device as an example, the drive module described in the embodiments of the present invention may include a rotating mechanism, and a one-dimensional rotating servo motor is used as an example for explanation.

[0098] Figure 5 A flowchart illustrating the process of adjusting the orientation of a display module via a rotating mechanism according to an embodiment of the present invention is shown below. Figure 5 As shown, it includes: S501 acquires audio and image signals.

[0099] The audio signal is obtained from the vehicle's microphone array, and the image information is captured by the in-vehicle camera.

[0100] Specifically, in this embodiment, the sound signal can be collected by an in-vehicle microphone array. Typically, multiple microphone arrays are arranged in the vehicle, in locations such as the driver's seat, front passenger seat, and rear seats. By arranging the microphones, the sound signal is collected, processed, and the interaction area is initially determined. This area is divided into the driver's seat, front passenger seat, and middle rear seat areas.

[0101] The image data can be collected by an in-vehicle camera, and the image area covered by the camera should include the front row of the driver's cabin and the middle rear row. After determining the initial interaction area through sound signals, the image information is analyzed to pinpoint the specific location of the interactive objects within that area, thereby distinguishing multiple users captured in the image.

[0102] The S502 calculates the user's eye position and converts the coordinate system to the servo angle in real time based on the sound source and image.

[0103] Specifically, after the image is located using an algorithm, the exact position of the interactive object is determined. The pixel coordinates of the image recognition are then converted to angular positions in the coordinate system of the rotating mechanism.

[0104] S503: The angle information is sent to the rotating mechanism. After receiving the angle information, the rotating mechanism calculates the rotation angle and determines whether rotation is necessary. If rotation is necessary, a rotation command is sent to control the rotating mechanism to rotate; if rotation is not necessary, the rotating mechanism remains in its current state.

[0105] By combining sound and image information, the sound information can initially determine the user's location, and the image information can be combined to determine the user's spatial location. This allows for accurate positioning of the user, improves the accuracy of the rotating mechanism, and provides a better user experience.

[0106] In some embodiments, the interactive device includes a touch feedback module, which provides feedback signals when the user interacts with the displayed image, similar to the vibration feedback of a touch screen. Specifically, the feedback signals can be provided by ultrasound, airflow, or other means.

[0107] The control method of the interactive device in this embodiment of the invention further includes: when the interactive device is in a gesture interaction state, based on the relative position information of the target user's interactive gesture to the image displayed on the interactive device in the touch area, controlling the touch feedback module to send a touch feedback signal to the location of the interactive gesture, so that the user's hand generates tactile sensation.

[0108] Specifically, the interactive device can provide displayed images, such as images displayed on a screen or projected images into the air. These displayed images can be interactive interfaces or virtual pets. Users can interact with the displayed images, such as function icons on the interface or interact with the displayed pet, specifically by petting a cat. When a user's gesture is detected, if the gesture is within the touch area, the relative position of the gesture is determined, and the touch feedback module is controlled to emit touch feedback signals, such as airflow or high-frequency ultrasonic waves, towards the gesture's location. The user's hand can feel the airflow or ultrasonic signals, thus generating tactile feedback when interacting with the displayed image. This is especially beneficial when interacting with virtual pets, such as in-car pet assistants, creating a realistic tactile experience and enhancing the interactive experience.

[0109] In one embodiment, the relative position information of the interactive gesture in the touch area can be obtained based on the image information of the target user's interactive gesture to the image displayed on the interactive device. For example, the user's state and behavior can be perceived by a sensing module. Specifically, the sensing module may include different devices such as a camera and a time-of-flight point cloud detector.

[0110] According to the control method of the interactive device of the present invention, based on the touch feedback module of the interactive device, when the user operates the displayed image, the touch feedback module is controlled to send a touch feedback signal to the location of the interactive gesture in response to the gesture information, thereby enabling the user's interactive hand to generate a real tactile sensation and improving the interactive experience.

[0111] In some embodiments, the touch feedback signal is a focused ultrasonic signal; that is, the touch feedback module may include an ultrasonic sensor array that emits high-frequency ultrasonic waves, which are reflected and scattered when they encounter the user's skin. By controlling the emission and reception of ultrasonic waves, specific vibration patterns can be formed on the user's skin, thereby simulating different tactile sensations.

[0112] Specifically, the ultrasonic array can generate ultrasonic waves within a certain range, allowing users to feel touch in the vicinity of the plane where the image is displayed on the interactive device. The range of tactile feedback generated by the touch feedback module can be determined based on the design of the ultrasonic sensor array. For example, the touch area is within 50cm in front of the module, and its width is determined by the width of the ultrasonic sensor array.

[0113] Furthermore, in some embodiments, the interactive device displays an image as a real image formed by a negative refractive index plate. Specifically, the interactive device may include a display module, which can be an aerial imaging module, specifically including a display screen and a negative refractive index plate. The display screen is used to display a target image, which may be an interface, video, or virtual pet, etc., without specific limitations. The negative refractive index plate is at a preset angle to the display screen. The negative refractive index plate is used to generate a real image of the target image on the imaging plane of the aerial target area when light containing the target image is incident on it. The imaging plane and the display screen are located on different sides of the negative refractive index plate, and the imaging plane and the display screen are symmetrical about the negative refractive index plate as an axis. The target image on the imaging plane is a real image of the image displayed on the display screen. Thus, when the user operates the displayed image, combined with touch feedback, the tactile experience becomes more realistic.

[0114] The following explanation uses a focused ultrasonic signal as a touch feedback signal to illustrate the tactile control process.

[0115] Figure 6 This is a flowchart of a control method for an interactive device according to an embodiment of the present invention, such as... Figure 6 As shown, it includes the following steps: S901, begin.

[0116] In this embodiment, the ultrasonic touch control is a passive module; its activation requires triggering or invocation by other systems. For example, when an image is already displayed and can be touched, the system invokes the touch feedback module, at which point it can be activated.

[0117] S902, acquire the imaging range.

[0118] When an image is displayed, its position in space and the volume it occupies can be calculated by the imaging module. The ultrasound module obtains this imaging range information to calculate the range of ultrasound waves emitted.

[0119] S903, obtains information related to user gestures.

[0120] Based on the user's gesture information, the displayed content is changed, and the relative coordinates of the gesture within the ultrasonic module are obtained. The gesture action can be acquired through a sensing module. Action classification information can be obtained using algorithms such as image recognition, and spatial coordinate information can be obtained from the point cloud data of a ToF camera.

[0121] S904 determines whether the user's hand coordinates are within the feedback coordinates.

[0122] Specifically, the user's hand position is acquired using a ToF camera to determine whether the user's hand is within the touchable range of the display area. If the hand is within the display range, the ultrasonic array sends a focus signal to that area, proceeding to step S906; if the hand is not within the display range, no ultrasonic signal is sent, proceeding to step S905.

[0123] In some embodiments, the control ultrasonic call signal can be a display system. If the display system does not require touch interaction, then the process ends at step S907.

[0124] S905 does not provide haptic feedback focusing.

[0125] S906, the ultrasonic array is focused towards the hand.

[0126] S907, End.

[0127] The control method of the interactive device in this embodiment of the invention uses a touch feedback function triggered based on interactive gesture information. When the interactive gesture is within the touch range, a focused ultrasonic signal is emitted to the interactive hand through a touch feedback module, such as an ultrasonic array. Through ultrasonic tactile interaction, the tactile experience of the user's gesture operation can be improved.

[0128] Based on the control method of the interactive device in the above embodiments, a second aspect of the present invention proposes an electronic device.

[0129] Figure 7 This is a block diagram of an electronic device according to an embodiment of the present invention, such as... Figure 7 As shown, the electronic device 10 includes at least one processor 11 and a memory 12 communicatively connected to the at least one processor.

[0130] The memory 12 stores a computer program that can be executed by at least one processor 11. When the at least one processor 11 executes the computer program, it implements the control method of the interactive device in the above embodiment.

[0131] According to the present invention, the electronic device 10 executes the control method of the interactive device of the above embodiment through at least one processor, and determines the location information of the target user by combining voice information and image information. The voice information can initially determine the user's location, and the image information can then be combined to determine the user's spatial location. By combining these two types of information, the user's location can be accurately located, improving the accuracy of the drive module in adjusting the angle of the display module and providing a better user experience.

[0132] In addition, when interactive devices perform gesture interactions, touch feedback signals are sent to the user's hand based on the gesture information, so that the hand can generate tactile sensation and improve the interactive experience.

[0133] In addition, the angle of the display module can be adjusted based on the target user's location to facilitate interaction with the target user.

[0134] A third aspect of the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed, implements the control method of the interactive device described above.

[0135] A fourth aspect of the present invention provides an interactive device.

[0136] Figure 8 This is a schematic diagram of an interactive device according to an embodiment of the present invention, such as... Figure 8 As shown, the interactive device 100 includes a display module 103 and a driving module 104.

[0137] Display module 103 is used to display the target screen, which can be an operating interface or a virtual pet image, such as a Pokémon.

[0138] The driving module 104 is connected to the display module 103. The driving module 104 is used to drive the display module to move to the position of the target user based on the target user's location information. The target user's location information is obtained based on voice information and image information in the space where the interactive device is located.

[0139] The interactive device 100 according to an embodiment of the present invention combines sound and image information. The sound information can initially determine the user's location, and the image information can be combined to determine the user's spatial location. By combining these two types of information, the user's location can be accurately located, improving the accuracy of the driving module driving the display module 103 and providing a better user experience.

[0140] like Figure 8 As shown, the interactive device 100 also includes a touch feedback module 102.

[0141] The haptic feedback module 102 is used to respond to the interactive gesture of the target user to the image displayed on the display module, and to emit haptic feedback signals to the touch area where the interactive gesture is located, so that the hand generates haptic sensation.

[0142] Specifically, the display module 103 displays the target screen, and the user can operate the displayed image, such as operating the function icons on the display interface or operating the displayed pet, specifically, petting a cat. When the user's gesture information is detected, if the interactive gesture is within the touch area, the relative position of the interactive gesture within the touch area is determined, and the touch feedback module is controlled to emit touch feedback signals, such as airflow or high-frequency ultrasonic signals, to the location of the interactive gesture. The user's interactive hand can feel the airflow or ultrasonic signals, that is, tactile feedback can be generated when operating the displayed image. Especially when interacting with virtual pets, such as in-car pet elves, a realistic tactile sensation is generated, thereby improving the interactive experience.

[0143] According to an embodiment of the present invention, the interactive device 100, based on the touch feedback module 102 of the interactive device, responds to the gesture information to control the touch feedback module to send a touch feedback signal to the location of the interactive gesture when the user operates the displayed image, thereby enabling the user's interactive hand to generate a real tactile sensation and improving the interactive experience.

[0144] Furthermore, in the embodiments, such as Figure 8 As shown, the touch feedback module 102 includes an ultrasonic array. The ultrasonic array is used to generate focused ultrasonic signals within the target touch area as touch feedback signals. The target touch area includes at least the display area of ​​the display module. Therefore, when the user performs gesture interactions, their hand will have a tactile sensation of touch operation, improving the interactive experience.

[0145] Specifically, the ultrasonic array can generate ultrasonic waves within a certain range, allowing users to feel touch in the space near the plane where the image is displayed on the interactive device. The range of tactile feedback generated by the touch feedback module can be determined based on the design of the ultrasonic sensor array. For example, the touch area is within 50cm in front of the module, and its width is determined by the width of the ultrasonic array.

[0146] Furthermore, in the embodiments, such as Figure 8 As shown, the interactive device 100 also includes a sensing module 101, which is used to sense the user's interactive gestures to the image displayed by the display module.

[0147] Specifically, the relative position information of the interactive gesture in the touch area can be obtained based on the image information of the target user's interactive gesture to the image displayed on the interactive device. For example, the user's state, behavior, etc., can be sensed by the sensing module 101. In an embodiment, the sensing module 101 may include different devices such as a camera and a time-of-flight point cloud detector.

[0148] In this embodiment, the display module may be an in-vehicle display screen, a control projection module, or a module with other display formats.

[0149] For example, Figure 9 This is a schematic diagram of the semi-transparent structure of a display module according to an embodiment of the present invention, as shown below. Figure 10 The diagram shown is a schematic diagram of the split structure of a display module according to an embodiment of the present invention. The display module 103 includes a display assembly 204 and a negative refractive index plate 202.

[0150] The display assembly 204 includes a display screen 2041 for displaying the target image; a negative refractive index plate 202 is at a preset angle to the display screen 2041, and the negative refractive index plate 202 is used to generate a real image of the target image on the imaging plane of the target area in the air when light containing the target image is incident on it.

[0151] Among them, such as Figure 11 As shown, the imaging plane 303 and the display screen 2041 are located on different sides of the negative refractive index plate 202, and the imaging plane 303 and the display screen 2041 are symmetrical about the negative refractive index plate 202.

[0152] Furthermore, such as Figure 10 As shown, the display module 103 also includes a housing 203, which has a receiving space. The top of the housing 203 has a window that communicates with the receiving space. The negative refractive index plate 202 is disposed at the window. The display assembly 204 is located in the receiving space.

[0153] In some embodiments, the cross-section of the housing 203 parallel to the window is circular on the outside and rectangular with rounded corners on the inside.

[0154] Specifically, such as Figure 10 As shown, the outer shell 203 is cylindrical on the outside and rectangular with rounded corners on the inside. The negative refractive index plate 202 is connected to the cylindrical shell 203 by adhesive bonding to the top of the cylindrical shell. The display assembly 204 is installed inside the cylindrical shell.

[0155] like Figure 10 As shown, the display module 103 also includes a connector 205, which is located at the bottom of the housing 203 and is used to connect the display module 103 and the drive module 104.

[0156] For example, the connector 205 can connect the cylindrical housing 203 and the drive module 104, such as the rotating mechanism, with screws. The connector 205 has a hollow structure in the middle, which can be used to display the wiring of the assembly 204, making it simple and practical.

[0157] In an embodiment, the interactive device 100 further includes a controller connected to the display module 103, the driving module 104, the touch feedback module 102, and the sensing module 101. The controller is used to execute the control method of the interactive device in the above embodiment to control the driving module 104 and the touch feedback module 102 to improve the interactive experience.

[0158] Specifically, taking an aerial imaging module as an example, the display module 103 receives a display image on a display plane 303, which is a real image plane. The image from the display assembly 204 is propagated to the display plane 303 by the negative refractive index plate 202. Users can see the image displayed on the display plane 303 within certain angle ranges of the display device. During gesture interaction, based on gesture interaction information, the touch feedback module 102 can generate ultrasonic waves, allowing the user to feel touch in the vicinity of the display plane 303, thus enhancing the tactile experience of gesture interaction. Furthermore, when it is determined that the angle of the display module 103 needs adjustment based on voice and image information, the display module 103 can be adjusted to the corresponding user position via a drive module 104, such as a rotation mechanism, facilitating interaction between the user and the interactive object.

[0159] Based on the electronic devices and interactive devices described above, a fifth aspect of the present invention proposes a vehicle.

[0160] In one embodiment, the vehicle includes the electronic device described above; or, the vehicle includes the interactive device described above.

[0161] For example, the interactive device could be an in-vehicle interactive device.

[0162] According to the vehicle of the present invention, by employing the electronic device or interactive device of the above embodiments, tactile feedback can be generated to the user's hand when the user performs gesture interaction, thereby improving the interactive experience.

[0163] Furthermore, the vehicle also includes a voice acquisition device and an image acquisition device. The voice acquisition device is used to acquire voice information in the space where the interactive device is located; the image acquisition device is used to acquire image information in the space where the interactive device is located.

[0164] Specifically, the voice acquisition device can be an in-vehicle microphone array, and the image acquisition device can be an in-vehicle camera. The user's position can be determined based on the voice and image information within the vehicle cabin, and the angle of the display module of the interactive device can be adjusted based on the user's position to facilitate interaction with the target user.

[0165] In an embodiment, such as Figure 12The diagram shows the installation position of an interactive device according to an embodiment of the present invention, wherein the center console armrest of the vehicle forms the housing 203 of the display module of the interactive device.

[0166] In an embodiment, such as Figure 12 As shown, the touch feedback module 102 of the interactive device is located on the center console armrest.

[0167] In some embodiments, the sensing module 101 of the interactive device can be positioned on the vehicle, for example, optionally on the vehicle's in-vehicle display screen. It can also be positioned above the touch feedback module 102, and multiple positions are possible.

[0168] Among them, such as Figure 12 As shown, if the sensing module 101 is placed at the vehicle display screen and embedded in the vehicle pad, it saves more space, but the related sensors can be customized, such as distance detectors, point cloud cameras, etc.

[0169] In some embodiments, the haptic feedback module can be installed below the in-vehicle display screen, and the display module can rotate left and right, with the display module embedded in the center console armrest. Figure 12 The dotted frame in the middle represents the display plane 30, i.e. the imaging position. This area is within the touch control range of the touch feedback module. Users can put their hands in this area, where there are touch feedback signals, such as ultrasonic waves. The interaction between the hand and the ultrasonic waves produces a tactile sensation.

[0170] In some embodiments, the controller of the interactive device may be an in-vehicle domain controller or a vehicle controller.

[0171] Figure 13 This is a schematic diagram illustrating the communication connection between vehicle components and components in an interactive device according to an embodiment of the present invention, as shown below. Figure 13 As shown, the system includes a drive module 104, a communication bus 1003, a display module 103, a touch feedback module 102, a microphone array 1006, a processor 11, a memory 12, and a sensing module 101.

[0172] The drive module 104 can be a rotating mechanism, providing drive and control. The communication bus 1003 provides data and signal transmission and communication between the drive module 104, display module 103, touch feedback module 102, microphone array 1006, processor 11 (e.g., vehicle controller or domain controller), memory 12, and sensing module 101 (e.g., a sensing module). The processor 11 has vector acceleration computing capabilities, enabling faster deep learning model calculations. The memory 12 stores computer programs, which, when executed by the processor, perform the aforementioned sound signal processing, image preprocessing, deep learning model prediction, angle calculation, touch position calculation, and rotating mechanism signal calculation.

[0173] The vehicle of this invention can achieve intelligent interaction and feedback in multiple dimensions, including vision, touch, and hearing, further enhancing user experience and driving safety.

[0174] In the description of this specification, any process or method described in the flowcharts or otherwise herein may be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order according to the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0175] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable storage medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0176] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0177] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0178] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0179] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

[0180] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example.

[0181] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A control method for an interactive device, characterized in that, The interactive device includes a movable display module, and the control method includes: The location information of the target user is obtained, which is based on voice and image information of the space where the interactive device is located; Based on the location information, the display module is controlled to move to the location corresponding to the target user.

2. The control method according to claim 1, characterized in that, The location information is determined based on the facial image information of the target user within the initial interaction area; The initial interaction area is determined based on the voice information; The facial image information of the target user is determined based on the image information within the initial interaction area.

3. The control method according to claim 2, characterized in that, The image information within the initial interaction area is obtained by preprocessing and cropping the image information within the space where the interaction device is located.

4. The control method according to claim 2, characterized in that, The facial image information of the target user is the output of the target image processing neural network model, which takes the image information within the initial interaction area as input.

5. The control method according to claim 4, characterized in that, The target image processing neural network model is constructed based on the target loss function, which is obtained based on the face binary classification loss function, the face target box loss function, and the face recognition information loss function.

6. The control method according to claim 5, characterized in that, The target loss function is obtained by summing the values ​​of the first loss function, the second loss function, and the third loss function. The first loss function value is obtained by multiplying the face binary classification loss function and the first coefficient; The second loss function value is obtained by multiplying the face bounding box loss function and the second coefficient. The third loss function value is obtained by multiplying the face recognition information loss function and the third coefficient.

7. The control method according to claim 6, characterized in that, The objective loss function is expressed as follows: ； in, Let the target loss function be... Let the face binary classification loss function be... Let the loss function be the face bounding box. Let the loss function be the facial recognition information. For binary classification, the label truth value. , and These are weighting coefficients, and the first coefficient is... The second coefficient is The third coefficient is .

8. The control method according to claim 4, characterized in that, The output of the target image processing neural network model also includes a face bounding box; The control method further includes: When multiple face bounding boxes are determined based on image information within the initial interaction area, the target user corresponds to the largest face bounding box among the multiple face bounding boxes.

9. The control method according to claim 1, characterized in that, Before controlling the display module to move to the position corresponding to the target user based on the location information, the control method further includes: When the position deviation of the display module is less than or equal to the position deviation threshold, the display module remains in the current position; Alternatively, when the position deviation of the display module is greater than the position deviation threshold, the display module can be moved based on the position information; The position deviation is the deviation between the current position of the display module and the position information of the target user.

10. The control method according to any one of claims 1-9, characterized in that, The control method further includes: The interactive device is in gesture interaction mode; Based on the relative position information of the target user's interactive gestures to the image displayed on the interactive device in the touch area, the touch feedback module is controlled to send touch feedback signals to the location of the interactive gestures, so that the user's hand generates tactile sensation.

11. The control method according to claim 10, characterized in that, The touch feedback signal is an ultrasonic signal.

12. The control method according to claim 10, characterized in that, The relative position information is obtained based on the image information of the target user's interactive gestures to the image displayed on the interactive device.

13. The control method according to claim 10, characterized in that, The interactive device displays a real image formed by a negative refractive index plate.

14. An electronic device, characterized in that, include: At least one processor; A memory that is communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which, when executing the computer program, implements the control method of the interactive device according to any one of claims 1-13.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the control method of the interactive device according to any one of claims 1-13.

16. An interactive device, characterized in that, include: Display module, used to display the target image; A driving module is connected to the display module, and the driving module is used to drive the display module to move to the position corresponding to the target user based on the target user's location information; The location information of the target user is obtained based on voice and image information of the space where the interactive device is located.

17. The interactive device according to claim 16, characterized in that, The interactive device also includes: A haptic feedback module is used to respond to an interactive gesture by a target user to an image displayed on the display module, by emitting a haptic feedback signal to the touch area where the interactive gesture is located, so that the hand generates haptic sensation.

18. The interactive device according to claim 17, characterized in that, The touch module includes: An ultrasonic array is used to generate a focused ultrasonic signal within a target touch area as a touch feedback signal, wherein the target touch area includes at least the display area of ​​the display module.

19. The interactive device according to claim 17, characterized in that, The interactive device also includes: A sensing module is used to sense user interaction gestures with the images displayed by the display module.

20. The interactive device according to any one of claims 16-19, characterized in that, The display module includes: The display assembly includes a display screen for displaying the target image. A negative refractive index plate is positioned at a preset angle to the display screen. The negative refractive index plate is used to generate a real image of the target image on the imaging plane of the target area in the air when light containing the target image is incident on it. The imaging plane and the display screen are located on different sides of the negative refractive index plate, and the imaging plane and the display screen are symmetrical about the negative refractive index plate.

21. The interactive device according to claim 20, characterized in that, The display module also includes: The housing has a receiving space, and the top of the housing has a window that communicates with the receiving space. The negative refractive index plate is disposed at the window. The display assembly is located in the receiving space.

22. The interactive device according to claim 21, characterized in that, The cross-section of the shell parallel to the window is circular on the outside and rectangular with rounded corners on the inside.

23. The interactive device according to claim 21, characterized in that, The display module also includes: A connector located at the bottom of the housing for connecting the display module and the drive module.

24. The interactive device according to claim 19, characterized in that, The interactive device also includes: A controller, connected to the display module, the driving module, the touch feedback module and the sensing module, is used to execute the control method of the interactive device according to any one of claims 1-13.

25. A vehicle, characterized in that, The vehicle includes the electronic equipment as described in claim 14; Alternatively, the vehicle may include the interactive device as described in any one of claims 16-24.

26. The vehicle according to claim 25, characterized in that, The vehicle also includes: A voice acquisition device is used to acquire voice information in the space where the interactive device is located; An image acquisition device is used to acquire image information of the space where the interactive device is located.

27. The vehicle according to claim 25, characterized in that, The center console of the vehicle forms the housing for the display module of the interactive device.

28. The vehicle according to claim 27, characterized in that, The touch module of the interactive device is located on the central control armrest box.

29. The vehicle according to claim 25, characterized in that, The sensing module of the interactive device is installed on the vehicle's in-vehicle display screen.

30. The vehicle according to claim 25, characterized in that, The controller of the interactive device is an in-vehicle domain controller or a vehicle controller.

Images