Data processing method, vehicle, storage medium and computer program product

Abstract

The present disclosure provides a data processing method, a vehicle, a storage medium and a computer program product. The method includes: obtaining the action state data of the target object and the multimedia signal of the vehicle input device, wherein the multimedia signal is used to determine the acoustic object to be played; generating a control signal to be executed based on the action state data and the multimedia signal; and using the control signal to control the vehicle output device to execute a control operation matching the action state data. The present disclosure solves the technical problems of poor interactive experience and low real-time performance in the data processing methods provided in the related art.

Description

Technical Field

[0001] The present disclosure relates to the field of intelligent interaction technology, and in particular, to a data processing method, a vehicle, a storage medium, and a computer program product. Background Art

[0002] As an important part of smart car cockpit entertainment and information transmission, the function of the car audio system directly affects the user's driving experience. The car audio system in related technologies mostly adopts the sound source playback mode, that is, the system plays the audio file according to the preset sound quality parameters. The user can only adjust the overall audio output effect through the pre-set parameters, and cannot directly interact with the sound objects in the music. At the same time, the optoelectronic and mechanical devices in the cockpit also lack a mechanism for real-time linkage with user actions, which reduces the user interaction experience.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present disclosure provide a data processing method, a vehicle, a storage medium, and a computer program product to at least solve the technical problems of poor interactive experience and low real-time performance in the data processing methods provided in the related arts.

[0005] According to one aspect of an embodiment of the present disclosure, a data processing method is provided, including: acquiring action state data of a target object and a multimedia signal of an on-board input device, wherein the multimedia signal is used to determine an acoustic object to be played; generating a control signal to be executed based on the action state data and the multimedia signal; and using the control signal to control an on-board output device to execute a control operation matching the action state data.

[0006] Optionally, obtaining the motion state data of the target object includes: obtaining the motion state data of the target object using a position sensing device and / or a vehicle-mounted visual input device.

[0007] Optionally, generating a control signal to be executed based on the action state data and the multimedia signal includes: determining a position change of the target object based on the action state data; adjusting a characteristic parameter of the multimedia signal according to the position change to obtain an adjustment result; and generating a control signal to be executed using the position change and the adjustment result.

[0008] Optionally, adjusting characteristic parameters of the multimedia signal according to the position change to obtain an adjustment result includes: in response to the multimedia signal being an audio signal of a vehicle-mounted audio input device, adjusting at least one of the frequency, volume and delay of the audio signal according to the position change to obtain an adjustment result.

[0009] Optionally, using the position change and the adjustment result to generate a control signal to be executed includes: in response to the on-board output device including a on-board speaker device, generating an acoustic control signal for the on-board speaker device based on the adjustment result; in response to the on-board output device including a on-board optoelectronic device, generating an optical control signal for the on-board optoelectronic device based on the position change; in response to the on-board output device including a on-board interactive device, generating a motor control signal for the on-board interactive device based on the position change.

[0010] Optionally, the control operation that matches the action status data by controlling the vehicle-mounted output device with the control signal includes: linkage control of the acoustic elements of the vehicle-mounted speaker device and the action status data based on the acoustic control signal; linkage control of the optical elements of the vehicle-mounted optoelectronic device and the action status data based on the optical control signal; linkage control of the interactive elements of the vehicle-mounted interactive device and the action status data based on the motor control signal.

[0011] Optionally, determining the position change of the target object based on the action state data includes: constructing a spatial coordinate system in the vehicle cabin based on the position of the target object; and determining the position change of the target object based on the action state data in the spatial coordinate system.

[0012] According to another aspect of an embodiment of the present disclosure, a data processing device is also provided, including: an acquisition module, used to acquire action state data of a target object and a multimedia signal of a vehicle-mounted input device, wherein the multimedia signal is used to determine an acoustic object to be played; a generation module, used to generate a control signal to be executed based on the action state data and the multimedia signal; and a control module, used to use the control signal to control the vehicle-mounted output device to perform a control operation matching the action state data.

[0013] Optionally, the acquisition module is further used to: acquire the motion state data of the target object using a position sensing device and / or a vehicle-mounted visual input device.

[0014] Optionally, the generation module is further used to: determine the position change of the target object based on the action state data; adjust the characteristic parameters of the multimedia signal according to the position change to obtain an adjustment result; and generate a control signal to be executed using the position change and the adjustment result.

[0015] Optionally, the generating module is further used to: in response to the multimedia signal being an audio signal of a vehicle-mounted audio input device, adjust at least one of the frequency, volume and delay of the audio signal according to the position change to obtain an adjustment result.

[0016] Optionally, the generation module is also used to: in response to the on-board output device including a on-board speaker device, generate an acoustic control signal for the on-board speaker device based on the adjustment result; in response to the on-board output device including a on-board optoelectronic device, generate an optical control signal for the on-board optoelectronic device based on the position change; in response to the on-board output device including a on-board interactive device, generate a motor control signal for the on-board interactive device based on the position change.

[0017] Optionally, the control module is also used to: link and control the acoustic elements and action status data of the vehicle-mounted speaker device according to the acoustic control signal; link and control the optical elements and action status data of the vehicle-mounted optoelectronic device according to the optical control signal; link and control the interactive elements and action status data of the vehicle-mounted interactive device according to the motor control signal.

[0018] Optionally, the generation module is further used to: construct a spatial coordinate system in the vehicle cabin based on the position of the target object; and determine the position change of the target object in the spatial coordinate system based on the action state data.

[0019] According to another aspect of the embodiment of the present disclosure, a vehicle is further provided, including: a memory storing an executable program; and a processor for running the program, wherein the data processing method in the embodiment of the present disclosure is executed when the program is running.

[0020] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is further provided, the computer-readable storage medium including a stored executable program, wherein when the executable program runs, the device where the computer-readable storage medium is located is controlled to execute the data processing method in the embodiment of the present disclosure.

[0021] According to another aspect of the embodiments of the present disclosure, a computer program product is also provided, including a computer program, and when the computer program is executed by a processor, the data processing method in the embodiments of the present disclosure is implemented.

[0022] In the disclosed embodiment, by acquiring and analyzing the motion state data of the target object and intelligently combining it with the multimedia signal of the vehicle-mounted input device, dynamic control of the acoustic object and the linkage response of the acoustic, optical and electrical devices in the cabin are achieved, thereby achieving the purpose of enhancing the interactivity and immersive experience in the cabin, thereby achieving the technical effect of improving the user's interactive experience and the real-time performance of the system, and further solving the technical problems of poor interactive experience and low real-time performance of the data processing method provided in the relevant technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0024] Figure 1 is a hardware structure diagram of a data processing method according to one embodiment of the present disclosure;

[0025] Figure 2 is a flow chart of a data processing method according to one embodiment of the present disclosure;

[0026] Figure 3 is a schematic diagram of a data processing system according to one embodiment of the present disclosure;

[0027] Figure 4 is a schematic diagram of a data processing method according to one embodiment of the present disclosure;

[0028] Figure 5 It is a structural block diagram of a data processing device according to one embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] In order to enable those skilled in the art to better understand the scheme of the present disclosure, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present disclosure.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0031] In-car audio systems in related technologies mostly use the sound source playback mode, that is, the system plays audio files according to preset sound quality parameters, and users can only adjust the overall audio output effect through pre-set parameters, but cannot directly interact with the sound objects in the music. At the same time, the optoelectronic and mechanical devices in the cockpit also lack a mechanism for real-time linkage with user actions, which reduces the user interaction experience.

[0032] Specifically, in the sound source playback mode, the car audio system plays the sound in the audio file as a whole, and the position and characteristics of the sound object are fixed during the playback process and cannot be adjusted in real time according to the user's needs or actions. For example, if the user is interested in the drum sound in a piece of music and hopes to feel stronger drum beats in the cockpit, the car audio system in the related art cannot meet this demand. In addition, although the optoelectronic equipment in the cockpit, such as ambient lights and display screens, and mechanical devices such as seat adjustment mechanisms and on-board robots, play a role in enhancing the cockpit atmosphere and providing convenience, their response mechanisms are usually pre-programmed or based on fixed trigger conditions, rather than real-time monitoring of user actions and intelligent linkage. For example, the color change or brightness adjustment of the ambient light, the automatic adjustment of the seat, etc., are often not synchronized with the user's specific actions or real-time position, which limits the ability of the cockpit environment to respond to the user's personalized needs, thereby affecting the user's overall interactive experience.

[0033] According to an embodiment of the present disclosure, a method embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] The method embodiment can be executed in an electronic device or a similar computing device including a memory and a processor. Taking running on a computer terminal as an example, the computer terminal may include one or more processors (processors may include but are not limited to central processing units (CPU), graphics processing units (GPU), digital signal processing (DSP) chips, microprocessors (MCU), programmable logic devices (Field Programmable Gate Array, FPGA), neural network processors (Neural-network Processor Unit, NPU), tensor processors (TensorProcessing Unit, TPU), artificial intelligence (Artificial Intelligence, AI) type processors and other processing devices) and a memory for storing data. Optionally, the above-mentioned computer terminal may also include a transmission device, an input and output device, and a display device for communication functions. It can be understood by those of ordinary skill in the art that the above-mentioned structural description is only for illustration, and it does not limit the structure of the above-mentioned computer terminal. For example, the computer terminal may also include more or fewer components than the above-mentioned structural description, or have a configuration different from the above-mentioned structural description.

[0035] The memory can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the data processing method in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the computer program stored in the memory, that is, realizing the above-mentioned data processing method. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0036] The transmission device is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0037] The display device may be, for example, a touch screen type liquid crystal display (LCD) and a touch display (also referred to as a "touch screen" or "touch display screen"). The liquid crystal display may enable a user to interact with a user interface of a mobile terminal. In some embodiments, the mobile terminal has a graphical user interface (GUI), and a user may interact with the GUI by finger contacts and / or gestures on a touch-sensitive surface, wherein the human-computer interaction functions here may optionally include the following interactions: creating web pages, drawing, word processing, making electronic documents, games, video conferencing, instant messaging, sending and receiving emails, call interfaces, playing digital videos, playing digital music and / or web browsing, etc. The executable instructions for executing the above human-computer interaction functions are configured / stored in a computer program product or readable storage medium executable by one or more processors.

[0038] Figure 1 is a hardware structure diagram of a data processing method according to one embodiment of the present disclosure, such as Figure 1 As shown, the hardware structure includes: terminal equipment, vehicles and roadside equipment.

[0039] Exemplarily, terminal devices generally refer to intelligent devices involved in communication, including but not limited to the target vehicle itself, mobile positioning terminals (such as smart phones, smart wearable devices, and vehicle-mounted communication systems of other vehicles), etc. The above-mentioned devices collect environmental data through built-in sensors, or collect information through user interaction, and then transmit the collected data to the cloud server or the vehicle-mounted control unit of the target vehicle through the network for analysis and processing. The network includes but is not limited to cellular networks (such as 4G, 5G), Wi-Fi, Bluetooth, Vehicle-to-Everything (V2X) and other technologies, which are responsible for real-time data transmission to ensure that the vehicle can receive the location information and status data shared by other vehicles, pedestrians, roadside equipment, etc. in a timely manner. Roadside equipment refers to equipment installed on the side of the road for monitoring and sensing the road environment, including but not limited to radars, cameras, infrared sensors, meteorological sensors, traffic light controllers, etc. The data collected by the roadside equipment is transmitted to the vehicle or cloud server through the network for vehicle control and traffic management.

[0040] Furthermore, a driving scene switching instruction can be issued by the terminal device, and the driving scene switching instruction can be sent to the vehicle through the network. The vehicle receives the driving scene switching instruction issued by the terminal device and sends an information subscription request to the roadside device. The roadside device receives the information subscription request from the vehicle and sends the continuously collected roadside perception information back to the vehicle.

[0041] Figure 2 is a flow chart of a data processing method according to one embodiment of the present disclosure, such as Figure 2 As shown, the method comprises the following steps:

[0042] Step S21, acquiring the action state data of the target object and the multimedia signal of the vehicle-mounted input device, wherein the multimedia signal is used to determine the acoustic object to be played;

[0043] Step S22, generating a control signal to be executed based on the action state data and the multimedia signal;

[0044] Step S23, using the control signal to control the vehicle-mounted output device to execute a control operation matching the action state data.

[0045] The above-mentioned vehicle-mounted input device refers to a device that can capture or receive user action state data and multimedia signals, including but not limited to position sensing devices, vehicle-mounted visual input devices, vehicle-mounted audio input devices and other types of sensors.

[0046] The above-mentioned acoustic object, i.e., a single-channel sound source, refers to a sound element in the audio content that can be independently controlled and processed. In multi-channel or spatial audio technology, the audio file is no longer simply viewed as a whole, but is broken down into multiple single-channel sound sources, each of which represents a sound source in the audio content, such as the performance of a musical instrument, a human voice conversation, a specific environmental sound effect, etc. A single-channel sound source has independent position information in a spatial audio system, i.e., a single-channel sound source has a specific coordinate in the three-dimensional spatial coordinate system of the cockpit.

[0047] The above-mentioned vehicle output device refers to a device in the cockpit that can receive the control signal generated by the central data processing unit and perform specific sound, light, electricity or mechanical operations according to the control signal. Vehicle output devices include but are not limited to optoelectronic actuators, acoustic actuators and other actuators. Among them, optoelectronic actuators, such as ambient lights, starry sky ceiling light arrays, reading lights, etc., can change the brightness, color or lighting mode according to the control signal to enhance the visual experience and atmosphere in the cockpit. For example, when the user's watch points to a certain direction, the lights in the corresponding area will automatically adjust to achieve the linkage between light and shadow and user actions; acoustic actuators mainly refer to vehicle audio systems, including speakers, subwoofers, etc., which can adjust the frequency, volume, delay and other parameters of the sound according to the control signal to achieve real-time positioning of acoustic objects and sound effect changes, so that the sound can change with the change of user actions to produce a dynamic auditory experience; other actuators, such as in-vehicle robots, rotating tweeters, seat vibrators, etc., can perform physical movements or changes according to the control signal, such as rotation, swinging, vibration, etc., to provide mechanical feedback or experience that matches the sound and visual effects.

[0048] Exemplarily, when the vehicle audio system is playing music, the smart watch worn by the passenger is used as a peripheral device to capture and transmit the motion state data of his arm in real time. At the same time, the vehicle's multimedia player is playing a song, in which the singer's main voice and accompaniment are used as independent single-channel sound sources, i.e., acoustic objects. When the passenger chooses to turn on the follow-up function on the vehicle interactive display screen and selects the singer's main voice as the interactive object, the central data processing unit analyzes the motion state data of the passenger's arm, identifies the change in the coordinates of the smart watch, and combines the acoustic object information in the multimedia signal to generate a control signal to be executed. After receiving the control signal, the vehicle speaker system changes the position of the singer's voice following the direction of movement of the passenger's arm, creating an immersive auditory interactive experience. At the same time, when the passenger's arm points to a specific area of ​​the vehicle, such as the starry sky ceiling light array on the roof, the central data processing unit matches the smart watch coordinates with the lamp bead coordinates. When the smart watch coordinates are close to or coincide with the lamp bead coordinates, the corresponding atmosphere light is turned on or the color of the atmosphere light is changed to form a visual effect that matches the position of the sound object.

[0049] Based on the above steps S21 to S23, by acquiring and analyzing the motion state data of the target object and intelligently combining it with the multimedia signal of the vehicle-mounted input device, dynamic control of the acoustic object and the linkage response of the acoustic, optical and electrical devices in the cabin are realized, thereby achieving the purpose of enhancing the interactivity and immersive experience in the cabin, thereby achieving the technical effect of improving the user's interactive experience and the real-time performance of the system, and further solving the technical problems of poor interactive experience and low real-time performance of the data processing method provided in the relevant technology.

[0050] Figure 3 is a schematic diagram of a data processing system according to one embodiment of the present disclosure, such as Figure 3 As shown, the system includes: peripheral equipment, vehicle-mounted audio input equipment, vehicle-mounted visual input equipment, a central data processing unit, vehicle-mounted speaker equipment, vehicle-mounted optoelectronic equipment and vehicle-mounted interactive equipment.

[0051] Optionally, in step S21, obtaining the motion state data of the target object includes: obtaining the motion state data of the target object using a position sensing device and / or a vehicle-mounted visual input device.

[0052] The above-mentioned position sensing device is a device with an independent position sensor, which refers to an external device that can detect and track the position and movement of a target object in three-dimensional space, including but not limited to digital wearable devices, handheld devices, vehicle-mounted positioning systems, seat sensors, and dedicated position trackers. Among them, digital wearable devices, such as smart watches and bracelets, are usually equipped with accelerometers, gyroscopes, or magnetometers, which can sense the wearer's arm movements, body movements, and changes in direction, and then convert the above changes into position data in the spatial coordinate system of the cabin; handheld devices, such as smart phones, have a variety of built-in sensors that can monitor the movement and tilt angle of the device, and are used to update the user's relative position in the cabin in real time; vehicle-mounted positioning systems, such as GPS receivers, are mainly used to determine the geographic location of the vehicle, and can also be combined with the position sensing system in the cabin to provide more accurate positioning information; seat sensors can detect the position and posture of passengers in the cabin through integrated pressure sensors or proximity sensors; dedicated position trackers, such as those based on ultrasonic, infrared, or radar technologies, are used to track the precise position of passengers or objects in the cabin in real time.

[0053] The above-mentioned on-board visual input device refers to a sensor or camera system that can obtain and analyze the visual features of users or objects in the cabin, and is mainly used to monitor and identify visual information in the cabin in real time, including but not limited to cameras, radars, visual sensor arrays, infrared thermal imagers, and beam scanners. Among them, the camera is mainly used to obtain image information inside the cabin, analyze the passenger's behavior, gestures or facial expressions through image recognition technology, and provide visual feature data; the radar is used to achieve non-contact monitoring of passengers or objects in the cabin; the visual sensor array is composed of multiple small cameras or image sensors, covering different angles and areas in the cabin, and can provide more comprehensive and blind-angle visual monitoring; the infrared thermal imager forms an image by capturing the infrared heat released by the object, and can monitor the passengers in the cabin in an environment with insufficient light or complete darkness.

[0054] For example, when the movements of passengers in the car change, the pressure sensor installed under the passenger seat works with the accelerometer and gyroscope in the smart watch. The former captures the slight changes in the passenger's sitting posture, while the latter records the specific movement trajectory of the passenger's arm. At the same time, the in-car camera and depth sensor array continuously scan the interior of the cabin to obtain the passenger's facial expressions and more detailed gestures. When the passenger starts the vehicle and puts on the smart watch, the in-vehicle system automatically recognizes and connects to the smart watch, and begins to monitor the passenger's arm movements through its built-in sensors. Suppose the passenger waves to the right, the sensor in the smart watch responds immediately, recording the acceleration and rotation angle of the passenger's arm movement.

[0055] Based on the above optional embodiments, by using position sensing devices and / or vehicle-mounted visual input devices to obtain the motion state data of the target object, it is possible to monitor and analyze the dynamic behavior of the target object in the cabin in real time, thereby improving the intelligence level of the vehicle and achieving a more intelligent driving experience.

[0056] Optionally, in step S22, generating a control signal to be executed based on the action state data and the multimedia signal includes:

[0057] Step S221, determining a position change of a target object based on the action state data;

[0058] Step S222, adjusting characteristic parameters of the multimedia signal according to the position change to obtain an adjustment result;

[0059] Step S223, generating a control signal to be executed using the position change and the adjustment result.

[0060] The above position change refers to the spatial three-coordinate change in the three-dimensional acoustic modeling space constructed in the cockpit, which is used to describe the dynamic position change of the target object, such as the passenger, in the cockpit. Through the position sensing device integrated in the smart wearable device or the seat, and the on-board visual input device such as the camera and radar, real-time capture and tracking are performed to obtain the tiny movements of the passenger's body or arm. The central data processing unit can convert the above dynamic behavior into numerical changes on the three coordinate axes of X, Y, and Z to form digital coordinates readable by the system.

[0061] For example, first, the dynamic behavior of passengers is captured in real time through position sensing devices and on-board visual input devices, and converted into coordinate changes in the three-dimensional acoustic modeling space; secondly, the central data processing unit calculates and adjusts the characteristic parameters of the multimedia signal according to the coordinate changes, so that the sound, image and light outputs are closer to the real-time actions and positions of the passengers; finally, based on the adjusted signal parameters, the system generates an execution control signal to drive the acoustic, optical and other execution equipment in the cabin to respond, thereby realizing immersive interaction between passenger actions and the cabin environment.

[0062] Based on the above optional embodiments, the position change of the target object is determined based on the action state data, and the characteristic parameters of the multimedia signal are adjusted according to the position change, and finally a control signal to be executed is generated, thereby realizing real-time linkage between the cabin environment and passenger behavior, and improving the interactivity of the cabin, the personalization of the user experience and the immersion.

[0063] Optionally, in step S222, characteristic parameters of the multimedia signal are adjusted according to the position change, and the adjustment result is obtained, including: in response to the multimedia signal being an audio signal of a vehicle-mounted audio input device, at least one of the frequency, volume and delay of the audio signal is adjusted according to the position change to obtain the adjustment result.

[0064] For example, the volume is adjusted dynamically according to the change in the position of the passenger relative to the sound source, ensuring that no matter where the passenger is in the cabin, they can get the best volume experience and avoid the volume fluctuation caused by the change in distance. When the passenger moves to the left, the central data processing unit responds to the change in the passenger's position, reduces the delay of the left speaker, and increases the volume at the same time, maintaining the sense of sound positioning and spatial effect.

[0065] Based on the above optional embodiments, when the multimedia signal is an audio signal of a vehicle-mounted audio input device, the central data processing unit can adjust characteristic parameters such as frequency, volume or delay of the audio signal in real time according to the position change of the target object in the cockpit, thereby providing accurate sound positioning and high-quality auditory experience, thereby enhancing the interactivity of the smart cockpit.

[0066] Optionally, in step S223, using the position change and the adjustment result to generate a control signal to be executed includes:

[0067] Step S2231, in response to the in-vehicle output device including an in-vehicle speaker device, generating an acoustic control signal for the in-vehicle speaker device based on the adjustment result;

[0068] Step S2232, in response to the vehicle-mounted output device including the vehicle-mounted optoelectronic device, generating an optical control signal for the vehicle-mounted optoelectronic device based on the position change;

[0069] Step S2233 , in response to the in-vehicle output device including the in-vehicle interactive device, generating a motor control signal for the in-vehicle interactive device based on the position change.

[0070] The above-mentioned in-vehicle speaker equipment refers to the sound system used for sound playback in the cockpit, including but not limited to surround sound speakers, tweeters, subwoofers, center speakers and overhead speakers. After the central data processing unit analyzes the action state data of the target object and adjusts the characteristic parameters such as frequency, volume or delay of the audio signal, it will generate an acoustic control signal for the in-vehicle speaker equipment based on the adjustment result. The acoustic control signal guides the in-vehicle speaker equipment to adjust the output, which is used to achieve stereo positioning of the sound, dynamic balance of the volume and optimization of playback synchronization, thereby providing a more personalized and immersive listening experience.

[0071] The above-mentioned on-board optoelectronic equipment refers to the lighting and display system in the cockpit, including ambient lights, starry sky ceiling light arrays, reading lights, instrument panel displays, head-up displays, etc. Based on the position changes of the target object in the cockpit, the central data processing unit can generate optical control signals for the on-board optoelectronic equipment. The optical control signal can indicate the brightness, color, display position and animation effects of the indicator beads or display pixels, so that the lights and display content change with the movements of the passengers, thereby enhancing the visual immersion and interactivity in the cockpit.

[0072] The above-mentioned in-vehicle interactive devices refer to cockpit devices that can interact with passengers through physical means, such as in-vehicle robots, adjustable seats, movable center consoles, rotating tweeters and other devices. In-vehicle interactive devices are usually equipped with servo motor systems that can achieve precise control of position, angle and motion trajectory. The central data processing unit can generate motor control signals for the in-vehicle interactive device based on the position change of the target object, and guide the servo motor of the in-vehicle interactive device to perform corresponding actions, such as adjusting the seat posture to adapt to the passenger's body position changes, or making the robot assistant's arms move with the passenger's gestures, thereby providing a more intelligent and humanized interactive experience.

[0073] Based on the above optional embodiments, through the real-time capture and analysis of the target object's action state by the central data processing unit, as well as the dynamic adjustment and control of audio signals, optoelectronic signals and interactive device signals, it is possible to achieve a high degree of synchronization and intelligent response between the multimedia environment in the cabin and the passenger's actions, creating a personalized, immersive interactive experience.

[0074] Optionally, in step S23, using the control signal to control the vehicle-mounted output device to perform a control operation matching the action state data includes:

[0075] Step S231, linking and controlling the acoustic elements and the action state data of the vehicle-mounted speaker device according to the acoustic control signal;

[0076] Step S232, linking and controlling the optical elements and the action state data of the vehicle-mounted optoelectronic device according to the optical control signal;

[0077] Step S233 , linking and controlling the interactive elements and the action state data of the vehicle-mounted interactive device according to the motor control signal.

[0078] The above acoustic elements refer to the sound characteristics that can be independently controlled in the vehicle-mounted speaker equipment, including but not limited to the direction of the sound, volume, frequency and delay time. When the acoustic control signal is issued according to the passenger's action status data, the vehicle-mounted speaker equipment can adjust these acoustic elements accordingly to achieve dynamic changes and precise positioning of the sound.

[0079] The above optical elements refer to the adjustable visual parameters in the on-board optoelectronic equipment, including but not limited to brightness, color, display position and animation effects. Based on the linkage between the optical control signal and the passenger's action status, the on-board optoelectronic equipment can change its own optical elements to enhance the visual immersion of the cockpit.

[0080] The above-mentioned interactive elements refer to the mechanical parts of the in-vehicle interactive device used for physical interaction with passengers, including but not limited to the adjustment mechanism of the seat, the motion joints of the robot in the car, the rotating device of the rotating tweeter, etc. When the linkage control is activated, the interactive elements of the in-vehicle interactive device will respond according to the passenger's action state data, such as adjusting the seat's tilt angle to adapt to the passenger's posture changes, or making the robot assistant imitate the passenger's gestures to perform actions, thereby providing passengers with a more intelligent, intuitive and humanized interactive experience.

[0081] For example, when a passenger activates the follow-up music playback function in the vehicle system, the central data processing unit quickly calculates the relative position between the audio object and the passenger based on the three-coordinate changes of the passenger's arm movement, and then adjusts the characteristic parameters of the audio signal to ensure that the audio output changes synchronously with the passenger's movement, creating an immersive auditory experience. By reducing the delay of the left speaker and increasing the delay of the right speaker, the simulated sound seems to be emitted naturally from the direction pointed by the passenger, and the volume and frequency are fine-tuned according to the relative distance and direction between the passenger and the sound source to ensure the harmony and coherence of the sound output. During the adjustment of the audio signal, the central data processing unit synchronously generates the optical control signal of the vehicle-mounted optoelectronic device. When the passenger's arm points to a specific area of ​​the cabin, such as the starry sky ceiling, the central data processing unit immediately sends an optical control signal to the corresponding lamp bead, adjusts its brightness and color, synchronizes the lamp bead with the rhythm of the music, and creates an immersive audio-visual linkage experience. At the same time, the three-axis robot in the vehicle-mounted interactive device also receives the motor control signal related to the passenger's movement changes. When the passenger's hand movements are associated with specific elements in the music, the robot accurately replicates the passenger's movements through a servo motor system, as if playing together with the passenger, making the passenger's experience in the cabin richer and more vivid.

[0082] Based on the above optional embodiments, by integrating the linkage control of acoustics, optics and physical interactions, a new way for passengers to deeply interact with the cabin environment is provided, thereby enhancing the personalization and immersion of the passengers' in-car entertainment experience.

[0083] Optionally, in step S221, determining the position change of the target object based on the action state data includes:

[0084] Step S2211, in the vehicle cabin, constructing a spatial coordinate system based on the location of the target object;

[0085] Step S2212, determining the position change of the target object based on the action state data in the spatial coordinate system.

[0086] For example, when the central data processing unit realizes the interaction between the user in the cabin and various actuators, it first constructs a spatial coordinate system with the specific position of the passenger in the cabin as the origin of the coordinate system. Then, the passenger's motion state data is obtained through position sensing devices, such as smart watches worn by passengers, or on-board visual input devices in the cabin, such as high-precision cameras, and then the passenger's position change is determined based on the passenger's motion state data and the relative position relationship between the coordinates of various actuators in the cabin.

[0087] Based on the above optional embodiments, a spatial coordinate system with the passenger position as the origin is constructed by the central data processing unit, and real-time capture and analysis of the passenger's motion status data, so as to achieve highly intelligent and personalized real-time interaction between the user in the cabin and various actuators.

[0088] Figure 4 is a schematic diagram of a data processing method according to one embodiment of the present disclosure, such as Figure 4 As shown, the method includes the following steps: first, using a smart watch as a peripheral device, a vehicle-mounted host as a vehicle-mounted audio input device, and an in-vehicle camera as a vehicle-mounted visual input device, and acquiring data in the vehicle cabin based on the peripheral device, the vehicle-mounted audio input device, and the vehicle-mounted visual input device; second, when a passenger in the vehicle wearing a smart watch turns on the follow-up function, the position data of the smart watch is connected to the cabin space, and all the acquired data is transmitted to a central data processing unit to capture the coordinate changes of the peripheral device; finally, the three spatial coordinates are calculated, and when the coordinates of each execution device are consistent with the angular components of the watch connection line and the angular components of the three spatial coordinates, the corresponding execution device performs an action, such as the lamp bead is lit up, otherwise, no response is made.

[0089] Exemplarily, when a passenger makes a specific action in the cabin, such as waving his arm, the smartwatch will send the recorded spatial three-coordinate change data to the central data processing unit. In response to the change in the passenger's action state, the central data processing unit converts the spatial three-coordinate change data into a control signal corresponding to the execution device in the cabin. For example, if the direction of the arm movement points to a certain lamp bead in the on-board photoelectric actuator, the central data processing device will calculate the corresponding coordinate change and generate a control signal to light up or adjust the brightness and color of the lamp bead in response to the passenger's action. At the same time, the central data processing device will adjust the frequency, volume and delay parameters of the sound based on the coordinates of the passenger's arm movement, so that the sound and image position of the sound source seems to move with the passenger's action, creating an immersive auditory experience. Alternatively, the data of the passenger's position change is converted into a control signal for the action of the robot in the car, such as adjusting its direction or the extension of the robotic arm, to achieve physical interactive feedback.

[0090] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0091] According to an embodiment of the present disclosure, a device embodiment of a data processing device is provided. It should be noted that the device can be used to execute the above-mentioned data processing method.

[0092] Figure 5 is a structural block diagram of a data processing device according to one embodiment of the present disclosure, such as Figure 5 As shown, the device comprises:

[0093] An acquisition module 501 is used to acquire the action state data of the target object and the multimedia signal of the vehicle-mounted input device, wherein the multimedia signal is used to determine the acoustic object to be played;

[0094] A generating module 502, configured to generate a control signal to be executed based on the action state data and the multimedia signal;

[0095] The control module 503 is used to control the vehicle-mounted output device to execute a control operation matching the action state data by using a control signal.

[0096] Optionally, the acquisition module 501 is further used to: acquire the motion state data of the target object by using a position sensing device and / or a vehicle-mounted visual input device.

[0097] Optionally, the generation module 502 is further used to: determine the position change of the target object based on the action state data; adjust the characteristic parameters of the multimedia signal according to the position change to obtain an adjustment result; and generate a control signal to be executed using the position change and the adjustment result.

[0098] Optionally, the generating module 502 is further configured to: in response to the multimedia signal being an audio signal of a vehicle-mounted audio input device, adjust at least one of the frequency, volume and delay of the audio signal according to the position change to obtain an adjustment result.

[0099] Optionally, the generation module 502 is also used to: in response to the on-board output device including a on-board speaker device, generate an acoustic control signal for the on-board speaker device based on the adjustment result; in response to the on-board output device including a on-board optoelectronic device, generate an optical control signal for the on-board optoelectronic device based on the position change; in response to the on-board output device including a on-board interactive device, generate a motor control signal for the on-board interactive device based on the position change.

[0100] Optionally, the control module 503 is also used to: perform linkage control on the acoustic elements and action status data of the vehicle-mounted speaker device according to the acoustic control signal; perform linkage control on the optical elements and action status data of the vehicle-mounted optoelectronic device according to the optical control signal; and perform linkage control on the interactive elements and action status data of the vehicle-mounted interactive device according to the motor control signal.

[0101] Optionally, the generating module 502 is further used to: construct a spatial coordinate system in the vehicle cabin based on the position of the target object; and determine the position change of the target object in the spatial coordinate system based on the action state data.

[0102] An embodiment of the present disclosure further provides a vehicle, comprising: a memory storing an executable program; and a processor for running the program, wherein the method in each embodiment of the present disclosure is executed when the program is running.

[0103] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0104] S1, obtaining the action state data of the target object and the multimedia signal of the vehicle input device, wherein the multimedia signal is used to determine the acoustic object to be played;

[0105] S2, generating a control signal to be executed based on the action state data and the multimedia signal;

[0106] S3, using the control signal to control the vehicle-mounted output device to perform a control operation that matches the action state data.

[0107] An embodiment of the present disclosure further provides a computer-readable storage medium, which includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present disclosure.

[0108] Optionally, in this embodiment, the storage medium may be configured to store a computer program for performing the following steps:

[0109] S1, obtaining the action state data of the target object and the multimedia signal of the vehicle input device, wherein the multimedia signal is used to determine the acoustic object to be played;

[0110] S2, generating a control signal to be executed based on the action state data and the multimedia signal;

[0111] S3, using the control signal to control the vehicle-mounted output device to perform a control operation that matches the action state data.

[0112] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store computer programs.

[0113] The embodiments of the present disclosure further provide a computer program product, including a computer program, which implements the methods in the various embodiments of the present disclosure when executed by a processor.

[0114] Optionally, in this embodiment, the computer program product may be configured as a computer program for executing the following steps:

[0115] S1, obtaining the action state data of the target object and the multimedia signal of the vehicle input device, wherein the multimedia signal is used to determine the acoustic object to be played;

[0116] S2, generating a control signal to be executed based on the action state data and the multimedia signal;

[0117] S3, using the control signal to control the vehicle-mounted output device to perform a control operation that matches the action state data.

[0118] In the above embodiments of the present disclosure, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0119] In the several embodiments provided in the present disclosure, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0120] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0121] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0122] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0123] The above is only a preferred embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present disclosure. These improvements and modifications should also be regarded as the scope of protection of the present disclosure.

Claims

1. A data processing method, characterized in that: include: Acquiring action state data of a target object and a multimedia signal of an in-vehicle input device, wherein the multimedia signal is used to determine an acoustic object to be played; generating a control signal to be executed based on the action state data and the multimedia signal; The control signal is used to control the vehicle-mounted output device to execute a control operation matching the action state data.

2. The data processing method according to claim 1, characterized in that: Acquiring the action state data of the target object includes: The action state data of the target object is acquired by using a position sensing device and / or a vehicle-mounted visual input device.

3. The data processing method according to claim 1, characterized in that: Generating the control signal to be executed based on the action state data and the multimedia signal includes: determining a position change of the target object based on the action state data; adjusting a characteristic parameter of the multimedia signal according to the position change to obtain an adjustment result; The control signal to be executed is generated by using the position change and the adjustment result.

4. The data processing method according to claim 3, characterized in that: Adjusting the characteristic parameter of the multimedia signal according to the position change to obtain the adjustment result includes: In response to the multimedia signal being an audio signal of an in-vehicle audio input device, at least one of the frequency, volume and delay of the audio signal is adjusted according to the position change to obtain the adjustment result.

5. The data processing method according to claim 3, characterized in that: Generating the control signal to be executed by using the position change and the adjustment result includes: In response to the in-vehicle output device including an in-vehicle speaker device, generating an acoustic control signal for the in-vehicle speaker device based on the adjustment result; In response to the in-vehicle output device comprising an in-vehicle optoelectronic device, generating an optical control signal for the in-vehicle optoelectronic device based on the position change; In response to the in-vehicle output device including an in-vehicle interactive device, a motor control signal for the in-vehicle interactive device is generated based on the position change.

6. The data processing method according to claim 5, characterized in that: Using the control signal to control the vehicle-mounted output device to perform a control operation matching the action state data includes: Linking control of the acoustic elements of the vehicle-mounted speaker device and the action state data according to the acoustic control signal; controlling the optical element of the vehicle-mounted optoelectronic device and the action state data in linkage with each other according to the optical control signal; The interactive elements of the vehicle-mounted interactive device and the action state data are linked and controlled according to the motor control signal.

7. The data processing method according to claim 3, characterized in that: Determining the position change of the target object based on the action state data includes: In the vehicle cabin, constructing a spatial coordinate system based on the location of the target object; The position change of the target object is determined within the spatial coordinate system based on the motion state data.

8. A vehicle, characterized in that: include: A memory storing an executable program; A processor, used to run the program, wherein the program executes the data processing method according to any one of claims 1 to 7 when running.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is run, the device where the storage medium is located is controlled to execute the data processing method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the data processing method according to any one of claims 1 to 7 when being executed by a processor.

Images