Device control method and apparatus, vehicle, medium, and chip

Abstract

The present disclosure relates to a device control method, device, vehicle, medium and chip. The method comprises: emitting a preset first ultrasonic signal in a vehicle interior space; determining, for each seat in the vehicle, a second ultrasonic signal corresponding to the first ultrasonic signal picked up by the seat; determining, according to the second ultrasonic signal, channel impulse response characteristic information corresponding to each seat, the channel impulse response characteristic information being used to reflect whether the propagation path of the ultrasonic signal is affected; determining, according to the channel impulse response characteristic information, a target seat at which a control action is made by a user; and controlling at least one of the devices associated with the target seat according to the control action corresponding to the target seat. Thus, individual identification and response of the action of each seat occupant can be achieved, and the action is not affected by external factors such as light, and accurate identification can be completed even at night.

Description

Technical Field

[0001] The present disclosure relates to the field of vehicle technology, and in particular to a device control method, apparatus, vehicle, medium, and chip. Background Art

[0002] Currently, users can control the vehicle contactlessly through hand gestures while in the vehicle. For gesture control inside the vehicle, when there are multiple people in the vehicle, it is necessary to respond separately to the actions of people in different positions. In related technologies, it is usually necessary to set up a camera in front of each seat to collect images of people in the vehicle, extract features from the images, and analyze them to identify the actions of people in the vehicle. However, the above method requires the addition of multiple cameras, which has the problem of high cost. At the same time, the camera has high requirements for light. When the light is poor, especially when driving at night, the recognition effect will be affected. In addition, the setting of the camera may also affect the riding experience of people in the vehicle. Summary of the Invention

[0003] To overcome the problems existing in the related art, the present disclosure provides a device control method, apparatus, vehicle, medium and chip.

[0004] According to a first aspect of an embodiment of the present disclosure, a device control method is provided, the method comprising:

[0005] emitting a preset first ultrasonic signal in the interior space of the vehicle;

[0006] For each seat in the vehicle, determining a second ultrasonic signal corresponding to the first ultrasonic signal picked up by the seat;

[0007] determining, based on the second ultrasonic signal, channel impulse response characterization information corresponding to each seat, the channel impulse response characterization information being used to reflect whether a propagation path of the ultrasonic signal is affected;

[0008] determining, based on the channel impulse response characterization information, a target seat for the occupant to perform a control action;

[0009] At least one of the devices associated with the target seat is controlled according to the control action corresponding to the target seat.

[0010] Optionally, each of the seats is provided with a signal acquisition device;

[0011] The determining of the second ultrasonic signal corresponding to the first ultrasonic signal picked up by the seat includes:

[0012] Obtaining an initial ultrasonic signal collected by a signal collection device of the seat;

[0013] performing bandpass filtering on the initial ultrasonic signal to obtain a processed signal;

[0014] Demodulating the processed signal to obtain a baseband signal;

[0015] Determine a target beam matrix corresponding to the seat;

[0016] The second ultrasonic signal is determined according to the baseband signal and the target beam matrix.

[0017] Optionally, the first ultrasonic signal is emitted by a signal transmitting device in the vehicle;

[0018] Determining the target beam matrix corresponding to the seat includes:

[0019] Determining the relative position between the signal collecting device of the seat and the signal transmitting device;

[0020] According to the relative position, a target beam matrix corresponding to the seat is determined.

[0021] Optionally, determining channel impulse response characterization information corresponding to each seat according to the second ultrasonic signal includes:

[0022] For each seat, perform the following operations:

[0023] determining a channel impulse response vector corresponding to the seat based on the first ultrasonic signal and a second ultrasonic signal corresponding to the seat;

[0024] According to the signal impulse response vector corresponding to the seat, intensity change information of the channel impulse response vector is determined as the channel impulse response representation information corresponding to the seat.

[0025] Optionally, determining, based on the channel impulse response characterization information, a target seat where the user performs a control action includes:

[0026] Determining, based on the channel impulse response characterization information, a recognition result corresponding to each seat using a pre-trained recognition model, wherein the recognition result is used to indicate a control action type corresponding to each seat, wherein the control action type is one of a plurality of preset control actions or no control action;

[0027] The seat whose control action type indicated by the recognition result is the preset control action is determined as the target seat.

[0028] Optionally, the preset control action is used to instruct control of a device of the vehicle;

[0029] The controlling of at least one of the devices associated with the target seat according to the control action corresponding to the target seat includes:

[0030] Determining a target preset control action corresponding to the target seat according to the recognition result of the target seat;

[0031] Determining a target device targeted by the target preset control action and a target control instruction corresponding to the target preset control action;

[0032] Sending the target control instruction to the target device.

[0033] Optionally, a speaker is provided inside the vehicle, and the first ultrasonic signal is emitted by the speaker inside the vehicle; and,

[0034] Each seat in the vehicle is provided with a microphone array for picking up ultrasonic signals.

[0035] According to a second aspect of an embodiment of the present disclosure, there is provided a device control apparatus, the apparatus comprising:

[0036] a transmitting module, configured to transmit a preset first ultrasonic signal in a space inside the vehicle;

[0037] a first determining module configured to determine, for each seat in the vehicle, a second ultrasonic signal corresponding to the first ultrasonic signal picked up by the seat;

[0038] a second determining module configured to determine, based on the second ultrasonic signal, channel impulse response characterization information corresponding to each seat, the channel impulse response characterization information being used to reflect whether a propagation path of the ultrasonic signal is affected;

[0039] a third determining module, configured to determine a target seat for the user to perform a control action based on the channel impulse response characterization information;

[0040] The control module is configured to control at least one of the devices associated with the target seat according to the control action corresponding to the target seat.

[0041] Optionally, each of the seats is provided with a signal acquisition device;

[0042] The first determining module includes:

[0043] an acquisition submodule, configured to acquire an initial ultrasonic signal collected by a signal acquisition device of the seat;

[0044] a first processing submodule, configured to perform bandpass filtering on the initial ultrasonic signal to obtain a processed signal;

[0045] A second processing submodule is configured to demodulate the processed signal to obtain a baseband signal;

[0046] A first determining submodule is configured to determine a target beam matrix corresponding to the seat;

[0047] The second determining submodule is configured to determine the second ultrasonic signal according to the baseband signal and the target beam matrix.

[0048] Optionally, the first ultrasonic signal is emitted by a signal transmitting device in the vehicle;

[0049] The first determination submodule is configured to: determine the relative position between the signal acquisition device of the seat and the signal transmission device; and determine the target beam matrix corresponding to the seat according to the relative position.

[0050] Optionally, the second determining module is configured to perform the following operations for each seat:

[0051] determining a channel impulse response vector corresponding to the seat based on the first ultrasonic signal and a second ultrasonic signal corresponding to the seat;

[0052] According to the signal impulse response vector corresponding to the seat, intensity change information of the channel impulse response vector is determined as the channel impulse response representation information corresponding to the seat.

[0053] Optionally, the third determining module includes:

[0054] a third determination submodule configured to determine, based on the channel impulse response representation information and using a pre-trained recognition model, a recognition result corresponding to each seat, wherein the recognition result is used to indicate a control action type corresponding to each seat, wherein the control action type is one of a plurality of preset control actions or no control action;

[0055] The fourth determining submodule is configured to determine the seat whose control action type indicated by the recognition result is the preset control action as the target seat.

[0056] Optionally, the preset control action is used to instruct control of a device of the vehicle;

[0057] The control module includes:

[0058] a fifth determining submodule, configured to determine a target preset control action corresponding to the target seat according to the recognition result of the target seat;

[0059] a sixth determining submodule, configured to determine a target device targeted by the target preset control action and a target control instruction corresponding to the target preset control action;

[0060] The sending submodule is configured to send the target control instruction to the target device.

[0061] Optionally, a speaker is provided inside the vehicle, and the first ultrasonic signal is emitted by the speaker inside the vehicle; and,

[0062] Each seat in the vehicle is provided with a microphone array for picking up ultrasonic signals.

[0063] According to a third aspect of an embodiment of the present disclosure, there is provided a vehicle, comprising:

[0064] processor;

[0065] a memory for storing processor-executable instructions;

[0066] The processor is configured to execute instructions in the memory to implement the steps of the method described in the first aspect of the present disclosure.

[0067] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the device control method provided in the first aspect of the present disclosure are implemented.

[0068] According to a fifth aspect of an embodiment of the present disclosure, a chip is provided, comprising a processor and an interface; the processor is configured to read instructions to execute the method described in the first aspect of the present disclosure.

[0069] The technical solutions provided by the embodiments of the present disclosure may have the following beneficial effects:

[0070] Through the above scheme, a preset first ultrasonic signal is emitted within the vehicle. For each seat in the vehicle, a second ultrasonic signal corresponding to the first ultrasonic signal picked up by the seat is determined. Based on the second ultrasonic signal, channel impulse response characterization information corresponding to each seat is determined. Based on the channel impulse response characterization information, the target seat where the occupant performed a control action is determined. Based on the control action corresponding to the target seat, at least one device associated with the target seat is controlled. The channel impulse response characterization information reflects whether the propagation path of the ultrasonic signal is affected. Thus, by emitting an ultrasonic signal within the vehicle and identifying whether a occupant in each seat is performing a control action based on the ultrasonic signal's pickup, individual occupant actions can be individually identified and responded to. Furthermore, since control action recognition is based on ultrasonic signals, it is not affected by external factors such as light, enabling accurate recognition even at night. Furthermore, the transmission and reception of ultrasonic signals can be implemented using existing vehicle equipment, eliminating the need for additional hardware and significantly reducing costs.

[0071] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0073] Figure 1 The figure is a flowchart of a device control method according to an exemplary embodiment.

[0074] Figure 2 is a block diagram of a device control apparatus according to an exemplary embodiment.

[0075] Figure 3 It is a functional block diagram of a vehicle shown in an exemplary embodiment. DETAILED DESCRIPTION

[0076] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure, as detailed in the appended claims.

[0077] It is noted that all actions of obtaining signals, information or data in the present application are carried out in compliance with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization given by the owner of the corresponding device.

[0078] Figure 1 is a flowchart of a device control method according to an exemplary embodiment. By way of example, the method provided by the present disclosure can be applied to a vehicle. As shown in Figure 1 The method can include steps 11-15.

[0079] In step 11, a preset first ultrasonic signal is emitted in the vehicle interior.

[0080] The first ultrasonic signal can be emitted by a signal emitting device in the vehicle. By way of example, a speaker can be provided in the vehicle interior, and the first ultrasonic signal can be emitted by the speaker in the vehicle interior.

[0081] By way of example, the first ultrasonic signal can use a CHIRP (Chirp) signal.

[0082] In step 12, for each seat in the vehicle, a second ultrasonic signal corresponding to the first ultrasonic signal picked up by the seat is determined.

[0083] Each seat in the vehicle can be provided with a signal acquisition device for picking up ultrasonic signals. The signal acquisition device can be a microphone array on the seat in the vehicle. By way of example, the signal acquisition device can be a 4-way microphone array.

[0084] The first ultrasonic signal emitted in step 11 propagates in the vehicle, and the signal acquisition device on each seat in the vehicle can acquire the ultrasonic signal propagating in the vehicle.

[0085] In one possible implementation, the determination of the second ultrasonic signal corresponding to the first ultrasonic signal picked up by the seat in step 12 can include the following steps:

[0086] Obtaining the initial ultrasonic signal acquired by the signal acquisition device of the seat;

[0087] Bandpass filtering the initial ultrasonic signal to obtain a processed signal;

[0088] Demodulating the processed signal to obtain a baseband signal;

[0089] Determining a target beam matrix corresponding to the seat;

[0090] Determining the second ultrasonic signal based on the baseband signal and the target beam matrix.

[0091] Typically, the signal acquisition device can directly capture the ultrasonic signal propagating within the vehicle, which is also known as the initial ultrasonic signal. For example, if the signal acquisition device is a 4-channel microphone array, the initial ultrasonic signal is the 4-channel microphone signal.

[0092] Initial ultrasonic signals are generally noisy and difficult to process. Therefore, the collected initial ultrasonic signals can undergo some processing. First, the initial ultrasonic signals can be bandpass filtered to obtain a processed signal. This bandpass filtering can be performed based on the primary frequency band of the first ultrasonic signal, thereby removing noise interference outside the ultrasonic band. The processed signal is then demodulated to obtain a baseband signal. This demodulation can be performed to lower the frequency of the processed signal to prevent high-frequency signals from being difficult to process.

[0093] After obtaining the baseband signal, the target beam matrix corresponding to the seat can be determined, and the second ultrasonic signal can be determined based on the baseband signal and the target beam matrix.

[0094] The target beam matrix corresponding to the seat is used to determine the ultrasonic signal that can be picked up in the direction of the seat.

[0095] For seats in a vehicle, the relative position of the signal acquisition device thereon and the signal transmitting device in the vehicle is known. Therefore, a beam matrix in a fixed direction can be pre-set. Using this beam matrix, it is possible to determine the signal obtained when the signal acquisition device only picks up the ultrasonic signal in the direction of the signal transmitting device. For example, for a left-hand drive vehicle, assuming that the signal transmitting device in the vehicle is located at the front of the interior of the vehicle (for example, near the center console), the relative position between the driver's seat in the vehicle (including the signal acquisition device provided on the driver's seat) and the signal transmitting device is known and fixed. If the signal transmitting device is located X degrees north-east of the signal acquisition device, the X degrees north-east is the fixed direction mentioned above.

[0096] For example, the target beam matrix corresponding to the seat can be determined as follows:

[0097] Determine the relative position between the signal acquisition device and the signal transmission device of the seat;

[0098] According to the relative position, the target beam matrix corresponding to the seat is determined.

[0099] In the present disclosure, the beam matrix corresponding to each relative position can be set based on experience, and then a corresponding relationship between the relative position and the beam matrix can be formed. Here, each relative position corresponds to the relative position of a signal acquisition device on a seat in the vehicle and a signal transmitting device in the vehicle. For example, if the vehicle contains 4 seats, each seat is provided with a signal acquisition device, then the signal acquisition devices and the signal transmitting devices on these 4 seats respectively have a relative position. Based on the corresponding relationship between the relative position and the beam matrix, according to the relative position between the signal acquisition device and the signal transmitting device on the seat, the beam matrix corresponding to the relative position between the signal acquisition device and the signal transmitting device on the seat can be determined, that is, the target beam matrix corresponding to the seat.

[0100] For example, if the driver's seat and the front passenger seat each correspond to a beam matrix. If the baseband signal obtained after the above processing is A, the beam matrix corresponding to the driver's seat is M0, and the beam matrix corresponding to the front passenger seat is M1, then the second ultrasonic signal B containing only the driver's seat can be determined by B=A*M0, and the second ultrasonic signal C containing only the front passenger seat can be determined by C=A*M1.

[0101] In step 13, channel impulse response characterization information corresponding to each seat is determined based on the second ultrasonic signal.

[0102] The channel response characterizes the transformation method that transforms the input signal into the output signal. The channel response in the time domain is called the channel impulse response (CIR). For example, in R(n) = S(t) * h(n), S(n) is the baseband transmitted signal, R(n) is the baseband received signal, h(n) is the channel impulse response vector, * is convolution, and h(n) can be called the channel impulse response vector transformed from S(n) to R(n).

[0103] Action recognition (for example, gesture recognition) can be implemented based on CIR and dCIR (differential channel impulse response). CIR can reflect the path impact of the ultrasonic signal from the signal transmitting device to the signal collecting device. When the person in the seat is not moving, CIR is a stable value and dCIR should be close to 0. When the person in the seat moves, it will cause disturbances to the propagation path of the ultrasonic signal, so dCIR will fluctuate accordingly. At the same time, the different fluctuations in dCIR can also indirectly reflect the type of action that caused the fluctuation.

[0104] Based on the above description, we can see that information related to the channel impulse response can assist in identifying the movements of the person in the seat. Therefore, the channel impulse response representation information can be used to reflect whether the propagation path of the ultrasonic signal is affected, and this information can be used for subsequent movement recognition.

[0105] In a possible implementation, in step 13, determining the channel impulse response characterization information corresponding to each seat according to the second ultrasonic signal may include the following steps.

[0106] For each seat, perform the following steps:

[0107] determining a channel impulse response vector corresponding to the seat based on the first ultrasonic signal and a second ultrasonic signal corresponding to the seat;

[0108] According to the signal impulse response vector corresponding to the seat, intensity change information of the channel impulse response vector is determined as the channel impulse response representation information corresponding to the seat.

[0109] As described above regarding the channel impulse response vector, for each seat, the corresponding channel impulse response vector can be determined based on the first ultrasonic signal and the corresponding second ultrasonic signal. Furthermore, based on the corresponding signal impulse response vector, the intensity variation information of the channel impulse response vector (i.e., dCIR) can be further determined as the channel impulse response representation information for the seat.

[0110] For example, the intensity variation information of the channel impulse response vector may be amplitude variance information of the channel impulse response vector or energy variance information of the channel impulse response vector.

[0111] In step 14, the target seat where the user performs the control action is determined based on the channel impulse response characterization information.

[0112] In a possible implementation, step 14 may include the following steps:

[0113] Based on the channel impulse response characterization information, the pre-trained recognition model is used to determine the recognition results corresponding to each seat;

[0114] The seat whose control action type indicated by the recognition result is a preset control action is determined as the target seat.

[0115] The recognition result indicates the corresponding control action type for each seat. The control action type can be one of multiple preset control actions or no control action. Preset control actions can be used to control vehicle devices. Preset control actions can be any available gesture, such as drawing a specific shape with a finger.

[0116] The recognition model may be a convolutional neural network (CNN) model. To train the recognition model, multiple sets of training data may be obtained, including training channel impulse response representation information and control action types corresponding to the training channel impulse response representation information. During model training, the training channel impulse response representation information is used as the model input, and the control action types corresponding to the training channel impulse response representation information are used as the model's expected output, thereby obtaining a trained recognition model.

[0117] Furthermore, based on the recognition result, a seat whose control action type indicated by the recognition result is a preset control action can be determined as a target seat. A seat whose control action type indicated by the recognition result is no control action will not be determined as a target seat.

[0118] In step 15 , at least one of the devices associated with the target seat is controlled according to the control action corresponding to the target seat.

[0119] In a possible implementation, step 15 may include the following steps:

[0120] Determine the target preset control action corresponding to the target seat according to the recognition result of the target seat;

[0121] Determine the target device targeted by the target preset control action and the target control instruction corresponding to the target preset control action;

[0122] Send target control instructions to the target device.

[0123] As described above, preset control actions can be used to instruct control of a vehicle's equipment, while users occupying seats typically only need to control equipment associated with that seat. Therefore, the target device targeted by the target preset control action corresponding to the target seat is the device associated with the target seat. At the same time, preset control actions typically have their corresponding instruction meanings. Therefore, based on the target preset control action corresponding to the target seat, the target device and target control instruction targeted by the target preset control action can be determined. Furthermore, control of the target device can be achieved by sending a target control instruction to the target device. This allows for separate responses to control actions by users in different seats.

[0124] Through the above scheme, a preset first ultrasonic signal is emitted within the vehicle. For each seat in the vehicle, a second ultrasonic signal corresponding to the first ultrasonic signal picked up by the seat is determined. Based on the second ultrasonic signal, channel impulse response characterization information corresponding to each seat is determined. Based on the channel impulse response characterization information, the target seat where the occupant performed a control action is determined. Based on the control action corresponding to the target seat, at least one device associated with the target seat is controlled. The channel impulse response characterization information reflects whether the propagation path of the ultrasonic signal is affected. Thus, by emitting an ultrasonic signal within the vehicle and identifying whether a occupant in each seat is performing a control action based on the ultrasonic signal's pickup, individual occupant actions can be individually identified and responded to. Furthermore, since control action recognition is based on ultrasonic signals, it is not affected by external factors such as light, enabling accurate recognition even at night. Furthermore, the transmission and reception of ultrasonic signals can be implemented using existing vehicle equipment, eliminating the need for additional hardware and significantly reducing costs.

[0125] Figure 2 FIG. 1 is a block diagram of a device control apparatus according to an exemplary embodiment. Figure 2 As shown, the device 20 includes:

[0126] The transmitting module 21 is configured to transmit a preset first ultrasonic signal in the space inside the vehicle;

[0127] A first determining module 22 is configured to determine, for each seat in the vehicle, a second ultrasonic signal corresponding to the first ultrasonic signal picked up by the seat;

[0128] A second determining module 23 is configured to determine channel impulse response characterization information corresponding to each seat based on the second ultrasonic signal, wherein the channel impulse response characterization information is used to reflect whether a propagation path of the ultrasonic signal is affected;

[0129] The third determining module 24 is configured to determine the target seat where the user performs the control action based on the channel impulse response characterization information;

[0130] The control module 25 is configured to control at least one of the devices associated with the target seat according to the control action corresponding to the target seat.

[0131] Optionally, each of the seats is provided with a signal acquisition device;

[0132] The first determining module 22 includes:

[0133] an acquisition submodule, configured to acquire an initial ultrasonic signal collected by a signal acquisition device of the seat;

[0134] a first processing submodule, configured to perform bandpass filtering on the initial ultrasonic signal to obtain a processed signal;

[0135] A second processing submodule is configured to demodulate the processed signal to obtain a baseband signal;

[0136] A first determining submodule is configured to determine a target beam matrix corresponding to the seat;

[0137] The second determining submodule is configured to determine the second ultrasonic signal according to the baseband signal and the target beam matrix.

[0138] Optionally, the first ultrasonic signal is emitted by a signal transmitting device in the vehicle;

[0139] The first determination submodule is configured to: determine the relative position between the signal acquisition device of the seat and the signal transmission device; and determine the target beam matrix corresponding to the seat according to the relative position.

[0140] Optionally, the second determining module 23 is configured to perform the following operations for each seat:

[0141] determining a channel impulse response vector corresponding to the seat based on the first ultrasonic signal and a second ultrasonic signal corresponding to the seat;

[0142] According to the signal impulse response vector corresponding to the seat, intensity change information of the channel impulse response vector is determined as the channel impulse response representation information corresponding to the seat.

[0143] Optionally, the third determining module 24 includes:

[0144] a third determination submodule configured to determine, based on the channel impulse response representation information and using a pre-trained recognition model, a recognition result corresponding to each seat, wherein the recognition result is used to indicate a control action type corresponding to each seat, wherein the control action type is one of a plurality of preset control actions or no control action;

[0145] The fourth determining submodule is configured to determine the seat whose control action type indicated by the recognition result is the preset control action as the target seat.

[0146] Optionally, the preset control action is used to instruct control of a device of the vehicle;

[0147] The control module 25 includes:

[0148] a fifth determining submodule, configured to determine a target preset control action corresponding to the target seat according to the recognition result of the target seat;

[0149] a sixth determining submodule, configured to determine a target device targeted by the target preset control action and a target control instruction corresponding to the target preset control action;

[0150] The sending submodule is configured to send the target control instruction to the target device.

[0151] Optionally, a speaker is provided inside the vehicle, and the first ultrasonic signal is emitted by the speaker inside the vehicle; and,

[0152] Each seat in the vehicle is provided with a microphone array for picking up ultrasonic signals.

[0153] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0154] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, which implement the steps of the device control method provided by the present disclosure when the program instructions are executed by a processor.

[0155] In addition to being an independent electronic device, the above-mentioned device can also be a part of an independent electronic device. For example, in one embodiment, the device can be an integrated circuit (IC) or a chip, wherein the integrated circuit can be a single IC or a collection of multiple ICs; the chip can include but is not limited to the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip, SoC, system on chip or system-on-chip), etc. The above-mentioned integrated circuit or chip can be used to execute executable instructions (or codes) to implement the above-mentioned device control method. The executable instructions can be stored in the integrated circuit or chip, or can be obtained from other devices or equipment. For example, the integrated circuit or chip includes a processor, a memory, and an interface for communicating with other devices. The executable instruction can be stored in the memory, and when the executable instruction is executed by the processor, the above-mentioned device control method is implemented; alternatively, the integrated circuit or chip can receive the executable instruction through the interface and transmit it to the processor for execution, so as to implement the above-mentioned device control method.

[0156] See Figure 3 , Figure 3 FIG6 is a functional block diagram of a vehicle 600, illustrating an exemplary embodiment. Vehicle 600 can be configured for fully or partially autonomous driving. For example, vehicle 600 can obtain environmental information about its surroundings through perception system 620 and, based on analysis of the environmental information, derive an autonomous driving strategy to achieve fully autonomous driving, or present the analysis results to the user to achieve partially autonomous driving.

[0157] Vehicle 600 may include various subsystems, such as an infotainment system 610, a perception system 620, a decision control system 630, a drive system 640, and a computing platform 650. Alternatively, vehicle 600 may include more or fewer subsystems, and each subsystem may include multiple components. Furthermore, each subsystem and component of vehicle 600 may be interconnected via wired or wireless means.

[0158] In some embodiments, infotainment system 610 may include a communication system 611 , an entertainment system 612 , and a navigation system 613 .

[0159] The communication system 611 may include a wireless communication system that can communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system can use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as LTE. Or 5G cellular communication. The wireless communication system can use WiFi to communicate with a wireless local area network (WLAN). In some embodiments, the wireless communication system can use an infrared link, Bluetooth, or ZigBee to communicate directly with the device. Other wireless protocols, such as various vehicle communication systems, for example, the wireless communication system may include one or more dedicated short range communications (DSRC) devices, which may include public and / or private data communications between vehicles and / or roadside stations.

[0160] The entertainment system 612 may include a display device, a microphone and speakers. Users can listen to the radio and play music in the car based on the entertainment system; or connect the mobile phone to the vehicle and project the mobile phone screen on the display device. The display device can be touch-sensitive and the user can operate it by touching the screen.

[0161] In some cases, the user's voice signal can be obtained through a microphone, and based on the analysis of the user's voice signal, the user can control certain aspects of the vehicle 600, such as adjusting the temperature inside the vehicle, etc. In other cases, music can be played to the user through a speaker.

[0162] The navigation system 613 may include a map service provided by a map provider, thereby providing navigation for the vehicle 600. The navigation system 613 may be used in conjunction with the vehicle's global positioning system 621 and inertial measurement unit 622. The map service provided by the map provider may be a two-dimensional map or a high-precision map.

[0163] The perception system 620 may include several sensors that sense information about the environment surrounding the vehicle 600. For example, the perception system 620 may include a global positioning system 621 (the global positioning system may be a GPS system, or a BeiDou system or other positioning system), an inertial measurement unit (IMU) 622, a lidar 623, a millimeter wave radar 624, an ultrasonic radar 625, and a camera 626. The perception system 620 may also include sensors of the internal systems of the monitored vehicle 600 (for example, an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). Sensor data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, direction, speed, etc.). Such detection and recognition are key functions for the safe operation of the vehicle 600.

[0164] The global positioning system 621 is used to estimate the geographic location of the vehicle 600 .

[0165] The inertial measurement unit 622 is used to sense the posture change of the vehicle 600 based on inertial acceleration. In some embodiments, the inertial measurement unit 622 can be a combination of an accelerometer and a gyroscope.

[0166] LiDAR 623 utilizes laser light to sense objects in the environment in which vehicle 600 is located. In some embodiments, LiDAR 623 may include one or more laser sources, a laser scanner, and one or more detectors, among other system components.

[0167] The millimeter wave radar 624 uses radio signals to sense objects in the surrounding environment of the vehicle 600. In some embodiments, in addition to sensing objects, the millimeter wave radar 624 can also be used to sense the speed and / or heading of the objects.

[0168] The ultrasonic radar 625 may sense objects around the vehicle 600 using ultrasonic signals.

[0169] The camera device 626 is used to capture image information of the surrounding environment of the vehicle 600. The camera device 626 may include a monocular camera, a binocular camera, a structured light camera, a panoramic camera, etc. The image information obtained by the camera device 626 may include static images or video stream information.

[0170] The decision control system 630 includes a computing system 631 that analyzes and makes decisions based on the information obtained by the perception system 620. The decision control system 630 also includes a vehicle controller 632 that controls the power system of the vehicle 600, as well as a steering system 633, throttle 634 and braking system 635 for controlling the vehicle 600.

[0171] The computing system 631 can be operated to process and analyze various information obtained by the perception system 620 in order to identify targets, objects and / or features in the environment surrounding the vehicle 600. Targets may include pedestrians or animals, and objects and / or features may include traffic signals, road boundaries and obstacles. The computing system 631 may use object recognition algorithms, Structure from Motion (SFM) algorithms, video tracking and other technologies. In some embodiments, the computing system 631 can be used to map the environment, track objects, estimate the speed of objects, and so on. The computing system 631 can analyze the various information obtained and derive a control strategy for the vehicle.

[0172] The vehicle controller 632 can be used to coordinate and control the vehicle's power battery and engine 641 to improve the power performance of the vehicle 600.

[0173] The steering system 633 is operable to adjust the forward direction of the vehicle 600. For example, in one embodiment, it may be a steering wheel system.

[0174] The throttle 634 is used to control the operating speed of the engine 641 and thereby control the speed of the vehicle 600 .

[0175] Braking system 635 is used to control the deceleration of vehicle 600. Braking system 635 can use friction to slow down wheels 644. In some embodiments, braking system 635 can convert the kinetic energy of wheels 644 into electric current. Braking system 635 can also take other forms to slow the rotation speed of wheels 644 and thus control the speed of vehicle 600.

[0176] Drive system 640 may include components that provide powered motion for vehicle 600. In one embodiment, drive system 640 may include an engine 641, an energy source 642, a transmission system 643, and wheels 644. Engine 641 may be an internal combustion engine, an electric motor, an air compression engine, or another combination of engines, such as a hybrid engine consisting of a gasoline engine and an electric motor, or a hybrid engine consisting of an internal combustion engine and an air compression engine. Engine 641 converts energy source 642 into mechanical energy.

[0177] Examples of energy source 642 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. Energy source 642 can also provide energy to other systems of vehicle 600.

[0178] The transmission system 643 can transmit mechanical power from the engine 641 to the wheels 644. The transmission system 643 may include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission system 643 may also include other components, such as a clutch. The drive shaft may include one or more shafts that can be coupled to one or more wheels 644.

[0179] Some or all functions of vehicle 600 are controlled by a computing platform 650. Computing platform 650 may include at least one processor 651 that can execute instructions 653 stored in a non-transitory computer-readable medium such as memory 652. In some embodiments, computing platform 650 may also be a plurality of computing devices that control individual components or subsystems of vehicle 600 in a distributed manner.

[0180] The processor 651 may be any conventional processor, such as a commercially available CPU. Alternatively, the processor 651 may include a graphics processor (GPU), a field programmable gate array (FPGA), a system on chip (SOC), an application specific integrated circuit (ASIC), or a combination thereof. Figure 3 Functionally, processor, memory, and other elements of the computer in the same block are illustrated, but those of ordinary skill in the art will appreciate that the processor, computer, or memory may in fact comprise a plurality of processors, computers, or memories that may or may not be stored in the same physical housing. For example, the memory may be a hard drive or other storage medium that is positioned in a housing that is different from the computer. Therefore, reference to a processor or computer will be understood to include reference to a collection of processors or computers or memories that may or may not operate in parallel. Different from using a single processor to perform the steps described herein, some assemblies such as steering assembly and deceleration assembly may each have their own processor that only performs the calculations relevant to the functions specific to the assembly.

[0181] In an embodiment of the present disclosure, the processor 651 may execute the above-mentioned device control method.

[0182] In various aspects described herein, the processor 651 can be located remotely from the vehicle and in wireless communication with the vehicle. In other aspects, some of the processes described herein are performed on a processor disposed within the vehicle while others are performed by a remote processor, including taking the necessary steps to perform a single maneuver.

[0183] In some embodiments, the memory 652 may include instructions 653 (e.g., program logic) that are executable by the processor 651 to perform various functions of the vehicle 600. The memory 652 may also include additional instructions, including instructions for sending data to, receiving data from, interacting with, and / or controlling one or more of the infotainment system 610, the perception system 620, the decision control system 630, and the drive system 640.

[0184] In addition to instructions 653, memory 652 may also store data such as road maps, route information, the vehicle's location, direction, speed, and other such vehicle data, as well as other information. This information may be used by vehicle 600 and computing platform 650 during operation of vehicle 600 in autonomous, semi-autonomous, and / or manual modes.

[0185] The computing platform 650 may control functions of the vehicle 600 based on input received from various subsystems, such as the drive system 640, the perception system 620, and the decision control system 630. For example, the computing platform 650 may utilize input from the decision control system 630 to control the steering system 633 to avoid an obstacle detected by the perception system 620. In some embodiments, the computing platform 650 may be operable to provide control over many aspects of the vehicle 600 and its subsystems.

[0186] Alternatively, one or more of the above components may be installed or associated separately from the vehicle 600. For example, the memory 652 may be partially or completely separate from the vehicle 600. The above components may be communicatively coupled together in a wired and / or wireless manner.

[0187] Optionally, the above components are just an example. In actual applications, the components in the above modules may be added or deleted according to actual needs. Figure 3 It should not be understood as limiting the embodiments of the present disclosure.

[0188] An autonomous vehicle traveling on a road, such as vehicle 600 above, can identify objects in its surroundings to determine adjustments to its current speed. Objects can be other vehicles, traffic control devices, or other types of objects. In some examples, each identified object can be considered independently, and the speed adjustment to be made to the autonomous vehicle can be determined based on its respective characteristics, such as its current speed, acceleration, and distance from the vehicle.

[0189] Optionally, the vehicle 600 or a sensing and computing device associated with the vehicle 600 (e.g., computing system 631, computing platform 650) can predict the behavior of the identified objects based on the characteristics of the identified objects and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of each other, so the behavior of all identified objects can also be considered together to predict the behavior of a single identified object. The vehicle 600 can adjust its speed based on the predicted behavior of the identified objects. In other words, the autonomous vehicle can determine what stable state the vehicle will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the objects. In this process, other factors can also be considered to determine the speed of the vehicle 600, such as the lateral position of the vehicle 600 in the road it is traveling on, the curvature of the road, the proximity of static and dynamic objects, etc.

[0190] In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device may also provide instructions to modify the steering angle of vehicle 600 so that the autonomous vehicle follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle (e.g., vehicles in adjacent lanes on the road).

[0191] The vehicle 600 may be any type of vehicle, such as a car, a truck, a motorcycle, a bus, a ship, an airplane, a helicopter, an RV, a train, etc., and the present disclosure does not impose any particular limitation thereto.

[0192] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a programmable device, and the computer program has a code portion for executing the above-mentioned device control method when executed by the programmable device.

[0193] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0194] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A device control method, characterized in that: The method comprises: emitting a preset first ultrasonic signal in the interior space of the vehicle; Each seat in the vehicle is provided with a signal acquisition device; obtaining an initial ultrasonic signal acquired by the signal acquisition device of the seat; performing bandpass filtering on the initial ultrasonic signal to obtain a processed signal; performing demodulation on the processed signal to obtain a baseband signal; determining a target beam matrix corresponding to the seat; and determining a second ultrasonic signal based on the baseband signal and the target beam matrix. determining, based on the second ultrasonic signal, channel impulse response characterization information corresponding to each seat, the channel impulse response characterization information being used to reflect whether a propagation path of the ultrasonic signal is affected; determining, based on the channel impulse response characterization information, a target seat for the occupant to perform a control action; At least one of the devices associated with the target seat is controlled according to the control action corresponding to the target seat.

2. The method according to claim 1, characterized in that The first ultrasonic signal is emitted by a signal transmitting device in the vehicle; Determining the target beam matrix corresponding to the seat includes: Determining the relative position between the signal collecting device of the seat and the signal transmitting device; According to the relative position, a target beam matrix corresponding to the seat is determined.

3. The method according to claim 1, characterized in that The determining, based on the second ultrasonic signal, channel impulse response characterization information corresponding to each seat includes: For each seat, perform the following operations: determining a channel impulse response vector corresponding to the seat based on the first ultrasonic signal and a second ultrasonic signal corresponding to the seat; According to the signal impulse response vector corresponding to the seat, intensity change information of the channel impulse response vector is determined as the channel impulse response representation information corresponding to the seat.

4. The method according to claim 1, wherein The determining, based on the channel impulse response characterization information, a target seat for the user to perform a control action comprises: Determining, based on the channel impulse response characterization information, a recognition result corresponding to each seat using a pre-trained recognition model, wherein the recognition result is used to indicate a control action type corresponding to each seat, wherein the control action type is one of a plurality of preset control actions or no control action; The seat whose control action type indicated by the recognition result is the preset control action is determined as the target seat.

5. The method according to claim 4, characterized in that The preset control action is used to instruct control of the equipment of the vehicle; The controlling of at least one of the devices associated with the target seat according to the control action corresponding to the target seat includes: Determining a target preset control action corresponding to the target seat according to the recognition result of the target seat; Determining a target device targeted by the target preset control action and a target control instruction corresponding to the target preset control action; Sending the target control instruction to the target device.

6. The method according to any one of claims 1 to 5, characterized in that A speaker is provided inside the vehicle, and the first ultrasonic signal is emitted by the speaker inside the vehicle; and Each seat in the vehicle is provided with a microphone array for picking up ultrasonic signals.

7. A device control device, characterized in that: The device comprises: a transmitting module, configured to transmit a preset first ultrasonic signal in a space inside the vehicle; The first determination module is configured to, for each seat in the vehicle, each seat being provided with a signal acquisition device; obtain an initial ultrasonic signal acquired by the signal acquisition device of the seat; perform bandpass filtering on the initial ultrasonic signal to obtain a processed signal; perform demodulation on the processed signal to obtain a baseband signal; determine a target beam matrix corresponding to the seat; and determine a second ultrasonic signal based on the baseband signal and the target beam matrix. a second determining module configured to determine, based on the second ultrasonic signal, channel impulse response characterization information corresponding to each seat, the channel impulse response characterization information being used to reflect whether a propagation path of the ultrasonic signal is affected; a third determining module, configured to determine a target seat for the user to perform a control action based on the channel impulse response characterization information; The control module is configured to control at least one of the devices associated with the target seat according to the control action corresponding to the target seat.

8. A vehicle, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute instructions in the memory to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A chip, characterized in that: The method comprises a processor and an interface; the processor is used to read instructions to execute the method according to any one of claims 1 to 6.

Images