Equipment control methods, devices, terminals, storage media and program products

Abstract

This application discloses a device control method, apparatus, terminal, storage medium, and program product, belonging to the field of human-computer interaction technology. The method includes: establishing a communication connection with a vehicle-mounted system; generating a target control request in response to a received control operation, the target control request being used to request adjustment of the device angle of a target device in the vehicle; and sending the control request to the vehicle-mounted system via the communication connection, so that the vehicle-mounted system controls the target device to adjust its angle based on the target control request. Using the solution provided in this application, when it is inconvenient for the user to directly interact with the vehicle-mounted system, the angle of devices such as cameras, speakers, and seats in the vehicle can be adjusted through the terminal, which helps to improve the convenience of controlling devices in the vehicle.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction technology, and in particular to a device control method, apparatus, terminal, storage medium, and program product. Background Technology

[0002] With the increasing intelligence of vehicle infotainment systems, drivers or passengers can use these systems to understand the vehicle's status and control the equipment within the vehicle.

[0003] However, as the only human-machine interaction point in a vehicle, some functions of the vehicle's infotainment system will be unavailable when users are unable to interact with it directly. Summary of the Invention

[0004] This application provides a device control method, apparatus, terminal, storage medium, and program product. The technical solution is as follows:

[0005] On one hand, embodiments of this application provide a device control method, the method being executed by a terminal, the method comprising:

[0006] Establish a communication connection with the vehicle's infotainment system;

[0007] In response to a received control operation, a target control request is generated, the target control request being used to request adjustment of the device angle of a target device in the vehicle;

[0008] The control request is sent to the vehicle's infotainment system via the communication connection, so that the vehicle's infotainment system controls the target device to adjust its angle based on the target control request.

[0009] On the other hand, embodiments of this application provide a device control apparatus, the apparatus comprising:

[0010] The connection establishment module is used to establish a communication connection with the vehicle's infotainment system.

[0011] A request generation module is used to generate a target control request in response to a received control operation. The target control request is used to request adjustment of the device angle of a target device in the vehicle.

[0012] The request sending module is used to send the control request to the vehicle system through the communication connection, so that the vehicle system controls the target device to adjust the device angle based on the target control request.

[0013] On the other hand, embodiments of this application provide a terminal, the terminal including a processor and a memory; the memory stores at least one instruction, the at least one instruction being executed by the processor to implement the device control method as described above.

[0014] On the other hand, embodiments of this application provide a computer-readable storage medium storing at least one piece of program code, which is loaded and executed by a processor to implement the device control method as described above.

[0015] On the other hand, embodiments of this application provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the device control methods provided in the various optional implementations of the above aspects.

[0016] In this embodiment of the application, when a communication connection is established with the vehicle's infotainment system, the terminal can generate a target control request based on the received control operation and send the target control request to the infotainment system through the communication connection. The infotainment system can then adjust the device angle of the target device in the vehicle based on the request. For example, using the solution provided in this embodiment of the application, when it is inconvenient for the user to interact directly with the infotainment system, the user can adjust the angle of devices such as cameras, speakers, and seats in the vehicle through the terminal, which helps to improve the convenience of controlling the devices in the vehicle. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic diagram of an implementation environment provided by an exemplary embodiment of this application is shown;

[0019] Figure 2 A flowchart illustrating a device control method provided in an exemplary embodiment of this application is shown;

[0020] Figure 3 This is a schematic diagram illustrating the attitude angle of a terminal in an exemplary embodiment of this application;

[0021] Figure 4 A flowchart of a device control method provided in another exemplary embodiment of this application is shown;

[0022] Figure 5 This is an exemplary embodiment of the process of a mobile phone controlling the camera to adjust the shooting angle, as shown in this application.

[0023] Figure 6 This is an exemplary embodiment of the present application illustrating the process of adjusting the backrest angle of a seat using a mobile phone.

[0024] Figure 7 This is a schematic diagram of the control interface shown in an exemplary embodiment of this application;

[0025] Figure 8 This is a flowchart illustrating a target device determination process in an exemplary embodiment of this application;

[0026] Figure 9 yes Figure 8 The diagram shows the implementation of the target equipment determination process;

[0027] Figure 10 This is a flowchart illustrating the target device determination process in another exemplary embodiment of this application;

[0028] Figure 11 yes Figure 10 The diagram shows the implementation of the target equipment determination process;

[0029] Figure 12 This paper shows a structural block diagram of a device control apparatus provided in one embodiment of the present application;

[0030] Figure 13 A structural block diagram of a terminal provided in an exemplary embodiment of this application is shown. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0032] Figure 1 A schematic diagram of an implementation environment provided by an exemplary embodiment of this application is shown. This implementation environment may include: a terminal 110 and a vehicle-mounted system 120.

[0033] Terminal 110 can be an electronic device such as a smartphone, tablet, personal computer, or wearable device. Figure 1 The example given is a smartphone, specifically the 110 terminal.

[0034] In this embodiment, terminal 110 has the function of data communication with vehicle infotainment system 120. The terminal 110 and vehicle infotainment system 120 can establish a communication connection via wired or wireless means, and then conduct data communication through this connection. This communication connection can be Bluetooth, Wireless Fidelity (WiFi), Universal Serial Bus (USB), or mobile data network connection, etc., and this embodiment does not limit it to any particular type.

[0035] In some embodiments, the terminal 110 is equipped with sensors. Based on the sensor parameters, the terminal 110 can acquire its own real-time attitude and orientation, and thus generate control requests for controlling devices in the vehicle.

[0036] The vehicle infotainment system 120 is an electronic device installed in a vehicle to control other equipment within the vehicle. Typically, the vehicle infotainment system 120 is located in the front driver's area of ​​the vehicle so that the driver or front passenger can control the equipment within the vehicle.

[0037] In some embodiments, the devices in the vehicle may include cameras disposed on the outer periphery of the vehicle, speakers disposed inside the vehicle, multimedia devices disposed inside the vehicle, seats disposed in the vehicle, etc.

[0038] Optionally, the devices in the vehicle can establish a connection with the vehicle-mounted unit 120 via Low-Voltage Differential Signaling (LVDS), Composite Video Broadcast Signal (CVBS), Controller Area Network (CAN), or Local Interconnect Network (LIN). Accordingly, the vehicle-mounted unit 120 can control the devices through this connection.

[0039] In addition, the device can also send data to the vehicle's infotainment system 120 via a connection, and the infotainment system 120 can display the data based on the received data. For example, the camera can transmit video streams to the infotainment system 120 via a CVBS connection, and the infotainment system 120 can decode and display the video streams.

[0040] In some usage scenarios, when passengers need to adjust the angle of a device in the vehicle, such as adjusting the angle of a camera (to view the surrounding real-time environment) or the angle of a speaker (for better listening experience), in order to control the device in the vehicle without directly operating the vehicle's infotainment system 120, in this embodiment, the terminal 110 can generate a control request based on the received control operation and send the control request to the vehicle's infotainment system 120 through the communication connection with the system. After receiving the request, the vehicle's infotainment system 120 sends an angle adjustment command to the device through the connection, thereby adjusting and controlling the device's angle.

[0041] Indicative, such as Figure 1 As shown, cameras 130 are installed in all four directions of the vehicle, front, rear, left, and right, and each camera 130 is connected to the vehicle's infotainment system 120. Since the infotainment system 120 is located in the front row, it is inconvenient for passengers in the rear seats to directly operate the front-seat infotainment system 120 to adjust the shooting angle of the cameras 130. After connecting the terminal 110 to the infotainment system 120, the passenger selects the desired camera 130 through the terminal 110, and then adjusts the posture of the terminal 110 to trigger it to send a control request to the infotainment system 120 based on its own posture. Based on the received control request, the infotainment system 120 controls the camera 130 to adjust its shooting angle, so that the shooting angle of the camera 130 changes with the posture of the terminal 110. Simultaneously, the camera 130 transmits a real-time video stream to the infotainment system 120, which then sends the video stream to the terminal 110 via a communication connection, allowing the passenger to view the external environment in real time through the terminal 110.

[0042] It should be noted that, Figure 1 The example of controlling a camera is provided for illustration only. In other examples, the terminal can use the vehicle's infotainment system to adjust the angle of other devices, and this embodiment does not constitute a limitation.

[0043] Please refer to Figure 2 It illustrates a flowchart of a device control method provided in an exemplary embodiment of this application, in which the method is applied to... Figure 1 Taking terminal 110 as an example, the method includes:

[0044] Step 201: Establish a communication connection with the vehicle's infotainment system.

[0045] The communication connection can be a wired connection or a wireless connection.

[0046] In one possible scenario, when the terminal is located inside the vehicle, or is close to the vehicle, a wired connection is established between the terminal and the vehicle's infotainment system via a data cable, or a wireless connection is established via a short-range communication component (such as Bluetooth).

[0047] In another possible scenario, when the terminal is far from the vehicle, a mobile network data connection is established between the terminal and the vehicle's infotainment system, thereby enabling communication via the mobile data network.

[0048] For ease of description, the following embodiments use a scenario where the terminal is located inside the vehicle and has a wireless connection with the vehicle's infotainment system as an example, but this does not constitute a limitation.

[0049] Step 202: In response to the received control operation, a target control request is generated, which is used to request adjustment of the device angle of the target device in the vehicle.

[0050] Optionally, the control operation may be a voice control operation, a trigger operation on a control control (a physical control set on the terminal or a virtual control displayed in the control interface), or a posture control operation on the terminal (such as changing the posture of the terminal by rotation), etc. This embodiment does not limit this to any particular type.

[0051] Optionally, the target device includes at least one of a camera, speaker, seat, screen, radar, and vehicle robot.

[0052] In some embodiments, the target device is at least one device of the same type. For example, the target device may be a camera located at the front exterior of the vehicle, or two rear seats in the vehicle.

[0053] In other embodiments, the target device is at least two devices of different device types. For example, the target device is a rear seat in a vehicle and the screen of a multimedia device located in front of the rear seat.

[0054] Based on the received control operation, the terminal generates a target control request. Optionally, the target control request may include at least the device identifier of the target device to be controlled and the control parameters.

[0055] In one possible implementation, the terminal collects the user's voice through a microphone, converts the user's voice into speech text, performs semantic recognition on the speech text, determines the target device to be controlled and the control parameters, and then generates a target control request.

[0056] In another possible implementation, the terminal displays a control interface. When it receives a selection operation on the target device in the control interface and a touch operation on the control control corresponding to the target device, the terminal determines the control parameters based on the touch operation on the control control, and then generates a target control request based on the control parameters and the device identifier of the target device.

[0057] In another possible implementation, once the target device to be controlled is determined, when the user rotates the terminal, the terminal determines the control parameters of the target device based on its own real-time attitude parameters, and then generates a target control request based on the control parameters and the device identifier of the target device.

[0058] It should be noted that, in addition to the control operations described above, the terminal can also generate target control requests based on other types of control operations (such as eye control operations and gesture control operations); and the terminal can generate target control requests from combined control operations based on at least two types of control operations (such as a combination of voice control operations and posture control operations), which is not limited in this embodiment.

[0059] Step 203: Send a target control request to the vehicle's infotainment system via the communication connection, so that the vehicle's infotainment system can control the target device to adjust the device angle based on the target control request.

[0060] In one possible implementation, after receiving a target control request, the vehicle's infotainment system verifies the sender of the request. If the sender has control authority, the system controls the target device to adjust its angle based on the request. If the sender does not have control authority, the vehicle's infotainment system does not respond to the target control request.

[0061] Optionally, if a control whitelist is configured in the vehicle's infotainment system, terminals on the whitelist have control permissions over the devices. In some embodiments, different devices may correspond to different control whitelists, or different devices may correspond to the same control whitelist.

[0062] In one possible implementation, for devices that support adjusting their angle via the vehicle's infotainment system, the corresponding control interface is encapsulated as a Service-Oriented Architecture (SOA) service and integrated into a Software Development Kit (SDK) for easy access. The terminal sends the target control request to the SDK on the vehicle's infotainment system via a communication connection, and the SDK on the vehicle's infotainment system then calls the SOA service interface to control the device to adjust its angle.

[0063] Optionally, the vehicle-mounted infotainment system can send data transmitted by the target device to the terminal via a communication connection for display. For example, when the target device is a camera, the terminal can send a target control request to the vehicle-mounted infotainment system instructing the camera angle to be adjusted, while simultaneously receiving the camera video stream from the vehicle-mounted infotainment system and decoding and displaying it.

[0064] In summary, in the embodiments of this application, when a communication connection is established with the vehicle's infotainment system, the terminal can generate a target control request based on the received control operation and send the target control request to the infotainment system through the communication connection. The infotainment system can then adjust the device angle of the target device in the vehicle based on the request. For example, using the solution provided in the embodiments of this application, when it is inconvenient for the user to interact directly with the infotainment system, the user can adjust the angle of devices such as cameras, speakers, and seats in the vehicle through the terminal, which helps to improve the convenience of controlling the devices in the vehicle.

[0065] To achieve more convenient and accurate device angle adjustment, when the terminal is equipped with a sensor that acquires attitude parameters, the user can adjust the terminal's attitude through attitude control operations, ensuring that the device angle changes of the target device are consistent with the terminal's attitude changes. In one possible implementation, in response to an attitude control operation on the terminal, the terminal generates a target control request based on its attitude parameters, which are acquired by the sensors within the terminal.

[0066] In some embodiments, the terminal is equipped with a gyroscope, and the attitude parameters collected by the gyroscope include the heading angle, roll angle, and pitch angle. (Illustrative example, such as...) Figure 3 As shown, when the terminal's attitude is adjusted, the attitude parameters obtained by the terminal include the pitch angle around the X-axis, the roll angle around the Y-axis, and the yaw angle around the Z-axis.

[0067] Furthermore, the terminal determines the control parameters corresponding to the target device based on at least one of the heading angle, roll angle, and pitch angle, and generates a target control request. The types of control parameters may differ depending on the type of target device.

[0068] Compared to voice control, posture control enables more precise and continuous angle adjustments (because it generates continuous control requests based on continuous posture parameters collected by sensors); compared to triggering controls, users only need to rotate the terminal to control the device without clicking on controls, making device angle adjustments more convenient.

[0069] The following exemplary embodiment illustrates the process of controlling a device using attitude control operations.

[0070] Please refer to Figure 4 The diagram illustrates a flowchart of a device control method provided in another exemplary embodiment of this application, the method comprising:

[0071] Step 401: Establish a communication connection with the vehicle's infotainment system.

[0072] The implementation method of this step can refer to step 201 above, and will not be repeated here.

[0073] Step 402: In response to the trigger operation of the initialization control in the control interface, obtain the initialization attitude parameters.

[0074] In order to make the changes in the device's angle consistent with the changes in the terminal's attitude, the terminal needs to determine its own initial attitude parameters so that the attitude changes can be determined subsequently based on the initial attitude parameters and real-time attitude parameters.

[0075] In one possible implementation, after establishing a communication connection with the vehicle's infotainment system, the terminal displays a control interface. This control interface is used to control the device and includes control options corresponding to the device. In this embodiment, the control interface includes an initialization control; when a trigger operation is received on the initialization control, the terminal determines the real-time attitude parameters at the time of the trigger operation as the initialization attitude parameters.

[0076] In an illustrative example, the control interface corresponding to the camera includes an initialization control. When a trigger operation is received on the initialization control, the terminal obtains the initial attitude parameters, including the pitch angle x_p0, the roll angle y_p0, and the yaw angle z_p0.

[0077] Step 403: In response to the triggering operation of the initialization control in the control interface, an initialization command is sent to the vehicle system so that the vehicle system controls the target device to initialize the device angle based on the initialization command.

[0078] In one possible implementation, in order to improve the adjustability of each adjustable direction during subsequent attitude control operations, when a trigger operation is received on the initialization control in the control interface, the terminal will also send an initialization command to the vehicle system. The initialization command contains the device identifier of the target device, and the vehicle system initializes the device angle of the target device according to the device identifier.

[0079] Optionally, the method of initializing the device angles may differ depending on the type of target device. This initialization method is related to the adjustable angles of the target device. For example, for target devices such as cameras and speakers, whose adjustable angles include pitch and yaw, the vehicle's infotainment system controls the target device to initialize the pitch angle to x_c = 0 and the yaw angle to z_c = 0 based on initialization commands. For target devices such as screens and seats, whose adjustable angles include pitch, the vehicle's infotainment system controls the target device to initialize the pitch angle to x_c = 0 based on initialization commands. For target devices such as in-vehicle robots, whose adjustable angles include pitch, roll, and yaw, the vehicle's infotainment system controls the target device to initialize the pitch angle to x_c = 0, the roll angle to y_z = 0, and the yaw angle to z_c = 0 based on initialization commands.

[0080] In an illustrative example, the control interface corresponding to the camera includes an initialization control. When a trigger operation is received on the initialization control, the terminal sends an initialization command to the vehicle system. Based on the initialization command, the vehicle system controls the camera to initialize the pitch angle to x_c = 0 and the yaw angle to z_c = 0, that is, the camera's shooting direction is directly forward.

[0081] It should be noted that this step is optional, meaning that the target device does not need to initialize the device angle, and the subsequent device angle adjustment is based on the current device angle. This embodiment will not elaborate on this.

[0082] Step 404: In response to the attitude control operation on the terminal, based on the initial attitude parameters and real-time attitude parameters of the terminal, determine the attitude change parameters, which are used to characterize the attitude change of the terminal.

[0083] In one possible implementation, upon receiving a trigger operation on the initialization control, the terminal acquires its real-time attitude parameters according to the sampling frequency, and determines an attitude change parameter characterizing the terminal's attitude change based on the real-time attitude parameters and the initialization attitude parameters. This attitude change parameter is defined as: Real-time attitude parameters - Initialization attitude parameters.

[0084] To illustrate, when the attitude parameters include pitch angle, roll angle and yaw angle, and the real-time attitude parameters are (x_p1, y_p1, z_p1), and the initial attitude parameters are (x_p0, y_p0, z_p0), the attitude change parameters are (x_p1-x_p0, y_p1-y_p0, z_p1-z_p0).

[0085] It should be noted that since the adjustable angle of the target device has an adjustable range, when the attitude change parameter is outside the adjustable range, the terminal will determine the extreme value of the adjustable range as the attitude change parameter. For example, if the adjustable range of the camera's pitch angle is (-120°, 120°), when x_p1-x_p0>120°, the terminal determines the pitch angle change in the attitude change parameter to be 120°; when x_p1-x_p0<-120°, the terminal determines the pitch angle change in the attitude change parameter to be -120°.

[0086] Optionally, the terminal stores information about the device's adjustable angle range, which includes a correspondence between the device and the adjustable range. Based on this correspondence, the terminal determines whether the attitude change parameters are within the adjustable range.

[0087] Optionally, when the attitude change parameters are outside the adjustable range, the terminal displays a prompt message on the control interface to remind the user that the target device has reached the maximum adjustable angle.

[0088] Step 405: Generate a target control request based on attitude change parameters, wherein the vehicle-mounted system is used to control the target device to adjust the device angle based on the attitude change parameters.

[0089] Since different devices support different adjustable angles, in order to ensure correct control of the target device subsequently, in one possible implementation, the terminal determines the target attitude change parameters from the attitude change parameters based on the adjustable angles supported by the target device, and then generates a target control request based on the target attitude change parameters. This target control request includes the target attitude change parameters and the device identifier of the target device.

[0090] In some embodiments, the terminal stores adjustable angle information (which can be synchronized to the terminal by the vehicle system after a communication connection is established between the terminal and the vehicle system). This adjustable angle information contains the correspondence between the device and the adjustable angle. The terminal then determines the target attitude change parameters from the attitude change parameters based on this adjustable angle information. The correspondence between the device and the adjustable angle is illustrated in Table 1.

[0091] Table 1

[0092] equipment Adjustable angle Camera, speaker, radar Pitch angle, yaw angle Seats, screens Pitch angle Vehicle-mounted robots Pitch angle, roll angle, yaw angle

[0093] Based on the correspondence shown in Table 1, when the target device is a camera, the terminal determines the target posture change parameters as (x_p1-x_p0, z_p1-z_p0); when the target device is a seat, the terminal determines the target posture change parameters as (x_p1-x_p0); and when the target device is a vehicle-mounted robot, the terminal determines the target posture change parameters as (x_p1-x_p0, y_p1-y_p0, z_p1-z_p0).

[0094] In another possible implementation, the terminal generates a target control request that includes attitude change parameters and the device identifier of the target device. After the target control request is sent to the vehicle system, the vehicle system determines the target attitude change parameters from the attitude change parameters and controls the target device based on the target attitude change parameters. This embodiment will not be described in detail here.

[0095] Step 406: Send a target control request to the vehicle's infotainment system via the communication connection, so that the vehicle's infotainment system can control the target device to adjust the device angle based on the target control request.

[0096] The implementation method of this step can refer to step 203 above, and will not be repeated here.

[0097] Based on the example in the above steps, when the target device is a camera, after the vehicle system adjusts the camera's device angle based on the target attitude change parameters (x_p1-x_p0, z_p1-z_p0), the camera's pitch angle is x_p1-x_p0 and its yaw angle is z_p1-z_p0.

[0098] In an illustrative application scenario, such as Figure 5 As shown, when the target device is camera 52, after the user rotates mobile phone 51 to change the heading angle, mobile phone 51 sends a target control request containing the heading angle change angle to the vehicle system. Based on the heading angle change angle, the vehicle system controls the shooting angle of camera 52 in the left and right directions.

[0099] In another illustrative application scenario, such as Figure 6 As shown, when the target device is seat 62, after the user rotates mobile phone 61 to change the pitch angle, mobile phone 61 sends a target control request containing the pitch angle change angle to the vehicle system. Based on the pitch angle change angle, the vehicle system controls seat 62 to adjust the backrest angle.

[0100] In this embodiment, the terminal obtains its own initial attitude parameters based on the received initialization operation, and determines the attitude change parameters based on its own real-time attitude parameters during the attitude control process. Then, it generates a control request based on the attitude change parameters, thereby realizing that the device angle of the target device changes synchronously with the terminal attitude.

[0101] The above embodiments are illustrated using the example of attitude change parameters being determined by the terminal. In another possible implementation, in response to a trigger operation on the initialization control in the control interface, the terminal obtains the initialization attitude parameters and sends them to the vehicle's infotainment system via a communication connection. Correspondingly, the vehicle's infotainment system receives and stores the initialization attitude parameters for subsequent determination of the terminal's attitude change parameters.

[0102] When the terminal generates a target control request, it does so based on the terminal's real-time attitude parameters. After receiving the target control request, the vehicle-mounted system determines the attitude change parameters based on the real-time attitude parameters and the terminal's initial attitude parameters, and then controls the target device to adjust its angle based on these attitude change parameters.

[0103] Optionally, the process by which the vehicle-mounted system determines the target attitude change parameters from the attitude change parameters can refer to the above-described terminal-side embodiment, which will not be elaborated upon here.

[0104] Typically, multiple devices of the same type are installed in different locations within a vehicle. For example, cameras are installed on the exterior of the vehicle in all four directions (front, back, left, and right) to capture images from different angles; speakers are installed inside the vehicle at the driver's seat, passenger seat, left rear seat, middle rear seat, and right rear seat. To improve the accuracy of device control, the terminal needs to identify the target device from a pool of candidate devices before controlling the device. These candidate devices are located in different locations within the vehicle.

[0105] In one possible implementation, the user can select a target device via interface instructions. Accordingly, in response to the device selection operation within the control interface, the terminal determines the candidate device indicated by the device selection operation as the target device.

[0106] Optionally, the control interface may display candidate devices, which can be devices of the same type. For example, the control interface may display cameras located at different positions outside the vehicle.

[0107] Indicative, such as Figure 7 As shown, the control interface 70 displays a first camera 71 located at the front of the vehicle, a second camera 72 located on the left side of the vehicle, a third camera 73 located on the right side of the vehicle, and a fourth camera 74 located at the rear of the vehicle. When the user selects the first camera 71, the terminal immediately identifies the first camera 71 as the target camera to be controlled. During subsequent control operations, only the first camera 71 adjusts its shooting angle based on the control operation; the other cameras do not adjust their shooting angles accordingly.

[0108] In another possible implementation, the terminal can also determine the target device from candidate devices based on its location within the vehicle, simplifying the target device determination process without requiring manual user input. For example... Figure 8 As shown, the process by which a terminal determines a target device based on its position within a vehicle may include the following steps.

[0109] Step 801: Determine the relative orientation between the terminal and the vehicle-mounted system.

[0110] In one possible implementation, since the vehicle-mounted unit's position within the vehicle is fixed, the terminal can use the vehicle-mounted unit as a positioning anchor point to determine its own position within the vehicle. Optionally, after establishing a communication connection with the vehicle-mounted unit, the terminal first determines its relative orientation to the unit.

[0111] Regarding the method of determining relative location, in one possible implementation, when the communication component between the terminal and the vehicle's infotainment system has positioning capabilities, the terminal determines its relative location with the vehicle's infotainment system through the communication component. For example, when the terminal and the vehicle's infotainment system establish a communication connection via Bluetooth, the terminal can use Bluetooth positioning to determine its relative location with the vehicle's infotainment system.

[0112] In another possible implementation, when positioning components are installed in both the terminal and the vehicle's infotainment system, the terminal can determine its phase orientation relative to the vehicle's infotainment system using these components. For example, when an Ultra Wideband (UWB) chip is installed in both the terminal and the vehicle's infotainment system, the terminal can determine its relative orientation relative to the vehicle's infotainment system based on the UWB signal.

[0113] Of course, the terminal can also determine its relative position to the vehicle's infotainment system through other means, and the embodiments of this application do not constitute a limitation thereto.

[0114] Indicative, such as Figure 9 As shown, terminal 91 determines its relative position to vehicle system 92 using UWB positioning.

[0115] Step 802: Determine the position of the terminal in the vehicle based on the vehicle's location and relative orientation.

[0116] Furthermore, using the vehicle-mounted unit as an anchor point, the terminal determines its position within the vehicle based on the unit's location within the vehicle and its relative orientation to the unit. The position of the vehicle-mounted unit within the vehicle can be provided by the unit itself.

[0117] In other possible implementations, the terminal can provide the determined relative orientation to the vehicle system, which then determines the terminal's position in the vehicle based on the relative orientation and its own position in the vehicle, and feeds it back to the terminal. This embodiment does not limit this.

[0118] Indicative, such as Figure 9 As shown, based on the position of the vehicle-mounted unit 92 in the vehicle and the relative orientation between the terminal 91 and the vehicle-mounted unit 92, the terminal 91 determines that the terminal 91 is located in the left seat of the second row in the vehicle.

[0119] Step 803: Determine the target device based on the location of the candidate device in the vehicle and the location of the terminal in the vehicle.

[0120] In one possible implementation, the terminal determines the distance between itself and each candidate device based on the position of the candidate device in the vehicle and its own position in the vehicle, thereby identifying the candidate device closest to the terminal as the target device.

[0121] In other possible implementations, the terminal identifies a candidate device located in the same area as the terminal as the target device based on the candidate device and the terminal's position within the vehicle. For example, when the terminal is in the left seat of the second row, the terminal identifies the speaker located in the second row as the target speaker.

[0122] Indicative, such as Figure 9 As shown, terminal 91 identifies the speaker 93 closest to the left-hand seat in the second row as the target speaker. During subsequent device control, the terminal controls the angle of the target speaker based on the received control commands.

[0123] In this embodiment, the terminal automatically determines the target device from the candidate devices based on its own position in the vehicle and the position of the candidate devices in the vehicle, and controls the target device without requiring manual instructions from the user, further simplifying the device control process.

[0124] Because the adjustable angle of a single device is limited, users may need to switch between multiple devices when using the terminal to control the device. For example, when controlling a camera using the terminal, controlling the front-facing camera only allows viewing a 240° fan-shaped area in front. When the user needs to view the environment on the right side of the vehicle, they need to switch to controlling the right-side camera. In some embodiments, users can manually switch target devices in the control interface.

[0125] In addition to manually switching target devices, to further improve the efficiency of switching control between different devices, in one possible implementation, the terminal can determine the target device based on the location of the candidate device in the vehicle and the terminal's location in the vehicle. For example... Figure 10 As shown, the process by which a terminal determines a target device may include the following steps.

[0126] Step 1001: Determine the terminal orientation based on sensor data.

[0127] In one possible implementation, during device control, the terminal determines its real-time orientation based on real-time sensor data collected by a sensor. This sensor can be an electronic compass sensor, and the terminal orientation is determined by cardinal directions (north, south, east, west).

[0128] Indicative, such as Figure 11 As shown, when the user uses terminal 1105 to control the camera outside the vehicle, the terminal orientation of terminal 1105 is determined to be due north.

[0129] Step 1002: Determine the device orientation of the candidate device based on the real-time vehicle orientation and the position of the candidate device in the vehicle.

[0130] In one possible implementation, the terminal obtains the real-time vehicle orientation through a communication connection with the vehicle's infotainment system, thereby determining the device orientation of each candidate device based on its position within the vehicle.

[0131] In another possible implementation, the vehicle-mounted system determines the real-time orientation of each candidate device based on its own real-time vehicle orientation and the location of the candidate devices, and then provides the device orientation to the terminal through a communication connection.

[0132] Optionally, the real-time vehicle orientation can be determined by the vehicle's infotainment system using data from built-in sensors.

[0133] Indicative, such as Figure 11 As shown, when the vehicle's real-time orientation is due north, the first camera 1101 located at the front of the vehicle is oriented due north, the second camera 1102 located on the left side of the vehicle is oriented due west, the third camera 1103 located on the right side of the vehicle is oriented due east, and the fourth camera 1104 located at the rear of the vehicle is oriented due south.

[0134] Step 1003: Determine the target device based on the terminal orientation and the device orientation.

[0135] Since users need to control devices within the terminal's orientation range, the terminal determines the orientation angle between the terminal's orientation and the orientation of each device, and identifies the candidate device corresponding to the smallest orientation angle as the target device.

[0136] Indicative, such as Figure 11As shown, since the device orientation of the first camera 1101 is the same as the terminal orientation of the terminal 1105, the terminal identifies the first camera 1101 as the target camera and sends a control request to the vehicle system to control the first camera 1101 in order to view the real-time image in front of the vehicle. When the user wants to view the real-time image on the right side of the vehicle, the terminal 1105 is rotated to make the terminal orientation due east. At this time, the terminal identifies the third camera 1103 as the target camera and sends a control request to the vehicle system to control the third camera 1103.

[0137] It should be noted that this embodiment is illustrated by taking the control of a camera as an example, but it does not constitute a limitation.

[0138] In this embodiment, the terminal determines the target device that matches its own orientation based on its own real-time orientation and the real-time orientation of each candidate device in the vehicle, thereby controlling the target device without having to manually switch the target device to be controlled, which helps to improve the switching efficiency of the device switching process.

[0139] In some embodiments, the above-described device control method can be applied to either a vehicle driving scenario or a vehicle stationary scenario. However, in a vehicle driving scenario, some devices may be used for assisted driving, and the user's adjustment of the device angle of such devices may affect the assisted driving. For example, the user adjusting the shooting angle of the front camera of the vehicle may affect the recognition of vehicles or pedestrians in front.

[0140] Therefore, in order to improve the safety of vehicle operation, in one possible implementation, while the vehicle is in motion, the terminal detects whether the target device to be controlled is a driving assistance device.

[0141] The vehicle's driving status can be fed back to the terminal by the vehicle-mounted system through a communication connection, or it can be automatically determined by the terminal.

[0142] Optionally, the terminal has a list of driving assistance devices, which it uses to determine whether a target device is a driving assistance device. The lists of driving assistance devices may be the same or different for different driving states.

[0143] For example, in a forward driving scenario, the list of driver assistance devices includes a front camera, a left-side camera, a right-side camera, and a rear camera; in a parking scenario, the list of driver assistance devices includes a left-side camera, a right-side camera, and a rear camera.

[0144] When the vehicle is in motion and the target device is not a driver assistance device, in response to the received control operation, the terminal generates a target control request, which enables the device angle to be adjusted.

[0145] When the vehicle is in motion and the target device is a driver assistance device, in response to the received control operation, the terminal will issue a control pause reminder, that is, remind the user that the device angle cannot be adjusted at present.

[0146] For example, if a control command is received for the camera while the vehicle is in motion, the user cannot control the camera while driving because the camera is used to assist driving; if a control command is received for the speaker, the user can adjust the speaker angle while driving because the speaker is not used to assist driving.

[0147] Optionally, to further improve the safety of vehicle operation, when a control end command is received, the terminal sends a device reset command to the vehicle control unit, which then controls the target device to reset its angle based on the device reset command.

[0148] In this embodiment, when the vehicle is in motion, the terminal detects whether the target device to be controlled is a driving assistance device, thereby avoiding adjusting the device angle of the driving assistance device while in motion, ensuring that the driving assistance device works normally during the driving process, and improving the safety of the vehicle's driving process.

[0149] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0150] Please refer to Figure 12 This illustration shows a structural block diagram of a device control apparatus according to an embodiment of this application. The apparatus may include:

[0151] The connection establishment module 1201 is used to establish a communication connection with the vehicle's infotainment system.

[0152] The request generation module 1202 is used to generate a target control request in response to a received control operation. The target control request is used to request adjustment of the device angle of the target device in the vehicle.

[0153] The request sending module 1203 is used to send the control request to the vehicle system through the communication connection, so that the vehicle system controls the target device to adjust the device angle based on the target control request.

[0154] Optionally, the request generation module 1202 is used to:

[0155] In response to an attitude control operation on the terminal, a target control request is generated based on the attitude parameters of the terminal, which are acquired by sensors in the terminal.

[0156] Optionally, the request generation module 1202 is used to:

[0157] Based on the initial attitude parameters and real-time attitude parameters of the terminal, attitude change parameters are determined, which are used to characterize the attitude change of the terminal; based on the attitude change parameters, the target control request is generated, wherein the vehicle system is used to control the target device to adjust the device angle based on the attitude change parameters;

[0158] or,

[0159] The target control request is generated based on the real-time attitude parameters of the terminal, wherein the vehicle system is used to determine attitude change parameters based on the real-time attitude parameters and the initial attitude parameters of the terminal, and to control the target device to adjust the device angle based on the attitude change parameters.

[0160] Optionally, if the target control request includes the attitude change parameters, the apparatus further includes:

[0161] The acquisition module is used to acquire the initialization attitude parameters in response to a trigger operation on the initialization control in the control interface;

[0162] If the target control request includes the real-time attitude parameters, the apparatus further includes:

[0163] The acquisition module is used to acquire the initialization attitude parameters in response to a trigger operation on the initialization control in the control interface;

[0164] An initialization module is used to send the initialization attitude parameters to the vehicle's infotainment system via the communication connection.

[0165] Optionally, the device further includes:

[0166] An initialization command sending module is used to send an initialization command to the vehicle system in response to a trigger operation of the initialization control in the control interface, so that the vehicle system controls the target device to initialize the device angle based on the initialization command.

[0167] Optionally, the device further includes:

[0168] The device determination module is used to determine the target device from candidate devices, wherein different candidate devices are set at different positions on the vehicle.

[0169] Optionally, the device determination module determines the target device in at least one of the following ways:

[0170] In response to a device selection operation within the control interface, the candidate device indicated by the device selection operation is determined as the target device;

[0171] The target device is determined from the candidate devices based on the position of the terminal in the vehicle;

[0172] The target device is determined from the candidate devices based on the terminal orientation of the terminal.

[0173] Optionally, in the process of determining the target device from candidate devices based on the position of the terminal in the vehicle, the device determination module is configured to:

[0174] Determine the relative orientation between the terminal and the vehicle-mounted system;

[0175] Based on the vehicle's location within the vehicle and its relative orientation, the location of the terminal within the vehicle is determined.

[0176] The target device is determined based on the location of the candidate device in the vehicle and the location of the terminal in the vehicle.

[0177] Optionally, during the process of determining the target device from candidate devices based on the terminal's orientation, the device determination module is configured to:

[0178] The orientation of the terminal is determined based on sensor data;

[0179] The device orientation of the candidate device is determined based on the real-time vehicle orientation and the position of the candidate device in the vehicle.

[0180] The target device is determined based on the terminal orientation and the device orientation.

[0181] Optionally, the request generation module 1202 is used to:

[0182] When the vehicle is in motion and the target device is not a driver assistance device, the target control request is generated in response to the received control operation.

[0183] Optionally, the device further includes:

[0184] The reminder module is used to provide a control pause reminder when the vehicle is in motion and the target device is the driving assistance device.

[0185] Optionally, the target device includes at least one of a camera, speaker, seat, screen, radar, and vehicle robot.

[0186] In summary, in the embodiments of this application, when a communication connection is established with the vehicle's infotainment system, the terminal can generate a target control request based on the received control operation and send the target control request to the infotainment system through the communication connection. The infotainment system can then adjust the device angle of the target device in the vehicle based on the request. For example, using the solution provided in the embodiments of this application, when it is inconvenient for the user to interact directly with the infotainment system, the user can adjust the angle of devices such as cameras, speakers, and seats in the vehicle through the terminal, which helps to improve the convenience of controlling the devices in the vehicle.

[0187] Please refer to Figure 13 This diagram illustrates a structural block diagram of a terminal provided in an exemplary embodiment of this application. The terminal 1300 can be implemented as the terminal in the various embodiments described above. The terminal 1300 may include one or more components such as a processor 1310 and a memory 1320.

[0188] Processor 1310 may include one or more processing cores. Processor 1310 connects to various parts within terminal 1300 using various interfaces and lines, and performs various functions and processes data of terminal 1300 by running or executing instructions, programs, code sets, or instruction sets stored in memory 1320, and by calling data stored in memory 1320. Optionally, processor 1310 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 1310 may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), Neural-network Processing Unit (NPU), and modem. Specifically, the CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required to be displayed on the touch screen; the NPU is used to implement Artificial Intelligence (AI) functions; and the modem is used to handle wireless communication. It is understandable that the aforementioned modem may not be integrated into the processor 1310, but may be implemented as a separate chip.

[0189] The memory 1320 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 1320 may include a non-transitory computer-readable storage medium. The memory 1320 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 1320 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the various method embodiments described below, etc.; the data storage area may store data created according to the use of the terminal 1300 (such as audio data, phone book, etc.).

[0190] In addition, those skilled in the art will understand that the structure of the terminal 1300 shown in the above figures does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the terminal 1300 may also include a display screen, camera assembly, microphone, speaker, radio frequency circuit, input unit, sensors (such as accelerometer, angular velocity sensor, light sensor, etc.), audio circuit, WiFi module, power supply, Bluetooth module, etc., which will not be described in detail here.

[0191] This application also provides a computer-readable storage medium storing at least one piece of program code, which is loaded and executed by a processor to implement the device control method described in the above embodiments.

[0192] This application provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the device control methods provided in various optional implementations of the above aspects.

[0193] It should be understood that "multiple" as used herein refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. Furthermore, the step numbers described herein are merely illustrative of one possible execution order. In some other embodiments, the steps may not be executed in numerical order, such as two steps with different numbers being executed simultaneously, or two steps with different numbers being executed in the reverse order of the illustration. This application does not limit this.

[0194] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A device control method, characterized in that, The method is executed by a terminal, and the method includes: Establish a communication connection with the vehicle's infotainment system; The target device is determined from the candidate devices based on the position of the terminal in the vehicle, wherein the target device is the candidate device that is closest to the terminal or is in the same area as the terminal; and / or, the target device is determined from the candidate devices based on the terminal orientation of the terminal, wherein the target device is the candidate device corresponding to the smallest orientation angle, wherein the orientation angle is the angle between the terminal orientation and the device orientation; When the vehicle is in motion and the target device is not a driving assistance device, in response to the attitude control operation on the terminal, attitude change parameters are determined based on the initial attitude parameters, real-time attitude parameters and device angle adjustable range information of the terminal. The attitude change parameters are used to characterize the attitude change of the terminal. The device angle adjustable range information includes the correspondence between the device and the adjustable range, and the attitude change parameters are within the adjustable range of the target device. The driving assistance device is determined based on a driving assistance device list, and different driving assistance device lists correspond to different driving states. Based on the adjustable angle information of the device, the target attitude change parameters are determined from the attitude change parameters, and a target control request is generated based on the target attitude change parameters. The target attitude change parameters are the adjustable angles supported by the target device, and the adjustable angle information of the device includes the correspondence between the device and the adjustable angle. The target control request is sent to the vehicle's infotainment system via the communication connection, so that the vehicle's infotainment system controls the target device to adjust its angle based on the target control request. The control interface corresponding to the device in the vehicle is encapsulated as an SOA service and integrated into the SDK for invocation. The target control request is sent to the SDK of the vehicle's infotainment system, and the SDK of the vehicle's infotainment system calls the SOA service to control the target device to adjust its angle.

2. The method according to claim 1, characterized in that, If the target control request includes the attitude change parameters, the method further includes: In response to a trigger operation on the initialization control in the control interface, the initialization attitude parameters are obtained; If the target control request includes the real-time attitude parameters, the method further includes: In response to a trigger operation on the initialization control in the control interface, the initialization attitude parameters are obtained; the initialization attitude parameters are then sent to the vehicle system via the communication connection.

3. The method according to claim 2, characterized in that, The method further includes: In response to the triggering operation of the initialization control in the control interface, an initialization command is sent to the vehicle system, so that the vehicle system controls the target device to initialize the device angle based on the initialization command.

4. The method according to claim 1, characterized in that, The step of determining the target device from candidate devices based on the position of the terminal in the vehicle includes: Determine the relative orientation between the terminal and the vehicle-mounted system; Based on the vehicle's location within the vehicle and its relative orientation, the location of the terminal within the vehicle is determined. The target device is determined based on the location of the candidate device in the vehicle and the location of the terminal in the vehicle.

5. The method according to claim 1, characterized in that, The process of determining the target device from candidate devices based on the terminal orientation of the terminal includes: The orientation of the terminal is determined based on sensor data; The device orientation of the candidate device is determined based on the real-time vehicle orientation and the position of the candidate device in the vehicle. The target device is determined based on the terminal orientation and the device orientation.

6. The method according to claim 1, characterized in that, The method further includes: When the vehicle is in motion and the target device is the driving assistance device, a control pause reminder is issued.

7. The method according to any one of claims 1 to 3, characterized in that, The target device includes at least one of a camera, speaker, seat, screen, radar, and vehicle robot.

8. A device control apparatus, characterized in that, The device includes: The connection establishment module is used to establish a communication connection with the vehicle's infotainment system. The device determination module is configured to determine a target device from candidate devices based on the position of the terminal in the vehicle, wherein the target device is a candidate device that is closest to the terminal or is in the same area as the terminal; and / or, to determine the target device from candidate devices based on the terminal orientation of the terminal, wherein the target device is a candidate device corresponding to the smallest orientation angle, wherein the orientation angle is the angle between the terminal orientation and the device orientation; The request generation module is configured to, in response to the attitude control operation on the terminal, determine attitude change parameters based on the initial attitude parameters, real-time attitude parameters, and adjustable angle range information of the terminal when the vehicle is in motion and the target device is not a driving assistance device, the attitude change parameters being used to characterize the attitude change of the terminal. The adjustable angle range information includes the correspondence between the device and the adjustable range, and the attitude change parameters are within the adjustable range of the target device. The driving assistance device is determined based on a driving assistance device list, and different driving assistance device lists correspond to different driving states. Based on the adjustable angle information of the device, the target attitude change parameters are determined from the attitude change parameters, and a target control request is generated based on the target attitude change parameters. The target attitude change parameters are the adjustable angles supported by the target device, and the adjustable angle information of the device includes the correspondence between the device and the adjustable angle. The request sending module is used to send the control request to the vehicle-mounted system through the communication connection, so that the vehicle-mounted system controls the target device to adjust the device angle based on the target control request. The control interface corresponding to the device in the vehicle is encapsulated as an SOA service and integrated into the SDK for invocation. The target control request is sent to the SDK of the vehicle-mounted system, and the SDK of the vehicle-mounted system calls the SOA service to control the target device to adjust the device angle.

9. A terminal, characterized in that, The terminal includes a processor and a memory; the memory stores at least one instruction, which is executed by the processor to implement the device control method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is loaded and executed by a processor to implement the device control method as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions to cause the computer device to perform the device control method as described in any one of claims 1 to 7.

Images