Method and apparatus for controlling central control system of vehicle, vehicle, and storage medium

Abstract

To provide a method for controlling a central control system of a vehicle, a control method applicable to the central control system of a vehicle, a computer apparatus, a vehicle, a computer-readable storage medium, and a computer program product relating to the technical field of central control of a vehicle.SOLUTION: A method for controlling a central control system of a vehicle includes the steps of: acquiring an image including a central control panel of the central control system of a vehicle; identifying at least one control component on the central control panel in the image; and, in response to detection of a first interactive operation for a first target control component of the at least one control component in the image, determining operation information associated with the first target control component; and transmitting the operation information to the central control system of a vehicle so that the central control system of a vehicle can determine a control command awaiting execution based on the operation information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to the technical field of vehicle central control, and in particular to a method for controlling a vehicle central control system, a control method applied to a vehicle central control system, a computer device, a vehicle, a computer-readable storage medium, and a computer program product. [Background technology]

[0002] As various technologies such as autonomous driving, advanced driver assistance, intelligent cabins, and intelligent vehicle networking advance in the field of vehicle technology, the functions of vehicle central control systems are becoming more diversified, and they can provide passengers with various functional modules, such as in-car audio and video entertainment, in-car navigation, and in-car air conditioning control settings.

[0003] Vehicles are typically provided with a central control panel that can be operated by the driver and passengers to enable the driver and passengers to control various in-vehicle functions. Interaction between the central control panel and the driver and passengers can be achieved through various types of control components, such as mechanical buttons and touchscreens. However, from the perspective of the vehicle driver, using control components such as screens or buttons during driving can cause distraction and even compromise the driving safety of the vehicle. From the perspective of passengers inside the vehicle, operating the central control panel inside the vehicle has limited flexibility and convenience due to the installation locations of control components such as mechanical buttons and touchscreens inside the vehicle.

[0004] The approaches described in this section are not necessarily approaches that have been previously conceived or adopted. Unless otherwise noted, it should not be assumed that any approach described in this section constitutes prior art merely by virtue of its inclusion in this section. Similarly, unless otherwise noted, any problem addressed in this section is not deemed to be disclosed in any prior art. Summary of the Invention [Problem to be solved by the invention]

[0005] The present disclosure aims to solve at least one of the technical problems existing in the background art. Therefore, one objective of the present disclosure is to provide a method for controlling a vehicle central control system, which uses a user terminal to acquire an image of a central control panel included in the vehicle central control system, identify control components therein, and detect a user's interactive operation of the control components in the image, thereby transmitting operation information to the vehicle central control system, thereby easily realizing the user's control of the vehicle central control system. By applying the above method, a user is no longer limited by the physical contact distance between the control components and the vehicle central control system when controlling the vehicle central control system, and can easily control the control components from a seating position far away from the central control panel. This alleviates the problem in the related art that only the vehicle driver can operate the central control panel, which compromises vehicle driving safety. [Means for solving the problem]

[0006] In an embodiment of a first aspect of the present disclosure, there is provided a method for controlling a central control system of a vehicle, the central control system including a central control panel, the method being applied to a user terminal communicatively connected to the central control system, the method including the steps of acquiring an image including the central control panel; identifying at least one control component on the central control panel in the image; determining operation information associated with the first target control component in response to detecting a first interactive operation on a first target control component among the at least one control component in the image; and transmitting the operation information to the central control system so that the central control system can determine a pending control command based on the operation information.

[0007] In a technical aspect of an embodiment of the present disclosure, a user terminal is used to acquire an image including a vehicle central control panel, and image identification is performed to obtain at least one control component included in the central control panel in the image. Furthermore, a user's interactive operation on a target control component in the image is detected, and operation information is transmitted to the vehicle central control system, whereby the vehicle central control system can determine a waiting control command based on the user's operation information, thereby easily realizing user control of the vehicle central control system. By applying the above method, even if a passenger in a vehicle is seated far away from the central control panel, the vehicle central control system can be easily controlled using a user terminal, and the vehicle driver does not need to operate the central control panel, thereby improving vehicle driving safety.

[0008] In some embodiments, identifying at least one control component on the central control panel in the image includes inputting the image into a computer vision model to obtain at least one control component output by the computer vision model, the computer vision model being obtained by training using a sample image including the reference control component.

[0009] In some embodiments, the computer vision model identifies at least one control component in the image by determining image features of the image; and determining the at least one control component based on the image features, wherein the image features corresponding to the at least one control component and the image features corresponding to a reference control component in the sample image can satisfy a predetermined similarity condition.

[0010] In some embodiments, in response to detecting a first interactive operation on a first target control component of the at least one control component in the image, determining operation information associated with the first target control component includes determining an operation type for the first target control component, wherein the operation type includes at least one of a click, a touch, and a rotate.

[0011] In some embodiments, the method for controlling a central control system of a vehicle further includes sending an authorization request to the central control system in response to detecting a second interactive operation on a second target control component of the at least one control component in the image, and the step of determining operation information associated with the first target control component in response to detecting a first interactive operation on a first target control component of the at least one control component in the image includes determining the operation information in response to receiving authorization information returned from the central control system of the vehicle.

[0012] In some embodiments, the step of sending an authorization request to the vehicle's central control system in response to detecting a second interactive operation on a second target control component among the at least one control component in the image includes determining an in-vehicle function module associated with the second target control component in the vehicle and sending an authorization request for the in-vehicle function module to the vehicle's central control system, and the step of determining operation information associated with the first target control component in response to detecting a first interactive operation on a first target control component among the at least one control component in the image includes determining the operation information in response to receiving authorization information for the in-vehicle function module returned from the vehicle's central control system and in response to determining that the first target control component is associated with the in-vehicle function module.

[0013] In an embodiment of a second aspect of the present disclosure, there is provided a control method applicable to a central control system of a vehicle, the central control system including a central control panel, the method including a step of determining a control command to be executed based on the operation information in response to receiving operation information transmitted by a user terminal communicatively connected to the central control system of the vehicle, the operation information being generated by the user terminal by acquiring an image including the central control panel, determining at least one control component on the central control panel in the image, and determining operation information associated with the first target control component in response to detecting a first interactive operation on a first target control component among the at least one control component in the image.

[0014] In some embodiments, the operation information includes an operation type of the first interactive operation, and the step of determining a control command to be executed based on the operation information includes determining an in-vehicle function module corresponding to the first target control component, and determining a control command for the in-vehicle function module based on the operation type for the first target control component.

[0015] In some embodiments, determining a control command for the vehicle function module based on an operation type for the first target control component includes determining a control command from an instruction table corresponding to the vehicle function module based on the operation type, wherein the instruction table stores mapping relationships between a plurality of pre-defined operation types and a plurality of pre-defined control commands.

[0016] In some embodiments, the control method applied to the central control system of the vehicle further includes receiving an authorization request sent by a user terminal, generating notification information for presenting the authorization request, generating authorization information in response to detecting an authorization interactive operation on the notification information, and returning the authorization information to the user terminal.

[0017] In an embodiment of a third aspect of the present disclosure, there is provided an apparatus for controlling a central control system of a vehicle, the central control system including a central control panel, the apparatus being applied to a user terminal communicatively connected to the central control system, the apparatus including: an acquisition unit configured to acquire an image including the central control panel; a first determination unit configured to identify at least one control component on the central control panel in the image; a second determination unit configured to determine operation information associated with the first target control component in response to detecting a first interactive operation on a first target control component among the at least one control component in the image; and a transmission unit configured to transmit the operation information to the central control system so that the central control system can determine a waiting control command to be executed based on the operation information.

[0018] In an embodiment of a fourth aspect of the present disclosure, there is provided an apparatus applicable to a central control system of a vehicle, the central control system including a central control panel, the apparatus including a third determination unit configured to determine a pending control instruction based on the operation information in response to receiving operation information transmitted by a user terminal communicatively connected to the central control system of the vehicle, the operation information being generated by the user terminal using an acquisition unit configured to acquire an image including the central control panel, a first determination unit configured to determine at least one control component on the control panel in the image, and a second determination unit configured to determine operation information associated with the first target control component in response to detecting a first interactive operation on a first target control component among the at least one control component in the image.

[0019] In an embodiment of the fifth aspect of the present disclosure, there is provided a computing device used to perform the method for controlling a central control system of a vehicle according to the first aspect of the present disclosure.

[0020] In an embodiment of the sixth aspect of the present disclosure, there is provided a computer device used to execute the control method applied to the central control system of a vehicle according to the second aspect of the present disclosure.

[0021] In an embodiment of the seventh aspect of the present disclosure, there is provided a vehicle including a central control system including a central control panel and a computing device according to the sixth aspect of the present disclosure.

[0022] In an embodiment of an eighth aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon instructions that, when executed alone or jointly by one or more processors of a computing device, cause the computing device to perform a method according to the first aspect of the present disclosure or a method according to the second aspect of the present disclosure.

[0023] In an embodiment of a ninth aspect of the present disclosure, there is provided a computer program product comprising instructions that, when executed alone or jointly by one or more processors of a computing device, cause the computing device to perform a method according to the first aspect of the present disclosure or a method according to the second aspect of the present disclosure. [Effects of the Invention]

[0024] The above description merely provides an outline of the technical aspects of the present disclosure. In order to more clearly understand the technical means of the present disclosure and to enable implementation based on the contents of the specification, and to make the above and other objects, features, and advantages of the present disclosure clearer and easier to understand, specific embodiments of the present disclosure are given below. [Brief explanation of the drawings]

[0025] In the drawings, unless otherwise specified, the same reference numerals used throughout the drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings are merely illustrative of some of the embodiments disclosed in the present disclosure and should not be considered limiting on the scope of the present disclosure. [Figure 1]1 is a flowchart of a method for controlling a central control system of a vehicle according to an embodiment of the present disclosure. [Figure 2] 10 is a flowchart of a method for controlling a central control system of a vehicle according to another embodiment of the present disclosure. [Figure 3] 1 is a flowchart of a control method applied to a central control system of a vehicle according to an embodiment of the present disclosure. [Figure 4] 1 is a flowchart of an interaction process for controlling a central control system of a vehicle according to an embodiment of the present disclosure. [Figure 5] 1 is a block diagram of a device for controlling a central control system of a vehicle according to an embodiment of the present disclosure. [Figure 6] 1 is a block diagram illustrating a configuration of a device applied to a central control system of a vehicle according to an embodiment of the present disclosure. [Figure 7] FIG. 1 is a block diagram of an exemplary computer device that can be applied to exemplary embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0026]

[0023] In the following, exemplary embodiments of the present disclosure will be described with reference to the accompanying drawings. For ease of understanding, various details of the embodiments of the present disclosure will be described, but these details should be considered as merely illustrative. Therefore, those skilled in the art will recognize that various modifications and changes can be made to the embodiments described herein without departing from the scope of the present disclosure. Similarly, for clarity and conciseness, the following description omits descriptions of well-known functions and structures.

[0027] In this disclosure, unless otherwise specified, the use of terms such as "first," "second," etc. to describe various elements is not intended to limit the location, time, or importance of these elements, and these terms are used only to distinguish one element from another. In some examples, a first element and a second element can refer to the same instance of the element, but in some cases, they can refer to different instances based on the context.

[0028] In this disclosure, the terms used in the description of the various examples are merely for the purpose of describing specific examples and are not intended to be limiting. Unless otherwise clearly indicated in the context, if no intentional limitation is intended on the number of elements, the elements may be one or more. Also, as used in this disclosure, the term "and / or" covers any and all possible combinations of the listed items.

[0029] As described above, when a vehicle driver uses central control components such as a screen or buttons during driving, they become distracted, further compromising the vehicle's driving safety. To solve this problem, one implementation of related technology involves using voice commands to interact with a central control system. However, the functionality that can be achieved through voice interaction is limited, making it impossible to achieve comprehensive and accurate control of the vehicle's central control system. Another implementation involves installing application programs that can interact with each other on the vehicle's central control system and a user terminal, respectively, and allowing the user to configure application programs (e.g., audio / video entertainment programs, navigation programs) on the vehicle's central control system by operating the application program on the user terminal. However, this implementation requires strict pairing between the vehicle's central control system and the application program on the user terminal, making it less versatile.

[0030] Based on this, the present disclosure provides a method for controlling a vehicle central control system, allowing a user terminal to obtain an image of a central control panel included in the vehicle central control system, identify control components therein, and further detect a user's interactive operation on the control components in the image, and convey the user's operation information to the vehicle central control system, so that the central control system can determine a waiting control command to be executed based on the operation information, thereby realizing user control over the in-vehicle central control system.

[0031] In some embodiments of the present disclosure, a method for controlling a central control system of a vehicle is provided, the central control system including a central control panel, and the method is applied to a user terminal in communication with the central control system.

[0032] 1 illustrates a method 100 for controlling a central control system of a vehicle according to an embodiment of the present disclosure. As shown in FIG. 1, the method 100 includes steps S101 to S104.

[0033] In step S101, an image including the central control panel is acquired.

[0034] In step S102, at least one control component on the central control panel in the image is identified.

[0035] In step S103, in response to detecting a first interactive operation on a first target control component of the at least one control component in the image, operation information associated with the first target control component is determined.

[0036] In step S104, the operation information is sent to the central control system so that the central control system can determine the control commands waiting to be executed based on the operation information.

[0037] According to the method 100 of the embodiment of the present disclosure, a user terminal is used to acquire an image including a vehicle central control panel, and image identification is performed to obtain at least one control component included in the central control panel in the image. Furthermore, a user's interactive operation on the target control component in the image is detected, and operation information is transmitted to the vehicle central control system, so that the vehicle central control system can determine a waiting control command based on the user's operation information, thereby easily realizing the user's control over the vehicle central control system. By applying the above method 100 to the user terminal, even if a passenger in the vehicle is seated far away from the central control panel, the passenger can easily control the vehicle central control system using the user terminal, and the vehicle driver does not need to operate the central control panel, thereby improving vehicle driving safety.

[0038] In some examples, the user terminal may be of various types, such as a mobile phone, a tablet computer, or a smart wearable device (e.g., a smart watch, smart glasses, etc.), as long as it is capable of providing human-machine interaction with the user.

[0039] In some examples, the communication connection between the user terminal and the vehicle's central control system may be achieved using various communication technologies, such as a Wi-Fi connection, a Bluetooth connection, a cellular communication connection, etc.

[0040] In some examples, a specific application program may be installed on the user terminal, and the steps of the above method 100 may be implemented using the application program.

[0041] In some examples, the user terminal includes a camera component, such that step S101 can utilize the camera component to capture images of the central control panel. In other examples, the user terminal can be communicatively coupled to an imaging device, such that images of the central control panel can be captured and transmitted by the imaging device.

[0042] In some examples, the image including the central control panel in step S101 may be obtained from a single frame photograph collected by a camera. In other examples, the image including the central control panel in step S101 may be obtained from a video collected by a camera, i.e., by obtaining at least one image frame from a video including the central control panel and identifying footage of the central control panel included in the image frame.

[0043] In some embodiments, identifying at least one control component on the central control panel in the image S102 includes inputting the image into a computer vision model to obtain at least one control component output by the computer vision model, the computer vision model being obtained by training with a sample image including a reference control component, thereby enabling the computer vision model to more efficiently and accurately identify the control component on the central control panel.

[0044] In some examples, the at least one control component on the central control panel may be of various types, such as, for example, a touch screen, a mechanical button, a mechanical knob, etc.

[0045] In some examples, the construction and training of the computer vision model may be performed as follows: An initial computer vision model is constructed, and the initial model can be constructed based on various types of neural networks, such as a convolutional neural network, a spatial attention network, etc. A sample image including a sample central control panel is input into the initial computer vision model. At least one predictive control component output by the initial computer vision model is obtained. A loss value is calculated based on the at least one pre-annotated reference control component and the at least one predictive control component output by the initial model. Based on the loss value, parameters of the initial computer vision model are adjusted. Using the pre-annotated sample images, supervised training of the initial computer vision model can be performed to obtain a computer vision model that can efficiently and accurately identify at least one control component in the control panel.

[0046] In some examples, sample images including a reference control component can be used to perform unsupervised training on the initial computer vision model, for example, unsupervised training can be performed based on consistency learning, which can reduce the cost of sample annotation while training the model and more efficiently and easily obtain a trained computer vision model. It can be appreciated that training the initial computer vision model can be achieved by applying various deep learning algorithms, but this disclosure is not limited thereto.

[0047] In some embodiments, the computer vision model identifies at least one control component in the image by: determining image features of the image; and determining at least one control component based on the image features, where the image features corresponding to the at least one control component and the image features corresponding to a reference control component in the sample image can satisfy a predetermined similarity condition, thereby achieving more accurate image identification through the process of extracting and matching image features.

[0048] In some examples, the computer vision model may include a feature extraction network for extracting image features, such as shape features, color features, and texture features, from the image. In some examples, the computer vision model may further include a text identification network for identifying text sequences contained in the central control panel in the image to more accurately identify the control component. After the features are extracted, the image features of the input image are matched with image features of reference control components in samples stored in the model. Image areas where the feature matching results satisfy a predetermined similarity condition can be output as the control component predicted by the model. For example, annotated samples for training the computer vision model may be annotated with N different types of reference control components. Supervised training may be performed to store the image features of the N reference control components, and similarity matching of the image features may be performed to output a confidence that the features in a particular area of ​​the image belong to the N reference control components. A control component prediction result may then be obtained based on a predetermined confidence threshold.

[0049] In some examples, the computer vision model can extract features and then use a classifier to predict the control component. In such examples, the model can be supervised trained using samples annotated with the reference control component category (which may be binary categorical information or multivariate categorical information), and the multi-class classifier, after parameter adjustment, can fit a linear or nonlinear classification function to output a classification result that can indicate whether an image area corresponds to a reference control component.

[0050] In some examples, the computer vision model can utilize a two-stage area prediction and category prediction algorithm to determine the area where the control components of the central control panel are located in the image and the category of the control components, and can also utilize an end-to-end object detection algorithm to output area and category information of the control components, although this disclosure is not limited thereto.

[0051] In some examples, the central control panel may include multiple control components, and the areas where the multiple control components are located in the image may be discrete. In such cases, the identification result of at least one control component in the image may include multiple controllable areas, and the identification result of the controllable areas may be output in the form of, for example, an identification frame, a highlight mask, an identification information label, a display area enlargement, etc., so that the user can accurately interact with the controllable areas.

[0052] In some embodiments, in response to detecting a first interactive operation on a first target control component of the at least one control component in the image, determining operation information associated with the first target control component S103 includes determining an operation type for the first target control component, where the operation type includes at least one of click, touch, and rotate, so that the operation type can be used to more accurately represent the user's interactive operation on the control component, and enable the vehicle's central control system to accurately determine pending control commands.

[0053] In some examples, the user can directly perform an operation on a first target control component in the image, such as a mechanical click (in the case of a mechanical button), a mechanical rotation (in the case of a mechanical knob), a touch tap, a touch slide, or a touch rotation (in the case of a touch screen). In other examples, after completing the identification of the control component, the user terminal can directly present pre-set operation buttons around the control component, for example, a left rotation button and a right rotation button can be presented near a mechanical knob, so that the user can click the left rotation button or the right rotation button to convey the operation intention, and the user terminal can more directly and accurately determine the operation information of the rotation operation on the mechanical knob.

[0054] In some examples, the operation information may further include more content, for example, if the control component is a touch screen, the operation information may further include information such as the user's touch coordinates on the touch screen and the strength of the touch, and if the control component is a mechanical knob, the operation information may further include information such as the user's rotation direction and rotation angle, thereby more accurately and comprehensively representing the user's interactive operation information.

[0055] By applying the above method 100, a person other than the vehicle driver can realize control of the vehicle's central control system at a location far away from the vehicle's central control system, but the control of the vehicle's central control system by another person may affect the safety of the vehicle.

[0056] Based on this, in some embodiments, the method 100 for controlling the central control system of the vehicle further includes the step of sending an authorization request to the central control system in response to detecting a second interactive operation on a second target control component among the at least one control component in the image, and the step of determining operation information associated with the first target control component in response to detecting a first interactive operation on a first target control component among the at least one control component in the image includes determining the operation information in response to receiving authorization information returned from the central control system of the vehicle. This allows the user terminal to obtain authorization from the central control system of the vehicle before controlling the central control system of the vehicle, thereby ensuring safety.

[0057] In some examples, the steps of requesting authorization and receiving authorization can be selectively implemented, for example, the user terminal can be preset to a mode in which it can control the central control system of the vehicle without obtaining authorization information, or the user terminal can be preset to a mode in which it cannot control the central control system of the vehicle without obtaining authorization information.

[0058] In actual application scenarios, a vehicle's central control system may include various types of control components, and depending on the control component, the degree of impact of its corresponding functional module on vehicle safety may vary. For example, an electronic shift component may have a direct impact on the vehicle's driving safety, a navigation component may affect the driver's driving operation, and an air conditioning setting component has a small impact on vehicle safety.

[0059] Based on this, in some embodiments, the step of sending an authorization request to the vehicle central control system in response to detecting a second interactive operation on a second target control component among the at least one control component in the image includes determining an in-vehicle function module associated with the second target control component in the vehicle and sending an authorization request for the in-vehicle function module to the vehicle central control system, and the step S103 of determining operation information associated with the first target control component in response to detecting a first interactive operation on a first target control component among the at least one control component in the image includes determining the operation information in response to receiving authorization information for the in-vehicle function module returned from the vehicle central control system and in response to determining that the first target control component is associated with the in-vehicle function module. When the control components of the central control panel correspond to different in-vehicle function modules, performing the above steps can obtain vehicle authorization for a specific in-vehicle function module, thereby allowing the terminal to control only the module for which authorization has been obtained, further improving flexibility and safety.

[0060] In some examples, the in-vehicle function modules may include an audio / video entertainment module, a navigation module, an air conditioning setting module, etc. The pending control commands may be of various types, such as a music playback command corresponding to the audio / video entertainment module, a navigation stop command corresponding to the navigation module, and an air conditioning start command corresponding to the air conditioning setting module. Each in-vehicle function module may correspond to one or more control components. For example, an air conditioning setting module may correspond to a mechanical button for starting / stopping the air conditioning, a mechanical button for cooling / heating, and a mechanical knob for adjusting the fan speed. It can be understood that the division of in-vehicle function modules corresponds to the division of the area authority of the control components. If the control authorities of multiple control components correspond to the same authorization request, these control components should be divided into the same function module, so that the user terminal and the vehicle's central control system can transmit the authorization request based on unique control component-to-in-vehicle function module mapping information.

[0061] As described above, the central control panel may include multiple control components located in multiple controllable areas. In such a case, the controllable areas on the central control panel may be divided based on the in-vehicle function modules associated with the control components. The user terminal's identification result of the control components in the image of the central control panel may include multiple function areas corresponding to different in-vehicle function modules, respectively.

[0062] In the above example, when using a computer vision model to identify multiple functional areas containing control components in a central control panel, the annotation information of the sample data for model training may include a reference functional area, thereby enabling the computer vision model to achieve efficient and accurate identification.

[0063] In some examples, the first target control component and the second target control component may be the same, and the first interactive operation and the second interactive operation may be the same. In such cases, in response to detecting a user's interactive operation with the target control component, an authorization request may be sent to a central control system of the vehicle, and after obtaining authorization information, operation information for the target control component may be determined based on the interactive operation.

[0064] In another example, the first target control component and the second target control component may be the same, and the first interactive operation and the second interactive operation may be different. In such a case, in response to detecting a first interactive operation by the user with respect to the target control component, an authorization request may be sent to a central control system of the vehicle, and after obtaining authorization information, in response to detecting a second interactive operation by the user with respect to the target control component, operation information for the target control component may be determined based on the second interactive operation.

[0065] In a further example, the first target control component and the second target control component may be different, and therefore the first interactive operation and the second interactive operation may also be different. In such a case, in response to detecting a first interactive operation by the user on the first target control component in functional area A, an authorization request for functional area A may be sent to the central control system of the vehicle, and after obtaining authorization information, in response to detecting a second interactive operation by the user on the second target control component in functional area A, operation information for the second target control component may be determined based on the second interactive operation. If the second target control component does not belong to functional area A, the central control system of the vehicle cannot be controlled based on the authorization information for functional area A.

[0066] 2 shows a flowchart of a method 200 for controlling a central control system of a vehicle according to another embodiment of the present disclosure. As shown in FIG. 2, the method 200 includes steps S201 to S205.

[0067] In step S201, an image including the central control panel is acquired.

[0068] In step S202, at least one control component on the central control panel in the image is identified.

[0069] In step S203, in response to detecting a second interactive operation on a second target control component of the at least one control component in the image, an authorization request is sent to the central control system.

[0070] In step S204, in response to receiving authorization information returned from the central control system and in response to detecting a first interactive operation on a first target control component among the at least one control component in the image, operation information associated with the first target control component is determined.

[0071] In step S205, the operation information is sent to the central control system so that the central control system can determine the control commands waiting to be executed based on the operation information.

[0072] By applying the above method 200, a user terminal can be used to acquire an image including a vehicle central control panel and perform image identification to obtain at least one control component included in the central control panel in the image. When an interactive operation performed by a user on a control component in the image is detected, an authorization request can be sent to the vehicle central control system. After obtaining authorization, the user's operation information can be transmitted to the vehicle central control system. Thus, the vehicle central control system can determine a waiting control command based on the user's operation information, and easily realize user control over the vehicle central control system.

[0073] In some embodiments of the present disclosure, a control method is provided for application to a central control system of a vehicle, the central control system including a central control panel.

[0074] 3 shows a flowchart of a control method 300 applied to a central control system of a vehicle according to an embodiment of the present disclosure. As shown in FIG. 3, the method 300 includes step S301.

[0075] In step S301, in response to receiving operation information transmitted by a user terminal communicatively connected to the vehicle's central control system, a waiting control command is determined based on the operation information. The operation information in step S301 is generated by the user terminal as follows: acquiring an image including a central control panel, determining at least one control component on the central control panel in the image, and determining operation information associated with the first target control component in response to detecting a first interactive operation on a first target control component among the at least one control component in the image.

[0076] By applying the above method 300, the vehicle central control system can determine the waiting control command based on the user's interactive operation information on the image of the central control panel, that is, the user's interactive operation on the virtual central control panel (the central control panel in the image) on the user terminal is mapped to the real central control panel of the central control system, thereby easily realizing the user's control of the vehicle central control system. In this implementation, even if the passengers in the vehicle are seated far away from the central control panel, they can easily control the vehicle central control system using the user terminal, and the vehicle driver does not need to operate the central control panel, thereby improving vehicle driving safety.

[0077] In some embodiments, the operation information includes an operation type of the first interactive operation, and the step of determining a waiting control command based on the operation information includes determining an in-vehicle function module corresponding to the first target control component, and determining a control command for the in-vehicle function module based on the operation type of the first interactive operation on the first target control component. This makes it possible to simply and efficiently determine the waiting command based on the mapping relationship between the control component and the in-vehicle function module and the mapping relationship between the operation type and the control command.

[0078] In some embodiments, determining a control command for the vehicle function module based on an operation type of the first interactive operation on the first target control component includes determining the control command from an instruction table corresponding to the vehicle function module based on the operation type, where the instruction table stores mapping relationships between a plurality of pre-defined operation types and a plurality of pre-defined control commands, thereby enabling the waiting control command to be determined simply and efficiently by looking up the table.

[0079] In some embodiments, the control method applied to the central control system of the vehicle further includes receiving an authorization request sent by a user terminal, generating notification information for presenting the authorization request, generating authorization information in response to detecting an authorization interactive operation on the notification information, and returning the authorization information to the user terminal, thereby determining the authorization information for the user terminal through the authorization interactive operation of the driver or a specific passenger, and ensuring safety.

[0080] In some examples, the notification information for presenting the authorization request may be a notification of the authorization request presented in the form of text on a display screen, and authorization and non-authorization interaction buttons may be simultaneously presented to the driver or a specific passenger. In such a case, the authorization interactive operation may be an operation in which the driver or a specific passenger clicks the authorization interaction button.

[0081] In another example, the notification information for presenting the authorization request may be voice announcement information, and in such a case, the authorization interactive operation may be an operation in which the driver or a specific passenger issues a voice command for authorization.

[0082] 4 shows a flowchart of an interaction process for controlling a central control system of a vehicle according to an embodiment of the present disclosure. As shown in FIG. 4, the interaction process for controlling a central control system of a vehicle includes steps S1 to S9.

[0083] In step S1, the user terminal acquires an image including a central control panel.

[0084] In step S2, the user terminal identifies at least one control component on the central control panel in the image.

[0085] In step S3, the user terminal transmits an authorization request to the central control system in response to detecting a second interactive operation on a second target control component of the at least one control component in the image.

[0086] In step S4, the central control system generates notification information for submitting an authorization request.

[0087] In step S5, the central control system generates authorization information in response to detecting an authorized interaction with the notification information.

[0088] In step S6, the central control system returns authorization information to the user terminal.

[0089] In step S7, the user terminal determines operation information associated with the first target control component in response to detecting a first interactive operation on a first target control component among the at least one control component in the image.

[0090] In step S8, the user terminal transmits the operation information to the central control system.

[0091] In step S9, the central control system determines a control command waiting to be executed based on the operation information.

[0092] By applying the above interaction flow, an image including the vehicle's central control panel can be acquired using a user terminal, and image recognition can be performed to obtain at least one control component included in the central control panel in the image. When an interactive operation performed by a user on a control component in the image is detected, an authorization request can be sent to the vehicle's central control system. After obtaining authorization, the user's operation information can be transmitted to the vehicle's central control system. This allows the vehicle's central control system to determine a control command waiting to be executed based on the user's operation information, thereby easily realizing user control over the vehicle's central control system.

[0093] In some embodiments of the present disclosure, an apparatus for controlling a central control system of a vehicle is provided. Fig. 5 is a block diagram of an apparatus 500 for controlling a central control system of a vehicle according to an embodiment of the present disclosure, where the central control system includes a central control panel, and the apparatus 500 is applied to a user terminal that is communicatively connected to the central control system. As shown in Fig. 5, the apparatus 500 includes an acquiring unit 501 to a transmitting unit 504.

[0094] The acquisition unit 501 is configured to acquire an image including the central control panel.

[0095] The first determining unit 502 is configured to identify at least one control component on the central control panel in the image.

[0096] The second determination unit 503 is configured to determine operation information associated with the first target control component in response to detecting a first interactive operation on the first target control component of the at least one control component in the image.

[0097] The sending unit 504 is configured to send the operation information to the central control system so that the central control system can determine the pending control instructions based on the operation information.

[0098] According to some embodiments, the first determining unit 502 is configured to input the image into a computer vision model to obtain at least one control component output by the computer vision model, the computer vision model being obtained by training using sample images annotated with reference control components.

[0099] According to some embodiments, the computer vision model identifies the control area in the image by determining image features of the image and determining at least one control component based on the image features, wherein the image features corresponding to the at least one control component and the image features corresponding to a reference control component in the sample image can satisfy a pre-set similarity condition.

[0100] According to some embodiments, the second determining unit 503 is configured to determine an operation type for the first target control component, where the operation type includes at least one of a click, a touch, and a rotation.

[0101] According to some embodiments, the sending unit 504 is further configured to send an authorization request to the central control system in response to detecting a second interactive operation on a second target control component of the at least one control component in the image, and the second determining unit 503 is configured to determine the operation information in response to receiving the authorization information returned from the central control system of the vehicle.

[0102] According to some embodiments, the sending unit 504 is configured to determine an in-vehicle function module associated with a second target control component in the vehicle and send an authorization request for the in-vehicle function module to the vehicle's central control system, and the second determination unit 503 is configured to determine operation information in response to receiving authorization information for the in-vehicle function module returned from the vehicle's central control system and in response to determining that the first target control component is associated with the in-vehicle function module.

[0103] It should be understood that the operations of each unit of the apparatus 500 for controlling a central control system of a vehicle shown in Figure 5 can correspond to each step of the method 100 for controlling a central control system of a vehicle described in Figure 1. Accordingly, the operations, features, and advantages described above for the method 100 also apply to the apparatus 500 and each unit included therein. For the sake of brevity, the description of some operations, features, and advantages will not be repeated here.

[0104] In some embodiments of the present disclosure, an apparatus adapted to a central control system of a vehicle is provided. Figure 6 is a block diagram of an apparatus 600 adapted to a central control system of a vehicle according to an embodiment of the present disclosure, where the central control system includes a central control panel. As shown in Figure 6, the apparatus 600 includes a third determination unit 601.

[0105] The third determination unit 601 is configured to, in response to receiving operation information transmitted by a user terminal communicatively connected to the vehicle's central control system, determine a control command waiting to be executed based on the operation information, where the operation information is determined by the user terminal using the acquisition unit 501, the first determination unit 502, and the second determination unit 503.

[0106] According to some embodiments, the operation information includes an operation type of the first interactive operation, and the third determination unit 601 is configured to determine an in-vehicle function module corresponding to the first target control component, and determine a control command for the in-vehicle function module based on the operation type for the first target control component.

[0107] According to some embodiments, the third determining unit 601 is configured to determine a control command from an command table corresponding to the vehicle function module based on an operation type, where the command table stores mapping relationships between a plurality of pre-defined operation types and a plurality of pre-defined control commands.

[0108] According to some embodiments, the apparatus 600 further includes an authorization unit configured to receive an authorization request sent by the user terminal, generate notification information for presenting the authorization request, and in response to detecting an authorization interactive operation on the notification information, generate authorization information and return the authorization information to the user terminal.

[0109] It should be understood that the operations of each unit of the apparatus 600 applied to the vehicle central control system shown in Figure 6 can correspond to each step of the method 300 applied to the vehicle central control system described in Figure 3. Thus, the operations, features, and advantages described above for the method 300 also apply to the apparatus 600 and each unit included therein. For the sake of brevity, the description of some operations, features, and advantages will be omitted here.

[0110] In some embodiments of the present disclosure, a computing device is provided for performing the method 100 for controlling a central control system of a vehicle as described above.

[0111] In some embodiments of the present disclosure, a computer device is provided for executing the control method 300 applied to the central control system of the vehicle described above.

[0112] Referring to FIG. 7 , computing device 7000 may include elements connected to or in communication with bus 7002, possibly via one or more interfaces. For example, computing device 7000 may include bus 7002, one or more processors 7004, one or more input devices 7006, and one or more output devices 7008. The one or more processors 7004 may be any type of processor, including, but not limited to, one or more general-purpose processors and / or one or more special-purpose processors (e.g., specialized processing chips). Processor 7004 may process instructions executed within computing device 7000, including instructions stored in or on a memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to an interface). In other embodiments, multiple processors and / or multiple buses and multiple memories may be used together, as desired. Similarly, multiple computing devices can be connected, each providing some required operation, such as in a server array, a group of blade servers, or a multi-processor system. Figure 7 illustrates one processor 7004.

[0113] Input device 7006 may be any type of device capable of inputting information into computing device 7000. Input device 7006 may receive input numeric or textual information and, in the event, generate key signal input related to user settings and / or function control of the computing device, and may include, but is not limited to, a mouse, keyboard, touchscreen, trackpad, trackball, joystick, microphone, and / or remote controller. Output device 7008 may be any type of device capable of displaying information, and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer.

[0114] The computing device 7000 may further include or be connected to a non-transitory storage device 7010, which may be any storage device that is non-transitory and capable of storing data, including, but not limited to, a disk drive, an optical storage device, a solid-state memory, a floppy disk, a flexible disk, a hard disk, a magnetic tape or any other magnetic medium, an optical disk or any other optical medium, a ROM (read-only memory), a RAM (random access memory), a cache memory and / or any other memory chip or cartridge, and / or any other medium from which a computer can read data, instructions, and / or code. The non-transitory storage device 7010 may be detachable from the interface. The non-transitory storage device 7010 may contain data, programs (including instructions), codes, or modules for implementing the above-described methods and steps.

[0115] The computing device 7000 may further include a communications device 7012. The communications device 7012 may be any type of device or system that enables communication with external devices and / or networks, and may include, but is not limited to, a modem, a network card, an infrared communications device, a wireless communications device and / or chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communications device, and / or the like.

[0116] The computing device 7000 may further include a working memory 7014, which may be any type of working memory capable of storing programs (including instructions) and / or data useful to the operation of the processor 7004, and may include, but is not limited to, random access memory and / or read-only memory devices.

[0117] Software elements (programs) may be located in the working memory 7014 and include, but are not limited to, an operating system 7016, one or more application programs 7018, driver programs, and / or other data and code. Instructions for carrying out the above methods and steps may be included in one or more application programs 7018, and the above methods may be realized by the processor 7004 reading and executing instructions of the one or more application programs 7018. Executable code or source code for instructions of the software elements (programs) may be downloaded from a remote location.

[0118] It should also be understood that various modifications can be made depending on specific needs. For example, customized hardware can be used and / or particular elements can be implemented in hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. For example, some or all of the disclosed methods and apparatus can be implemented by programming hardware (e.g., programmable logic circuitry including field programmable gate arrays (FPGAs) and / or programmable logic arrays (PLAs)) in assembly language or hardware programming languages ​​(e.g., VERILOG, VHDL, C++) using the logic and algorithms of the present disclosure.

[0119] It should also be understood that the above method can be implemented in a server-client mode. For example, the client can receive data entered by a user and send the data to the server. The client can receive the data entered by the user, process part of the above method, and send the processed data to the server. The server can receive the data from the client, execute one or more of the above method, and return the execution results to the client. The client can receive the execution results from the server and present them to the user, for example, via an output device. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on corresponding computing devices and having a client-server relationship with each other. The server may be a server in a distributed system or a server incorporating blockchain technology. The server may be a cloud server, an intelligent cloud computing server equipped with artificial intelligence technology, or an intelligent cloud host. A cloud server is a type of host product in a cloud computing service system that addresses the drawbacks of traditional physical hosts and virtual private server (VPS) services, such as difficult management and poor scalability.

[0120] It should also be understood that components of computer device 7000 may be distributed across a network. For example, some processing may be performed using one processor while other processing is performed by another processor that is remote from the one processor. Other components of computer device 7000 may be similarly distributed. In this manner, computer device 7000 may be considered a distributed computing system in which processing is performed at multiple locations.

[0121] In some embodiments of the present disclosure, a vehicle is provided that includes a central control system that includes a central control panel and the above-described computing device capable of executing the method 100 for controlling the central control system of the vehicle.

[0122] In some embodiments of the present disclosure, a computer-readable storage medium is provided having stored thereon instructions that, when executed alone or in concert by one or more processors of a computing device, cause the computing device to perform the above method 100 or the above method 300.

[0123] A computer-readable medium may be a tangible medium that contains or can store a program used by or in connection with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination thereof. More specific examples of machine-readable storage media include an electrical connection using one or more cables, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0124] In some embodiments of the present disclosure, a computer program product is provided that includes instructions that, when executed alone or jointly by one or more processors of a computing device, cause the computing device to perform the above method 100 or the above method 300.

[0125] Finally, it should be noted that the above examples are merely intended to illustrate the technical aspects of the present disclosure and are not intended to limit the same. Although the present disclosure has been described in detail with reference to the above examples, it should be understood by those skilled in the art that the technical aspects described in the above examples can be modified or some or all of the technical features can be equivalently replaced. These modifications or replacements do not deviate from the essence of the corresponding technical aspects of the technical aspects of the examples of the present disclosure, and all are intended to be included in the scope of the claims and specification of the present disclosure. In particular, as long as there is no structural contradiction, the technical features mentioned in the examples can be combined in any manner. The present disclosure is not limited to the specific examples disclosed herein, but includes all technical aspects within the scope of the claims.

Claims

1. 1. A method for controlling a central control system of a vehicle, comprising: the central control system includes a central control panel, and the method is applied to a user terminal in communication with the central control system; acquiring an image including the central control panel; identifying at least one control component on the central control panel in the image; determining operation information associated with a first target control component of the at least one control component in response to detecting a first interactive operation on the first target control component in the image; sending the operation information to the central control system so that the central control system can determine a pending control command based on the operation information; A method comprising:

2. The step of identifying at least one control component on the central control panel in the image comprises:

2. The method of claim 1, comprising inputting the image into a computer vision model to obtain the at least one control component output by the computer vision model, the computer vision model being obtained by training using a sample image including a reference control component.

3. The computer vision model identifying a control area in the image may include: determining image characteristics of the image; determining the at least one control component based on the image features, wherein the image features corresponding to the at least one control component and the image features corresponding to a reference control component in the sample image can satisfy a predetermined similarity condition; The method according to claim 2 .

4. In response to detecting a first interactive operation on a first target control component of the at least one control component in the image, the step of determining operation information associated with the first target control component includes: The method of any one of claims 1 to 3, further comprising determining an operation type for the first target control component, the operation type including at least one of a click, a touch, and a rotation.

5. and transmitting an authorization request to the central control system in response to detecting a second interactive operation on a second target control component of the at least one control component in the image; In response to detecting a first interactive operation on a first target control component of the at least one control component in the image, the step of determining operation information associated with the first target control component includes: A method according to any preceding claim, comprising determining the operational information in response to receiving authorization information returned from a central control system of the vehicle.

6. the step of transmitting an authorization request to the central control system in response to detecting a second interactive operation on a second target control component of the at least one control component in the image, further comprising: determining an in-vehicle function module associated with the second target control component in the vehicle; sending an authorization request to the vehicle function module to a central control system of the vehicle; In response to detecting a first interactive operation on a first target control component of the at least one control component in the image, the step of determining operation information associated with the first target control component includes:

6. The method of claim 5, comprising determining the operation information in response to receiving authorization information for the vehicle function module returned from a central control system of the vehicle and in response to determining that the first target control component is associated with the vehicle function module.

7. 1. A method applied to a central control system of a vehicle, comprising: The central control system includes a central control panel, and the method comprises: and determining a waiting control command based on the operation information in response to receiving operation information transmitted by a user terminal communicatively connected to the central control system of the vehicle, the operation information being transmitted by the user terminal, acquiring an image including the central control panel; determining at least one control component on the central control panel in the image; In response to detecting a first interactive operation on a first target control component of the at least one control component in the image, determining operation information associated with the first target control component; It is generated as follows: method.

8. the operation information includes an operation type of the first interactive operation, and the step of determining a waiting control command based on the operation information includes: determining an in-vehicle function module corresponding to the first target control component; and determining a control command for the vehicle function module based on the operation type of the first interactive operation on the first target control component.

9. determining a control command for the vehicle function module based on the operation type of the first interactive operation on the first target control component; 9. The method of claim 8, further comprising: determining the control command from an command table corresponding to the vehicle function module based on the operation type, wherein the command table stores mapping relationships between a plurality of preset operation types and a plurality of preset control commands.

10. receiving an authorization request sent by a user terminal; generating notification information for presenting the authorization request; generating authorization information in response to detecting an authorized interactive operation on the notification information; The method according to any one of claims 7 to 9, further comprising the step of returning said authorization information to said user terminal.

11. 1. An apparatus for controlling a central control system of a vehicle, comprising: the central control system includes a central control panel, and the device is adapted for use in a user terminal communicatively connected to the central control system; an acquisition unit configured to acquire an image including the central control panel; a first determination unit configured to identify at least one control component on the central control panel in the image; a second determination unit configured to determine operation information associated with a first target control component of the at least one control component in response to detecting a first interactive operation on the first target control component in the image; and a sending unit configured to send the operation information to the central control system so that the central control system can determine a pending control command based on the operation information; 1. An apparatus comprising:

12. A device applied to a central control system of a vehicle, The central control system includes a central control panel, and the device includes: a third determination unit configured to, in response to receiving operation information transmitted by a user terminal in communication with the central control system of the vehicle, determine a waiting-to-execute control command based on the operation information, wherein the operation information is transmitted by the user terminal: an acquisition unit configured to acquire an image including the central control panel; a first determination unit configured to determine at least one control component on the control panel in the image; a second determination unit configured to determine operation information associated with a first target control component of the at least one control component in response to detecting a first interactive operation on the first target control component in the image; and It is generated using Device.

13. A computer device comprising: one or more processors; a memory on which are stored instructions which, when executed alone or jointly by said one or more processors, cause said computing device to carry out the method of any one of claims 1 to 6; including computer equipment.

14. A computer device comprising: one or more processors; a memory on which are stored instructions which, when executed alone or jointly by said one or more processors, cause said computing device to carry out the method of any one of claims 7 to 10; including computer equipment.

15. a central control system including a central control panel; A computer device according to claim 14; Vehicles including.

16. A computer-readable storage medium having stored thereon instructions which, when executed alone or in concert by one or more processors of a computing device, cause said computing device to carry out the method of any one of claims 1 to 10.

17. A computer program product comprising instructions which, when executed alone or in concert by one or more processors of a computing device, cause said computing device to carry out the method of any one of claims 1 to 10.

Images