Method and device for controlling central control system of vehicle, vehicle and storage medium

Abstract

The invention provides a method for controlling a central control system of a vehicle, a control method and device applied to the central control system of the vehicle, computer equipment, the vehicle, a computer readable storage medium and a computer program product, and relates to the technical field of central control of the vehicle. The method for controlling the central control system of the vehicle comprises the steps that an image including a central control panel of the central control system of the vehicle is obtained; identifying at least one control component on the central control panel in the image; in response to detection of a first interaction operation for a first target control component in the at least one control component in the image, determining operation information associated with the first target control component; and sending the operation information to a central control system of the vehicle, so that the central control system of the vehicle can determine a to-be-executed control instruction based on the operation information.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of center control of vehicles, and in particular to a method for controlling a center control system of a vehicle, a control method and device applied to a center control system of a vehicle, a computer device, a vehicle, a computer readable storage medium and a computer program product. BACKGROUND

[0002] With the development of various technologies in the technical field of vehicles, such as automatic driving, intelligent auxiliary driving, intelligent cockpit, and vehicle intelligent networking, the functions of the center control system of a vehicle are more diverse, and can provide various different functional modules to the occupants, such as vehicle-mounted audio and video entertainment, vehicle navigation, and vehicle control air conditioning settings.

[0003] A center control panel that can be operated by the driver and passengers is usually provided in a vehicle to enable the driver and passengers to control various vehicle functions. The interaction between the center control panel and the driver and passengers can be achieved through various types of control components, such as mechanical buttons and touch screens. However, for the driver of the vehicle, using screen or button control components during driving will cause distraction, which will affect the driving safety of the vehicle. For passengers in the vehicle, operating the center control panel in the vehicle relies on the installation position of the mechanical buttons and touch screens in the vehicle, which has certain limitations in flexibility and convenience.

[0004] The methods described in this section can not necessarily be the methods previously conceived or used. Unless otherwise indicated, nothing in this section should be assumed to be prior art merely because of its inclusion in this section. Similarly, issues mentioned in this section should not be assumed to have been admitted to be prior art in any jurisdiction unless otherwise indicated. SUMMARY

[0005] The present disclosure aims to at least solve one of the technical problems existing in the background art. To this end, one object of the present disclosure is to provide a method for controlling a center control system of a vehicle, which acquires an image of a center control panel included in the center control system of the vehicle using a user terminal and identifies control components therein, and then transmits operation information to the center control system of the vehicle by detecting the interactive operation of the user with respect to the control components in the image, thereby conveniently controlling the center control system of the vehicle. By applying the above method, the user is no longer limited by the physical contact distance with the control components when controlling the center control system, and can conveniently control the center control panel from a seat position far away from the center control panel, thereby improving the problem of relying on the driver of the vehicle to operate the center control panel, which affects the driving safety of the vehicle in the related art.

[0006] The embodiments of the first aspect of the present disclosure provide a method for controlling a central control system of a vehicle, the central control system comprising a central control panel, the method being applied to a user terminal in communication connection with the central control system, the method comprising: acquiring an image comprising the central control panel; identifying at least one control component on the central control panel in the image; in response to detecting a first interaction operation on a first target control component in the at least one control component in the image, determining operation information associated with the first target control component; and sending the operation information to the central control system to enable the central control system to determine a control instruction to be executed based on the operation information.

[0007] In the technical solution of the embodiments of the present disclosure, a user terminal is used to acquire an image comprising a central control panel of a vehicle and perform image recognition to obtain at least one control component included in the central control panel in the image, and then operation information can be transmitted to the central control system of the vehicle by detecting an interaction operation of a user on a target control component in the image, so that the central control system of the vehicle can determine a control instruction to be executed based on the operation information of the user, thereby conveniently realizing control of the central control system of the vehicle by the user. By applying the above method, even if the distance between the occupant of the vehicle and the central control panel is far, the central control system of the vehicle can be conveniently controlled by the user terminal, without the need for the driver of the vehicle to operate the central control panel, thereby improving the driving safety of the vehicle.

[0008] In some embodiments, identifying the at least one control component on the central control panel in the image comprises: inputting the image into a computer vision model to obtain the at least one control component output by the computer vision model, wherein the computer vision model is obtained by training using sample images comprising reference control components.

[0009] In some embodiments, the computer vision model identifies the at least one control component in the image based on the following process: determining image features of the image; and based on the image features, determining the at least one control component, wherein the image features corresponding to the at least one control component and the image features corresponding to the reference control components in the sample images satisfy a preset similarity condition.

[0010] In some embodiments, in response to detecting the first interaction operation on the first target control component in the at least one control component in the image, determining the operation information associated with the first target control component comprises: determining an operation type for the first target control component, wherein the operation type comprises at least one of clicking, touching, and rotating.

[0011] In some embodiments, the method for controlling the center control system of the vehicle further includes: in response to detecting the second interaction operation on the second target control component of the at least one control component in the image, sending an authorization request to the center control system, wherein the determining the operation information associated with the first target control component in response to detecting the first interaction operation on the first target control component of the at least one control component in the image includes: in response to receiving the authorization information returned by the center control system of the vehicle, determining the operation information.

[0012] In some embodiments, the sending the authorization request to the center control system of the vehicle in response to detecting the second interaction operation on the second target control component of the at least one control component in the image includes: determining a vehicle function module associated with the second target control component in the vehicle; sending the authorization request for the vehicle function module to the center control system of the vehicle, and wherein the determining the operation information associated with the first target control component in response to detecting the first interaction operation on the first target control component of the at least one control component in the image includes: in response to receiving the authorization information for the vehicle function module returned by the center control system of the vehicle, and in response to determining that the first target control component is associated with the vehicle function module, determining the operation information.

[0013] Embodiments of the second aspect of the present disclosure provide a control method applied to a center control system of a vehicle, the center control system including a center control panel, the method including: in response to receiving operation information sent by a user terminal in communication connection with the center control system of the vehicle, determining a control instruction to be executed based on the operation information, wherein the operation information is generated by the user terminal through the following process: acquiring an image including the center control panel; determining at least one control component on the center control panel in the image; and in response to detecting a first interaction operation on a first target control component of the at least one control component in the image, determining operation information associated with the first target control component.

[0014] In some embodiments, the operation information includes an operation type of the first interaction operation, and the determining the control instruction to be executed based on the operation information includes: determining a vehicle function module corresponding to the first target control component; and determining a control instruction for the vehicle function module based on the operation type of the first target control component.

[0015] In some embodiments, the determining the control instruction for the vehicle function module based on the operation type of the first target control component includes: based on the operation type, determining the control instruction from an instruction table corresponding to the vehicle function module, wherein the instruction table stores a mapping relationship between a plurality of preset operation types and a plurality of preset control instructions.

[0016] In some embodiments, the control method applied to the central control system of the vehicle further includes: receiving an authorization request sent by the user terminal; generating notification information for displaying the authorization request; in response to detecting an authorization interaction operation for the notification information, generating authorization information; and returning the authorization information to the user terminal.

[0017] Embodiments of the third aspect of the present disclosure provide a device for controlling a central control system of a vehicle, the central control system comprising a central control panel, the device being applied to a user terminal in communication connection with the central control system, the device comprising: an acquisition unit configured to acquire an image comprising the central control panel; a first determination unit configured to identify at least one control component on the central control panel in the image; a second determination unit configured to determine operation information associated with a first target control component in the at least one control component in the image in response to detecting a first interaction operation for the first target control component; and a sending unit configured to send the operation information to the central control system to enable the central control system to determine a control instruction to be executed based on the operation information.

[0018] Embodiments of the fourth aspect of the present disclosure provide a device applied to a central control system of a vehicle, the central control system comprising a central control panel, the device comprising: a third determination unit configured to determine a control instruction to be executed based on operation information in response to receiving the operation information sent by a user terminal in communication connection with the central control system of the vehicle, wherein the operation information is generated by the user terminal using: an acquisition unit configured to acquire an image comprising the central control panel; a first determination unit configured to determine at least one control component on the control panel in the image; and a second determination unit configured to determine operation information associated with a first target control component in the at least one control component in the image in response to detecting a first interaction operation for the first target control component.

[0019] Embodiments of the fifth aspect of the present disclosure provide a computer device for executing the method for controlling a central control system of a vehicle in the first aspect of the present disclosure.

[0020] Embodiments of the sixth aspect of the present disclosure provide a computer device for executing the control method applied to the central control system of the vehicle in the second aspect of the present disclosure.

[0021] Embodiments of the seventh aspect of the present disclosure provide a vehicle comprising a central control system comprising a central control panel, and the computer device in the sixth aspect of the present disclosure.

[0022] The embodiment of the eighth aspect of the present disclosure provides a computer readable storage medium, which stores instructions. When the instructions are executed by one or more processors of a computer device alone or jointly, the computer device executes the method in the first aspect of the present disclosure or the method in the second aspect of the present disclosure.

[0023] The embodiment of the ninth aspect of the present disclosure provides a computer program product, which comprises instructions. When the instructions are executed by one or more processors of a computer device alone or jointly, the computer device executes the method in the first aspect of the present disclosure or the method in the second aspect of the present disclosure.

[0024] The above description is only a summary of the technical solutions of the present disclosure. In order to enable one skilled in the art to better understand the technical means of the present disclosure, the following specific embodiments of the present disclosure are implemented according to the content of the description, and in order to make the above and other purposes, characteristics and advantages of the present disclosure more obvious and easy to understand, the following specific embodiments of the present disclosure are implemented. BRIEF DESCRIPTION OF DRAWINGS

[0025] In the drawings, like reference numerals refer to same or similar functionalities throughout the several views. The drawings are not necessarily to scale. It is to be understood that these drawings only depict some embodiments in accordance with the disclosure and should not be considered as limiting the scope of the disclosure.

[0026] Figure 1 Flowchart of the method for controlling the central control system of the vehicle according to an embodiment of the present disclosure;

[0027] Figure 2 Flowchart of the method for controlling the central control system of the vehicle according to another embodiment of the present disclosure;

[0028] Figure 3 Flowchart of the control method applied to the central control system of the vehicle according to an embodiment of the present disclosure;

[0029] Figure 4 Flowchart of the interaction process for controlling the central control system of the vehicle according to an embodiment of the present disclosure;

[0030] Figure 5 Structural block diagram of the device for controlling the central control system of the vehicle according to an embodiment of the present disclosure;

[0031] Figure 6 Structural block diagram of the device applied to the central control system of the vehicle according to an embodiment of the present disclosure;

[0032] Figure 7 Structural block diagram of an exemplary computer device according to the present disclosure, which can be applied to the exemplary embodiments of the present disclosure. DETAILED DESCRIPTION

[0033] Exemplary embodiments of the present disclosure are described herein below with reference to the accompanying drawings, in which various details are set forth to facilitate an understanding of the present disclosure. It should be apparent to those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present disclosure. Also, descriptions of well-known functions and constructions are omitted for clarity and conciseness.

[0034] In the present disclosure, the terms "first", "second", and the like are used to describe various elements only for the purpose of distinguishing one element from another, and the terms are not intended to limit the positions, sequence, or importance of the elements. In some examples, a first element and a second element can refer to the same instance of the element, and in some cases, they can refer to different instances of the element based on the context of the description.

[0035] The terms used in the description of various described examples in the present disclosure are only for the purpose of describing particular examples and are not intended to be limiting. Unless specifically defined otherwise, an element that is a singular can be plural and vice versa. Also, the term "and / or" used in the present disclosure encompasses any and all possible combinations of the listed items.

[0036] As mentioned above, the use of a screen or a button and the like as a central control component during driving by a vehicle driver can cause distraction, which in turn affects the driving safety of the vehicle. To solve this problem, one implementation in the related art is to use voice instructions to interact with the central control system. However, the voice interaction can only implement limited functions and cannot achieve comprehensive and accurate control of the vehicle central control system. Another implementation is to install application programs that can be matched with each other on the vehicle central control system and the user terminal, so that the user can set the application program (such as an audio and video entertainment program, a navigation program) on the vehicle central control system side by operating the application program on the user terminal side. However, this method requires the application programs on the vehicle central control system and the user terminal to be strictly matched, and the versatility is insufficient.

[0037] Based on this, the present disclosure provides a method for controlling a central control system of a vehicle, which causes a user terminal to acquire an image of a central control panel included in the central control system of the vehicle and to identify a control component therein, and then can detect an interactive operation of the user on the control component in the image, and by transmitting the operation information of the user to the central control system of the vehicle, the central control system can determine a control instruction to be executed based on the operation information, thereby achieving the control of the vehicle central control system by the user.

[0038] Some embodiments of the present disclosure provide a method for controlling a center control system of a vehicle, the center control system comprising a center control panel, the method being applied to a user terminal in communication connection with the center control system.

[0039] Figure 1 A method 100 for controlling a center control system of a vehicle according to an embodiment of the present disclosure is shown. As shown in Figure 1 The method 100 comprises the following steps:

[0040] Step S101, acquiring an image comprising a center control panel;

[0041] Step S102, identifying at least one control component on the center control panel in the image;

[0042] Step S103, in response to detecting a first interaction operation on a first target control component in the at least one control component in the image, determining operation information associated with the first target control component; and

[0043] Step S104, sending the operation information to the center control system to enable the center control system to determine a control instruction to be executed based on the operation information.

[0044] The method 100 according to an embodiment of the present disclosure is to acquire an image comprising a center control panel of a vehicle by using a user terminal and perform image recognition to obtain at least one control component included in the center control panel in the image, and then the operation information can be transmitted to the center control system of the vehicle by detecting the interaction operation of the user on the target control component in the image, so that the center control system of the vehicle can determine the control instruction to be executed based on the operation information of the user, thereby conveniently realizing the control of the center control system of the vehicle by the user. By applying the above-mentioned method 100 in the user terminal, even if the distance between the occupant in the vehicle and the center control panel is far, the center control system of the vehicle can be conveniently controlled by using the user terminal, without the need for the driver of the vehicle to operate the center control panel, thereby improving the driving safety of the vehicle.

[0045] In some examples, the user terminal can be various types, for example, can be a mobile phone, a tablet computer, a smart wearable device (such as a smart watch, smart glasses, etc.), as long as it can realize human-computer interaction with the user.

[0046] In some examples, the communication connection between the user terminal and the center control system of the vehicle can be realized by using various communication technologies, for example, can be a Wi-Fi connection, a Bluetooth connection, a cellular communication connection, etc.

[0047] In some examples, a specific application program can be installed on the user terminal, so that the application program can be used to realize each step in the above-mentioned method 100.

[0048] In some examples, the user terminal comprises a camera component, so that the image of the control panel can be captured by the camera component in step S101. In other examples, the user terminal can be communicatively connected with a camera device, and then the image of the control panel captured and sent by the camera device can be acquired.

[0049] In some examples, the image comprising the control panel in step S101 can be obtained based on a single frame of photo captured by the camera. In other examples, the image comprising the control panel in step S101 can be obtained based on a video captured by the camera, i.e., at least one image frame is acquired from the video comprising the control panel, and then the image of the control panel included in the image frame is recognized.

[0050] In some embodiments, the step S102 of recognizing at least one control component on the control panel in the image comprises: inputting the image into a computer vision model to obtain at least one control component output by the computer vision model, wherein the computer vision model is obtained by training a sample image comprising a reference control component. Thus, the control component on the control panel can be more efficiently and accurately recognized by using the computer vision model.

[0051] In some examples, the at least one control component on the control panel can be of various types, such as a touch screen, a mechanical button, a mechanical knob, etc.

[0052] In some examples, the computer vision model can be constructed and trained by the following process: constructing an initial computer vision model, which can be constructed based on various types of neural networks such as convolutional neural networks, spatial attention networks, etc.; inputting a sample image comprising a sample control panel into the initial computer vision model; acquiring at least one predicted control component output by the initial computer vision model; calculating a loss value based on at least one reference control component and the at least one predicted control component output by the initial model; and adjusting the parameters of the initial computer vision model based on the loss value. By using the pre-labeled sample image to supervise the training of the initial computer vision model, a computer vision model capable of efficiently and accurately recognizing at least one control component on the control panel can be obtained.

[0053] In some examples, the initial computer vision model can also be unsupervised trained based on sample images comprising reference control components, e.g., unsupervised trained based on consistency learning, so that the model training can be performed to obtain a trained computer vision model more efficiently and conveniently while saving the cost of sample labeling. It can be understood that the training of the initial computer vision model can be implemented by various deep learning algorithms, which are not limited in the present disclosure.

[0054] In some embodiments, the computer vision model is configured to identify the at least one control component in the image based on a process including: determining image features of the image; and determining the at least one control component based on the image features, wherein the image features of the at least one control component correspond to image features of a reference control component in the sample image and satisfy a preset similarity condition. In this way, more accurate image recognition can be achieved through the extraction and comparison of image features.

[0055] In some examples, the computer vision model can include a feature extraction network configured to extract shape features, color features, texture features, and the like of the image. In some examples, the computer vision model can further include a text recognition network configured to recognize a text sequence included in the center console in the image, so as to more accurately identify the control component. Based on the feature extraction, the image features of the input image can be compared with the image features of the reference control component in the sample image memorized by the model, and an image region whose feature comparison result satisfies a preset similarity condition can be output as the control component predicted by the model. For example, N different types of reference control components can be labeled in the labeled sample used to train the computer vision model, and the model can be trained to memorize the image features of the N reference control components. Then, the confidence that the features of a specific region in the image belong to the N reference control components can be output through the comparison of the similarity of the image features, and the control component prediction result can be obtained based on a preset confidence threshold.

[0056] In some examples, the computer vision model can also be configured to predict the control component based on the feature extraction using a classifier. In this example, the model can be supervised trained using samples labeled with reference control component categories (which can be binary category information or multi-category information), so that the multi-classifier after parameter adjustment can fit a linear or non-linear classification function to output a classification result indicating whether an image region can correspond to a reference control component.

[0057] In some examples, the computer vision model can be configured to determine the region of the control component of the center console in the image and the category of the control component using a two-stage region prediction-category prediction algorithm, or can be configured to output the region and category information of the control component using an end-to-end object detection algorithm, and the present disclosure does not limit the same.

[0058] In some examples, the center console can include a plurality of control components, and the regions of the plurality of control components in the image can be discrete. In this case, the identification result of the at least one control component of the image can include a plurality of controllable regions, which can be output in the form of an identification frame, a highlight mask, an identification information label, a display region magnification, and the like, so as to enable a user to accurately interact with the controllable regions.

[0059] In some embodiments, in response to detecting the first interaction operation on the first target control component among the at least one control component in the image in step S103, determining the operation information associated with the first target control component comprises: determining an operation type for the first target control component, wherein the operation type comprises at least one of clicking, touching, and rotating. In this way, the operation type can be used to more accurately characterize the user's interaction operation on the control component, so that the central control system of the vehicle can accurately determine the control instruction to be executed.

[0060] In some examples, the user can directly perform mechanical clicking (for mechanical buttons), mechanical rotation (for mechanical knobs), touch clicking, touch sliding, touch rotation (for touch screens), and the like operations on the first target control component in the image. In other examples, after completing the identification of the control component, the user terminal can directly display preset operation buttons around the control component, for example, left rotation buttons and right rotation buttons can be displayed near the mechanical knob, so that the user can pass the operation intention by clicking the left rotation button or the right rotation button, so that the user terminal can more directly and accurately determine the operation information of the rotation operation on the mechanical knob.

[0061] In some examples, the operation information can also include more content. For example, when the control component is a touch screen, the operation information can also include touch coordinates, touch force, and the like information of the user on the touch screen, and when the control component is a mechanical knob, the operation information can also include rotation direction, rotation angle, and the like information of the user, so as to more accurately and comprehensively indicate the user's interaction operation information.

[0062] By applying the above method 100, a person other than the driver of the vehicle can control the central control system of the vehicle at a position far away from the central control system of the vehicle. However, the control of the central control system of the vehicle by other persons can affect the safety of the vehicle.

[0063] Based on this, in some embodiments, the method 100 for controlling the central control system of the vehicle further comprises: in response to detecting a second interaction operation on a second target control component among the at least one control component in the image, sending an authorization request to the central control system, wherein in response to detecting the first interaction operation on the first target control component among the at least one control component in the image, determining the operation information associated with the first target control component comprises: in response to receiving the authorization information returned by the central control system of the vehicle, determining the operation information. In this way, the control of the central control system of the vehicle by the user terminal can be realized after obtaining the authorization of the central control system of the vehicle, so as to ensure safety.

[0064] In some examples, the steps of requesting authorization and receiving authorization described above can be selectively set. For example, it can be pre-set that the user terminal does not need to obtain authorization information to control the mode of the vehicle center control system, or it can be pre-set that the user terminal needs to obtain authorization information to control the mode of the vehicle center control system.

[0065] In actual application scenarios, the vehicle center control system can include various types of control components, and the influence of different control components corresponding to the function modules on the safety of the vehicle can be different. For example, the electronic gear shifting component can directly affect the driving safety of the vehicle, the navigation component can affect the driving operation of the driver, and the air conditioning setting component has less influence on the safety of the vehicle.

[0066] Based on this, in some embodiments, in response to detecting a second interaction operation on a second target control component of at least one control component in the image, sending an authorization request to the center control system of the vehicle includes: determining a vehicle function module associated with the second target control component in the vehicle; sending an authorization request for the vehicle function module to the center control system of the vehicle, and wherein in step S103, in response to detecting a first interaction operation on a first target control component of at least one control component in the image, determining operation information associated with the first target control component includes: in response to receiving authorization information returned by the center control system of the vehicle for the vehicle function module, and in response to determining that the first target control component is associated with the vehicle function module, determining the operation information. When the control components of the center control panel correspond to different vehicle function modules, by performing the above steps, the authorization of the vehicle for a specific vehicle function module can be obtained, so that the terminal can only control the module that has obtained the authorization, further improving flexibility and safety.

[0067] In some examples, the vehicle function module can include an audio and video entertainment module, a navigation module, an air conditioning setting module, etc. The control instruction to be executed can be of various types, such as a play music instruction corresponding to the audio and video entertainment module, a close navigation instruction corresponding to the navigation module, an open air conditioner instruction corresponding to the air conditioning setting module, etc. Each vehicle function module can correspond to one or more control components, for example, the air conditioning setting module can correspond to the mechanical buttons of the air conditioner on / off, the mechanical buttons of the refrigeration / heat and the mechanical knob of the air speed adjustment. It can be understood that the division method of the vehicle function module corresponds to the division method of the area authority of the control component, when the control authority of several control components corresponds to the same authorization request, that is, these several control components should be divided into the same function module, so that the user terminal and the center control system of the vehicle can deliver the authorization request based on the unified mapping information of the control component-vehicle function module.

[0068] As mentioned above, the center control panel can include a plurality of control components in a plurality of controllable regions. In this case, the controllable regions on the center control panel can be divided based on the vehicle function modules to which the control components are associated. The identification result of the control components in the center control panel image by the user terminal can include a plurality of functional regions to respectively correspond to a plurality of different vehicle function modules.

[0069] In the above examples, when a plurality of functional regions including control components in the center control panel are identified by using the computer vision model, the annotation information of the sample data used to train the model can include reference functional regions, so that efficient and accurate identification can be achieved by using the computer vision model.

[0070] In some examples, the first target control component and the second target control component can be the same, and the first interaction operation and the second interaction operation can be the same. In this case, the authorization request can be sent to the center control system of the vehicle in response to detecting the interaction operation of the user on the target control component, and the operation information for the target control component can be determined based on the interaction operation after obtaining the authorization information.

[0071] In other examples, the first target control component and the second target control component can be the same, and the first interaction operation and the second interaction operation can be different. In this case, the authorization request can be sent to the center control system of the vehicle in response to detecting the first interaction operation of the user on the target control component, and the operation information for the target control component can be determined based on the second interaction operation in response to detecting the second interaction operation of the user on the target control component after obtaining the authorization information.

[0072] In yet other examples, the first target control component and the second target control component can be different, so that the first interaction operation and the second interaction operation are also different. In this case, the authorization request for the functional region A can be sent to the center control system of the vehicle in response to detecting the first interaction operation of the user on the first target control component in the functional region A, and the operation information for the second target control component can be determined based on the second interaction operation in response to detecting the second interaction operation of the user on the second target control component in the functional region A after obtaining the authorization information. When the second target control component does not belong to the functional region A, the center control system of the vehicle cannot be controlled based on the authorization information for the functional region A.

[0073] Figure 2 A flowchart of a method 200 for controlling a center control system of a vehicle according to another embodiment of the present disclosure is shown. As shown in the figure, Figure 2 The method 200 includes the following steps:

[0074] Step S201, acquiring an image including a center control panel;

[0075] Step S202, identifying at least one control component on the center control panel in the image.

[0076] Step S203, in response to detecting a second interaction operation on a second target control component in the at least one control component in the image, sending an authorization request to the center control system.

[0077] Step S204, in response to receiving the authorization information returned by the center control system, and in response to detecting a first interaction operation on a first target control component in the at least one control component in the image, determining operation information associated with the first target control component; and

[0078] Step S205, sending the operation information to the center control system to enable the center control system to determine a control instruction to be executed based on the operation information.

[0079] By applying the above method 200, the user terminal can be used to obtain an image including a center control panel of a vehicle and perform image recognition to obtain at least one control component included in the center control panel in the image, and when detecting a user's execution of an interaction operation on the control component in the image, an authorization request is sent to the center control system of the vehicle. After obtaining authorization, the user's operation information can be transmitted to the center control system of the vehicle to enable the center control system of the vehicle to determine a control instruction to be executed based on the user's operation information, thereby conveniently realizing the user's control of the center control system of the vehicle.

[0080] Some embodiments of the present disclosure provide a control method applied to a center control system of a vehicle, the center control system comprising a center control panel.

[0081] Figure 3 A flowchart of a control method 300 applied to a center control system of a vehicle according to an embodiment of the present disclosure is shown. As shown in the flowchart, the method 300 comprises: Figure 3

[0082] Step S301, in response to receiving operation information sent by a user terminal in communication connection with the center control system of the vehicle, determining a control instruction to be executed based on the operation information. The operation information in step S301 is generated by the user terminal through the following processes: obtaining an image including a center control panel; identifying at least one control component on the center control panel in the image; and in response to detecting a first interaction operation on a first target control component in the at least one control component in the image, determining operation information associated with the first target control component.

[0083] ​By applying the above method 300, the vehicle's central control system can determine the control instructions to be executed based on the user's interactive operation information on the central control panel image. This is equivalent to mapping the user's interactive operations on the virtual central control panel (the central control panel in the image) on the user terminal to the physical central control panel of the central control system, thereby conveniently enabling the user to control the vehicle's central control system. In this implementation, even if the passenger's seat is far from the central control panel, the user terminal can still conveniently control the vehicle's central control system, eliminating the need for the driver to operate the central control panel, thereby improving vehicle driving safety.

[0084] In some embodiments, the operation information includes an operation type of the first interactive operation, and determining the control instruction to be executed based on the operation information includes: determining the vehicle-mounted functional module corresponding to the first target control component; and determining the control instruction for the vehicle-mounted functional module based on the operation type of the first interactive operation for the first target control component. Thus, the instruction to be executed can be determined simply and efficiently based on the mapping relationship between the control component and the vehicle-mounted functional module and the mapping relationship between the operation type and the control instruction.

[0085] In some embodiments, determining a control instruction for an in-vehicle functional module based on an operation type of a first interactive operation for a first target control component includes: determining a control instruction from an instruction table corresponding to the in-vehicle functional module based on the operation type, wherein the instruction table stores a mapping relationship between a plurality of preset operation types and a plurality of preset control instructions. Thus, the control instruction to be executed can be determined simply and efficiently by looking up the table.

[0086] In some embodiments, the control method for a vehicle's central control system further includes: receiving an authorization request from a user terminal; generating notification information for displaying the authorization request; generating authorization information in response to detecting an authorization interaction operation in response to the notification information; and returning the authorization information to the user terminal. Thus, authorization information for the user terminal can be determined based on the authorization interaction operation of the driver or a specific passenger, thereby ensuring safety.

[0087] In some examples, the notification information for displaying the authorization request may be a textual notification of the authorization request displayed on a display screen, and may also display authorization and non-authorization interaction buttons to the driver or specific passenger. In this case, the authorization interaction operation may be an operation of the driver or specific passenger clicking the authorization interaction button.

[0088] In other examples, the notification information for displaying the authorization request may be a voice broadcast information. In this case, the authorization interaction operation may be an operation in which the driver or a specific passenger issues an authorized voice instruction.

[0089] Figure 4A flowchart of an interaction process for controlling a center control system of a vehicle is shown according to an embodiment of the present disclosure. As shown in Figure 4 The interaction process for controlling the center control system of the vehicle includes:

[0090] Step S1, the user terminal acquires an image including a center control panel;

[0091] Step S2, the user terminal identifies at least one control component on the center control panel in the image;

[0092] Step S3, the user terminal sends an authorization request to the center control system in response to detecting a second interaction operation on a second target control component in the at least one control component in the image;

[0093] Step S4, the center control system generates notification information for displaying the authorization request;

[0094] Step S5, the center control system generates authorization information in response to detecting an authorization interaction operation on the notification information;

[0095] Step S6, the center control system returns the authorization information to the user terminal;

[0096] Step S7, the user terminal determines operation information associated with a first target control component in the at least one control component in the image in response to detecting a first interaction operation on the first target control component;

[0097] Step S8, the user terminal sends the operation information to the center control system;

[0098] Step S9, the center control system determines a control instruction to be executed based on the operation information.

[0099] By applying the above interaction process, the user terminal can be used to acquire an image including a center control panel of a vehicle and perform image recognition to obtain at least one control component included in the center control panel in the image. When detecting a user's execution interaction operation on the control component in the image, an authorization request is sent to the center control system of the vehicle. After obtaining authorization, the user's operation information can be transmitted to the center control system of the vehicle, so that the center control system of the vehicle can determine a control instruction to be executed based on the user's operation information, thereby conveniently realizing the user's control of the center control system of the vehicle.

[0100] Some embodiments of the present disclosure provide a device for controlling a center control system of a vehicle. Figure 5 A structural block diagram of the device 500 for controlling the center control system of the vehicle according to an embodiment of the present disclosure is shown. The center control system includes a center control panel, and the device 500 is applied to a user terminal in communication connection with the center control system. As shown in Figure 5 The device 500 includes:

[0101] The acquisition unit 501 is configured to acquire an image including a center control panel.

[0102] The first determination unit 502 is configured to identify at least one control component on the center control panel in the image.

[0103] The second determination unit 503 is configured to, in response to detecting a first interaction operation on a first target control component of the at least one control component in the image, determine operation information associated with the first target control component.

[0104] The sending unit 504 is configured to send the operation information to the center control system, so that the center control system can determine a control instruction to be executed based on the operation information.

[0105] According to some embodiments, the first determination unit 502 is configured to input the image into a computer vision model to obtain at least one control component output by the computer vision model, wherein the computer vision model is obtained by training sample images labeled with reference control components.

[0106] According to some embodiments, the computer vision model is configured to identify the control region in the image based on the following process: determining image features of the image; and determining at least one control component based on the image features, wherein the image features corresponding to the at least one control component satisfy a preset similarity condition with the image features corresponding to the reference control components in the sample images.

[0107] According to some embodiments, the second determination unit 503 is configured to determine an operation type for the first target control component, wherein the operation type includes at least one of clicking, touching, and rotating.

[0108] According to some embodiments, the sending unit 504 is further configured to, in response to detecting a second interaction operation on a second target control component of the at least one control component in the image, send an authorization request to the center control system, wherein the second determination unit 503 is configured to, in response to receiving authorization information returned by the center control system of the vehicle, determine the operation information.

[0109] According to some embodiments, the sending unit 504 is configured to: determine a vehicle-mounted function module associated with the second target control component in the vehicle; send an authorization request for the vehicle-mounted function module to the center control system of the vehicle, and the second determination unit 503 is configured to: in response to receiving authorization information for the vehicle-mounted function module returned by the center control system of the vehicle, and in response to determining that the first target control component is associated with the vehicle-mounted function module, determine the operation information.

[0110] It should be understood that Figure 5 The operations of each unit of the device 500 for controlling the center control system of the vehicle shown in the above description mayFigure 1 The various steps in the method 100 for controlling the center control system of a vehicle correspond. Thus, the operations, features and advantages described above for the method 100 also apply to the apparatus 500 and the various units included therein. For the sake of brevity, certain operations, features and advantages are not described again here.

[0111] Some embodiments of the present disclosure provide an apparatus applied to a center control system of a vehicle. Figure 6 A structural block diagram of the apparatus 600 applied to a center control system of a vehicle of an embodiment of the present disclosure is shown in FIG. 6. The center control system includes a center control panel. As shown in FIG. 6, the apparatus 600 includes: Figure 6

[0112] The third determining unit 601 is configured to determine, in response to receiving operation information sent by a user terminal in communication connection with the center control system of the vehicle, a control instruction to be executed based on the operation information, wherein the operation information is determined by the user terminal using the foregoing obtaining unit 501, the first determining unit 502 and the second determining unit 503.

[0113] According to some embodiments, the operation information includes an operation type of the first interaction operation, and the third determining unit 601 is configured to determine a vehicle-mounted function module corresponding to the first target control component, and determine a control instruction for the vehicle-mounted function module based on the operation type for the first target control component.

[0114] According to some embodiments, the first determining unit 601 is configured to determine the control instruction from an instruction table corresponding to the vehicle-mounted function module based on the operation type, wherein the instruction table stores a mapping relationship between a plurality of preset operation types and a plurality of preset control instructions.

[0115] According to some embodiments, the apparatus 600 further includes an authorization unit configured to receive an authorization request sent by the user terminal, generate notification information for displaying the authorization request, generate authorization information in response to detecting an authorization interaction operation for the notification information, and return the authorization information to the user terminal.

[0116] It should be understood that Figure 6 The operations of the various units of the apparatus 600 applied to the center control system of the vehicle shown in FIG. 6 can correspond to the operations of the method 300 for controlling the center control system of a vehicle. Figure 3 The various steps in the method 300 for controlling the center control system of a vehicle correspond. Thus, the operations, features and advantages described above for the method 300 also apply to the apparatus 600 and the various units included therein. For the sake of brevity, certain operations, features and advantages are not described again here.

[0117] Some embodiments of the present disclosure provide a computer device for executing the above-described method 100 for controlling the center control system of a vehicle.​

[0118] Some embodiments of the present disclosure provide a computer device for executing the above-mentioned control method 300 of a center control system applied to a vehicle.

[0119] With reference to Figure 7 The computer device 7000 can include elements connected to or in communication with the bus 7002 (possibly via one or more interfaces), for example. The computer device 7000 can include a bus 7002, one or more processors 7004, one or more input devices 7006, and one or more output devices 7008, for example. The one or more processors 7004 can be any type of processors and can include, but are not limited to, one or more general-purpose processors and / or one or more special-purpose processors (e.g., special purpose processing chips). The processor(s) 7004 can process instructions for execution within the computer device 7000, including instructions stored in a memory or instructions that cause a graphical information to be displayed on a GUI on an external input / output device (such as a display device coupled to an interface). In other implementations, multiple processors and / or buses can be employed with multiple memories as needed. Also, multiple computer devices can be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). Figure 7 The processor 7004 is taken as an example in the middle.

[0120] The input device 7006 can be any type of device that can input information to the computer device 7000. The input device 7006 can receive inputted digital or character information, and generate key signal inputs related to event extraction of the user settings and / or function controls of the computer device, and can include, but is not limited to, a mouse, a keyboard, a touch screen, a trackpad, a trackball, a joystick, a microphone, and / or a remote controller. The output device 7008 can be any type of device that can present information, and can include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer.

[0121] The computer device 7000 can also include or be connected to a non-transitory storage device 7010, which can be any storage device(s) (can be non-transitory and can enable data storage) and can include, but is not limited to, a magnetic disk drive, an optical storage device, a solid-state memory, a floppy disk, a flexible disk, a hard disk, a magnetic tape, or any other magnetic medium, an optical disk, or any other optical medium, a ROM (read only memory), a RAM (random access memory), a cache, and / or any other memory chip or cartridge, and / or any other medium from which a computer can read data, instructions, and / or code. The non-transitory storage device 7010 can be detachable from the interface. The non-transitory storage device 7010 can have data / programs (including instructions) / code / modules for implementing the above-described methods and steps.

[0122] The computer device 7000 can also include a communication device 7012. The communication device 7012 can be any type of device or system that enables communication with external devices and / or with networks, and can include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication device, and / or a chipset such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0123] The computer device 7000 can also include a working memory 7014, which can be any type of working memory that can store programs (including instructions) and / or data useful for the operation of the processor 7004, and can include, but is not limited to, a random access memory and / or a read only memory device.

[0124] The software elements (programs) can be located within working memory 7014, including, but not limited to, an operating system 7016, one or more application programs 7018, other program modules, and / or other data and code. Instructions for implementing the above-described methods and steps can be included in one or more of the application programs 7018, and the above-described methods can be implemented by the processor 7004 reading and executing the instructions of the one or more application programs 7018. The executable code or source code of the instructions of the software elements (programs) can also be downloaded from a remote location.

[0125] It should also be understood that a variety of changes can be made without departing from the essence of the disclosed embodiments. Such changes are also encompassed within the spirit of the present disclosure and are intended to be within the scope of the disclosure. For example, the disclosed methods and devices can be implemented using custom hardware, and / or custom software, firmware, middleware, microcode, hardware description languages, and / or any combination thereof. For example, some or all of the disclosed methods and devices can be implemented using hardware (e.g., programmable logic circuitry, including field programmable gate arrays (FPGAs) and / or programmable logic arrays (PLAs), which can be programmed using assembly or hardware programming languages (such as VERILOG, VHDL, C++) according to the teachings of the present disclosure.

[0126] It should also be appreciated that the foregoing methods can be implemented by a server-client mode. For example, a client can receive data input by a user and send the data to a server. The client can also receive data input by a user, perform part of the processing in the foregoing method, and send the data obtained by processing to the server. The server can receive data from the client, and perform the foregoing method or another part of the foregoing method, and return the execution result to the client. The client can receive the execution result of the method from the server, and can present it to the user, for example, through an output device. The client and the server are generally far away from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on corresponding computer devices and having a client-server relationship with each other. The server can be a server of a distributed system, or a server combined with a blockchain. The server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The cloud server is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and virtual private server (VPS, Virtual Private Server) services.

[0127] It should also be appreciated that the components of the computer device 7000 can be distributed over a network. For example, some processing can be performed using one processor, while other processing can be performed by another processor that is remote from the one processor. Other components of the computer device 7000 can be similarly distributed. In this way, the computer device 7000 can be interpreted as a distributed computing system that performs processing at multiple locations.

[0128] Some embodiments of the present disclosure provide a vehicle, comprising a central control system, the central control system comprising a central control panel; and the computer device capable of executing the method 300 for controlling the central control system of the vehicle.

[0129] Some embodiments of the present disclosure provide a computer-readable storage medium, which stores instructions, when executed by one or more processors of a computer device individually or collectively, causes the computer device to execute the foregoing method 100 or the foregoing method 300.

[0130] The computer readable medium can be a tangible medium that can contain or store the program for use by or in connection with the instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more of: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0131] Some embodiments of the present disclosure provide a computer program product comprising instructions which, when executed by one or more processors of a computer device, individually or collectively, cause the computer device to perform the method 100 described above or the method 300 described above.

[0132] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present disclosure, and not to limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some or all of the technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present disclosure, and they should all be covered in the scope of the claims and the specification of the present disclosure. In particular, the technical features mentioned in each embodiment can be combined in any way as long as there is no structural conflict. The present disclosure is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for controlling a center system of a vehicle, the center system comprising a center panel, the method being applied to a user terminal communicatively connected to the center system, the method comprising: obtaining an image comprising the center panel; identifying at least one control component on the center panel in the image; in response to detecting a first interaction operation directed to a first target control component among the at least one control component in the image, determining operation information associated with the first target control component; and sending the operation information to the center system to enable the center system to determine a control instruction to be executed based on the operation information. The identifying the at least one control component on the center panel in the image comprises:

2. The method of claim 1, wherein, inputting the image into a computer vision model to obtain the at least one control component output by the computer vision model, wherein the computer vision model is obtained by training using sample images comprising reference control components. The computer vision model is based on a process of identifying a control region in the image by:

3. The method of claim 2, wherein, determining image features of the image; and based on the image features, determining the at least one control component, wherein the image features corresponding to the at least one control component satisfy a preset similarity condition with the image features corresponding to the reference control components in the sample images. The determining the operation information associated with the first target control component in response to detecting the first interaction operation directed to the first target control component among the at least one control component in the image comprises:

4. The method of any one of claims 1-3, wherein, determining an operation type for the first target control component, wherein the operation type comprises at least one of clicking, touching, and rotating.

5. The method of any one of claims 1-3, further comprising: in response to detecting a second interaction operation directed to a second target control component among the at least one control component in the image, sending an authorization request to the center system, wherein the determining the operation information associated with the first target control component in response to detecting the first interaction operation directed to the first target control component among the at least one control component in the image comprises: in response to receiving authorization information returned by the center system of the vehicle, determining the operation information. The sending the authorization request to the center system in response to detecting the second interaction operation directed to the second target control component among the at least one control component in the image comprises:

6. The method of claim 5, wherein, determining a vehicle-mounted function module associated with the second target control component in the vehicle; sending an authorization request for the vehicle-mounted function module to the center system of the vehicle, and wherein the determining the operation information associated with the first target control component in response to detecting the first interaction operation directed to the first target control component among the at least one control component in the image comprises: ​ in response to receiving authorization information for the in-vehicle function module returned by the central control system of the vehicle, and in response to determining that the first target control component is associated with the in-vehicle function module, determining the operation information. 7.A method applied to a central control system of a vehicle, the central control system comprising a center console panel, the method comprising: in response to receiving operation information sent by a user terminal communicatively connected with the central control system of the vehicle, determining a control instruction to be executed based on the operation information, wherein the operation information is generated by the user terminal through a process comprising: obtaining an image comprising the center console panel; determining at least one control component on the center console panel in the image; and in response to detecting a first interaction operation for a first target control component in the at least one control component in the image, determining operation information associated with the first target control component.

8. The method of claim 7, wherein, the operation information comprises an operation type of the first interaction operation, and the determining the control instruction to be executed based on the operation information comprises: determining an in-vehicle function module corresponding to the first target control component; and determining a control instruction for the in-vehicle function module based on the operation type of the first interaction operation for the first target control component.

9. The method of claim 8, wherein, the determining the control instruction for the in-vehicle function module based on the operation type of the first interaction operation for the first target control component comprises: determining the control instruction from an instruction table corresponding to the in-vehicle function module based on the operation type, wherein the instruction table stores a mapping relationship between a plurality of preset operation types and a plurality of preset control instructions. 10.The method of any one of claims 7-9, further comprising: receiving an authorization request sent by the user terminal; generating notification information for presenting the authorization request; in response to detecting an authorization interaction operation for the notification information, generating authorization information; and returning the authorization information to the user terminal. 11.An apparatus for controlling a central control system of a vehicle, the central control system comprising a center console panel, the apparatus being applied to a user terminal communicatively connected with the central control system, the apparatus comprising: an obtaining unit configured to obtain an image comprising the center console panel; a first determining unit configured to identify at least one control component on the center console panel in the image; a second determining unit configured to, in response to detecting a first interaction operation for a first target control component in the at least one control component in the image, determine operation information associated with the first target control component; and a sending unit configured to send the operation information to the central control system, so that the central control system can determine a control instruction to be executed based on the operation information. 12.An apparatus applied to a central control system of a vehicle, the central control system comprising a center console panel, the apparatus comprising: ​ ​ a third determining unit, configured to determine a control instruction to be executed based on operation information in response to receiving the operation information sent by the user terminal in communication connection with the central control system of the vehicle, wherein the operation information is generated by the user terminal using the following units: an obtaining unit, configured to obtain an image including the central control panel; a first determining unit, configured to determine at least one control component on the control panel in the image; and a second determining unit, configured to determine operation information associated with a first target control component in the at least one control component in the image in response to detecting a first interactive operation on the first target control component.

13. A computer device, comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors individually or collectively, cause the computer device to perform the method of any one of claims 1 to 6.

14. A computer device, comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors individually or collectively, cause the computer device to perform the method of any one of claims 7 to 10.

15. A vehicle, comprising: a central control system, the central control system comprising a central control panel; and the computer device of claim 14.

16. A computer-readable storage medium storing instructions that, when executed by one or more processors of a computer device individually or collectively, cause the computer device to perform the method of any one of claims 1 to 10.

17. A computer program product comprising instructions that, when executed by one or more processors of a computer device individually or collectively, cause the computer device to perform the method of any one of claims 1 to 10.

Images