Vehicle cockpit interlocking method, device, medium, and computer program product

Abstract

To provide a vehicle cockpit interlocking method, a device, a medium, and a computer program product.SOLUTION: A vehicle cockpit interlocking method includes the steps of: acquiring user interaction with respect to an application screen displayed inside a vehicle cockpit; and triggering interlocking of the vehicle cockpit, based on the application screen and the user interaction. Accordingly, an application scene and an interaction object can be determined based on the application screen and the user interaction, without calling a data interface from an application provider, thus an on-vehicle infotainment system provider can achieve interlocking of a vehicle cockpit independently, and save costs.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates generally to the field of vehicle control, and more particularly to methods, apparatus, media and computer program products for interlocking a vehicle cockpit. [Background technology]

[0002] With the continuous development of science and technology, automobiles are gradually evolving from a means of transportation used only for driving to a "mobile intelligent space." The widespread use of in-vehicle infotainment systems and smart mobile devices in vehicle cockpits brings greater convenience and a more comfortable experience to users. Therefore, modern in-vehicle technology is beginning to evolve in a more intelligent and personalized direction, going beyond providing basic driving information and entertainment functions.

[0003] With the widespread use of smart mobile devices and the upgrade of in-vehicle infotainment systems, users are becoming accustomed to using smart devices such as mobile phones and tablet computers in the vehicle cockpit to operate a variety of applications, from navigation to music, videos, and games. Meanwhile, with the advancement of technology, the vehicle cockpit environment can be synchronized based on the user's application operations, providing a more personalized and comfortable driving environment for users and improving the user experience. Summary of the Invention

[0004] In a first aspect of an embodiment of the present disclosure, a vehicle cockpit interaction method is provided, the method including: obtaining a user interaction with an application screen displayed within the vehicle cockpit; and triggering an interaction of the vehicle cockpit based on the application screen and the user interaction.

[0005] In a second aspect of an embodiment of the present disclosure, there is provided an electronic device including one or more processors and a memory, coupled to the at least one processor, storing instructions that, when executed by the at least one processor, cause the electronic device to perform operations including obtaining a user interaction with an application screen displayed within a vehicle cockpit and triggering an engagement of the vehicle cockpit based on the application screen and the user interaction.

[0006] In a third aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, realizes a vehicle cockpit interlocking method, the method including: obtaining a user interaction with an application screen displayed in the vehicle cockpit; and triggering an interlocking of the vehicle cockpit based on the application screen and the user interaction.

[0007] In a fourth aspect of an embodiment of the present disclosure, there is provided a computer program product tangibly stored on a non-transitory computer-readable medium and including machine-executable instructions that, when executed, cause a machine to implement a vehicle cockpit interlocking method, the method including obtaining a user interaction with an application screen displayed within the vehicle cockpit, and triggering an interlocking of the vehicle cockpit based on the application screen and the user interaction.

[0008] It should be noted that the contents described in the Summary of the Invention are not intended to limit the essential or important features of the embodiments of the present disclosure, and do not limit the scope of the present disclosure. Other features of the present disclosure will be easily understood from the following description. [Brief explanation of the drawings]

[0009] These and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent with reference to the drawings and the following detailed description, in which the same or similar reference numerals indicate the same or similar elements. [Figure 1] 1 shows a schematic diagram of an exemplary environment in which several embodiments of the present disclosure can be implemented. [Figure 2] 1 illustrates a flowchart of a vehicle cockpit interlocking method according to some embodiments of the present disclosure. [Figure 3A] 1 shows a schematic diagram of an example of recognizing a character selection scene using a generic template and a dedicated template, according to some embodiments of the present disclosure. [Figure 3B] 1 shows a schematic diagram of an example of recognizing a character selection scene using a generic template and a dedicated template, according to some embodiments of the present disclosure. [Figure 4] 1 illustrates a schematic diagram of an example process for obtaining an object contour, according to some embodiments of the present disclosure. [Figure 5] 1 shows a schematic diagram of an example process for extracting color features of an object contour, according to some embodiments of the present disclosure. [Figure 6] 1 illustrates a schematic diagram of an example process for adjusting a vehicle cockpit environment by recognizing characters corresponding to object contours, according to some embodiments of the present disclosure. [Figure 7] 1 illustrates a schematic diagram of an example process for interfacing with a vehicle cockpit based on user interactions on a user's mobile device, according to some embodiments of the present disclosure. [Figure 8] 1 shows a schematic diagram of an interlocking arrangement for a vehicle cockpit according to some embodiments of the present disclosure. [Figure 9] 1 shows a block diagram of a device capable of implementing several embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described in more detail with reference to the drawings. Although several embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be realized in various forms and should not be understood as being limited to the embodiments described herein. On the contrary, these embodiments are provided for a clearer and more complete understanding of the present disclosure. It should also be understood that the drawings and embodiments of the present disclosure are merely illustrative and do not limit the scope of protection of the present disclosure.

[0011] In describing embodiments of the present disclosure, the term "comprising" and similar terms should be understood as an open-ended inclusion, i.e., "including, but not limited to." The term "based on" should be understood as "based at least in part on." The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment." The terms "first," "second," etc. may refer to different objects or the same object. Other explicit and implicit definitions may also be included below.

[0012] As described above, vehicle cockpits are evolving into mobile multimedia entertainment spaces, where drivers and passengers can operate various application programs, such as navigation, music, videos, and games, via an in-vehicle infotainment system or a user's mobile device, to meet different user needs. Some vehicle cockpits are equipped with lighting systems for creating atmospheres, such as ambient lights with adjustable color and brightness. Control of the ambient lights is typically managed by an in-vehicle infotainment system, which can adjust parameters such as the color and brightness of the ambient lights by sending control commands based on a preset mode or a data signal received in real time. Some vehicles are also equipped with a fragrance system that can emit different fragrances to create different atmosphere experiences within the vehicle, and the in-vehicle infotainment system can control the fragrance emission time and intensity. In addition, the in-vehicle infotainment system can also send control commands to systems that can generate sensory interactions with the user, such as the vehicle's audio system, seat massage system, or air conditioning system, to change their operating states. By centrally controlling these environmental hardware systems, the in-vehicle infotainment system can adjust the in-cockpit environment based on the external data it receives, providing an immersive experience for the user.

[0013] However, interaction data between a user and an application is typically owned and managed by the application provider. To achieve a connected vehicle cockpit environment, some related technologies require the application provider to provide a related data interface to an in-vehicle infotainment system provider to obtain real-time status data of the application the user is currently operating. This data may include, for example, the application's identifier, type, current scene, interface elements, etc. However, developing such a data interface requires significant time and resources, forcing the in-vehicle infotainment system provider to incur high costs for such a data interface. Another related technology utilizes artificial intelligence (AI)-based object detection technology to perform target detection on a single-frame screen, thereby recognizing an object within an application with which the user is interacting. However, if the in-vehicle infotainment system has limited computing resources, it may be impossible to deploy an AI model. Furthermore, the inference process of the AI ​​model may cause problems such as latency and response delays. Furthermore, different AI models need to be trained for different applications, which requires a large amount of training data, making the cost of obtaining trained models very high.

[0014] Therefore, an embodiment of the present disclosure provides a vehicle cockpit interlocking aspect, which can acquire application screens displayed in the vehicle cockpit and user interactions with the application screens, and then trigger vehicle cockpit interlocking based on the application screens and user interactions. In this way, application scenes and interaction objects can be determined based on the application screens and user interactions without invoking a data interface from the application provider, allowing in-vehicle infotainment system providers to independently implement vehicle cockpit interlocking and save costs. Furthermore, compared to aspects using artificial intelligence-based object detection technology, this aspect uses user interaction data to assist in determining interaction objects on the screen, thereby improving processing speed, shortening response time, improving versatility, and saving costs spent on training artificial intelligence models.

[0015] FIG. 1 illustrates a schematic diagram of an exemplary environment 100 in which several embodiments of the present disclosure can be implemented. As illustrated in FIG. 1, the environment 100 includes a vehicle 102, which includes a control unit 104, an application device 106, and an interlocking unit 108. The control unit 104 may be any device having computing or processing capabilities. For example, the control unit 104 may be an in-vehicle infotainment system, a desktop computer, a laptop computer, a tablet computer, a server, a mobile device, a vehicle control unit (including an overall vehicle control unit and a subsystem control unit), etc. The control unit 104 may send control commands to the interlocking unit 108 to change the state of the interlocking unit 108.

[0016] The application device 106 is a device that executes the application 110; for example, the application device 106 may be a user's personal mobile device (e.g., a mobile phone, a tablet computer, etc.). The application 110 may be any application that executes on the user's mobile device (e.g., a game application, a music application, a video application, a navigation application, etc.). In some embodiments of the present disclosure, the application device 106 may be the same component as the control unit 104; for example, the control unit 104 and the application device 106 may simultaneously refer to the in-vehicle infotainment system of the vehicle 102. In these embodiments, the application 110 may be any application that executes on the in-vehicle infotainment system (e.g., a game application, a music application, a video application, a navigation application, etc.).

[0017] The interlocking unit 108 is any component within the cockpit of the vehicle 102 that can interact sensory with the user. For example, the interlocking unit 108 may include ambient lighting, audio, air conditioning, seats, fragrance, etc. within the cockpit. For example, the ambient lighting may change color, brightness, or flash in response to a user's operation of the application 110, the audio may change volume or play a specific sound in response to a user's operation of the application 110, and the air conditioning may change the strength of the air conditioning in response to a user's operation of the application 110.

[0018] 1, in environment 100, a user can operate application 110 via application device 106, and more specifically, a user can perform user interactions 114 on application screen 112. Application screen 112 may include multiple objects, such as object 116, object 118, etc. User interaction 114 may be a tap or touch on application screen 112, where the location of the tap or touch is within a region associated with object 116, thereby selecting object 116.

[0019] In some conventional techniques, the control unit 104 can obtain an identifier of a scene of the application screen 112, a description of the scene of the application screen 112, an identifier of an object selected by the user, and a description of the object selected by the user from an interface provided by a provider of the application 110. In other conventional techniques, the control unit 104 can recognize the scene of the application screen 112 and the object selected by the user using an object detection technique based on the application screen 112 alone. However, the method of invoking an interface requires high costs, while the artificial intelligence method not only requires a large amount of training data to train models for different applications, but also introduces delays in the inference stage of the model, degrading the user experience.

[0020] In an embodiment provided by the present disclosure, the control unit 104 can acquire the application screen 112 and the user interaction 114 from the application device 106 (for example, the application screen 112 and the related user interaction 114 of the application 110 executed thereon can be captured by an in-vehicle infotainment system, or the application screen 112 and the user interaction 114 can be acquired by another application, such as a screen projection application, on the application device 106 and transmitted to the control unit 104), and can analyze information such as the scene of the application screen 112, the area tapped or touched by the user interaction 114, whether a valid object is selected in the area, and the characteristics of the selected object based on the application screen 112 and the user interaction 114. Then, the control unit 104 can send corresponding control commands to the interlocking unit 108 based on the information, thereby realizing interlocking of the vehicle cockpit. In this way, vehicle cockpit interlocking can be achieved while avoiding the need for data interfaces from application providers and the use of artificial intelligence-based target detection technology, reducing costs, reducing delays, increasing versatility, and improving the user experience.

[0021] FIG. 2 illustrates a flowchart of a vehicle cockpit interlocking method 200 according to some embodiments of the present disclosure. The method 200 may be performed, for example, by the control unit 104 in the environment 100 illustrated in FIG. 1. As illustrated in FIG. 2, in block 202, the method 200 may acquire user interactions with an application screen displayed within the vehicle cockpit. For example, in the environment 100 illustrated in FIG. 1, the control unit 104 may acquire the application screen 112 and the user interactions 114 with the application screen 112. For example, if the control unit 104 and the application device 106 are two separate components (e.g., the control unit 104 is an in-vehicle infotainment system and the application device 106 is a user's mobile device), the control unit 104 may acquire the application screen 112 and the user interactions 114 from the application device 106 (e.g., via a screen projection application on the application device 106). If the control unit 104 is the same component as the application device 106 (e.g., both are in-vehicle infotainment systems), the control unit 104 can directly obtain the application screens 112 and user interactions 114 of the application 110 running on it.

[0022] In block 204, the method 200 can trigger a vehicle cockpit interaction based on the application screen and the user interaction. For example, in the environment 100 shown in FIG. 1 , the control unit 104 can analyze information based on the application screen 112 and the user interaction 114, such as the scene of the application screen 112, the area tapped or touched by the user interaction 114, whether a valid object was selected in the area, and the characteristics of the selected object. Then, the control unit 104 can realize the vehicle cockpit interaction by sending a corresponding control command to the interaction unit 108 based on the information. The vehicle cockpit interaction may include, for example, adjusting the color and brightness of ambient lights, flashing ambient lights, adjusting the volume of audio, playing specific sound effects in audio, turning on air conditioning, changing the strength of air conditioning, vibrating seats, changing the type and strength of fragrance, etc., in response to the user interaction 114.

[0023] In this way, the application scene and interaction object can be determined based on the application screen and user interaction without invoking a data interface from the application provider, allowing the in-vehicle infotainment system provider to independently realize vehicle cockpit interlocking and save costs. Furthermore, compared to using artificial intelligence-based object detection technology, the user interaction data can be used to assist in determining the interaction object on the screen, thereby improving processing speed, shortening response time, improving versatility, and saving costs spent on training artificial intelligence models.

[0024] Aspects of the present disclosure may be applied to any type of application and any appropriate scene in an application. For example, aspects of the present disclosure may be applied to a character selection scene or a map selection scene in a game application, a playlist selection scene or a preferred music type selection scene in a music application, a restaurant selection scene in a life service application, etc. Aspects of the present disclosure are particularly suitable for a character selection scene in a game application, where the user's attention is not yet fully focused on the game process. In this case, the interaction with the vehicle cockpit can quickly immerse the user in the game atmosphere and improve the user experience. Therefore, other embodiments of the present disclosure will be described below using a character selection scene in a game application as an example.

[0025] In some examples, based on the application screen and user interaction, it may be determined whether the application screen matches a target scene to trigger an engagement of the vehicle cockpit. Then, in response to the application screen matching the target scene, it may be triggered to engage the vehicle cockpit based on the application screen and user interaction. In some examples, to determine whether the application screen matches the target scene, a template for the target scene may be obtained, the template including target features corresponding to the target scene. Then, in response to determining that the application screen has features corresponding to the target features, it may be determined that the application screen matches the target scene.

[0026] 3A-3B illustrate schematic diagrams of an example of recognizing a character selection scene using a generic template and a specific template, according to some embodiments of the present disclosure. In some embodiments, if there is no specific template for a target scene in a particular application, a generic template for the target scene can be used to determine whether an application screen matches the target scene. FIG. 3A illustrates a schematic diagram of an example 300 of recognizing a character selection scene in a game application using a generic template, according to some embodiments of the present disclosure. As shown in FIG. 3A, the example 300 includes a game screen 302 in a game application. The game screen 302 includes character selection text 304, character 306, character 308, character 310, character 312, a character preview 314 for character 306, a back button 316, and a selection confirmation button 318.

[0027] As shown in FIG. 3A , example 300 further includes a generic template 320, which is used to determine whether game screen 302 belongs to a character selection scene. Generic template 320 includes region 322 and target text 324 (i.e., “character selection”). Generic template 320 may indicate that if target text 324 appears in region 322 of the game screen, the game screen can be determined to belong to a character selection scene. In example 300, after obtaining a region of game screen 302 corresponding to region 322 of generic template 320, it can be determined that the region contains target text 324, and therefore game screen 302 can be determined to belong to a character selection scene. In some embodiments, the text content in the region can be obtained by image recognition, and then the image of the region can be compared with target text 324 instead of comparing the image of region 322 of generic template 320. This allows accurate determination that game screen 302 matches generic template 320 even if character selection text 304 has a different font.

[0028] In this way, if the game screen 302 has the target feature indicated by the generic template 320 (i.e., if the target region 322 has the target text 324), it can be determined that the game screen 302 belongs to a character selection scene without recognizing other elements (e.g., character 306, character 308, character 310, character 312, character preview 314, back button 316, and selection confirmation button 318) in the game screen 302. In this way, the speed and versatility of target scene recognition can be improved.

[0029] For ease of understanding, the example 300 shows the generic template 302 in the form of an image, however, the generic template 302 may be stored in other data formats, for example, the target area 322 may be stored as the coordinates of two diagonal corners of a rectangular box, and the target text 324 may be stored as separate text.

[0030] In some embodiments, the control unit may obtain an identifier of the running application, then read a pre-stored dedicated template for the application based on the identifier, and determine whether the application screen matches the target scene based on the dedicated template. Figure 3B shows a schematic diagram of an example 320 of recognizing a character selection scene in a game application using a dedicated template, according to some embodiments of the present disclosure. As shown in Figure 3B, the example 320 includes a game screen 322. The game screen 322 includes character selection text 324, a character 326, a character 328, a character 330, a character 332, a character preview 334 of character 326, a back button 336, and a selection confirmation button 338.

[0031] 3B , the example 320 further includes a dedicated template 340, which is used to determine whether the game screen 322 belongs to a character selection scene of a specific game. The dedicated template 340 includes a region 342, target text 344 (i.e., "character selection") corresponding to the region 342, a region 346, and a target object 348 (i.e., a selection confirmation button) corresponding to the region 346. The dedicated template 340 may indicate that when the target text 344 appears in the region 342 (or a specific position) and the target object 348 appears in the region 346 (or a specific position) of the game screen, it can be determined that the game screen belongs to a character selection scene. In example 320, after obtaining an area of ​​game screen 322 corresponding to area 342 of dedicated template 340, it can be determined that the area contains target text 344, and after obtaining an area of ​​game screen 322 corresponding to area 346 of dedicated template 340, it can be determined that the area contains target object 348, and therefore it can be determined that game screen 322 belongs to the character selection scene.

[0032] Because the dedicated template 340 is generated specifically for the currently running game, the dedicated template 340 can include more target features than a general-purpose template, which means stricter matching conditions, allowing for more accurate recognition of whether a game screen belongs to a character selection scene and reducing the chance of misidentifying a screen that is not a character selection scene as a character selection scene. Using a template to determine whether an application screen matches a target scene can improve the processing speed of the process, reducing delays and improving the user experience.

[0033] In some examples, to trigger an engagement of the vehicle cockpit based on the application screen and the user interaction, a region of interest associated with the user interaction can be determined based on the application screen and the user interaction, and then an object contour within the region of interest can be determined and the engagement of the vehicle cockpit can be triggered based on the object contour. In some examples, to determine the region of interest associated with the user interaction, coordinates associated with the user interaction in the application screen can be obtained, and then the region of interest in the application screen can be determined based on the coordinates. In some examples, to trigger an engagement of the vehicle cockpit, a color characteristic of the object contour can be determined and then vehicle ambient light can be adjusted based on the color characteristic of the object contour.

[0034] FIG. 4 illustrates a schematic diagram of an exemplary process 400 for obtaining an object contour, according to some embodiments of the present disclosure. As illustrated in FIG. 4 , a game screen 402 is a game screen in a character selection scene, and includes characters 404, 406, 408, and 410. In the exemplary process 400, a user taps on an area where the character 404 is located. Then, the process 400 determines coordinates 412 of a touch event (i.e., a touch event) corresponding to the tap (i.e., a user interaction) and can determine a region of interest 414 based on the coordinates 412. For example, the region of interest 414 may be a circular region whose center is the coordinate 412 of the touch event and whose radius is a specific predefined value. Alternatively, the region of interest 414 may be a square region whose center is the coordinate 412 of the touch event and whose side lengths are specific predefined values.

[0035] After determining the region of interest 414, contour segmentation can be performed on the region of interest 414 to obtain an object contour 416 within the region of interest 414. Because the object contour 416 corresponds to the character 404, an interaction of the vehicle cockpit can be triggered based on the object contour 416 to achieve the effect that when a user selects the object 414, a corresponding change occurs in the vehicle cockpit environment. For example, the process 400 can perform grayscale processing on the region of interest 414 to obtain a grayscaled region of interest 414. The process 400 can then execute an edge detection algorithm on the grayscaled region of interest 414 to generate a binarized edge image including edge information for the region of interest 414, where edge pixel points may be marked as white and non-edge pixel points may be marked as black. The process 400 can then apply a contour segmentation algorithm to the binarized edge image to generate the object contour 416 in the image.

[0036] After generating the object contour 416, the process 400 can determine color characteristics of the object contour 416 and then adjust the vehicle's ambient lighting based on the color characteristics of the object contour 416. In some embodiments, the color characteristics may include at least one of a dominant color, a strip light ratio, or a brightness ratio. In some embodiments, the process 400 can determine a dominant color of the interior image of the object contour 416 and adjust the color of the ambient lighting based on the dominant color. For example, if the dominant color inside the object contour 416 is blue, the vehicle's ambient lighting can be adjusted to blue. In some embodiments, the vehicle cockpit can support simultaneous illumination of ambient lights of different colors, and the process 400 can determine a strip light ratio inside the object contour 416 and adjust the ambient lighting based on the strip light ratio. For example, if the ratio of blue to red inside the object contour 416 is the highest, blue and red ambient lights can be illuminated simultaneously. In some embodiments, process 400 can determine the brightness ratio of each strip light within object outline 416 and adjust the color and brightness of the ambient light based on the brightness ratio. For example, if the ratio of blue to red is highest within object outline 416 and the red is more bright than the blue, then the blue and red ambient lights can be turned on simultaneously, with the red set to bright and the blue set to dark.

[0037] In some embodiments, after determining the strip light ratio within object outline 416, the strip light ratio can be compared to a predefined strip light ratio for each character. If the strip light ratio within object outline 416 matches the strip light ratio for a character, the color of the ambient light can be adjusted based on the color scheme of the character.

[0038] In this way, the object selected by the user can be determined based on the touch event coordinates of the user interaction and the application screen, and the characteristics of the object can be determined by contour segmentation and contour color feature extraction without using target detection technology based on artificial intelligence, and the vehicle cockpit environment can be adjusted according to the object characteristics. In this way, the vehicle cockpit environment can be adapted to the characteristics of the object selected by the user, and the user can be more immersed in the atmosphere of the application.

[0039] In some embodiments, to determine the color feature of the object contour, a plurality of strip lights that the vehicle ambient light can display can be determined, and pixels in the object contour can be clustered into a plurality of pixel sets based on the plurality of strip lights, and the color feature of the object contour can be determined based on the number of pixels in the plurality of pixel sets.

[0040] FIG. 5 illustrates a schematic diagram of an example process 500 for extracting color features of an object contour, according to some embodiments of the present disclosure. As illustrated in FIG. 5 , the process 500 may determine all strip lights that the ambient light can display, and then determine multiple target strip lights from all the strip lights. These target strip lights may be, for example, strip lights with large color differences. For example, a vehicle may support 20 types of strip lights. Then, five types of strip lights with large color differences, such as red, yellow, purple, blue, and green, may be selected from these 20 types of strip lights, as illustrated in FIG. 5 . The process 500 may then cluster pixels in the object contour 502 into five pixel sets according to color: a pixel set 504 corresponding to red, a pixel set 506 corresponding to yellow, a pixel set 508 corresponding to purple, a pixel set 510 corresponding to blue, and a pixel set 512 corresponding to green. During clustering, the process 500 may calculate the distances from the colors of the pixels to be clustered to the five color centers, with the values ​​representing red, yellow, purple, blue, and green at the centers, respectively. If the minimum value of the five calculated distances is smaller than a predetermined threshold, it indicates that the color of the pixel is closest to the color corresponding to the center value, and the pixel can be clustered into the corresponding set. If the minimum values ​​of all of the five calculated distances are equal to or greater than a predetermined threshold, it indicates that none of the five colors is close to the pixel, and the pixel will not be clustered into any of the five sets.

[0041] In the example shown in FIG. 5 , pixels in object outline 502 that are closer to red (which may include, for example, light red or dark red) may be clustered into set 504, pixels that are closer to yellow may be clustered into set 506, pixels that are closer to purple may be clustered into set 508, pixels that are closer to blue may be clustered into set 510, and pixels that are closer to green may be clustered into set 512 (in the example shown in FIG. 5 , there are no pixels that are closer to green, and therefore no pixels are clustered into set 512). As shown in FIG. 5 , set 510 has the largest number of pixels, followed by set 514, set 516, and set 518. In some embodiments, a blue strip light of the ambient light may be illuminated with a predominant color tone of blue corresponding to set 510. In some embodiments, multiple strip lights of the ambient light, for example, blue, red, and yellow, may be illuminated based on the strip light ratio.

[0042] In this way, color features can be extracted from the object contours, and the color of the ambient light in the vehicle cockpit can be adjusted based on the color features. Because the process requires a small amount of calculation, it can achieve both calculation speed and implementation effectiveness, reducing calculation delays while immersing the user in the target scene and improving the user experience.

[0043] In some embodiments, a similar method can be used to determine whether a user interaction selects an object in an application. In these embodiments, all strip lights that can be displayed by the vehicle ambient light can be determined, and then multiple target strip lights can be determined from all of the strip lights, and white and black can be added to the multiple target strip lights. Then, pixels in the object outline can be clustered into multiple pixel sets corresponding to the multiple target strip lights (including white and black), and whether a user interaction is a target interaction can be determined based on the number of pixels in the multiple pixel sets. For example, a target interaction can be the selection of an object. If a user interaction does not select an object but taps an invalid background area, the user interaction is not a target interaction. Because the background color of most application screens is close to white or black, if the number of pixels in the pixel set corresponding to white exceeds a predetermined percentage or the number of pixels in the pixel set corresponding to black exceeds a predetermined percentage, it can be determined that the user interaction taps an invalid background area rather than a valid object area. If the user interaction is the target interaction, the color of the ambient light in the vehicle cockpit can be adjusted based on the main color tone, strip light ratio or brightness ratio.

[0044] In this way, if the user interaction is not the target interaction, the false triggering of the vehicle cockpit interlock can be reduced, and the calculation process can be terminated early, saving calculation resources and improving the user experience.

[0045] In some examples, based on the object contour, a target contour of a predetermined target object can be obtained to trigger an engagement of the vehicle cockpit. Then, based on the object contour and the target contour, a presence of the target object can be determined within the region of interest. Then, based on the predetermined setting information corresponding to the target object, an engagement of the vehicle cockpit can be triggered. In some examples, to trigger an engagement of the vehicle cockpit, at least one of vehicle ambient lighting, cockpit lighting, car audio, seats, air conditioning, or fragrance can be adjusted based on the predetermined setting information corresponding to the target object.

[0046] FIG. 6 illustrates a schematic diagram of an example process 600 for adjusting a vehicle cockpit environment by recognizing a character corresponding to an object contour, according to some embodiments of the present disclosure. As illustrated in FIG. 6 , the process 600 can determine a region of interest based on coordinates of a game screen and a user interaction, and then obtain an object contour 602 from the region of interest through contour segmentation. Before determining color characteristics of the object contour 602, the object contour 602 can be compared with characters 606-1, 606-2, ..., 606-N (collectively referred to as characters 606) pre-stored in a character library 604. Each character 606 includes a character contour and setting information for that character. For example, character 606-1 includes character contour 608 and setting information 610. The process 600 can match the object contour 602 with the character contour of each character 606.

[0047] 6 , since the object contour 602 matches the character contour 614 of the character 612 in the character library 604, it can be determined that the user interaction selected the character 612, and the vehicle cockpit can be triggered to interact with the character 612 based on the setting information 616 of the character 612. The setting information 616 may include, for example, the main color tone of the ambient light, the strip light ratio, the brightness ratio, and the sound effects, seat vibration intensity, the air conditioning strength, the type of fragrance, etc., for the character 612. In this way, the interaction form of the cockpit environment can be enriched, and further, the vehicle cockpit can be triggered to interact with a specific character based on the predetermined setting information, so that the cockpit environment can better fit the character and the user can be more immersed in the game.

[0048] As described above, a game application (or other application) may be installed in an in-vehicle infotainment system, allowing the in-vehicle infotainment system to easily acquire game screens and user interactions of the game application running thereon. However, there are often more game applications on a user's personal mobile device than on the in-vehicle infotainment system, and in some scenarios, users tend to play games using their own mobile device rather than the in-vehicle infotainment system. In such cases, the in-vehicle infotainment system needs to acquire game screens and user interactions from the user's mobile device.

[0049] In some embodiments, the application that the user is operating and that is installed on the user's mobile device is a first application, and to obtain user interactions on the application screen displayed in the vehicle cockpit, the application screen and the user interactions can be obtained from a second application that is installed on the user's mobile device and that is related to the vehicle, and the application screen and the user interactions are from the first application.

[0050] FIG. 7 illustrates a schematic diagram of an example process 700 for interfacing with a vehicle cockpit based on user interactions on a user's mobile device, according to some embodiments of the present disclosure. As illustrated in FIG. 7 , a user can operate a game application 706 using a user's mobile device 704 in a vehicle 702. The user's mobile device 704 may be a mobile device such as a mobile phone or a tablet computer, and the game application 706 may be a game application executable on the user's mobile device 704 from any game provider. The user's mobile device 704 also has a control application 708 installed thereon. The control application 708 is capable of acquiring game screens and user interactions from the game application 706 and communicating with an in-vehicle infotainment system 710 to transmit the acquired game screens and user interactions to the in-vehicle infotainment system 710. For example, the control application 708 may be a screen projection application, and the user can transmit the game screens and user interactions to the in-vehicle infotainment system 710 using a screen projection or screen mirroring function. The control application 708 may be a communication application that simply transmits game screens and user interactions to the in-vehicle infotainment system 710 and does not require screen projection.

[0051] In this manner, the in-vehicle infotainment system 710 can acquire the game screen and user interaction from the control application 708, analyze the game screen and user interaction, and then send a control command to the interlocking unit 712 to adjust the cockpit environment of the vehicle 702. The interlocking unit 712 may include, for example, ambient lighting, audio, air conditioning, seats, fragrance, etc. in the cockpit. In this manner, the interlocking of the vehicle cockpit can be triggered even when the user operates the game application on their mobile device (e.g., selects a character on a character selection screen), thereby improving the versatility of the cockpit interlocking, and not being limited to only being triggered when operating a game on the in-vehicle infotainment system.

[0052] FIG. 8 illustrates a block diagram of a vehicle cockpit interlocking device 800 according to some embodiments of the present disclosure. As illustrated in FIG. 8, the device 800 includes a user interaction acquisition unit 802 configured to acquire a user interaction with an application screen displayed within the vehicle cockpit. The device 800 further includes a vehicle cockpit interlocking unit 804 configured to trigger vehicle cockpit interlocking based on the user interaction with the application screen. It can be appreciated that the device 800 of the present disclosure can achieve at least one of many advantages that can be achieved by the above-described method or process. For example, vehicle cockpit interlocking can be achieved while avoiding the need for a data interface from an application provider and the use of artificial intelligence-based target detection technology, thereby reducing costs, delays, improving versatility, and improving the user experience.

[0053] FIG. 9 shows a schematic block diagram of an exemplary device 900 suitable for implementing embodiments of the present disclosure. For example, the control unit 104 of FIG. 1 may be the exemplary device 900 shown in FIG. 9. As shown, the device 900 includes a computing unit 901 that can perform various appropriate operations and processes in accordance with computer program instructions stored in a read-only memory (ROM) 902 or loaded from a storage unit 908 into a random access memory (RAM) 903. The RAM 903 may further store various programs and data necessary for the operation of the device 900. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other by a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0054] Several components in device 900 are connected to I / O interface 905, including input units 906 such as a keyboard, a mouse, etc., output units 907 such as various types of displays, speakers, etc., storage units 908 such as magnetic disks, optical disks, etc., and communication units 909 such as a network card, modem, wireless communication transceiver, etc. The communication units 909 allow device 900 to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunication networks.

[0055] The computing unit 901 may be various general-purpose and / or special-purpose processing components having processing and computing capabilities. Examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, computing units that execute various machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs each of the methods and processes described above, such as method 200. For example, in some embodiments, method 200 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 908. In some embodiments, some or all of the computer program may be loaded and / or installed into the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, it may perform one or more steps of the method 200 described above. Preferably, in alternative embodiments, the computing unit 901 may be configured to perform the method 200 in any other suitable manner (eg, by firmware).

[0056] The functionality described herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc.

[0057] Program code implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus, such that when the program code is executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are performed. The program code may be executed entirely on a machine, partially on a machine, partially on a machine as a separate software package and partially on a remote machine, or entirely on a remote machine or server.

[0058] In the context of this disclosure, a machine-readable medium may be a tangible medium that contains or can store a program for use by or in connection with an instruction execution system, device, or apparatus. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination thereof. More specific examples of machine-readable storage media include one or more wire-based electrical connections, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a convenient compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. Additionally, although operations are described in a particular order, this should not be understood as requiring such operations to be performed in the particular order or sequence shown, or that all illustrated operations be performed. In some cases, multitasking and parallel processing may be advantageous. Similarly, although the above discussion includes details of specific implementations, these should not be construed as limiting the scope of the present disclosure. Certain features described in the context of a single embodiment may also be implemented in combination in one embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments alone or in any suitable subcombination.

[0059] Although the present subject matter has been described in language specific to structural features and / or logical operations of methods, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or operations described above. Rather, the specific features and operations described above are merely example forms of implementing the claims.

Claims

1. A vehicle cockpit interlocking method, comprising: acquiring user interactions with application screens displayed within the vehicle cockpit; and triggering vehicle cockpit interaction based on the application screen and the user interaction.

2. Triggering an engagement of a vehicle cockpit based on the application screen and the user interaction includes: determining whether the application screen matches a target scene; and in response to the application screen matching the target scene, triggering an engagement of a vehicle cockpit based on the application screen and the user interaction.

3. determining whether the application screen matches the target scene; obtaining a template for the target scene, the template including target features corresponding to the target scene; and determining that the application screen matches the target scene in response to determining that the application screen has a feature that corresponds to the target feature.

4. Triggering an engagement of a vehicle cockpit based on the application screen and the user interaction in response to the application screen matching the target scene includes: determining, based on the application screen and the user interaction, a region of interest associated with the user interaction; determining an object contour within the region of interest; and triggering a vehicle cockpit engagement based on the object contour.

5. determining the region of interest associated with the user interaction based on the application screen and the user interaction, obtaining coordinates associated with the user interaction on the application screen; and determining an area of ​​interest on the application screen based on the coordinates.

6. Triggering an engagement of a vehicle cockpit based on the object contour includes: determining color characteristics of the object contour; and adjusting vehicle ambient lighting based on the color features of the object contour.

7. The step of determining the color features of the object contour comprises: determining a plurality of strip lights that the vehicle ambient light is capable of displaying; clustering pixels within the object contour into a plurality of pixel sets based on the plurality of strip lights; and determining the color feature of the object contour based on the number of pixels in the plurality of pixel sets.

8. The method of claim 6 , wherein the color characteristics include at least one of a dominant color tone, a strip light ratio, or a brightness ratio.

9. Triggering an engagement of a vehicle cockpit based on the object contour includes: obtaining a target contour of a predetermined target object; determining the presence of the target object within the region of interest based on the object contour and the target contour; and triggering an engagement of a vehicle cockpit based on predetermined configuration information corresponding to the target object.

10. Triggering an engagement of a vehicle cockpit based on the predetermined setting information corresponding to the target object includes:

10. The method of claim 9, further comprising adjusting at least one of vehicle ambient lighting, cockpit lighting, car audio, seats, air conditioning, or fragrance based on the predetermined setting information corresponding to the target object.

11. The application is a first application installed on a user's mobile device, and the step of acquiring the user's interaction with the application screen displayed in the vehicle cockpit includes:

2. The method of claim 1, further comprising: obtaining the user interactions on the application screen from a second application related to the vehicle that is installed on the user's mobile device, the application screen and the user interactions being from the first application.

12. The method of claim 2 , wherein the application is a game application and the target scene is a game character selection scene.

13. An electronic device, at least one processor; and a memory coupled to the at least one processor and storing instructions that, when executed by the at least one processor, cause the electronic device to perform the method of any one of claims 1 to 12.

14. A computer readable storage medium having stored thereon a computer program, the program being configured, when executed by a processor, to cause the processor to perform the method of any one of claims 1 to 12.

15. 13. A computer program product, the computer program product tangibly stored on a non-transitory computer readable medium and comprising machine-executable instructions that, when executed, cause a machine to perform the method of any one of claims 1 to 12.

Images