Method, apparatus, medium and computer program product for vehicle cabin linkage

Abstract

The disclosure relates to a method, an apparatus, a medium and a computer program product for vehicle cabin linkage. The method includes obtaining a user interaction for an application screen displayed within a vehicle cabin, and triggering a vehicle cabin linkage based on the application screen and the user interaction. Through the mode, the scene and the interaction object of the application can be determined based on the application picture and the user interaction under the condition that a data interface from an application provider is not called, so that the vehicle engine system provider can independently realize vehicle cabin linkage, and the cost can be saved. In addition, compared with a scheme of using an object detection technology based on artificial intelligence, the scheme uses user interaction data to help determine an interaction object in a picture, so that the processing speed can be improved, the response time can be shortened, training of a special artificial intelligence model is avoided, and the universality is improved.

Description

Technical Field

[0001] The present disclosure relates generally to the field of vehicle control, and more particularly to methods, devices, media, and computer program products for vehicle-cockpit linkage. Background Art

[0002] With the continuous advancement of technology, cars are no longer simply vehicles; they are gradually becoming mobile intelligent spaces. The prevalence of in-cabin computer systems and smart mobile devices has brought greater convenience and comfort to users. Therefore, in addition to providing basic driving information and entertainment functions, modern in-vehicle technology is also developing in a more intelligent and personalized direction.

[0003] With the prevalence of smart mobile devices and the upgrade of vehicle-mounted systems, users have become accustomed to using smart devices such as mobile phones and tablets in the vehicle cabin to operate various applications, ranging from navigation to music, video, and games. With technological advances, the vehicle cabin environment can also be linked based on user operation of applications, providing users with a more personalized and comfortable driving environment and enhancing the user experience. Summary of the Invention

[0004] In a first aspect of an embodiment of the present disclosure, a vehicle-cockpit linkage method is provided, comprising obtaining a user interaction with an application screen displayed in a vehicle cockpit, and triggering the vehicle-cockpit linkage based on the application screen and the user interaction.

[0005] In a second aspect of an embodiment of the present disclosure, an electronic device is provided. The electronic device includes one or more processors; and a memory coupled to the at least one processor and having instructions stored therein. When executed by the at least one processor, the instructions cause the electronic device to perform an action, the action including obtaining a user interaction with an application screen displayed in a vehicle cabin, and triggering vehicle cabin linkage based on the application screen and the user interaction.

[0006] In a third aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, storing a computer program that, when executed by a processor, implements a method for vehicle-cockpit linkage. The method includes obtaining user interactions with an application screen displayed in the vehicle cockpit, and triggering vehicle-cockpit linkage based on the application screen and the user interactions.

[0007] In a fourth aspect of an embodiment of the present disclosure, a computer program product is provided. The computer program product is tangibly stored on a non-volatile computer-readable medium and includes machine-executable instructions that, when executed, cause a machine to implement a method for vehicle-cockpit linkage. The method includes obtaining user interactions with an application screen displayed in a vehicle cockpit, and triggering vehicle-cockpit linkage based on the application screen and the user interactions.

[0008] It should be understood that the contents described in the Summary of the Invention section are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram illustrating an example environment in which various embodiments of the present disclosure may be implemented;

[0011] Figure 2 A flowchart showing a method for vehicle-cabin linkage according to some embodiments of the present disclosure is shown;

[0012] Figure 3A-3B A schematic diagram illustrating an example of using a general template and a dedicated template to identify a character selection scenario according to some embodiments of the present disclosure;

[0013] Figure 4 A schematic diagram illustrating an example process of obtaining an object contour according to some embodiments of the present disclosure;

[0014] Figure 5 A schematic diagram illustrating an example process of extracting color features of an object contour according to some embodiments of the present disclosure;

[0015] Figure 6 A schematic diagram illustrating an example process of adjusting a vehicle cabin environment by identifying a role corresponding to an object outline according to some embodiments of the present disclosure;

[0016] Figure 7 A schematic diagram illustrating an example process of interfacing with a vehicle cockpit based on user interaction on a user's mobile device according to some embodiments of the present disclosure;

[0017] Figure 8 A schematic diagram showing a vehicle-cabin linkage apparatus according to some embodiments of the present disclosure; and

[0018] Figure 9 A block diagram of a device in which various embodiments of the present disclosure may be implemented is shown. DETAILED DESCRIPTION

[0019] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0020] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to." The term "based on" should be understood as "based at least in part on." The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment." The terms "first," "second," etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0021] As mentioned above, vehicle cabins have gradually evolved into mobile multimedia entertainment spaces, where drivers and passengers can access a variety of applications, such as navigation, music, video, and games, through the vehicle's head-mounted system or their mobile devices to meet diverse user needs. Some vehicles are equipped with ambient lighting systems, such as adjustable color and brightness. Control of these ambient lighting is typically managed by the vehicle's head-mounted system, which issues commands to adjust parameters such as color and brightness based on preset patterns or real-time data signals. Furthermore, some vehicles are equipped with fragrance systems that distribute different scents to create a diverse ambiance. The system can control the release time and intensity of the fragrances. Furthermore, the head-mounted system can issue control commands to systems that interact with the user's senses, such as the vehicle's audio system, seat massage system, or air conditioning system, to change their operating states. By centrally controlling these environmental hardware systems, the head-mounted system can adjust the cabin environment based on external data, creating an immersive experience for the user.

[0022] However, the interaction data between users and applications is typically owned and managed by the application provider. To achieve vehicle cockpit environment linkage, some related technologies allow application providers to provide relevant data interfaces to the vehicle system provider to obtain real-time status data of the application currently being operated by the user. This data may include, for example, the application's identification, type, current scene, interface elements, and so on. However, developing such data interfaces requires a significant amount of time and resources, making them expensive for the vehicle system provider. In other related technologies, artificial intelligence-based object detection technology can be used to detect objects in a single frame, thereby identifying objects within the application that the user is interacting with. However, if the vehicle system has limited computing resources, deploying artificial intelligence models may not be possible. Furthermore, the inference process of the artificial intelligence model may lead to problems such as delays and untimely responses. Furthermore, different artificial intelligence models need to be trained for different applications, and a large amount of training data is required, making the cost of obtaining trained models very expensive.

[0023] To this end, an embodiment of the present disclosure provides a vehicle cockpit linkage solution, which can obtain the application screen displayed in the vehicle cockpit and the user interaction with the application screen, and then trigger the vehicle cockpit linkage based on the application screen and the user interaction. In this way, the application scenario and the object of interaction can be determined based on the application screen and the user interaction without calling the data interface from the application provider, so that the vehicle system provider can independently implement the vehicle cockpit linkage, thereby saving costs. In addition, compared to the solution using artificial intelligence-based object detection technology, this solution uses user interaction data to help determine the interactive objects in the screen, so it can improve processing speed, reduce response time, improve versatility, and save the cost consumed in training artificial intelligence models.

[0024] Figure 1 1 shows a schematic diagram of an example environment 100 in which various embodiments of the present disclosure may be implemented. Figure 1 As shown, environment 100 includes a vehicle 102, which includes a control unit 104, a linkage unit 106, and an application device 108. Control unit 104 can be any device with computing or processing capabilities. For example, control unit 104 can be a vehicle-mounted system, a desktop computer, a laptop computer, a tablet computer, a server, a mobile device, or a vehicle control unit (including the entire vehicle control unit and subsystem control units). Control unit 104 can send control instructions to linkage unit 108 to change the state of linkage unit 108.

[0025] The application device 106 is a device for running the application 110. For example, the application device 106 may be a user's personal mobile device (e.g., a mobile phone, tablet computer, etc.). The application 110 may be any application running on the user's mobile device (e.g., a game application, a music application, a video application, a navigation application, etc.). In some embodiments of the present disclosure, the application device 106 may be the same component as the control unit 104. For example, the control unit 104 and the application device 106 may both refer to the vehicle-mounted system of the vehicle 102. In these embodiments, the application 110 may be any application running on the vehicle-mounted system (e.g., a game application, a music application, a video application, a navigation application, etc.).

[0026] The linkage unit 108 is any component within the cabin of the vehicle 102 that can interact with the user's senses. For example, the linkage unit 108 may include the cabin's ambient lighting, audio system, air conditioning, seats, and fragrance. For example, the ambient lighting may change color, brightness, or flash in response to user operations on the application 110; the audio system may change volume or play specific sounds in response to user operations on the application 110; and the air conditioning system may change wind speed in response to user operations on the application 110.

[0027] like Figure 1 As shown, in environment 100, a user can operate application 110 through application device 106. More specifically, the user can perform user interaction 114 on application screen 112. Application screen 112 may include multiple objects, such as object 116, object 118, etc. User interaction 114 may be a click or touch on application screen 112, and because the click or touch position is within the area associated with object 116, object 116 is selected.

[0028] In some related technologies, control unit 104 can obtain an identifier of the scene of application screen 112, a description of the scene of application screen 112, an identifier of the object selected by the user, and a description of the object selected by the user from an interface provided by the provider of application 110. In other related technologies, control unit 104 can identify the scene of application screen 112 and the object selected by the user based solely on application screen 112 and utilizing artificial intelligence-based object detection technology. However, calling an interface is expensive, and artificial intelligence-based methods not only require a large amount of training data to train models for different applications, but also introduce delays in the model's inference phase, reducing the user experience.

[0029] In the embodiment provided by the present disclosure, the control unit 104 can obtain the application screen 112 and the user interaction 114 from the application device 106 (for example, the application screen 112 and the related user interaction 114 of the application 110 running thereon are captured by the vehicle system, or the application screen 112 and the user interaction 114 are obtained by another application such as the projection application on the application device 106 and transmitted to the control unit 104), so that the scene of the application screen 112, the area clicked or touched by the user interaction 114, whether the area selects a valid object, and the characteristics of the selected object can be analyzed based on the application screen 112 and the user interaction 114. Then, the control unit 104 can send corresponding control instructions to the linkage unit 108 based on this information, thereby realizing vehicle cockpit linkage. In this way, vehicle cockpit linkage can be realized without using the data interface from the application provider and the target detection technology based on artificial intelligence, thereby reducing costs, reducing delays, improving versatility, and enhancing user experience.

[0030] Figure 2 FIG. 2 shows a flow chart of a method 200 for vehicle-cabin linkage according to some embodiments of the present disclosure. The method 200 may be performed by, for example Figure 1 The control unit 104 in the environment 100 shown is executed. Figure 2 As shown, at block 202, method 200 may obtain user interaction with an application screen displayed in a vehicle cabin. Figure 1 In the illustrated environment 100, the control unit 104 can obtain an application screen 112 and a user interaction 114 with respect to the application screen 112. For example, when the control unit 104 and the application device 106 are two separate components (e.g., the control unit 104 is a vehicle-mounted system and the application device 106 is a user's mobile device), the control unit 104 can obtain the application screen 112 and the user interaction 114 from the application device 106 (e.g., via a screen projection application on the application device 106). When the control unit 104 and the application device 106 are the same component (e.g., both are vehicle-mounted systems), the control unit 104 can directly obtain the application screen 112 and the user interaction 114 of the application 110 running thereon.

[0031] At block 204, method 200 may trigger vehicle cockpit linkage based on the application screen and user interaction. Figure 1In the illustrated environment 100, the control unit 104 can analyze the scene of the application picture 112, the area clicked or touched by the user interaction 114, whether the clicked area is selected on a valid object, and the features of the selected object, based on the application picture 112 and the user interaction 114. Then, the control unit 104 can send corresponding control instructions to the linkage unit 108 based on these information, so as to realize the linkage of the vehicle cabin. The linkage of the vehicle cabin may, for example, include adjusting the color and brightness of the ambient light, making the ambient light flicker, adjusting the volume of the sound, making the sound play a specific sound effect, turning on the air conditioner, changing the intensity of the air conditioner, making the seat vibrate, changing the type and intensity of the fragrance, and the like, in response to the user interaction 114.

[0032] In this way, the scene of the application and the object of the interaction can be determined based on the application picture and the user interaction without calling the data interface from the application provider, so that the car machine system provider can independently realize the linkage of the vehicle cabin, thereby saving costs. In addition, compared with the solution using the object detection technology based on artificial intelligence, the data of the user interaction can be used to help determine the interactive object in the picture, so that the processing speed can be improved, the response time can be reduced, the universality can be improved, and the cost consumed for training the artificial intelligence model can be saved.

[0033] The solution provided by the present disclosure can be applied to any type of application and any suitable scene in the application. For example, the solution provided by the present disclosure can be applied to a character selection scene or a map selection scene in a game application, a playlist selection scene or a favorite music type selection scene in a music application, a restaurant selection scene in a life service application, and the like. Among them, the solution provided by the present disclosure is particularly suitable for a character selection scene in a game application, in which the user's attention has not yet fully focused on the game process, and at this time, the linkage of the vehicle cabin can quickly immerse the user into the atmosphere of the game, thereby better improving the user experience. Therefore, the following describes other embodiments of the present disclosure by taking the character selection scene of the game application as an example.

[0034] In some embodiments, in order to trigger the linkage of the vehicle cabin based on the application picture and the user interaction, it can be determined whether the application picture matches a target scene. Then, in response to the application picture matching the target scene, the linkage of the vehicle cabin can be triggered based on the application picture and the user interaction. In some embodiments, in order to determine whether the application picture matches the target scene, a template for the target scene can be obtained, the template including target features corresponding to the target scene. Then, in response to determining that the application picture has features corresponding to the target features, it can be determined that the application picture matches the target scene.

[0035] Figure 3A-3BA schematic diagram illustrating an example of using a general template and a dedicated template to identify a character selection scene according to some embodiments of the present disclosure is shown. In some embodiments, in the absence of a dedicated template for a target scene of a specific application, a general template for the target scene can be used to determine whether the application screen matches the target scene. Figure 3A FIG. 3 is a schematic diagram showing an example 300 of using a general template to identify a character selection scene in a game application according to some embodiments of the present disclosure. Figure 3A As shown, example 300 includes a game screen 302 in a game application. Game screen 302 includes select character text 304, character 306, character 308, character 310, character 312, character preview 314 of character 306, return button 316, and confirm selection button 318.

[0036] like Figure 3A As shown, example 300 also includes a universal template 320, which is used to determine whether the game screen 302 belongs to the character selection scene. Universal template 320 includes area 322 and target text 324 (i.e., "select a role"). Universal template 320 can indicate that if target text 324 appears in area 322 of the game screen, it can be determined that the game screen belongs to the character selection scene. In example 300, an area corresponding to area 322 in universal template 320 in the game screen 302 can be obtained, and then it is determined that the target text 324 is included in the area, so it can be determined that the game screen 302 belongs to the character selection scene. In some embodiments, the text content in the area can be obtained by image recognition, and then compared with the target text 324, rather than comparing the image of the area with the image in area 322 of universal template 320. In this way, it can be accurately determined that the game screen 302 matches the universal template 320 even when the character selection text 304 has different fonts.

[0037] In this way, as long as the game screen 302 has the target feature indicated by the universal template 320 (that is, the target text 324 is in the target area 322), it can be determined that the game screen 302 belongs to the character selection scene, without having to identify other elements in the game screen 302 (for example, character 306, character 308, character 310, character 312, character preview 314, return button 316, and confirm selection button 318). In this way, the speed and versatility of target scene recognition can be improved.

[0038] For ease of understanding, the general template 302 is shown in the form of an image in example 300, but the general template 302 can also be stored in other data formats. For example, the target area 322 can be stored as the coordinates of the two diagonal corners of the rectangular box, and the target text 324 can be stored as independent text.

[0039] In some embodiments, the control unit may obtain an identifier of the running application, and then read a pre-stored dedicated template for the application based on the identifier, and determine whether the application screen matches the target scene based on the dedicated template. Figure 3B FIG. 3 is a schematic diagram showing an example 320 of using a dedicated template to identify a character selection scene in a game application according to some embodiments of the present disclosure. Figure 3B As shown, example 320 includes a game screen 322. Game screen 322 includes select character text 324, character 326, character 328, character 330, character 332, character preview 334 of character 326, return button 336, and confirm selection button 338.

[0040] like Figure 3B As shown, example 320 also includes a dedicated template 340, which is used to determine whether a game screen 322 belongs to a character selection scene for a specific game. Dedicated template 340 includes area 342, target text 344 (i.e., "Select a character") corresponding to area 342, area 346, and target object 348 (i.e., a confirm selection button) corresponding to area 346. Dedicated template 340 can indicate that if target text 344 appears in area 342 of the game screen (or at a specific location) and target object 348 appears in area 346 (or at a specific location), then the game screen can be determined to belong to a character selection scene. In example 320, an area corresponding to area 342 in dedicated template 340 can be obtained in the game screen 322, and then the target text 344 is determined to be included in the area. An area corresponding to area 348 in dedicated template 340 can also be obtained in the game screen 322, and then the target object 348 is determined to be included in the area. Therefore, it can be determined that the game screen 322 belongs to a character selection scene.

[0041] Because dedicated template 340 is generated specifically for the currently running game, it can include more target features than a general template. This means stricter matching conditions, allowing for more accurate identification of game screens as character selection scenes, reducing the chance of incorrectly identifying non-character selection scenes as such. Using a template to determine whether an application screen matches a target scene can speed up the process, reducing latency and improving the user experience.

[0042] In some embodiments, to trigger vehicle-cockpit linkage based on the application screen and the user interaction, a region of interest associated with the user interaction can be determined based on the application screen and the user interaction, and then the outline of an object within the region of interest can be determined, and the vehicle-cockpit linkage can be triggered based on the object outline. In some embodiments, to determine the region of interest associated with the user interaction, coordinates associated with the user interaction in the application screen can be obtained, and then the region of interest in the application screen can be determined based on the coordinates. In some embodiments, to trigger vehicle-cockpit linkage, color features of the object outline can be determined, and then the vehicle ambient light can be adjusted based on the color features of the object outline.

[0043] Figure 4 FIG. 4 is a schematic diagram illustrating an example process 400 for obtaining an object contour according to some embodiments of the present disclosure. Figure 4 As shown, game screen 402 is a game screen in a character selection scene, and game screen 402 includes character 404, character 406, character 408, and character 410. In the example process 400, the user clicks on the area where character 404 is located, and then process 400 can determine the coordinates 412 of the touch event (i.e., touch event) corresponding to the click (i.e., user interaction), and determine the area of ​​interest 414 based on the coordinates 412. For example, the area of ​​interest 414 can be a circular area with the coordinates 412 of the touch event as the center and a predefined specific value as the radius. In addition, the area of ​​interest 414 can also be a square area with the coordinates 412 of the touch event as the center and a predefined specific value as the side length.

[0044] After determining the region of interest 414, the region of interest 414 can be segmented to obtain an object outline 416 within the region of interest 414. Since the object outline 416 corresponds to the character 404, the vehicle cabin linkage can be triggered based on the object outline 416 to achieve the effect of a corresponding change in the vehicle cabin environment when the user selects the object 414. For example, the process 400 can perform grayscale processing on the region of interest 414 to obtain a grayscale region of interest 414. The process 400 can then perform an edge detection algorithm on the grayscale region of interest 414 to generate a binary edge image containing edge information for the region of interest 414. In the binary edge image, edge pixels can be marked as white, and non-edge pixels can be marked as black. The process 400 can then apply a contour segmentation algorithm to the binary edge image to generate the object outline 416 in the image.

[0045] After generating object outline 416, process 400 may determine the color characteristics of object outline 416 and then adjust the vehicle's ambient lighting based on the color characteristics of object outline 416. In some embodiments, the color characteristics may include at least one of a dominant color, a ratio of color bands, or a brightness ratio. In some embodiments, process 400 may determine the dominant color of the image within object outline 416 and adjust the color of the ambient lighting based on this dominant color. For example, if the dominant color within object outline 416 is blue, the vehicle's ambient lighting may be adjusted to blue accordingly. In some embodiments, the vehicle cabin may support simultaneous activation of ambient lighting of different colors. In this case, process 400 may determine the ratio of color bands within object outline 416 and adjust the ambient lighting based on this ratio. For example, if the ratio of blue to red is highest within object outline 416, both blue and red ambient lighting may be activated simultaneously. In some embodiments, process 400 may determine the brightness ratio of each color band within object outline 416 and then adjust the color and brightness of the ambient lighting based on this brightness ratio. For example, if the ratio of blue to red is the highest inside the object outline 416 and red has a higher brightness than blue, the blue and red ambient lights may be turned on at the same time, with red set to be brighter and blue set to be darker.

[0046] In some embodiments, after determining the color ratio within the object outline 416, the color ratio can be compared with predefined color ratios for various characters. If the color ratio within the object outline 416 matches the color ratio of a certain character, the color of the ambient light can be adjusted based on the color scheme of the character.

[0047] In this way, the user's selected object can be determined based on the coordinates of the touch event and the application screen. Without using AI-based object detection technology, the object's characteristics can be determined through contour segmentation and extraction of color features. This allows the vehicle cabin environment to be adjusted based on the object's characteristics. This allows the vehicle cabin environment to match the characteristics of the selected object, further immersing the user in the application's atmosphere.

[0048] In some embodiments, to determine the color characteristics of an object's outline, multiple color bands that can be displayed by the vehicle's ambient lighting can be determined, and based on the multiple color bands, pixels within the object's outline can be clustered into multiple pixel sets. The color characteristics of the object's outline can then be determined based on the number of pixels in the multiple pixel sets.

[0049] Figure 5 FIG. 5 is a schematic diagram illustrating an example process 500 for extracting color features of an object contour according to some embodiments of the present disclosure. Figure 5As shown, process 500 can determine all color strips that can be presented by the vehicle ambient light, and then determine multiple target color strips from all the color strips. These target color strips can be, for example, color strips with large color differences. For example, if the vehicle can support 20 color strips, then 5 color strips with large color differences can be selected from these 20 color strips, such as Figure 5 , yellow, purple, blue, and green as shown in . Then, process 500 can cluster the pixels in object outline 502 into five pixel sets based on color, namely, pixel set 504 corresponding to red, pixel set 506 corresponding to yellow, pixel set 508 corresponding to purple, pixel set 510 corresponding to blue, and pixel set 512 corresponding to green. During the clustering process, process 500 can calculate the distances from the color of the pixel to be clustered to the five color centers, with the values ​​representing red, yellow, purple, blue, and green as the centers. If the minimum of the five calculated distances is less than a predetermined threshold, it means that the color of the pixel is closest to the color corresponding to the center value, and the pixel can be clustered into the corresponding set. If the minimum of the five calculated distances is not less than the predetermined threshold, it means that none of the five colors is close to the pixel, and the pixel will not be clustered into any of the five sets.

[0050] In such Figure 5 In the example shown, pixels close to red (e.g., light red or dark red, etc.) within the object outline 502 may be clustered into a set 504, pixels close to yellow may be clustered into a set 506, pixels close to purple may be clustered into a set 508, pixels close to blue may be clustered into a set 510, and pixels close to green may be clustered into a set 512 (in the example shown). Figure 5 In the example shown, there are no pixels that are close to green, so no pixels are clustered into set 512). Figure 5 As shown, set 510 has the largest number of pixels, followed by set 514, set 516, and set 518. In some embodiments, the blue color corresponding to set 510 can be used as the primary color to illuminate the blue color band of the ambient light. In some embodiments, multiple color bands of the ambient light can be illuminated based on the color band ratio, such as blue, red, and yellow.

[0051] This method extracts color features from object outlines and uses them to adjust the color of the vehicle's cabin ambient lighting. This process requires minimal computation, balancing computational speed and performance. This reduces computational latency while immersing users in the target scene, enhancing the user experience.

[0052] In some embodiments, a similar method can be used to determine whether the user interaction selects an object in the application. In these embodiments, all color bands that the vehicle ambient light can present can be determined, and then multiple target color bands are determined from all the color bands, and white and black are additionally added to the multiple target color bands. Then, the pixels in the object outline can be clustered into multiple pixel sets corresponding to multiple target color bands (including white and black), and whether the user interaction is a target interaction is determined based on the number of pixels in the multiple pixel sets. For example, the target interaction can be to select an object. If the user interaction does not select the object but clicks on an invalid background area, the user interaction is not a target interaction. Since the background color of most application screens is close to white or black, if the number of pixels in the pixel set corresponding to white exceeds a predetermined ratio, or the number of pixels in the pixel set corresponding to black exceeds a predetermined ratio, it can be determined that the user interaction clicked on an invalid background area rather than a valid object area. In the case where the user interaction is a target interaction, the color of the ambient light in the vehicle cabin can be adjusted according to the main color tone, color band ratio or brightness ratio.

[0053] In this way, it is possible to reduce the erroneous triggering of vehicle-cockpit linkage when the user interaction is not the target interaction, and to terminate the calculation process in advance, thereby saving computing resources and improving the user experience.

[0054] In some embodiments, to trigger vehicle-cabin linkage based on an object outline, a predetermined target outline of a target object may be obtained. The presence of the target object within a region of interest may then be determined based on the object outline and the target outline. Vehicle-cabin linkage may then be triggered based on predetermined configuration information corresponding to the target object. In some embodiments, to trigger vehicle-cabin linkage, at least one of the following may be adjusted based on the predetermined configuration information corresponding to the target object: vehicle ambient lighting, cabin lighting, vehicle audio, seats, air conditioning, or fragrance.

[0055] Figure 6 FIG. 6 is a schematic diagram illustrating an example process 600 for adjusting a vehicle cabin environment by identifying a role corresponding to an object outline according to some embodiments of the present disclosure. Figure 6As shown, process 600 can determine a region of interest based on the coordinates of the game screen and user interaction, and then obtain an object outline 602 from the region of interest through outline segmentation. Before determining the color features of object outline 602, object outline 602 can be compared with characters 606-1, 606-02, ..., 606-N (collectively referred to as characters 606) pre-stored in a character library 604. Each character 606 includes a character outline and configuration information for that character. For example, character 606-1 includes a character outline 608 and configuration information 610. Process 600 can match object outline 602 with the character outlines of each character 606.

[0056] exist Figure 6 In the example shown, the object outline 602 matches the character outline 614 of the character 612 in the character library 604. Therefore, it can be determined that the user interaction has selected the character 612, and the vehicle cockpit linkage can be triggered based on the configuration information 616 of the character 612. The configuration information 616 may include, for example, the main color tone, color band ratio, brightness ratio of the ambient light for the character 612, as well as sound effects, seat vibration intensity, air conditioning wind intensity, fragrance type, etc. In this way, the interactive form of the cockpit environment can be enriched, and the vehicle cockpit linkage can be triggered for a specific character based on predetermined configuration information, so that the cockpit environment is more compatible with the character, thereby enabling the user to better immerse themselves in the game.

[0057] As mentioned above, game applications (or other applications) can be installed on the in-vehicle system, allowing the in-vehicle system to easily access the game screen and user interactions of the game applications running on it. However, compared to the game applications on the in-vehicle system, the game applications on the user's personal mobile device are often richer, and in some scenarios, users prefer to use their own mobile devices rather than the in-vehicle system to play games. In this case, the in-vehicle system needs to obtain the game screen and user interactions from the user's mobile device.

[0058] In some embodiments, the application that the user is operating and is installed on the user's mobile device is a first application. In order to obtain user interaction with the application screen displayed in the vehicle cabin, user interaction with the application screen can be obtained from a second application installed on the user's mobile device and associated with the vehicle, where the application screen and user interaction come from the first application.

[0059] Figure 7 FIG. 7 is a diagram illustrating an example process 700 for interfacing with a vehicle cockpit based on user interaction on a user's mobile device according to some embodiments of the present disclosure. Figure 7As shown, a user can use a user mobile device 704 in a vehicle 702 to operate a game application 706, where the user mobile device 704 can be a mobile device such as a mobile phone or tablet computer, and the game application 706 can be a game application from any game provider that can be run on the user mobile device 706. A control application 708 is also installed on the user mobile device 704. The control application 708 is an application that can obtain game images and user interactions from the game application 706 and can communicate with the vehicle system 710 to transmit the game images and user interactions obtained from the game application 706 to the vehicle system 710. For example, the control application 708 can be a screen casting application, and the user can transmit the game images and user interactions to the vehicle system 710 through the screen casting function or screen mirroring function. The control application 708 can also be a communication application that only transmits the game images and user interactions to the vehicle system 710 without screen casting.

[0060] In this way, after obtaining the game screen and user interaction from the control application 708, the vehicle system 710 can analyze the game screen and user interaction and send control instructions to the linkage unit 712 to adjust the cabin environment of the vehicle 702. The linkage unit 712 may include, for example, ambient lighting, audio, air conditioning, seats, fragrances, etc. in the cabin. In this way, when the user operates the game application on his or her mobile device (for example, selecting a character on the character selection screen), the vehicle-cockpit linkage can also be triggered, rather than being limited to triggering the cockpit linkage only when operating the game on the vehicle system, thereby improving the versatility of the cockpit linkage.

[0061] Figure 8 FIG. 8 is a block diagram of a vehicle cabin linkage apparatus 800 according to some embodiments of the present disclosure. Figure 8 As shown, the device 800 includes a user interaction acquisition unit 802, which is configured to acquire user interaction with the application screen displayed in the vehicle cockpit. The device 800 also includes a vehicle cockpit linkage unit 804, which is configured to trigger vehicle cockpit linkage based on the application screen and user interaction. It can be understood that by utilizing the device 800 of the present disclosure, at least one of the many advantages that can be achieved by the method or process described above can be achieved. For example, vehicle cockpit linkage can be achieved without using a data interface from an application provider and an artificial intelligence-based target detection technology, thereby reducing costs, reducing delays, increasing versatility, and enhancing user experience.

[0062] Figure 9 1 shows a schematic block diagram of an example device 900 that can be used to implement embodiments of the present disclosure. Figure 1 The control unit 104 in the embodiment may be as follows Figure 9900 is an example device shown. As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 902 or loaded from a storage unit 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the device 900 can also be stored in the RAM 903. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0063] Various components in the device 900 are connected to the I / O interface 905, including an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a magnetic disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0064] The computing unit 901 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as method 200. For example, in some embodiments, method 200 can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the method 200 described above can be performed. Alternatively, in other embodiments, the computing unit 901 can be configured to perform the method 200 in any other appropriate manner (e.g., by means of firmware).

[0065] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), and the like.

[0066] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0067] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In addition, although each operation is depicted in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in a sequential order, or requiring that all illustrated operations be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous. Likewise, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of separate embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented in multiple implementations individually or in any suitable subcombination.

[0068] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A vehicle cockpit linkage method, comprising: Obtaining user interaction with an application screen displayed in the vehicle cabin; as well as The vehicle cockpit linkage is triggered based on the application screen and the user interaction.

2. The method according to claim 1, wherein triggering vehicle cockpit linkage based on the application screen and the user interaction comprises: Determining whether the application screen matches the target scene; as well as In response to the application screen matching the target scene, vehicle cockpit linkage is triggered based on the application screen and the user interaction.

3. The method according to claim 2, wherein determining whether the application screen matches the target scene comprises: Acquire a template for the target scene, where the template includes target features corresponding to the target scene; as well as In response to determining that the application screen has a feature corresponding to the target feature, it is determined that the application screen matches the target scenario.

4. The method according to claim 2, wherein in response to the application screen matching the target scenario, triggering vehicle cockpit linkage based on the application screen and the user interaction comprises: determining a region of interest associated with the user interaction based on the application screen and the user interaction; determining an object contour within the region of interest; as well as A vehicle-cockpit linkage is triggered based on the object outline.

5. The method according to claim 4, wherein determining the region of interest associated with the user interaction based on the application screen and the user interaction comprises: Acquire coordinates associated with the user interaction in the application screen; as well as An area of ​​interest in the application screen is determined based on the coordinates.

6. The method according to claim 4, wherein triggering vehicle-cabin linkage based on the object outline comprises: determining color characteristics of the object's outline; as well as A vehicle ambient light is adjusted based on the color characteristic of the object outline.

7. The method of claim 6, wherein determining the color characteristics of the object contour comprises: determining a plurality of color bands that the vehicle ambient light can present; clustering pixels within the object outline into a plurality of pixel sets based on the plurality of color bands; as well as The color characteristic of the object contour is determined based on the number of pixels in the plurality of pixel sets. The method according to claim 6 , wherein the color characteristics include at least one of the following: a main hue, a color band ratio, or a brightness ratio.

9. The method according to claim 4, wherein triggering vehicle-cabin linkage based on the object outline comprises: Acquire a target contour of a predetermined target object; determining that the target object exists in the region of interest based on the object outline and the target outline; as well as The vehicle-cockpit linkage is triggered based on predetermined configuration information corresponding to the target object.

10. The method according to claim 9, wherein triggering vehicle cockpit linkage based on the predetermined configuration information corresponding to the target object comprises: At least one of the following items is adjusted based on the predetermined configuration information corresponding to the target object: vehicle ambient lighting, cabin lighting, vehicle audio, seats, air conditioning, or fragrance.

11. The method of claim 1 , wherein the application is a first application installed on a user's mobile device, and obtaining the user interaction with respect to the application screen displayed in the vehicle cabin comprises: The user interaction with the application screen is obtained from a second application associated with the vehicle and installed on the user mobile device, wherein the application screen and the user interaction are from the first application. 12 . The method according to claim 2 , wherein the application is a game application, and the target scene is a scene of selecting a game character.

13. An electronic device comprising: at least one processor; as well as A memory is coupled to the at least one processor and has instructions stored thereon, which, when executed by the at least one processor, cause the electronic device to perform the method according to any one of claims 1 to 12. 14 . A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the processor is caused to perform the method according to claim 1 .

15. A computer program product tangibly stored on a non-transitory computer readable medium and comprising machine-executable instructions which, when executed, cause a machine to perform the method according to any one of claims 1 to 12.

Images