Vehicle-mounted device interaction method and system and storage medium

Abstract

The invention provides a vehicle-mounted device interaction method and system and a storage medium. The method comprises the following steps: acquiring a real-time image of a user on a vehicle; based on the acquired real-time image, utilizing artificial intelligence to identify a touchable area of the user in the vehicle and a user intention of the user; a first vehicle-mounted display device and a display interface of the first vehicle-mounted display device are determined according to the recognized touchable area and the user intention, the first vehicle-mounted display device is at least partially located in the touchable area, and the display interface comprises content related to the user intention.

Description

Technical Field

[0001] This application relates to the field of human - machine interaction, and more specifically, to a method, system, and storage medium for in - vehicle device interaction. Background Art

[0002] With the continuous development of vehicle electrification and intelligence, a growing variety of in - vehicle devices are equipped on vehicles, such as various display screens, to meet the diverse needs of users for vehicle driving and riding. However, in some cases, users may not be able to or may not find it convenient to access these rich in - vehicle devices, thus reducing the user experience. Summary of the Invention

[0003] This application provides a method, system, and storage medium for in - vehicle device interaction. The following introduces each aspect related to this application.

[0004] In a first aspect, a method for in - vehicle device interaction is provided, including: obtaining a real - time image of a user in the vehicle; based on the obtained real - time image, using artificial intelligence to identify the reachable area of the user in the vehicle and the user's intention; according to the identified reachable area and user intention, determining a first in - vehicle display device and its display interface, where the first in - vehicle display device is at least partially within the reachable area, and the display interface contains content related to the user intention.

[0005] In a possible implementation, the first in - vehicle display device includes a first display area and a second display area, the first display area is completely within the reachable area, and the second display area is at least partially outside the reachable area; the determining of the first in - vehicle display device and its display interface according to the identified reachable area and user intention includes: adjusting the layout of the display interface of the first in - vehicle display device so that content related to the user intention is displayed in the first display area.

[0006] In a possible implementation, the user intention relates to a second in - vehicle display device outside the reachable area; and the determining of the first in - vehicle display device and its display interface according to the identified reachable area and user intention includes: determining that the display interface includes the content displayed on the second in - vehicle display device.

[0007] In a possible implementation, the user intention relates to an in - vehicle non - display device outside the reachable area; the determining of the first in - vehicle display device and its display interface according to the identified reachable area and user intention includes: determining that the display interface includes a control switch for controlling the in - vehicle non - display device.

[0008] In a possible implementation, there is also a third in-vehicle display device at least partially within the reachable area, and the user intention relates to an in-vehicle operation object outside the reachable area; determining the first in-vehicle display device and its display interface according to the user's reachable area and user intention includes: determining that the first in-vehicle display device is more ergonomic for the user than the third in-vehicle display device, and the display interface includes content related to the in-vehicle operation object.

[0009] In a possible implementation, based on the acquired real-time image, using artificial intelligence to identify the user's reachable area in the vehicle and the user's intention includes: identifying the user's posture, movement, gesture, eye contact and / or expression to determine the reachable area and the user intention.

[0010] In a possible implementation, determining the first in-vehicle display device and its display interface according to the identified reachable area and user intention includes: further determining the first in-vehicle display device and / or its display interface according to the user's historical behavior data.

[0011] In a second aspect, an in-vehicle device interaction system is provided, including: an acquisition module for acquiring a real-time image of a user in the vehicle; an identification module for, based on the acquired real-time image, using artificial intelligence to identify the user's reachable area in the vehicle and the user's intention; a determination module for determining a first in-vehicle display device and its display interface according to the identified reachable area and user intention, where the first in-vehicle display device is at least partially within the reachable area, and the display interface contains content related to the user intention.

[0012] In a third aspect, an in-vehicle device interaction system is provided, including: a memory for storing a program; a processor for calling and running the program from the memory, so that the in-vehicle device interaction system can implement the method according to the first aspect or any one of its implementations above.

[0013] In a fourth aspect, a non-transitory computer-readable storage medium is provided, on which a program is stored, and the program is used to execute the method according to the first aspect or any one of its implementations above.

[0014] In a fifth aspect, a chip is provided, including a processor, and the processor can be used to call and run a computer program from a memory, so that a device installed with the chip can execute the method according to the first aspect or any one of its implementations.

[0015] In a sixth aspect, there is provided a computer program product including a program for performing the method according to the first aspect or any of its implementations.

[0016] In the above implementations of the present application, when the user intends to operate an operation object outside the reachable area of the user, the content related to the operation object can be displayed on a display device that is at least partially within the reachable area of the user, so that the user can operate the operation object outside the reachable area within the reachable area without having to frequently adjust their posture or perform other cumbersome operations. This can ensure that the user's operation not only meets the user's intention but also conforms to the user's ergonomics, thereby enhancing the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a schematic block diagram of an in-vehicle device interaction method provided by an embodiment of the present application.

[0018] Figure 2 is a schematic diagram of adjusting the layout of the display interface of a first in-vehicle display device provided by an embodiment of the present application.

[0019] Figure 3 is a schematic diagram of displaying the content displayed on a second in-vehicle display device on a first in-vehicle display device provided by an embodiment of the present application.

[0020] Figure 4 is a schematic diagram of displaying content related to an in-vehicle non-display device on a first in-vehicle display device provided by an embodiment of the present application.

[0021] Figure 5 is a schematic structural block diagram of an in-vehicle device interaction system provided by an embodiment of the present application.

[0022] Figure 6 is a schematic structural block diagram of an in-vehicle device interaction system provided by an embodiment of the present application.

[0023] Figure 7 is a schematic structural block diagram of an in-vehicle device interaction system provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] In order to enable those skilled in the art to better understand the solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application should fall within the scope of protection of the present application.

[0025] Reference to "embodiments" in this document means that the specific features, structures, or characteristics described in connection with the embodiments may be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment each time, nor are they independent or alternative embodiments mutually exclusive of other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein may be combined with other embodiments.

[0026] In the description of the present application, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention; the terms "first", "second", "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance; in addition, unless otherwise clearly defined and limited, the terms "mounted", "connected", "coupled" should be understood in a broad sense, for example, it may be a fixed connection, a detachable connection; it may be directly connected, or indirectly connected through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0027] With the continuous development of vehicle electrification and intelligence, vehicles are equipped with a growing variety of in-vehicle devices, especially intelligent in-vehicle systems. With the popularization of intelligent in-vehicle systems, users (also known as car owners, drivers, etc., and these terms may be used interchangeably hereinafter) have put forward higher requirements for the convenience and comfort of in-vehicle operations. Moreover, although devices such as intelligent in-vehicle systems (e.g., intelligent screens) and touch panels in the vehicle can provide rich functions, due to environmental and postural limitations during driving, users may not be able to or may not be convenient to maintain contact with all in-vehicle devices (e.g., in-vehicle screens, control interfaces, etc.) or all areas of in-vehicle devices, resulting in some operations that may require cumbersome or complex steps or operations (e.g., gestures) to achieve, which affects the user experience. Furthermore, when interacting with in-vehicle devices during driving, users often need to frequently shift their line of sight or change their posture, which will interfere with normal driving, increase the driving burden on users, and reduce driving safety.

[0028] To solve the above technical problems, embodiments of the present application propose to use in-vehicle cameras and artificial intelligence to identify the reachable areas of users in the vehicle and user intentions, and then determine at least some in-vehicle display devices located within the reachable areas based on the identified reachable areas and user intentions, and then display content related to the user intentions on the determined in-vehicle display devices.

[0029] Through the above solution proposed by the embodiments of the present application, when the user's intended operation is on an operation object outside the user's reach area (for example, a display screen outside the user's reach area), the content related to the operation object (for example, the content displayed on the display screen) can be displayed on a display screen at least partially within the user's reach area, so that the user can operate the operation object outside the reach area within the reach area without having to frequently adjust the posture or perform other cumbersome operations. This can ensure that the user's operation not only meets the user's intention but also conforms to the user's ergonomics, thus enhancing the user experience.

[0030] The embodiments of the present application and their principles will be introduced and described in more detail below with reference to the accompanying drawings of the specification.

[0031] Figure 1 The schematic block diagram of the in-vehicle device interaction method 100 provided by an embodiment of the present application is shown. As Figure 1 shown, the method 100 may include step S110 to step S130.

[0032] Step S110: Obtain a real-time image of the user in the vehicle.

[0033] Step S120: Based on the obtained real-time image, use artificial intelligence to identify the reachable area of the user in the vehicle and the user's intention.

[0034] Step S130: According to the identified reachable area and the user's intention, determine the first in-vehicle display device and its display interface, where at least part of the first in-vehicle display device is within the reachable area, and the display interface includes content related to the user's intention.

[0035] In step S110, the real-time image of the user can be collected by various in-vehicle camera devices. As an example, the in-vehicle camera device can be an ordinary two-dimensional (2D) camera, and the obtained real-time image is a 2D image. Also, in some embodiments, the in-vehicle camera device can be a depth camera, and the obtained real-time image is a three-dimensional (3D) image. It should be understood that those skilled in the art can select different camera devices according to actual needs as long as they can obtain data representing the user's posture or movement. The present application does not limit this.

[0036] In step S120, artificial intelligence can be utilized to identify the user's posture, movement, gesture, eye contact, and / or expression based on the acquired real-time image of the user, and then determine the reachable area of the user in the vehicle and the user's intention. For example, in some embodiments, artificial intelligence can be used to identify the user's body posture and / or movement, and in the case where the spatial dimensions of various in-vehicle devices of the vehicle and the relative positional relationship between them are determined, the reachable area of the user in the vehicle can be identified in real time, that is, to identify which in-vehicle devices the user can reach. For another example, in some embodiments, artificial intelligence can be used to identify the user's posture, movement, gesture, eye contact, and / or expression based on the acquired real-time image of the user, and then determine the user's intention. For example, the artificial intelligence algorithm can not only identify the user's posture, but also judge the user's intention through the user's gesture, eye contact, and / or body movement (or body movement), etc. It should be understood that those skilled in the art can use various artificial intelligence technologies as needed. For example, deep learning methods (such as OpenPose, etc.) can be used for skeleton recognition to achieve the recognition of the reachable area and the user's intention. The present application does not make any limitations on this.

[0037] In step S130, after determining the reachable area of the user, at least part of the in-vehicle display device located within the reachable area can be determined, and then content related to the user's intention can be displayed on the determined in-vehicle display device. It should be understood that those skilled in the art can select various types of in-vehicle display devices as needed. For example, in some embodiments, the in-vehicle display device can be a touch display device.

[0038] Although some in-vehicle systems in the related art have interactive devices such as touch screens, they still do not fully consider the user's ergonomics and lack intelligent judgment of the user's intention. As a result, when the user operates, they may need to frequently adjust their posture or perform cumbersome operations, thus reducing the user experience. Through the above embodiments, in the case where the in-vehicle device that the user intends to operate cannot be reached or is inconvenient to reach, content related to the in-vehicle device that the user intends to operate can be displayed on the reachable in-vehicle display device, so that the user can operate the in-vehicle device that cannot be reached or is inconvenient to reach on the reachable in-vehicle display device. Therefore, the embodiments of the present application can accurately identify the user's needs through artificial intelligence and intelligently present convenient operations corresponding to the user's needs, improving the user experience.

[0039] Due to limitations such as the body shape or sitting posture of some users, even the in-vehicle display device closest to the user may still have only a part within the reachable area of the user. In this case, among the content displayed on the in-vehicle display device, there may still be a part that cannot be easily reached by the user, thus reducing the user experience. To solve this problem, in some embodiments of the present application, content related to the user's intention can be displayed only on the part of the in-vehicle display device within the reachable area, so as to ensure that the user can easily reach the content related to the user's intention displayed on the in-vehicle display device.

[0040] For example, Figure 2 FIG. shows a schematic diagram of adjusting the layout of the display interface of the first in-vehicle display device 200 provided by an embodiment of the present application.

[0041] As Figure 2 shown, the first in-vehicle display device 200 may include a first display area 231 and a second display area 232 divided by a first dividing line 210. The second dividing line 220 may represent the dividing line between the reachable area and the non-reachable area of the user, and the left area of the second dividing line 220 represents the reachable area of the user.

[0042] In this embodiment of the present application, the layout of the display interface of the first in-vehicle display device 200 can be adjusted such that content related to the user's intention is displayed in the first display area 231. Therefore, in the case where a part of the first in-vehicle display device 200 is within the reachable area and another part is outside the reachable area, through the embodiments of the present application, the user can still easily touch the content related to the user's intention within the reachable area, improving the user experience.

[0043] In some cases, the user may intend to operate another in-vehicle display device that cannot be touched. In some embodiments of the present application, the display content of the other in-vehicle display device that the user cannot touch can be presented on the first in-vehicle display device, so that the user can operate the other in-vehicle display device that cannot be touched by touching the first in-vehicle display device within the reachable area.

[0044] For example, Figure 3 FIG. shows a schematic diagram of displaying the content displayed on the second in-vehicle display device 300 on the first in-vehicle display device 200 provided by an embodiment of the present application.

[0045] As shown in the figure, Figure 3Among (a) and (b), they respectively represent the content display situations on the first vehicle-mounted display device 200 and the second vehicle-mounted display device 300 before content adjustment. As shown in the figure, the third dividing line 320 can represent the dividing line between the touchable area and the non-touchable area of the user, and the left area of the third dividing line 320 represents the touchable area of the user. As shown in the figure, the first vehicle-mounted display device 200 is within the touchable area, and the second vehicle-mounted display device 300 is outside the touchable area. Additionally, as Figure 3 shown in (a) among them, there is a display content 310 on the third vehicle-mounted display device 300 before content adjustment. After content adjustment, as Figure 3 shown in (b) among them, the display content 310 can be displayed on the first vehicle-mounted display device 200. Therefore, when the user intends to operate the second vehicle-mounted display device 300 outside the touchable area, the display content 310 can be presented on the first vehicle-mounted display device 200, so that the user can operate the first vehicle-mounted display device 200 to achieve the operation of the third vehicle-mounted display device 300. It should be understood that when the display content 310 of the second vehicle-mounted display device 300 is displayed on the first vehicle-mounted display device 200 simultaneously or afterwards, the second vehicle-mounted display device 300 may no longer present the display content 310, or may present other content, such as presenting a prompt message related to "the display content 310 is presented on the first vehicle-mounted display device 200". This application does not make a limitation on this.

[0046] In some cases, the user may expect to operate a vehicle-mounted non-display device that cannot be touched. In some embodiments of this application, the control switch (also referred to as a soft switch) of the vehicle-mounted non-display device that the user cannot touch can be displayed on the first vehicle-mounted display device 200, so that the user can touch the first vehicle-mounted display device 200 within the touchable area to achieve the operation of the vehicle-mounted non-display device.

[0047] For example, Figure 4 is a schematic diagram showing the content related to the vehicle-mounted non-display device 400 displayed on the first vehicle-mounted display device 200 provided by an embodiment of this application.

[0048] As shown in the figure, the fourth dividing line 420 may represent the dividing line between the touchable area and the non-touchable area of the user, and the area on the left side of the fourth dividing line 420 represents the touchable area of the user. As shown in the figure, the first in-vehicle display device 200 is within the touchable area, and the in-vehicle non-display device 400 is outside the touchable area. In this embodiment, a control switch 410 for controlling the in-vehicle non-display device 400 may be displayed on the first in-vehicle display device 200, so that the user can control the in-vehicle non-display device 400 outside the touchable area by touching the first in-vehicle display device 200 within the touchable area.

[0049] In particular, as an example, the in-vehicle non-display device 400 may be an in-vehicle reading lamp and is outside the touchable area of the user. In an embodiment of the present application, after determining that the user intends to operate the in-vehicle reading lamp, a control switch 410 for controlling the in-vehicle reading lamp may be presented on the first in-vehicle display device 200, so that the user can operate the untouchable in-vehicle reading lamp within the touchable range, which facilitates the user's operation and improves the user experience. Similarly, the in-vehicle non-display device 400 may also be a seat adjustment button or a seat massage start button. For example, during driving, if the system may recognize that the user is fatigued based on the user's real-time image, a control switch for adjusting the in-vehicle seat or starting the seat massage may be presented on the first in-vehicle display device 200. In addition, the in-vehicle non-display device 400 may also be a music player, an in-vehicle air conditioner, etc., and the present application does not limit this.

[0050] In some cases, the user intends to operate an in-vehicle operation object outside the touchable area, and in addition to the first in-vehicle display device 200, the in-vehicle display devices within the touchable area may further include a third in-vehicle display device. At this time, it is necessary to determine which in-vehicle display device is used to display the content related to the in-vehicle operation object (that is, the content related to the user's intention).

[0051] In some embodiments of the present application, it may be determined which of the first in-vehicle display device and the third in-vehicle display device is more ergonomic for the user, and the more ergonomic in-vehicle display device is used to display the content related to the in-vehicle operation object. For example, the ergonomic in-vehicle display device may include: the in-vehicle display device is closer to the user, the in-vehicle display device requires a smaller rotation amplitude of the user's body, the in-vehicle display device requires a smaller eye movement amplitude of the user, and so on. Therefore, the user's ergonomics is fully considered and the user experience is improved.

[0052] Return for reference Figure 1, in some embodiments, step S130 of method 100 may include: further determining the first in-vehicle display device and / or its display interface according to the user's historical behavior data. As an example, in some embodiments, the user's historical behavior data indicates that the user more frequently uses a specific in-vehicle display device, which indicates that this specific in-vehicle display device may better conform to the user's operating habits or ergonomics. Therefore, this in-vehicle display device can be determined as the first in-vehicle display device. For another example, in some embodiments, the user's historical behavior data indicates that the user more frequently operates a specific in-vehicle device outside the reachable area. Therefore, the content related to this specific in-vehicle device can be displayed on the first in-vehicle display device. In this way, it can be ensured that the first in-vehicle display device and / or its display interface better conform to the user's ergonomics and personalized needs.

[0053] In some embodiments of the present application, the analysis and processing of real-time images are both performed locally. Therefore, user privacy can be strictly protected during the data processing process.

[0054] To facilitate the understanding of the present application, the method embodiments of the present application will be described in more detail below with specific examples. It should be noted that this example is only to help those skilled in the art understand the embodiments of the present application, rather than to limit the embodiments of the present application to the specific numerical values or specific scenarios illustrated. Those skilled in the art can clearly make various equivalent modifications or changes according to the examples given below, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0055] In this embodiment of the present application, it mainly includes a real-time data monitoring and processing process, an intelligent display and recommendation process, and a dynamic optimization and adjustment process.

[0056] In the real-time data monitoring and processing process, it includes: identifying the owner's posture, for example, through an in-vehicle camera and a deep learning algorithm, analyzing the owner's bone data in real time for bone recognition, so as to identify the owner's current posture and hand movements to determine the reachability of the owner to each operation interface in the vehicle; discovering the user's intention, for example, by analyzing the owner's actions and behaviors, using an AI algorithm to judge the owner's intention, such as whether the owner intends to adjust the volume of the in-vehicle speaker, the in-vehicle air conditioner, whether the owner intends to find the seat adjustment button, etc., so as to be able to actively adjust the interaction operation mode of the in-vehicle device according to the owner's intention.

[0057] In the intelligent display and recommendation process, it includes: obtaining camera data, for example, using an in-vehicle camera to monitor the owner's actions and postures in real time; data analysis and feedback, for example, using an AI system to analyze the owner's action intention and adjust the display interface according to the owner's reachable area; and intelligent recommendation, for example, the system can recommend the operations that the owner can perform and display the recommended soft switches at appropriate positions.

[0058] During the dynamic optimization and adjustment process, it includes: the system adjusts the layout of the display interface according to the real-time feedback and behavior of the vehicle owner to ensure the convenience and comfort of operation.

[0059] Through this embodiment of the present application, the in-vehicle camera and AI algorithm can be used for user skeleton recognition, real-time monitoring of the vehicle owner's posture, movements and intentions, and analysis of the vehicle owner's current needs and operation intentions, thus fully considering the user's ergonomics and user intention exploration.

[0060] In addition, through this embodiment of the present application, an operation interface that the user cannot touch can be presented on the surrounding intelligent surfaces that the user can touch (for example, the central control screen, the armrest screen, etc.), and the position of the interface can be intelligently adjusted, thus facilitating the vehicle owner's operation and realizing intelligent interface adjustment and recommendation.

[0061] Furthermore, through this embodiment of the present application, according to the user's intention, soft switches that can be operated by the user can be intelligently recommended, and a convenient way to trigger the operation can be provided, thus reducing gesture operations during driving and dependence on physical buttons.

[0062] In addition, through this embodiment of the present application, through intelligent recommendation and display adjustment, the need for the driver to frequently change the line of sight and change the posture during driving can be reduced, thus improving driving safety.

[0063] As described above in combination with Figures 1 to 4 Detailed description of the method embodiments of the present application has been given above. Next, the device embodiments of the present application will be described in detail in combination with Figures 5 to 7 It should be understood that the description of the device embodiments corresponds to the description of the method embodiments. Therefore, the parts not described in detail can be referred to the previous method embodiments.

[0064] Figure 5 Fig. shows a schematic structural block diagram of an in-vehicle device interaction system 500 provided by an embodiment of the present application. As shown in the figure, the in-vehicle device interaction system 500 may include an acquisition module 510, an identification module 520, and a determination module 530.

[0065] The acquisition module 510 can be used to acquire real-time images of the users in the vehicle. The identification module can be used to identify the reachable area of the user in the vehicle and the user's intention by using artificial intelligence based on the acquired real-time images. The determination module 530 can be used to determine the first in-vehicle display device and its display interface according to the identified reachable area and user intention, where at least part of the first in-vehicle display device is within the reachable area, and the display interface contains content related to the user intention.

[0066] Figure 6The schematic structural block diagram of the vehicle-mounted device interaction system 600 provided by an embodiment of the present application is shown. As shown in the figure, the vehicle-mounted device interaction system 600 may include a processor 610 and a memory 620. The memory 620 can be used to store programs. The processor 610 can be used to call and run programs from the memory 620, so that the vehicle-mounted device interaction system 600 implements the following method: obtaining a real-time image of a user in the vehicle; based on the obtained real-time image, using artificial intelligence to identify the reachable area of the user in the vehicle and the user's intention; according to the identified reachable area and user intention, determining a first vehicle-mounted display device and its display interface, wherein the first vehicle-mounted display device is at least partially within the reachable area, and the display interface contains content related to the user intention.

[0067] Optionally, in some embodiments, the processor 610 can be used to call and run programs from the memory 620, so that the vehicle-mounted device interaction system 600 implements the following method: The first vehicle-mounted display device includes a first display area and a second display area, the first display area is completely within the reachable area, and the second display area is at least partially outside the reachable area; the determining the first vehicle-mounted display device and its display interface according to the identified reachable area and user intention includes: adjusting the layout of the display interface of the first vehicle-mounted display device so that content related to the user intention is displayed in the first display area.

[0068] Optionally, in some embodiments, the processor 610 can be used to call and run programs from the memory 620, so that the vehicle-mounted device interaction system 600 implements the following method: The user intention involves a second vehicle-mounted display device outside the reachable area; and the determining the first vehicle-mounted display device and its display interface according to the identified reachable area and user intention includes: determining that the display interface includes the content displayed on the second vehicle-mounted display device.

[0069] Optionally, in some embodiments, the processor 610 can be used to call and run programs from the memory 620, so that the vehicle-mounted device interaction system 600 implements the following method: The user intention involves a vehicle-mounted non-display device outside the reachable area; the determining the first vehicle-mounted display device and its display interface according to the identified reachable area and user intention includes: determining that the display interface includes a control switch for controlling the vehicle-mounted non-display device.

[0070] Optionally, in some embodiments, the processor 610 may be used to call and run a program from the memory 620, so that the vehicle-mounted device interaction system 600 implements the following method: There is also at least a third vehicle-mounted display device partially within the reachable area, and the user's intention involves a vehicle-mounted operation object outside the reachable area; determining the first vehicle-mounted display device and its display interface according to the user's reachable area and user intention includes: determining that the first vehicle-mounted display device is more ergonomic for the user than the third vehicle-mounted display device, and the display interface includes content related to the vehicle-mounted operation object.

[0071] Optionally, in some embodiments, the processor 610 may be used to call and run a program from the memory 620, so that the vehicle-mounted device interaction system 600 implements the following method: Based on the acquired real-time image, using artificial intelligence to identify the user's reachable area in the vehicle and the user's intention, including: identifying the user's posture, movement, gesture, eye movement, and / or expression to determine the reachable area and user intention.

[0072] Optionally, in some embodiments, the processor 610 may be used to call and run a program from the memory 620, so that the vehicle-mounted device interaction system 600 implements the following method: Determining the first vehicle-mounted display device and its display interface according to the identified reachable area and user intention includes: further determining the first vehicle-mounted display device and / or its display interface according to the user's historical behavior data.

[0073] Optionally, in some embodiments, the vehicle-mounted device interaction system 600 may further include a transceiver 630 for receiving data from other devices and / or sending data to other devices.

[0074] For ease of understanding of the present application, the following describes the device embodiments of the present application in more detail with specific examples and Figure 7 , It should be noted that this example is only for helping those skilled in the art to understand the embodiments of the present application, rather than limiting the embodiments of the present application to the specific numerical values or specific scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or changes according to the examples given below, and such modifications or changes also fall within the scope of the embodiments of the present application.

[0075] Figure 7 FIG. shows a schematic structural block diagram of a vehicle-mounted device interaction system 700 provided by an embodiment of the present application. As shown in the figure, the vehicle-mounted device interaction system 700 may include a camera device 710, a control device 720, and a display device 730.

[0076] The imaging device 710 can collect human body posture data, analyze and process it through the control device 720, and combine the analysis results of the user's intention to push the generated adaptive operation interface to the display device 730.

[0077] Through the in-vehicle device interaction system 700, the combination of in-vehicle cameras and AI algorithms can be realized, intelligent display interface and adjustment, as well as intelligent recommendation of soft switches.

[0078] Regarding the combination of in-vehicle cameras and AI algorithms, the following can be achieved: a bone recognition system, for example, installing a camera in the vehicle, analyzing the body posture, hand movements, and movement trajectories of the vehicle owner through AI algorithms, and in real time identifying whether the vehicle owner can reach certain operation interfaces, and the system can judge the user's current position, angle, and intention, and automatically adjust the display content based on this information; and, action intention analysis, for example, the AI algorithm can not only recognize the posture of the vehicle owner, but also judge the user's operation intention through signals such as gestures, eye movements, and body movements. For example, when the vehicle owner faces a certain operation interface, the system can infer that the vehicle owner intends to perform the relevant operation.

[0079] Through intelligent display interface and adjustment, the following can be achieved: an intelligent display interface, for example, the system can automatically display operation options that cannot be reached in the area that the user can reach (for example, the central control screen, the armrest screen) according to the recognition results of the vehicle owner, and the intelligent display interface can adjust the layout of the display interface according to the body posture and line of sight of the vehicle owner, so that the vehicle owner can easily complete the touch operation without moving the body.

[0080] Through intelligent recommendation of soft switches, the following can be achieved: dynamic operation establishment, for example, the system can recommend the most suitable operation for the current scenario according to the intention of the vehicle owner and real-time data. For example, during driving, if the system recognizes that the vehicle owner is fatigued, it can recommend that the vehicle owner adjust the seat position or activate the seat massage function.

[0081] Through the above in-vehicle device interaction system, the following can be achieved: ergonomic optimization, for example, the system performs bone recognition through AI algorithms, and combines the body posture and hand movements of the vehicle owner to intelligently recommend the most ergonomic operation interface; accurate identification of user intention, for example, through in-vehicle cameras and AI algorithms, it can judge the needs and intentions of the vehicle owner in real time, and automatically adjust the operation interface, reducing the time required for the vehicle owner to manually search and operate; convenient operation experience, for example, combining the operation interface that cannot be reached and the intelligently recommended soft switches can improve the operation convenience of the vehicle owner during driving, reducing interference and operation burden.

[0082] In an embodiment of the present application, a chip is further provided, which includes a processor. The processor can be used to call and run a computer program from a memory, so that a device installed with the chip executes the method described in the above method embodiment. It can be understood that the processor can be any type of processor. It can be understood that the memory can be independent of the chip or integrated in the chip.

[0083] An embodiment of the present application further provides a computer-readable storage medium for storing a program, and the program is used to execute the method of any one of the method embodiments of the present application.

[0084] An embodiment of the present application further provides a computer program product. The computer program product includes a program, and the program enables a computer to execute the method of any one of the method embodiments of the present application.

[0085] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present disclosure are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a machine-readable storage medium or transmitted from one machine-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The machine-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a digital video disc (DVD)), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0086] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments of the present disclosure can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present disclosure.

[0087] In several embodiments provided by the present disclosure, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be in electrical, mechanical, or other forms.

[0088] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0089] In addition, the functional units in each embodiment of the present disclosure can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0090] As described above, the above is only the specific implementation manner of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present disclosure can easily think of changes or substitutions, which should all be covered by the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the appended claims.

Claims

1. An in-vehicle device interaction method, characterized in that, including: Obtaining a real-time image of a user in the vehicle; Based on the obtained real-time image, using artificial intelligence to identify the reachable area of the user in the vehicle and the user's intention; According to the identified reachable area and user intention, determining a first in-vehicle display device and its display interface, wherein the first in-vehicle display device is at least partially within the reachable area, and the display interface contains content related to the user intention.

2. The in-vehicle device interaction method according to claim 1, wherein The first in-vehicle display device includes a first display area and a second display area, the first display area is completely within the reachable area, and the second display area is at least partially outside the reachable area; The determining the first in-vehicle display device and its display interface according to the identified reachable area and user intention includes: adjusting the layout of the display interface of the first in-vehicle display device so that content related to the user intention is displayed in the first display area.

3. The vehicle-mounted device interaction method according to claim 1 or 2, characterized in that The user intention relates to a second in-vehicle display device outside the reachable area; and The determining the first in-vehicle display device and its display interface according to the identified reachable area and user intention includes: determining that the display interface includes the content displayed on the second in-vehicle display device.

4. The in-vehicle device interaction method according to claim 1 or 2, characterized in that, The user intention relates to an in-vehicle non-display device outside the reachable area; The determining the first in-vehicle display device and its display interface according to the identified reachable area and user intention includes: determining that the display interface includes a control switch for controlling the in-vehicle non-display device.

5. The in-vehicle device interaction method according to claim 1 or 2, characterized in that There is also a third in-vehicle display device at least partially within the reachable area, and the user intention relates to an in-vehicle operation object outside the reachable area; The determining the first in-vehicle display device and its display interface according to the user's reachable area and user intention includes: determining that the first in-vehicle display device is more ergonomic for the user than the third in-vehicle display device, and the display interface includes content related to the in-vehicle operation object.

6. The in-vehicle device interaction method according to claim 1 or 2, characterized in that The identifying the reachable area of the user in the vehicle and the user's intention based on the obtained real-time image using artificial intelligence includes: identifying the user's posture, movement, gesture, eye contact and / or expression to determine the reachable area and the user intention.

7. The in-vehicle device interaction method according to claim 1 or 2, wherein The determining the first in-vehicle display device and its display interface according to the identified reachable area and user intention includes: further determining the first in-vehicle display device and / or its display interface according to the user's historical behavior data.

8. An in-vehicle device interaction system, characterized in that, including: An obtaining module for obtaining a real-time image of a user in the vehicle; An identifying module for identifying the reachable area of the user in the vehicle and the user's intention based on the obtained real-time image using artificial intelligence; A determining module for determining a first in-vehicle display device and its display interface according to the identified reachable area and user intention, wherein the first in-vehicle display device is at least partially within the reachable area, and the display interface contains content related to the user intention.

9. A vehicle-mounted device interaction system, characterized in that, including: A memory for storing programs; A processor for calling and running the program from the memory, so that the in-vehicle device interaction system implements the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that, A program is stored thereon, and the program is used to execute the method according to any one of claims 1 to 7.

Images