Vehicle intelligent cabin, interaction method and vehicle intelligent cabin control system

Abstract

The invention discloses an interaction method for an intelligent cabin of a vehicle. The interaction method comprises the following steps: acquiring information released by a user and environment information around the vehicle (S1); determining, based on the information released by the user and the environmental information, a target object outside the vehicle to which the user may pay attention, and presuming, based on the information released by the user, a potential intention of the user except for understanding attribute information of the target object itself (S2); and generating feedback information interacting with the user based on the potential intent (S3). The invention further discloses a vehicle intelligent cabin, a control system and a computer program product. According to the invention, the actual demand of the user in the vehicle driving process can be met, and the intelligent driving experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent vehicles, and in particular to a vehicle intelligent cockpit and an interaction method, a vehicle intelligent cockpit control system and a computer program product. Background Art

[0002] With the development of vehicle intelligence, vehicle cockpits are beginning to be able to realize artificial intelligence interaction with users, but the current related technologies are only limited to simple human-computer dialogue, and their applications in vehicle scenarios are no different from those in home or office scenarios. Therefore, they have no relevance to the actual driving environment in which users are currently located, making it difficult to provide users with artificial intelligence interaction that is close to their actual needs during vehicle driving.

[0003] Therefore, it is expected to provide a vehicle intelligent cockpit and interaction method that can respond to the vehicle's surrounding environment, so as to promptly meet the user's actual needs during vehicle driving and enhance the intelligent driving experience. Summary of the invention

[0004] In order to overcome one of the above-mentioned shortcomings and / or other possible shortcomings in the prior art not mentioned in this article, the purpose of the present invention is to provide a vehicle intelligent cockpit and interaction method, a vehicle intelligent cockpit control system and a computer program product.

[0005] The present invention particularly includes the following technical concepts: during driving, artificial intelligence technology is used to infer the user's focus on the surrounding environment information of the vehicle, to enhance the perception and recognition of the environmental information around the vehicle, and based on the user's focus or other interest points on the surrounding environment information of the vehicle, in-depth information mining is performed on the environmental information around the vehicle or other related information, so as to provide feedback information for artificial intelligence interaction, thereby meeting the actual needs of the user during driving.

[0006] According to one aspect of the present invention, there is provided an interaction method for a vehicle intelligent cockpit, the interaction method comprising: step S1: acquiring information released by a user and environmental information around the vehicle; step S2: determining a target object outside the vehicle that the user may be concerned about based on the information released by the user and the environmental information, and inferring the user's potential intentions other than understanding the target object's own attribute information based on the information released by the user; and step S3: generating feedback information for interacting with the user based on the potential intentions.

[0007] According to an optional embodiment of the present invention, the step S2 also includes step S21: using an artificial intelligence model to mine the target object's own attribute information, and inferring the potential intention in combination with the target object's own attribute information; and / or, the step S2 also includes step S22: using an artificial intelligence model to mine other related information except the target object's own attribute information, and inferring the potential intention in combination with the other related information.

[0008] According to an optional embodiment of the present invention, in step S2, based on the information released by the user, at least one feature point of the target object is obtained by means of semantic analysis, and the target object is determined based on the at least one feature point and the environmental information.

[0009] According to an optional embodiment of the present invention, the interaction method further includes step S4: the user asks questions to the smart cockpit based on the feedback information, and then uses an artificial intelligence model to further mine information of interest to the user based on the questions to provide feedback on the questions.

[0010] According to an optional embodiment of the present invention, the information released by the user is specifically communication information between the user and a third party other than the vehicle.

[0011] According to an optional embodiment of the present invention, the potential intention is specifically that the user expects to find the target object; and / or the feedback information is specifically image information of the current location of the target object.

[0012] According to an optional embodiment of the present invention, the target object is a passenger outside the vehicle, and the potential intention is specifically that the user expects to find the passenger outside the vehicle; the feedback information is specifically image information of the current location of the passenger outside the vehicle and navigation information for picking up the passenger outside the vehicle.

[0013] According to a second aspect of the present invention, a vehicle intelligent cockpit is provided, which includes: a human-computer interaction module, which is configured to obtain information released by a user; an environmental perception module, which is configured to obtain environmental information around the vehicle; an artificial intelligence module, which is configured to determine a target object outside the vehicle that the user may be concerned about based on the information released by the user and the environmental information, and to infer the user's potential intentions other than understanding the target object's own attribute information based on the information released by the user; the artificial intelligence module is also configured to generate feedback information for interacting with the user based on the potential intentions.

[0014] According to a third aspect of the present invention, a control system for a vehicle intelligent cockpit is provided, the control system comprising an intelligent driving controller and an intelligent cockpit controller which are communicatively connected, and the intelligent cockpit controller and the intelligent driving controller use the same central processing unit, thereby being integrated into a cockpit integrated controller; and the control system is communicatively connected to a display device, a microphone, a speaker, an image acquisition device, a radar system, and a global navigation satellite system.

[0015] According to a fourth aspect of the present invention, there is provided a computer program product, such as a machine-readable storage medium, which includes or stores computer program instructions, wherein the computer program is suitable for executing an interaction method for a vehicle intelligent cockpit when running on a computer.

[0016] According to some exemplary embodiments of the present invention, the intelligence level is high and the user experience is good. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The present invention will be described in more detail below with reference to the accompanying drawings, so that the principles, features and advantages of the present invention can be better understood. The accompanying drawings include:

[0018] Figure 1 A workflow diagram of an interaction method for a vehicle intelligent cockpit according to an exemplary embodiment of the present invention is shown.

[0019] Figure 2 A schematic block diagram of a vehicle intelligent cockpit according to an exemplary embodiment of the present invention is shown.

[0020] Figure 3 A schematic block diagram of a control system of a vehicle intelligent cockpit according to an exemplary embodiment of the present invention is shown. DETAILED DESCRIPTION

[0021] In order to make the technical problems, technical solutions and beneficial technical effects to be solved by the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and multiple exemplary embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the scope of protection of the present invention.

[0022] Figure 1 A workflow diagram of an interactive method for a vehicle intelligent cockpit according to an exemplary embodiment of the present invention is shown, and the method includes S1, S2 and S3.

[0023] In step S1 of this exemplary embodiment, the information released by the user and the environmental information around the vehicle are obtained. The information released by the user can be voice, picture or text information sent by the user in the vehicle to the vehicle intelligent cockpit in the form of questions, inquiries, instructions, etc., or voice, picture or text information between the user and a third party outside the vehicle. The target object outside the vehicle that the user may be concerned about, that is, the target object concerned in the information released by the user can be a person or object in the surrounding environment, specifically, including pedestrians, passengers outside the vehicle, motor vehicles (cars, motorcycles), non-motor vehicles (bicycles, battery vehicles, elderly scooters, tricycles), traffic facilities (traffic signs, intersections, highway exit ramps), buildings, commercial facilities, electric vehicle charging stations, parking lots, etc.

[0024] In step S1 of this exemplary embodiment, the environmental information around the vehicle can also be obtained with the help of the environmental perception module 20, and further, the objects existing in the vehicle's surrounding environment and at least part of their attribute information can be obtained from the environmental information around the vehicle. The environmental information around the vehicle includes image information of the vehicle's surrounding environment, the relative distance and speed between the vehicle and each object existing in its surrounding environment, the vehicle's current geographical location information, and commercial facilities, traffic control information and road condition information around the current geographical location. The objects existing in the vehicle's surrounding environment and at least part of their attribute information that can be obtained with the help of the environmental perception module 20 can specifically include:

[0025] a. Information on the speed, distance, color, model, brand, license plate number, etc. of other vehicles in the vicinity;

[0026] b. Information on the speed, distance, color, brand, specifications and model of nearby bicycles, battery vehicles, motorcycles and other non-motor vehicles;

[0027] c. Estimated recognition results of surrounding pedestrians, including distance, gender, height, clothing color, body shape, age group (e.g., infants, children, young people, elderly people, etc.), speed, and various states (standing, walking, running, squatting, sitting in a wheelchair, lying down);

[0028] d. The location, distance, color, content classification, scope of application of surrounding traffic signs, and road and traffic conditions around the location of the signs;

[0029] e. The location, brand, category, business status, etc. of surrounding commercial facilities;

[0030] f. The location, shape, structure, color, etc. of surrounding buildings.

[0031] Additionally, the collected image information of the vehicle's surroundings may be processed with the aid of image recognition technology, so as to more accurately and efficiently obtain objects existing in the vehicle's surroundings and at least part of their attribute information from the image information.

[0032] In step S2 of this exemplary embodiment, the interaction method determines a target object outside the vehicle that the user may be concerned about based on the information released by the user and the environmental information, and infers the user's potential intentions other than understanding the target object's own attribute information based on the information released by the user.

[0033] Optionally, based on the information released by the user, at least one feature point of the target object is obtained by semantic analysis, and the target object outside the vehicle that the user is concerned about is determined based on the at least one feature point and the environmental information. The one or more feature points related to the target object can be the color, structure, position, state parameter characteristics, morphology or dynamic characteristics of people or objects emphasized in the information released by the user. Combined with the one or more feature points of the identified target object, and the objects existing in the vehicle surroundings and at least part of their attribute information, according to the matching degree of the one or more feature points of the target object and the objects existing in the vehicle surroundings and at least part of their attribute information, the target object outside the vehicle that the user may be concerned about is determined and identified from the vehicle surroundings. Determining and identifying the target object outside the vehicle that the user is concerned about also includes grasping at least part of the target object's own attribute information such as the relative position relationship, relative distance and speed between the target object and the vehicle.

[0034] In the interaction method, the user's potential intention refers to the user's unstated or undirected desire to achieve a certain goal. To achieve this intention, the user will generally be accompanied by relevant behavioral performance, which may be retrieving information, collecting images, identifying information, turning on driving assistance functions, setting parameters, opening an operation interface, etc.; the inferred user's potential intention may be to deduce the user's possible thought trends based on the context of the interaction process, and then determine the user's possible expected purpose based on the deduction, and may determine the user's behavioral performance that may be accompanied by the user in order to achieve his or her expected purpose.

[0035] In step S3 of this exemplary embodiment, feedback information for interaction with the user is generated based on the potential intention. The feedback information is different from the information required in the information released by the user, but is generated by screening and combining information from various sources with the help of an artificial intelligence model in order to meet the user's potential intention after the user's potential intention is inferred. The feedback information can come from the target object's own attribute information (the target object's own attribute information perceived by the environmental perception module 20, or the target object's own attribute information obtained by deep mining of the interaction method), or from other related information mined by the interaction method other than the target object's own attribute information, or from the information indicated in the information released by the user. As long as it is necessary to meet the user's potential intention, it can be used as the source of generating the feedback information, and when generating feedback information, the principle of screening information from various sources is to ensure that the feedback information can quickly and efficiently meet the user's potential intention, so that the interaction between the vehicle's intelligent cockpit and the user meets the needs of intelligence.

[0036] Additionally, the interaction method may also be coordinated with other application software systems, such as a reservation system, an image processing system, an information query system, a navigation system, etc., to provide the user with further feedback information of needs, thereby enriching the user's interactive experience functions.

[0037] Optionally, the step S2 also includes step S21: performing in-depth information mining on the target object with the help of an artificial intelligence model, so as to further mine more attribute information of the target object itself, and infer the potential intention based on the attribute information of the target object.

[0038] In the interactive method, for different objects, the target object's own attribute information may include the object's inherent attribute information and the state attribute information displayed by the object at the current time and scene. For example, when the object is a traffic sign, its inherent attribute information includes its identification type, identification meaning, etc., and the state attribute information includes the surrounding traffic flow, current geographical location, etc.; when the object is a person, its inherent attribute information includes its gender, height, age, etc., and the state attribute information includes action, relative distance, relative speed, etc.; when the object is a car, its inherent attribute information includes its brand, model, store location, etc., and the state attribute information includes vehicle condition, relative distance, sales information, etc.; when the object is a building, its inherent attribute information includes its shape structure, geographical location, cultural background, affiliation, layout of administrative and commercial facilities contained inside, etc., and the state attribute information includes human flow and commercial activity information.

[0039] Additionally, for different objects, the target object's own attribute information may also include external attribute information and internal attribute information. For example, when the object is a traffic sign, its external attribute information includes its graphics, shape, etc., and its internal attribute information includes the meaning of the sign, geographical location, surrounding road conditions, etc.; when the object is a person, its external attribute information includes his or her clothing, height, etc., and its internal information includes age preference, reaction speed, etc.; when the object is a car, its external attribute information includes its color, model, brand, etc., and its internal attribute information includes vehicle condition, store location, sales information, etc.; when the object is a building, its external attribute information includes shape structure, layout of administrative and commercial facilities, geographical location, etc., and its internal attribute information includes the cultural and historical background of the building, commercial and cultural value, and activity information.

[0040] Optionally, the step S2 also includes step S22: using an artificial intelligence model to mine other related information of the target object in addition to the target object's own attribute information, and inferring the potential intention in combination with the other related information. Therefore, the interaction method can combine the target object's own attribute information to infer the user's potential intention, and can also combine other related information to comprehensively infer the user's potential intention. Other related information can be pre-stored information, such as user personalized information that reflects user habits and preferences, historical driving route information, historical consumption information, map information, and other related information can also be related information searched online based on the current time and location, such as sales information, pedestrian flow, traffic conditions information, etc., and can also be the user's current operations and facial expressions.

[0041] Additionally, after inferring the user's potential intention, the interaction method can determine the user's behavior that may be accompanied by the user in order to achieve the desired purpose. At this time, the interaction method can automatically complete the behavior on behalf of the user, such as retrieving information, collecting images, identifying information, turning on driving assistance functions, setting parameters, opening the operation interface, etc. However, in order to enhance the human-machine intelligent interaction experience and prevent the user from feeling too hasty, the interaction method generally needs to first clearly indicate its inferred potential intention to the user, and only after the user confirms the potential intention will it actively complete the corresponding behavior on behalf of the user, especially when these behaviors affect vehicle driving or involve related functions that assist vehicle driving.

[0042] Here are some specific application scenarios:

[0043] Scenario 1: Recognition of surrounding vehicles. For example, the user asks "What is the brand of the new blue car in front of you?" The vehicle's intelligent cockpit responds "The new blue car about 10 meters in front of you is the A model sedan newly released by Company A in June this year. It has two configurations, hybrid and pure electric. Now we are showing several pictures of this model", and at the same time displays pictures of the model and promotional information on the in-car display device. It can be seen that the interactive method can respond according to the user's query information, and can also infer that the user has the potential intention to understand the attribute information of the target object itself based on the user's query information.

[0044] Scenario 2: Traffic sign reminder. For example, the user asks, "What is the traffic sign I just passed on the right? Am I speeding?" The vehicle's intelligent cockpit replies, "The traffic sign on the right side of the road you passed 50 meters ago is a speed limit sign for entering the highway exit ramp. You are now driving on the main highway and are not restricted by this sign." At the same time, the interactive method mines out the driver's current driving route, which is not included in the historical driving data and is a new route, or mines out the associated information that the driver is a novice. Then the vehicle's intelligent cockpit further feedbacks the following information, "You will enter the speed limit section of the main highway 200 meters ahead, and you will enter the tunnel 1000 meters ahead." It can be seen that the interactive method can respond according to the user's inquiry information, and can also infer that the user has a potential intention to understand future driving road conditions information other than the target object's own attribute information based on the user's inquiry information and the environmental information around the vehicle, and in combination with other associated information other than the target object.

[0045] Scenario 3: Non-motor vehicle identification. For example, the user asks "What brand is the bicycle that the lady in white is riding on the bicycle lane on the right?" The vehicle intelligent cockpit replies "There are three bicycles on your right, and the second bicycle that the lady in white is riding may be a model B road bicycle from company B." At the same time, the interactive method further mines the product introduction information, price and store location information of the road bicycle, and then the vehicle intelligent cockpit further feedbacks the following information: "This bicycle has a full carbon body and was launched in August last year. The lowest price is 17,999 yuan. The nearest brand B bicycle store is No. xx, xx Road. The planned navigation route is displayed for you. Do you need to start navigation now?" It can be seen that the interactive method can respond according to the user's inquiry information, and can also infer that the user has a potential intention to understand more of the target object's own attribute information based on the user's inquiry information and the attribute information of the target object. At the same time, it can also infer that the user has a potential intention to start navigation and obtain a navigation planned route in addition to the attribute information of the target object itself. After inferring that the user has a potential intention to start navigation, the interactive method first clearly indicates the inferred potential intention to the user and waits for the user to confirm.

[0046] Scenario 4: Commercial facility identification. For example, the user asks, "Did you pass by a tea shop just now?" The vehicle's smart cockpit responds, "There is a C tea shop on your right." At the same time, the interactive method mines out information that the tea shop is not open and there are other chain tea shops nearby. It also mines out pre-stored information about the user's preference for tea drinks, or information that the user has multiple tea consumption records. The vehicle's smart cockpit then further feeds back the following information, "This shop is not open, but there is a D tea shop 200 meters ahead of you that is open for business and you can park on the roadside. Do you need me to open the D tea shop's mini program and place an order in advance?" At the same time, voice and related graphic information about tea discounts and promotions can be fed back to the user through voice or a human-computer interaction interface, and the corresponding order information and navigation route information to the D tea shop can be displayed on the in-vehicle display device. It can be seen that the interactive method can respond according to the user's inquiry information, and can further mine the attribute information and other related information of the target object, and based on the user's inquiry information and the attribute information of the target object, as well as other related information other than the target object, it is inferred that the user has a potential intention to understand the attribute information (discount information) of the target object, and has a potential intention to open the ordering interface, and it is also inferred that the user has a potential intention to set up navigation and obtain navigation route planning.

[0047] Optionally, the information released by the user may be communication information between the user and a third party other than the vehicle, and the target object of interest in the call information is identified from the vehicle surroundings based on the communication information between the user and the third party other than the vehicle and environmental information around the vehicle.

[0048] Optionally, the interaction method can be used to assist the user in finding a target object outside the vehicle. When the user's potential intention is inferred to be to find the target object outside the vehicle based on the information released by the user, the feedback information generated for interaction with the user based on the potential intention is specifically image information of the current location of the target object.

[0049] Optionally, the target object may be highlighted in the image information in a striking manner. Preferably, the target object is circled in a prompt box in the image information, or the target object is highlighted in the image information.

[0050] Specifically, the third party other than the vehicle may be a taxi-hailing platform or a passenger outside the vehicle, and the communication information may be voice call information or graphic information between the user in the vehicle and the taxi-hailing platform or the passenger by means of a personal mobile terminal or broadcast, and based on the communication information, it is inferred that the potential intention of the user is to find out the passenger outside the vehicle. In particular, the interaction method may also combine the target object's own attribute information and / or other related information to infer that the potential intention of the user is to find out the passenger outside the vehicle, and specifically may combine the gestures and actions of the user and the passenger, as well as the operations performed by the user, for example, the gestures and actions of the user looking out the window to search for pedestrians, the gestures and actions of the passenger eagerly checking the passing vehicles, and the user turning on the map query function, honking the horn, turning on the double flash, etc. In this case, based on the communication information, at least one feature point of the passenger may be obtained by means of semantic analysis, such as the passenger's gender, clothing, accessories and appearance characteristics, and landmark facilities nearby. Then, the interaction method may also combine these feature points and use image recognition technology to find and identify the passenger in the image information of the vehicle's surrounding environment. The interactive method can also generate feedback information for interaction with the user based on the user's desire to find out the potential intentions of the passengers outside the vehicle. At this time, the interactive method can further obtain higher resolution and clearer local environment image information for the target object, that is, the local environment where the passengers outside the vehicle are currently located, and provide the local environment image information to the user as feedback information, thereby assisting the user to quickly and efficiently find the passengers outside the vehicle from the environment outside the vehicle.

[0051] Additionally, the interaction method can further infer that the user wants to drive to the location of the target object, for example, to pick up the passenger outside the car. Therefore, the recommended navigation route can be displayed on the in-car display device, and the user can be confirmed whether navigation needs to be turned on.

[0052] The following is a specific application scenario:

[0053] Scenario five, pedestrian recognition reminder, for example, the user talks to a passenger waiting to board the bus through a hands-free phone while looking for the passenger, and the call information includes the passenger's voice "I'm next to Gate 3, wearing a white dress, and carrying a large shopping bag"; the vehicle's intelligent cockpit infers that the user hopes to find the passenger, and automatically searches for qualified pedestrians in the surrounding crowd, and switches to displaying the image information of the location of the only passenger who meets the conditions of "white dress plus shopping bag" on the central control screen, and circles the passenger's location in the picture with a prompt box; at the same time, the interactive method also infers that the user needs to drive to the passenger's location to pick him up, and displays the navigation route information on the in-vehicle display device. The user finds the passenger based on the image information and drives to the passenger's location based on the navigation route information.

[0054] Optionally, the interaction method may further include step S4, in which the user asks questions to the smart cockpit based on the feedback information, and then uses the artificial intelligence model to further mine information of interest to the user based on the questions to provide feedback on the questions. For example, in the above scenario 2, after the user asks about the car, the vehicle's smart cockpit responds and displays a picture of the car model on the in-car display device, the user can further ask "What is the price?" based on the feedback information, and the vehicle's smart cockpit further mines feedback information related to the vehicle's price based on the above questions, and provides feedback on the question "The price of plug-in hybrids ranges from 210,000 to 280,000, and the price of pure electrics ranges from 180,000 to 270,000. You can get a discount of 10,000 yuan if you place an order now."

[0055] Additionally, before the vehicle smart cockpit implements the interaction method, the user can first wake up the interactive response function of the vehicle smart cockpit through a wake-up word.

[0056] Figure 2 FIG. 1 shows a schematic block diagram of a vehicle intelligent cockpit according to an exemplary embodiment of the present invention. Figure 2 As shown, the vehicle intelligent cockpit includes the following components:

[0057] A human-computer interaction module 10, which is configured to obtain information released by a user;

[0058] An environment perception module 20, which is configured to obtain environment information around the vehicle;

[0059] The artificial intelligence module 30 is configured to determine a target object outside the vehicle that the user may be concerned about based on the information released by the user and the environmental information, and to infer the user's potential intention other than knowing the attribute information of the target object itself based on the information released by the user;

[0060] The artificial intelligence module 30 is also configured to generate feedback information for interacting with the user based on the potential intention.

[0061] The human-computer interaction module 10 may include an in-vehicle voice device, which is configured to obtain voice information released by the user. The voice information released by the user may be inquiry information for the smart cockpit, or call information between the user and a third party outside the vehicle. The in-vehicle voice device may be an in-vehicle microphone or other sound pickup device. The human-computer interaction module 10 may also include a human-computer interaction interface, which is configured to obtain images, text information or gesture information released by the user.

[0062] Optionally, the human-computer interaction module 10 may also include an in-vehicle display device, which is configured to display relevant image information or text information of the target object to the user; the in-vehicle display device may be a head-up display, an instrument display group, a vehicle-mounted host display or a projection device, etc., and the in-vehicle display device is preferably a touch display screen.

[0063] Optionally, the human-computer interaction module 10 may provide feedback information to the user in the form of voice response, image display, text display, or any combination of the above forms.

[0064] The environmental perception module 20 can obtain objects and at least part of their attribute information in the environment around the vehicle from the environmental information around the vehicle. The environmental perception module 20 can obtain image information of the environment around the vehicle with the help of an image acquisition system, preferably with the help of a vehicle-mounted camera system or a surround view system. In addition, the environmental perception module 20 can also process the collected image information of the environment around the vehicle with the help of image recognition technology, so as to more accurately and efficiently obtain objects and at least part of their attribute information in the environment around the vehicle from the image information; it can also obtain the relative distance and speed of people or objects around the vehicle with the help of a speed measurement system, preferably with the help of a radar system; it can also obtain the current geographical location information of the vehicle with the help of a roadside communication unit of a global navigation satellite system or a vehicle-road-cloud integrated system, as well as commercial facilities, road conditions and traffic control information around the current geographical location of the vehicle.

[0065] An artificial intelligence algorithm is deployed in the artificial intelligence module 30, which can obtain at least one feature point of the target object with the help of semantic analysis, and determine the target object of concern in the information released by the user from the environment around the vehicle based on the at least one feature point and the environmental information around the vehicle. It can organize and provide feedback information for response to the user based on the information released by the user, as well as the attribute information of the target object and / or other related information other than the attribute information of the target object. It can also infer whether the user has a potential intention to understand the attribute information of the target object itself, whether he has a potential intention other than understanding the attribute information of the target object itself, and can also generate feedback information for interaction with the user based on the potential intention.

[0066] Optionally, the artificial intelligence module 30 can identify the target object from the image information of the vehicle's surroundings with the help of image recognition technology after inferring the user's potential intention to find a target object outside the vehicle based on the communication information between the user in the vehicle and a third party outside the vehicle, and can control the environmental perception module 20 according to relevant instructions to further obtain higher resolution and clearer local environment image information for the local environment where the target object is currently located, and feed back the local environment image information to the user to assist the user in quickly locking and finding the target object outside the vehicle.

[0067] Figure 3 A schematic block diagram of a control system for a vehicle intelligent cockpit according to an exemplary embodiment of the present invention is shown.

[0068] The control system of the vehicle intelligent cockpit includes an intelligent driving controller 1 and an intelligent cockpit controller 2 which are communicatively connected. The intelligent driving controller 1 is deployed with an artificial intelligence algorithm module 11 and a driving assistance system algorithm module 12 which are communicatively connected to each other. The artificial intelligence algorithm module 11 and the driving assistance system algorithm module 12 deployed on the intelligent driving controller 1 are both communicatively connected to the image acquisition system, so that image information of the image acquisition system can be obtained.

[0069] In addition to the original perception recognition and artificial intelligence algorithms in the intelligent driving domain, the intelligent driving controller 1 can perform additional processing on the image signal of the image acquisition system, identify the target objects of interest to the user in the vehicle's surrounding environment from the image information, and combine the perception results of other sensors and big data to perform in-depth information mining on the target objects in the vehicle's surrounding environment to obtain detailed attribute information and other related information of the target objects in the vehicle's surrounding environment. Other sensors may include vehicle speed sensors, radar sensor systems, etc. Big data can be obtained through network search, and relevant information pre-stored locally or in the cloud can also be searched.

[0070] The intelligent cockpit controller 2 is equipped with an artificial intelligence algorithm module 21, a human-computer interaction module 22 and an image processing module 23 that are in communication with each other. The intelligent driving controller 1 and the intelligent cockpit controller 2 in the control system of the vehicle intelligent cockpit can be separated independently or integrated into one.

[0071] Optionally, the intelligent driving controller 1 and the intelligent cockpit controller 2 are relatively independent and separated, and each of the intelligent driving controller 1 and the intelligent cockpit controller 2 has its own central processing unit. The intelligent driving controller 1 can transmit the perception information, i.e., the acquired image information, the identified target object and its detailed attribute information, and other related information to the intelligent cockpit controller 2. After receiving the above information, the intelligent cockpit controller 2 can perform secondary processing of the AI ​​algorithm in combination with the current vehicle position (coordinates and lane positioning results) and the surrounding geographic information of the navigation map to achieve complete perception of the surrounding environment in the vehicle's intelligent cockpit, and ultimately achieve the same perception ability as the information visible to the naked eye of the passengers in the car, and even supplement and deeply expand some information that is limited in the user's naked eye perception. At the same time, the intelligent driving controller 1 transmits the acquired image information to the intelligent cockpit controller, which can provide users with video interactive display and voice interaction.

[0072] Optionally, the intelligent driving controller 1 and the intelligent cockpit controller 2 can be integrated into one, using the same central processing unit, so as to be integrated into a cabin-driver integrated controller (also called a cabin-driver fusion controller). In this case, the control system of the vehicle's intelligent cockpit only needs to use one central processing unit to complete the perception recognition, regulatory control calculation and instrument display, navigation map, multimedia playback, and voice interaction functions of all image acquisition systems. This cabin-driver integrated controller is very convenient for the implementation of this solution, because the signals of all sensors enter the same central processing unit for processing, and the deployment of the artificial intelligence model can be complete at one time without the need to transmit perception information between the intelligent driving controller and the intelligent cockpit controller, making the response faster and the interaction more natural, which is beneficial for the perception of vehicle surrounding environment information, deep information mining, and artificial intelligence interaction.

[0073] In addition, the vehicle intelligent cockpit and driving control system are also connected to the in-vehicle display screen 3, microphone, speaker 4, and in-cabin camera in communication to achieve human-computer interaction with the user.

[0074] The vehicle's intelligent cockpit and driving control system are also connected to the radar system and the Global Navigation Satellite System (GNSS), and connected to the AI ​​Cloud through TBOX5 to enable information transmission and storage between local and remote servers.

[0075] Although specific embodiments have been described above, these embodiments are not intended to limit the scope of the present disclosure, even when only a single embodiment is described with respect to specific features. The feature examples provided in the present disclosure are intended to be illustrative, not limiting, unless otherwise expressly stated. In specific implementations, multiple features can be combined with each other where technically feasible, depending on actual needs. Various substitutions, changes, and modifications may also be conceived without departing from the spirit and scope of the present invention.

Claims

1. An interactive method for a vehicle intelligent cockpit, the interactive method comprising: Step S1: Acquire information released by the user and environmental information around the vehicle; Step S2: determining a target object outside the vehicle that the user may be concerned about based on the information released by the user and the environmental information, and inferring the user's potential intention other than knowing the attribute information of the target object itself based on the information released by the user; and Step S3: Generate feedback information for interaction with the user based on the potential intention.

2. The interactive method according to claim 1, characterized in that: The step S2 further includes step S21: mining the attribute information of the target object itself with the help of an artificial intelligence model, and inferring the potential intention in combination with the attribute information of the target object; and / or, The step S2 also includes step S22: using an artificial intelligence model to mine other related information except the attribute information of the target object itself, and inferring the potential intention in combination with the other related information.

3. The interactive method according to claim 1, characterized in that: In the step S2, based on the information released by the user, at least one feature point of the target object is obtained by means of semantic analysis, and the target object is determined based on the at least one feature point and the environmental information.

4. The interactive method according to claim 1, characterized in that: The interaction method also includes step S4: the user asks questions to the smart cockpit according to the feedback information, and then uses an artificial intelligence model to further mine information of interest to the user based on the questions to provide feedback on the questions.

5. The interactive method according to any one of claims 1 to 4, characterized in that: The information released by the user is specifically communication information between the user and a third party other than the vehicle.

6. The interactive method according to claim 5, characterized in that: The potential intention is specifically that the user expects to find the target object; and / or, The feedback information is specifically image information of the current position of the target object.

7. The interactive method according to claim 5, characterized in that: The target object is a passenger outside the vehicle, and the potential intention is specifically that the user wants to find the passenger outside the vehicle; The feedback information specifically includes image information of the current location of the passenger outside the vehicle and navigation information for picking up the passenger outside the vehicle.

8. A vehicle intelligent cockpit, the vehicle intelligent cockpit comprising: A human-computer interaction module, which is configured to obtain information released by a user; An environment perception module, which is configured to obtain environment information around the vehicle; an artificial intelligence module, which is configured to determine a target object outside the vehicle that the user may be concerned about based on the information released by the user and the environmental information, and to infer the user's potential intention other than knowing the attribute information of the target object itself based on the information released by the user; The artificial intelligence module is also configured to generate feedback information for interacting with the user based on the potential intention.

9. A control system for a vehicle intelligent cockpit according to claim 8, the control system comprising an intelligent driving controller and an intelligent cockpit controller that are communicatively connected, and the intelligent cockpit controller and the intelligent driving controller use the same central processing unit, thereby being integrated into a cockpit integrated controller; and The control system is communicatively connected with a display device, a microphone, a speaker, an image acquisition device, a radar system and a global navigation satellite system.

10. A computer program product, such as a machine-readable storage medium, comprising or storing computer program instructions, wherein the computer program is suitable for executing the interaction method for a vehicle intelligent cockpit according to any one of claims 1 to 7 when running on a computer.

Images