Information processing method and device, electronic equipment and storage medium

Abstract

This disclosure relates to an information processing method, apparatus, electronic device, and storage medium. The method includes: acquiring entities in the vehicle's surrounding environment; acquiring in-vehicle voice information; extracting key information from the voice information, the key information being used to describe a target entity; and if the entities in the vehicle's surrounding environment include the target entity, generating interactive information about the target entity. Thus, once a user's voice information is detected to be related to the vehicle's surrounding environment, indicating that the user's points of interest are related to a target entity in the vehicle's surrounding environment, interactive information with that target entity can be generated promptly to provide the user with more services.

Description

Technical Field

[0001] This disclosure relates to the field of information, and more particularly to an information processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] As technology continues to develop and vehicles become more widely used, more and more vehicles are also becoming increasingly intelligent. For example, while driving, users can input their destination into the in-car navigation system, which can then plan a route for them and even continuously optimize the route based on road conditions.

[0003] However, existing methods, including many related smart devices in car navigation systems, can usually only provide users with simple services such as route planning, and cannot provide users with personalized interactive information. This makes the existing methods relatively limited in function and unable to better meet the needs of users. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides an information processing method, apparatus, electronic device, and storage medium.

[0005] According to a first aspect of the present disclosure, an information processing method is provided, comprising:

[0006] Obtain entities in the environment surrounding the vehicle;

[0007] Obtain in-vehicle voice information;

[0008] Key information is extracted from the voice information, and the key information is used to describe the target entity;

[0009] If the entities in the environment surrounding the vehicle include the target entity, interactive information about the target entity is generated.

[0010] Optionally, acquiring entities in the environment surrounding the vehicle includes:

[0011] Multimedia information during the vehicle's operation is acquired in real time using camera equipment, including images and / or videos;

[0012] The multimedia information is subjected to real-scene recognition using a preset recognition algorithm to obtain the first entity recognition result;

[0013] Obtain the preset entities contained in the current location area of ​​the vehicle in the preset map data;

[0014] Based on the first entity recognition result and the preset entity, the entities in the environment surrounding the vehicle are determined.

[0015] Optionally, the method further includes:

[0016] If the target entity is not included in the entities in the environment surrounding the vehicle, obtain the vehicle's historical multimedia information;

[0017] The historical multimedia information is subjected to real-scene recognition using a preset recognition algorithm to obtain the second entity recognition result;

[0018] Based on the second entity recognition result and the preset entity, the entities in the environment surrounding the vehicle are determined.

[0019] Optionally, the method further includes:

[0020] Detect whether the voice information contains a question;

[0021] If the voice information contains a question, perform the step of extracting key information from the voice information.

[0022] Optionally, detecting whether the voice information contains question information includes:

[0023] The speech information is converted into text information, and the text information is segmented to obtain multiple words;

[0024] Determine whether the multiple word segments contain a preset interrogative word;

[0025] If the multiple word segments contain preset question words, it is determined that the voice information contains question information.

[0026] Optionally, extracting key information from the voice information includes:

[0027] Obtain the attribute information of each word in the multiple word segments;

[0028] The target word whose attribute information is a preset attribute among the multiple word segments is used as the key information in the question information, and the preset attribute includes entity nouns.

[0029] Optionally, the target entity is a target location, and the generation of interactive information for the target entity includes:

[0030] Obtain the current location of the vehicle;

[0031] Input the target location into a preset map to obtain the target location;

[0032] Based on the current location and the target location, route information from the current location to the target location is generated, and the route information is used as the interaction information.

[0033] Optionally, the target entity is scene information, and the generation of interactive information for the target entity includes:

[0034] Obtain comments on the scenery information from the voice information;

[0035] Save the scenery information and associate the comment information with the scenery information.

[0036] Optionally, the method further includes:

[0037] Extract the target voiceprint features from the speech information;

[0038] Based on the pre-established relationship between voiceprint features and accounts, the target account corresponding to the target voiceprint features is determined, and the interaction information of the target entity is associated with the target account.

[0039] According to a second aspect of the present disclosure, an information processing apparatus is provided, comprising:

[0040] An entity acquisition module is used to acquire entities in the environment surrounding the vehicle;

[0041] The voice information acquisition module is used to acquire voice information from inside the vehicle;

[0042] An information extraction module is used to extract key information from the voice information, and the key information is used to describe the target entity.

[0043] An interactive information generation module is used to generate interactive information of the target entity if the entities in the environment surrounding the vehicle include the target entity.

[0044] Optionally, the entity acquisition module includes:

[0045] A multimedia information acquisition unit is used to acquire multimedia information during the vehicle's driving process in real time through a camera device, wherein the multimedia information includes images and / or videos;

[0046] The recognition unit is used to perform real-scene recognition on the multimedia information using a preset recognition algorithm to obtain a first entity recognition result.

[0047] An entity acquisition unit is used to acquire a preset entity contained in the current location area of ​​the vehicle in the preset map data;

[0048] An entity determination unit is used to determine entities in the environment surrounding the vehicle based on the first entity recognition result and the preset entity.

[0049] Optionally, the device further includes:

[0050] The historical multimedia information acquisition module is used to acquire the historical multimedia information of the vehicle when the target entity is not included in the entities in the environment surrounding the vehicle.

[0051] The real-scene recognition module is used to perform real-scene recognition on the historical multimedia information using a preset recognition algorithm to obtain a second entity recognition result.

[0052] An entity determination module is used to determine entities in the environment surrounding the vehicle based on the second entity recognition result and the preset entity.

[0053] Optionally, the device further includes:

[0054] The information detection module is used to detect whether the voice information contains a question.

[0055] Optionally, the information detection module includes:

[0056] A text processing unit is used to convert the speech information into text information and perform word segmentation on the text information to obtain multiple words;

[0057] A vocabulary judgment unit is used to determine whether the plurality of word segments contain a preset question word;

[0058] The question information determination unit is used to determine that the speech information contains question information when the multiple word segments contain preset question words.

[0059] Optionally, the information extraction module includes:

[0060] The attribute information acquisition unit is used to acquire the attribute information of each word in the plurality of word segments;

[0061] The key information determination unit is used to identify target words whose attribute information is a preset attribute among the multiple word segments as key information in the question information, wherein the preset attribute includes entity nouns.

[0062] Optionally, the target entity is a target location, and the interactive information generation module includes:

[0063] Current position acquisition unit, used to acquire the current position of the vehicle;

[0064] The template location acquisition unit is used to input the target location into a preset map and obtain the target location where the target location is located;

[0065] The route information generation unit is used to generate route information from the current location to the target location based on the current location and the target location, and to use the route information as the interaction information.

[0066] Optionally, the target entity is scene information, and the interactive information generation module includes:

[0067] Comment information acquisition unit, used to acquire comment information on the scene information in the voice information;

[0068] An information processing unit is used to collect the scene information and associate the comment information with the scene information.

[0069] Optionally, the device further includes:

[0070] The feature extraction unit is used to extract the target voiceprint features from the speech information;

[0071] The account processing unit is used to determine the target account corresponding to the target voiceprint feature based on the pre-established relationship between voiceprint features and accounts, and associate the interaction information of the target entity with the target account.

[0072] According to a third aspect of the present disclosure, an electronic device is provided, comprising:

[0073] processor;

[0074] Memory used to store processor-executable instructions;

[0075] The processor is configured to execute any of the information processing methods described above.

[0076] A fourth aspect of this disclosure provides a non-transitory computer-readable storage medium that, when instructions in the storage medium are executed by a processor of a mobile terminal, enables the mobile terminal to perform the aforementioned information processing method.

[0077] According to a fifth aspect of the present disclosure, an application / computer program product is provided that, when run on a computer, causes the computer to perform the steps of any of the information processing methods described in the above embodiments.

[0078] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0079] The information processing method, apparatus, electronic device, and storage medium provided in this disclosure acquire entities in the vehicle's surrounding environment and in-vehicle voice information. When an entity in the vehicle's surrounding environment contains a target entity extracted from the voice information, interactive information about that target entity is generated. Thus, once a connection is detected between a user's voice information and the vehicle's surrounding environment, indicating that the user's points of interest are related to a target entity in the surrounding environment, interactive information with that target entity can be generated promptly to provide the user with more services.

[0080] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0081] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0082] Figure 1 This is a flowchart illustrating an information processing method according to an exemplary embodiment;

[0083] Figure 2 yes Figure 1 A flowchart of one embodiment of step S110;

[0084] Figure 3 This is another flowchart illustrating an information processing method according to an exemplary embodiment;

[0085] Figure 4 This is a block diagram illustrating an information processing apparatus according to an exemplary embodiment;

[0086] Figure 5 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0087] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0088] Figure 1 This is a flowchart illustrating an information processing method according to an exemplary embodiment, such as... Figure 1 As shown, this method, used in a terminal, may include the following steps:

[0089] In step S110, entities in the environment surrounding the vehicle are acquired.

[0090] In the embodiments provided in this disclosure, image acquisition devices can be installed on the vehicle to acquire multimedia data such as images and videos of the vehicle's surrounding environment. For example, cameras or video cameras can be installed on the vehicle to process the acquired image or video data, and existing image recognition algorithms can be used to identify entities in the multimedia data. Entities in the vehicle's surrounding environment can specifically be buildings, people, animals, or natural scenery, etc., and can be set as needed or targeted according to user preferences. Alternatively, users can use handheld terminals to acquire multimedia data such as images and / or videos of the vehicle's surrounding environment to identify entities. The embodiments of this disclosure are not limited to these methods.

[0091] In step S120, the voice information inside the vehicle is acquired.

[0092] Specifically, sound acquisition devices can be installed inside the vehicle, or voice information inside the vehicle can be obtained through users' mobile phones and other terminal devices.

[0093] In step S130, key information is extracted from the speech information. This key information is used to describe the target entity.

[0094] In this embodiment, voice information can be processed via a terminal or uploaded to the cloud for processing via a server.

[0095] In step S140, if the entities in the environment surrounding the vehicle include the target entity, interaction information of the target entity is generated.

[0096] In this embodiment, when users inside the vehicle converse with other users, they often discuss objects of interest in the surrounding environment. Therefore, by collecting the user's voice information, extracting key information from this voice information, and matching this key information with entities in the vehicle's surrounding environment, it can be determined whether the entities in the surrounding environment contain the target entity. Additionally, while a user is driving, interactive information can also be generated by acquiring the user's voice information. For example, while driving, a user might exclaim, "What a beautiful lake!" upon seeing the scenery outside the car. By processing the user's voice information and acquiring entities in the vehicle's surrounding environment, interactive information including the lake's name, area, altitude, and local specialties can be generated and displayed to the user. In other scenarios, this could also involve processing the user's voice information during conversations in other languages ​​while driving.

[0097] The key information here refers to the description of the target entity. For example, while the vehicle is moving or stationary, if a user sees a magnificent building, such as an ancient tower, in the external environment, the user inside the vehicle is likely to discuss it. By acquiring the voice information from inside the vehicle and extracting key information from it, if this key information contains the noun "ancient tower" or "tower," and by recognizing the target entity "tower" through video or images captured by the camera around the vehicle, it indicates that the user inside the vehicle is discussing the building. At this point, interactive information can be generated targeting this entity. For example, more information about the target entity can be provided to increase the user's interest, achieving an interactive effect and providing more services to meet their needs.

[0098] The information processing method provided in this disclosure acquires entities in the vehicle's surrounding environment and in-vehicle voice information. When an entity in the vehicle's surrounding environment contains a target entity extracted from the voice information, interactive information about that target entity is generated. Thus, once a user's voice information is detected to be related to the vehicle's surrounding environment, indicating that the user's points of interest are related to a target entity in the surrounding environment, interactive information with that target entity can be generated promptly to provide the user with more services.

[0099] To illustrate in detail how entities in the vehicle's surrounding environment are acquired, in another embodiment provided in this disclosure, in conjunction with the above embodiments, such as... Figure 2 As shown, step S110 may further include the following steps:

[0100] Step S111: Acquire multimedia information during the vehicle's operation in real time using a camera device. This multimedia information includes images and / or video.

[0101] Step S112: Perform real-scene recognition on the multimedia information using a preset recognition algorithm to obtain the first entity recognition result.

[0102] In this embodiment, when acquiring multimedia information during vehicle operation, existing image recognition algorithms can be used for real-world recognition to obtain a first entity recognition result, such as identifying the presence of an "ancient pagoda" or "temple" in the real-world scene. For example, images or videos can be preprocessed to extract features from the multimedia information, and recognition can be performed using template matching or similar methods. Specific existing recognition algorithms can be used, which will not be elaborated upon here.

[0103] Step S113: Obtain the preset entities contained in the current location area of ​​the vehicle in the preset map data.

[0104] In this embodiment, relevant entities around the current location can be found in preset map data. Since current navigation map applications are quite mature, information such as relevant buildings and natural attractions is generally included in navigation maps. Therefore, preset entities contained in the vehicle's current location area can be obtained through preset map data.

[0105] Step S114: Based on the first entity recognition result and the preset entities, determine the entities in the environment surrounding the vehicle.

[0106] Typically, preset map data includes as much existing entity and other map information as possible. Therefore, the preset map data usually contains a relatively large number of preset entities in the area where the vehicle is currently located. However, real-time acquisition of multimedia information during the vehicle's movement via camera equipment is usually limited by the field of view of the surrounding environment, such as whether there are obstructions, and can generally only capture a limited number of entities. Therefore, the entities identified in the first recognition result can be matched with preset entities to determine the entities in the vehicle's surrounding environment, and then entities that the user inside the vehicle may be interested in can be filtered out.

[0107] In another embodiment provided in this disclosure, based on the above embodiments, such as Figure 3 As shown, the method may further include the following steps:

[0108] Step S115: If the entities in the environment surrounding the vehicle do not include the target entity, obtain the vehicle's historical multimedia information.

[0109] Step S116: Real-scene recognition is performed on historical multimedia information using a preset recognition algorithm to obtain the second entity recognition result.

[0110] Step S117: Based on the second entity recognition result and the preset entity, determine the entities in the environment surrounding the vehicle.

[0111] If the entities in the vehicle's surrounding environment do not include the target entity, it means that the target entity the user is interested in is likely no longer within the current location area, and it is necessary to obtain the vehicle's historical multimedia information. For example, if the vehicle is constantly moving, the target entity the user is talking about has already passed by and is no longer in the current field of view. A second entity result can be obtained by tracing back to previously recorded multimedia information, such as information from a few minutes ago, and performing real-scene recognition on the traced historical multimedia information.

[0112] The second entity recognition result obtained by real-scene recognition of historical multimedia information can be obtained by first checking if there is already a recognized result. If so, the existing recognition result can be directly used as the second entity recognition result to improve processing efficiency and avoid duplicate recognition. If not, existing related recognition algorithms can be used for recognition. Furthermore, by matching the second recognition result with preset entities in preset map data, entities in the vehicle's surrounding environment can be determined, and entities that the user may be interested in can be filtered out.

[0113] In yet another embodiment provided in this disclosure, based on the above embodiments, the method may further include the following steps:

[0114] In step S150, it is detected whether the voice information contains a question.

[0115] In step S160, if the voice information contains a question, step S130 is executed.

[0116] In this embodiment provided in this disclosure, when a question is detected in the voice information, it indicates that the user needs an answer, and interaction is more necessary to meet the user's needs. Therefore, the presence of a question in the voice information can be used as a trigger condition to process the user's voice information, avoiding the need to process all of the user's voice information, thereby effectively improving processing efficiency.

[0117] Specifically, speech information can be converted into text information, and the text information can be segmented to obtain multiple words. It is then determined whether any of these multiple words contain a preset interrogative word. If multiple words contain the preset interrogative word, it is determined that the speech information contains a question. This embodiment of the disclosure determines whether the speech information contains a question by detecting whether it contains a preset interrogative word. For example, if the user's speech information contains words such as "why," "what," "I don't know," or "do you know," it is very likely that the user has asked a question that needs to be answered. Therefore, in this embodiment of the disclosure, detecting the presence of a question in the speech information can be used as a trigger condition for extracting key information from the speech information, thereby greatly improving processing efficiency when the user's conversation is lengthy.

[0118] In this embodiment, during the extraction of key information from speech information, the speech information can be converted into text information, and the text information can be segmented to obtain multiple segments. The attribute information of each segment can then be obtained, such as which segments are verbs, which are names, etc. The target segments with preset attributes based on the attribute information of these multiple segments are used as key information in the query information. The preset attribute can be an entity name. Since users are usually interested in certain objects, such as buildings, scenic spots, etc., or locations, this embodiment extracts the target segments with entity names as attribute information as key information. The extracted key information is likely to be entities that users are interested in.

[0119] In another embodiment provided in this disclosure, in conjunction with the above embodiments, the key information extracted from the voice information is very likely a description of a location, that is, the target entity can be a target location. Therefore, the above step S140 may include the following steps:

[0120] Step S141: Obtain the current location of the vehicle.

[0121] Step S142: Input the target location into the preset map to obtain the target location.

[0122] Step S143: Based on the current location and the target location, generate route information from the current location to the target location, and use the route information as interactive information.

[0123] In this embodiment, if a user needs to reach a target location or is interested in a target location, the current location of the vehicle is obtained, and the target location is input into a preset map to obtain the target location's location. The preset map can generate route information from the current location to the target location, and this route information is displayed as interactive information to the user in the vehicle so that the user can refer to the route to determine if they need to go to the target location. This route information may include a route map, traffic conditions such as traffic congestion along the route, and the travel time, among other information.

[0124] In another embodiment provided in this disclosure, the target entity may also be scene information, and step S140 may further include the following steps:

[0125] Step S144: Obtain the comment information on the scene information from the voice information.

[0126] Step S145: Save the scenery information and associate the comment information with the scenery information.

[0127] In this example, users can save attraction information and reviews for future reference by themselves or other users.

[0128] Furthermore, in the embodiments provided in this disclosure, target voiceprint features can be extracted from the voice information; based on the pre-established relationship between voiceprint features and accounts, the target account corresponding to the target voiceprint features is determined, and the interaction information of the target entity is associated with the target account. Since voiceprints are unique, in these embodiments, corresponding accounts can be bound through voiceprint features.

[0129] Figure 4 This is a block diagram of an information processing apparatus according to an exemplary embodiment. (Refer to...) Figure 4 The device includes an entity acquisition module 10, a voice information acquisition module 20, an information extraction module 30, and an interactive information generation module 40.

[0130] The entity acquisition module is used to acquire entities in the environment surrounding the vehicle;

[0131] The voice information acquisition module is used to acquire voice information from inside the vehicle;

[0132] An information extraction module is used to extract key information from the voice information, and the key information is used to describe the target entity.

[0133] An interactive information generation module is used to generate interactive information of the target entity if the entities in the environment surrounding the vehicle include the target entity.

[0134] Optionally, the entity acquisition module includes:

[0135] A multimedia information acquisition unit is used to acquire multimedia information during the vehicle's driving process in real time through a camera device, wherein the multimedia information includes images and / or videos;

[0136] The recognition unit is used to perform real-scene recognition on the multimedia information using a preset recognition algorithm to obtain a first entity recognition result.

[0137] An entity acquisition unit is used to acquire a preset entity contained in the current location area of ​​the vehicle in the preset map data;

[0138] An entity determination unit is used to determine entities in the environment surrounding the vehicle based on the first entity recognition result and the preset entity.

[0139] Optionally, the device further includes:

[0140] The historical multimedia information acquisition module is used to acquire the historical multimedia information of the vehicle when the target entity is not included in the entities in the environment surrounding the vehicle.

[0141] The real-scene recognition module is used to perform real-scene recognition on the historical multimedia information using a preset recognition algorithm to obtain a second entity recognition result.

[0142] An entity determination module is used to determine entities in the environment surrounding the vehicle based on the second entity recognition result and the preset entity.

[0143] Optionally, the device further includes:

[0144] The information detection module is used to detect whether the voice information contains a question.

[0145] Optionally, the information detection module includes:

[0146] A text processing unit is used to convert the speech information into text information and perform word segmentation on the text information to obtain multiple words;

[0147] A vocabulary judgment unit is used to determine whether the plurality of word segments contain a preset question word;

[0148] The question information determination unit is used to determine that the speech information contains question information when the multiple word segments contain preset question words.

[0149] Optionally, the information extraction module includes:

[0150] The attribute information acquisition unit is used to acquire the attribute information of each word in the plurality of word segments;

[0151] The key information determination unit is used to identify target words whose attribute information is a preset attribute among the multiple word segments as key information in the question information, wherein the preset attribute includes entity nouns.

[0152] Optionally, the target entity is a target location, and the interactive information generation module includes:

[0153] Current position acquisition unit, used to acquire the current position of the vehicle;

[0154] The template location acquisition unit is used to input the target location into a preset map and obtain the target location where the target location is located;

[0155] The route information generation unit is used to generate route information from the current location to the target location based on the current location and the target location, and to use the route information as the interaction information.

[0156] Optionally, the target entity is scene information, and the interactive information generation module includes:

[0157] Comment information acquisition unit, used to acquire comment information on the scene information in the voice information;

[0158] An information processing unit is used to collect the scene information and associate the comment information with the scene information.

[0159] Optionally, the device further includes:

[0160] The feature extraction unit is used to extract the target voiceprint features from the speech information;

[0161] The account processing unit is used to determine the target account corresponding to the target voiceprint feature based on the pre-established relationship between voiceprint features and accounts, and associate the interaction information of the target entity with the target account.

[0162] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0163] The information processing apparatus provided in this disclosure acquires entities in the vehicle's surrounding environment and in-vehicle voice information. When an entity in the vehicle's surrounding environment contains a target entity extracted from the voice information, it generates interactive information for that target entity. Thus, once it detects that a user's voice information is associated with the vehicle's surrounding environment, indicating that the user's conversation is related to a target entity in the surrounding environment, interactive information with that target entity can be generated promptly to provide the user with more services.

[0164] Figure 5 This is a block diagram illustrating an information processing apparatus 800 according to an exemplary embodiment. For example, apparatus 800 is an electronic device, specifically a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0165] Reference Figure 5 The device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0166] Processing component 802 typically controls the overall operation of device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0167] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of this data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0168] Power supply component 806 provides power to various components of device 800. Power supply component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to device 800.

[0169] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0170] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0171] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0172] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0173] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as WiFi, carrier networks (such as 2G, 3G, 4G, or 5G), or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0174] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the information processing method described above.

[0175] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of the device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0176] This disclosure also provides a non-transitory computer-readable storage medium, which, when the instructions in the storage medium are executed by the processor of a mobile terminal, enables the mobile terminal to perform the aforementioned information processing method.

[0177] According to embodiments of this disclosure, an application / computer program product is also provided. In yet another embodiment provided by this disclosure, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the steps of any of the information processing methods described in the above embodiments.

[0178] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, DSL (Digital Subscriber Line)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs (Digital Versatile Discs)), or semiconductor media (e.g., SSDs (Solid State Disks)).

[0179] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0180] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0181] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An information processing method characterized by comprising: include: Obtain entities in the environment surrounding the vehicle; Obtain in-vehicle voice information; Key information is extracted from the voice information, and the key information is used to describe the target entity; If the entities in the environment surrounding the vehicle include the target entity, generate interactive information for the target entity; Extract the target voiceprint features from the speech information; Based on the pre-established relationship between voiceprint features and accounts, the target account corresponding to the target voiceprint features is determined, and the interaction information of the target entity is associated with the target account, and the corresponding target account is bound through the voiceprint features. The target entity includes target location or scenery information; When the target entity is a target location, generating the interactive information of the target entity includes: obtaining the current location of the vehicle; Input the target location into a preset map to obtain the target location; Based on the current location and the target location, generate route information from the current location to the target location, and use the route information as the interaction information; When the target entity is scene information, generating the interactive information of the target entity includes: Obtain comments on the scenery information from the voice information; Save the scenery information and associate the comment information with the scenery information.

2. The method according to claim 1, wherein acquiring entities in the environment surrounding the vehicle comprises: Multimedia information during the vehicle's operation is acquired in real time using camera equipment, including images and / or videos; The multimedia information is subjected to real-scene recognition using a preset recognition algorithm to obtain the first entity recognition result; Obtain the preset entities contained in the current location area of ​​the vehicle in the preset map data; Based on the first entity recognition result and the preset entity, the entities in the environment surrounding the vehicle are determined.

3. The method according to claim 2, further comprising: If the target entity is not included in the entities in the environment surrounding the vehicle, obtain the vehicle's historical multimedia information; The historical multimedia information is subjected to real-scene recognition using a preset recognition algorithm to obtain the second entity recognition result; Based on the second entity recognition result and the preset entity, the entities in the environment surrounding the vehicle are determined.

4. The method according to claim 1, further comprising: Detect whether the voice information contains a question; If the voice information contains a question, perform the step of extracting key information from the voice information.

5. The method according to claim 4, wherein detecting whether the voice information contains question information includes: The speech information is converted into text information, and the text information is segmented to obtain multiple words; Determine whether the multiple word segments contain a preset interrogative word; If the multiple word segments contain preset question words, it is determined that the voice information contains question information.

6. The method according to claim 5, wherein extracting key information from the voice information includes: Obtain the attribute information of each word in the multiple word segments; The target word whose attribute information is a preset attribute among the multiple word segments is used as the key information in the question information, and the preset attribute includes entity nouns.

7. An information processing apparatus, characterized by comprising: include: The entity acquisition module is used to acquire entities in the environment surrounding the vehicle; The voice information acquisition module is used to acquire voice information from inside the vehicle; An information extraction module is used to extract key information from the voice information, and the key information is used to describe the target entity. An interactive information generation module is used to generate interactive information of the target entity if the entities in the environment surrounding the vehicle include the target entity. Extract the target voiceprint features from the speech information; Based on the pre-established relationship between voiceprint features and accounts, the target account corresponding to the target voiceprint feature is determined, and the interaction information of the target entity is associated with the target account. The corresponding target account is then bound through the voiceprint feature. The target entity includes target location or scenery information; When the target entity is a target location, the interactive information generation module includes: Current position acquisition unit, used to acquire the current position of the vehicle; The template location acquisition unit is used to input the target location into a preset map and obtain the target location where the target location is located; A route information generation unit is used to generate route information from the current location to the target location based on the current location and the target location, and to use the route information as the interactive information; When the target entity is scene information, the interactive information generation module includes: Comment information acquisition unit, used to acquire comment information on the scene information in the voice information; An information processing unit is used to collect the scene information and associate the comment information with the scene information.

8. An electronic device, comprising: include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the information processing method according to any one of claims 1-6.

9. A non-transitory computer-readable storage medium, comprising: When the instructions in the storage medium are executed by the processor of the mobile terminal, the mobile terminal is able to perform an information processing method, the method comprising the steps of the information processing method according to any one of claims 1-6.

10. A computer program product, characterised in that, When it is run on a computer, it causes the computer to perform the steps of the information processing method according to any one of claims 1-6.

Images