Scene state feedback method, in-vehicle infotainment system and storage medium

Abstract

The invention provides a scene state feedback method, an in-vehicle infotainment system and a storage medium. The scene state feedback method comprises the following steps: monitoring a target scene interface to obtain scene state information; generating a corresponding target feedback signal according to the scene state information, and sending the target feedback signal to the vehicle machine equipment; further converting the target feedback signal into a feedback instruction and executing the feedback instruction through the vehicle machine equipment; wherein the vehicle machine equipment at least comprises cabin equipment and handle equipment, and the feedback instructions at least comprise a visual instruction, an auditory instruction, a touch instruction and an olfactory instruction. According to the application, richer and more immersive interaction experience can be provided through multi-dimensional feedback; and handle operation can be expanded to the whole cabin environment, so that more comprehensive interactive experience is provided for users.

Description

Technical Field

[0001] The present application relates to the field of computer human-computer interaction, and in particular to a scene state feedback method, a vehicle system and a storage medium. Background Art

[0002] With the rapid development of smart cars and in-car entertainment systems, users' demand for playing games in the car is gradually increasing. Especially when driving long distances or waiting, in-car gaming entertainment has become an important way to relax.

[0003] Currently, most of the handle devices on the market are designed for PCs, consoles or mobile devices, and cannot meet the usage requirements in vehicle scenarios. Although these handle devices have basic buttons and vibration feedback, they are limited by the scene status form and cannot be effectively linked with the handle devices in the vehicle environment, nor can they mobilize the vehicle starting equipment, expanding from handle operation to the entire vehicle space, giving the game a more dimensional experience. Summary of the invention

[0004] In order to solve the above-mentioned technical problems, the present application provides a scene status feedback method, a vehicle system and a storage medium that can enhance the interactive experience.

[0005] Specifically, the present application provides a scene state feedback method, comprising the following steps: Monitor the target scene interface to obtain scene status information.

[0006] A corresponding target feedback signal is generated according to the scene state information, and the target feedback signal is sent to the vehicle machine device.

[0007] And, the target feedback signal is converted into a feedback instruction and executed through the vehicle-mounted device; wherein the vehicle-mounted device includes at least a cockpit device and a handle device, and the feedback instruction includes at least a visual instruction, an auditory instruction, a tactile instruction and an olfactory instruction.

[0008] In the above technical solution, through multi-dimensional feedback instructions, the present application can provide a richer and more immersive interactive experience; and the present application can realize the organic linkage between the handle device and the vehicle-mounted device, so that the handle operation is no longer limited to a single input and feedback, but can be extended to the entire cockpit environment, providing users with a more comprehensive interactive experience.

[0009] In addition, the feedback method of this application is not only applicable to games, but can also be extended to other in-vehicle entertainment scenarios, such as movies, music, etc., to provide a more diverse entertainment experience.

[0010] Furthermore, the scene state information includes at least scene image information and scene audio information; and the target feedback signal includes at least an image feedback signal and an audio feedback signal.

[0011] In the above technical solution, the scene image information mainly comes from the display screen or projection equipment in the car, capturing the visual elements in the current game or entertainment interface, such as the character position, environmental changes (such as weather, terrain), special effects (such as explosions, flashes), etc. in the combat game; the scene audio information comes from the audio system in the car, capturing the sound elements in the current game or entertainment interface, such as changes in background music, character dialogues, specific sound effects (such as gunshots, wind sounds), etc.

[0012] The two feedback signals correspond to two types of scene information, so as to facilitate the subsequent generation of targeted feedback instructions according to different feedback signals, thereby providing a more immersive interactive experience.

[0013] Further, generating a target feedback signal includes: An image processing model is mobilized based on the scene image information to perform image understanding to generate an image feedback signal.

[0014] And, based on the scene audio information, an audio processing model is mobilized to perform audio processing to generate an audio feedback signal; wherein the audio feedback signal at least includes an audio time domain signal and an audio frequency domain signal.

[0015] In the above technical solution, by mobilizing the image processing model and the audio processing model, the system can conduct in-depth analysis and understanding of the current scene image and audio information, thereby generating more accurate and rich feedback signals and realizing intelligent feedback; through the processing of audio time domain and frequency domain signals, the stereoscopic and spatial sense of the audio is enhanced, allowing users to obtain more multi-dimensional and multi-level feedback, and enhance the immersiveness and realism of the interaction.

[0016] Among them, the audio time domain signal is the original audio signal, which is used to directly adjust the output of the in-car audio system; the audio frequency domain signal is a signal that has been analyzed and processed in the frequency domain, which is used to enhance the stereoscopic and immersive feeling of the audio.

[0017] Furthermore, converting the target feedback signal into a feedback instruction includes: According to the image feedback signal, the feedback action rule is configured to output a corresponding feedback instruction.

[0018] And, according to the audio time domain signal and the audio frequency domain signal, the feedback action rule is configured to output a corresponding feedback instruction.

[0019] In the above technical solution, the feedback action rules set a series of mapping relationships between image feedback signals, audio feedback signals and vehicle equipment feedback behaviors. For example, the rules may be defined as: when a specific image change is detected (such as an explosion scene), the vehicle's ambient lights should flash and special effects should be presented on the display screen.

[0020] By configuring the target feedback signal with the feedback action rules, users can obtain a personalized feedback experience. The system can generate corresponding feedback instructions based on different scene states and user preferences to achieve efficient interaction.

[0021] In addition, the feedback mechanism of the present application has good scalability and can easily adjust the feedback action rules according to the new game status or device functions to adapt to different application scenarios and entertainment needs.

[0022] Furthermore, before performing image understanding, the scene image information is converted into a preset format in advance to obtain first scene information; and the image processing model at least performs: Feature extraction is performed based on the first scene information to obtain second scene information.

[0023] A plurality of groups of candidate scene areas are generated according to the second scene information, so as to perform target detection and target classification based on the candidate scene areas to obtain target positions and target categories.

[0024] And, the target position and target category are screened, and an image feedback signal is output.

[0025] In the above technical solution, by pre-converting the scene image information into a preset format, the standardization and consistency of subsequent processing are ensured, which helps to improve the recognition accuracy and stability of the image processing model; the deep learning model is used for feature extraction, which can effectively capture subtle features in complex scenes and ensure that subsequent target detection and classification can maintain high performance in a variety of environments; by generating multiple groups of candidate scene areas, the model's target detection and classification capabilities are significantly enhanced, effectively reducing the probability of not detecting meaningful targets, and correspondingly improving the system's adaptability to complex scenes; by screening target locations and target categories, the system can flexibly decide which targets should be fed back to users based on user needs and scene background, thereby enhancing the user's interactive experience.

[0026] Furthermore, before performing audio processing, an audio sampling period is predetermined; the audio processing module at least performs: An audio maximum value and an audio minimum value in the scene audio information are obtained based on the audio sampling period, so as to output an audio time domain signal according to the audio maximum value and the audio minimum value.

[0027] And, based on the audio sampling period, Fourier transform is performed on the scene audio information to output an audio frequency domain signal.

[0028] In the above technical scheme, by predetermining the audio sampling period, it is ensured that the audio signal can be sampled and analyzed with high precision, so as to capture the subtle changes and fluctuations in the audio, and improve the accuracy and precision of audio processing; the audio maximum and minimum extraction methods are adopted, so as to comprehensively capture the characteristic changes of the audio signal in the time domain, so that the generated audio time domain signal is richer and more diverse, which helps to improve the diversity and accuracy of audio feedback; through Fourier transform, the system can convert the audio signal into the frequency domain for multi-dimensional analysis, revealing the energy distribution of the audio at different frequencies, and providing richer characteristic information for audio processing.

[0029] In addition, by combining audio time domain and frequency domain signals, the system can provide fuller and more three-dimensional sound feedback; for example, it can perform equalization and enhancement processing based on the energy distribution of different frequency bands, making the sound effects more realistic and layered.

[0030] Furthermore, the target feedback signal is transmitted to the vehicle-mounted device via the vehicle-mounted system, and further includes: The operation signal of the handle device is monitored, and the operation signal is transmitted to the terminal device through the vehicle system, so as to display the scene state corresponding to the operation signal through the target scene interface of the terminal device.

[0031] In the above technical solution, two-way interaction between the handle device and the terminal device is realized through the vehicle system, and the operation signal of the handle device can be quickly transmitted to the terminal, thereby improving the smoothness of the interactive operation.

[0032] Based on the same concept, the present application also provides a vehicle system, which is used to execute the scene state feedback method, and the vehicle system includes: The monitoring module is used to monitor the target scene interface to obtain scene status information.

[0033] The feedback module is used to generate a corresponding target feedback signal according to the scene state information, and send the target feedback signal to the vehicle machine device, so that the target feedback signal is converted into a feedback instruction and executed by the vehicle machine device.

[0034] The vehicle-mounted device at least includes a cockpit device and a handle device, and the feedback instructions at least include visual instructions, auditory instructions, tactile instructions and olfactory instructions.

[0035] In the above technical solution, the vehicle system extracts scene status information from the target scene interface and generates feedback, thereby improving the linkage effect between the handle device and the target scene; and the present application expands the feedback range of the handle device, so that users can not only feel the feedback in the game from the handle device, but also get a more immersive interactive experience through the cockpit device.

[0036] Furthermore, the monitoring module is also used to monitor the operation signal of the handle device; the feedback module is also used to transmit the operation signal to the terminal device so as to display the scene state corresponding to the operation signal through the target scene interface of the terminal device.

[0037] In the above technical solution, the vehicle-computer system realizes two-way interaction, and the operation signals of the handle device can be quickly transmitted to the terminal device, thereby improving the smoothness of control.

[0038] Based on the same concept, the present application also provides a readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the scene state feedback method is implemented.

[0039] Compared with the prior art, the beneficial effects of this application are: The present application first monitors the target scene interface to obtain scene status information; then generates a corresponding target feedback signal based on the scene status information, and sends the target feedback signal to the vehicle-mounted device; further converts the target feedback signal into a feedback instruction through the vehicle-mounted device and executes it; wherein the vehicle-mounted device includes at least a cockpit device and a handle device, and the feedback instruction includes at least a visual instruction, an auditory instruction, a tactile instruction, and an olfactory instruction. The present application can provide a richer and more immersive interactive experience through multi-dimensional feedback; and can realize the expansion of handle operations to the entire cockpit environment, providing users with a more comprehensive interactive experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a flow chart of the scene state feedback method described in this application.

[0041] Figure 2 This is a framework diagram of the vehicle system described in this application. DETAILED DESCRIPTION

[0042] The following is a further detailed description of a scene state feedback method, a vehicle system and a storage medium of the present application in conjunction with specific embodiments and drawings.

[0043] See also Figure 1 , the embodiment of the present application provides a scene state feedback method, comprising the following steps: Monitor the target scene interface to obtain scene status information.

[0044] Furthermore, the scene state information includes at least scene image information and scene audio information.

[0045] Among them, scene image information mainly comes from the display screen or projection equipment in the car, capturing the visual elements in the current game or entertainment interface, such as the character position, environmental changes (such as weather, terrain), special effects (such as explosions, flashes), etc. in combat games; scene audio information comes from the audio system in the car, capturing the sound elements in the current game or entertainment interface, such as changes in background music, character dialogues, specific sound effects (such as gunshots, wind sounds), etc.

[0046] Furthermore, after the scene state information is obtained, a corresponding target feedback signal is generated according to the scene state information, and the target feedback signal is sent to the vehicle machine device.

[0047] Wherein, the target feedback signal at least includes an image feedback signal and an audio feedback signal.

[0048] Further, generating a target feedback signal includes: An image processing model is mobilized based on the scene image information to perform image understanding to generate an image feedback signal.

[0049] Furthermore, before performing image understanding, the scene image information is converted into a preset format in advance to obtain first scene information.

[0050] In some embodiments, the scene image information is standardized and formatted in advance to meet the input requirements of the image processing model. The preset format may include image size, color space conversion, data enhancement, etc.; in one embodiment of the present application, the scene image information is uniformly converted into RGB format.

[0051] It should be noted that those skilled in the art can select a preprocessing method for scene image information according to actual application requirements, and is not limited to converting to RGB format.

[0052] The image processing model at least performs: Feature extraction is performed based on the first scene information to obtain second scene information.

[0053] In some embodiments, an image processing model (such as a backbone neural network) is used to perform feature extraction on the preprocessed scene image information to identify important features in the image, such as edges, textures, and shapes. The second scene information generated at this stage contains a higher-level feature representation and can better distinguish different elements of the scene.

[0054] A plurality of groups of candidate scene areas are generated according to the second scene information, so as to perform target detection and target classification based on the candidate scene areas to obtain target positions and target categories.

[0055] In some embodiments, based on the second scene information, the image processing model generates multiple groups of candidate scene regions and identifies areas that may contain targets; this process involves a region proposal network or a sliding window technique to identify parts of the image that may be of interest.

[0056] For the generated candidate scene areas, the model will perform target detection (identify objects in the image) and target classification (determine the category of the object); this stage outputs the location and category information of the target, and includes data such as coordinates and confidence.

[0057] And, the target position and target category are screened, and an image feedback signal is output.

[0058] In some embodiments, after obtaining the target location and target category, the system will filter according to preset rules and thresholds, and only retain those targets that meet specific conditions (such as targets with confidence levels higher than a certain threshold). Ultimately, the generated image feedback signal will contain the location information and category information of these filtered targets for use in subsequent feedback instruction generation.

[0059] And, based on the scene audio information, an audio processing model is mobilized to perform audio processing to generate an audio feedback signal; wherein the audio feedback signal at least includes an audio time domain signal and an audio frequency domain signal.

[0060] Furthermore, before performing audio processing, an audio sampling period is predetermined; the audio processing module at least performs: An audio maximum value and an audio minimum value in the scene audio information are obtained based on the audio sampling period, so as to output an audio time domain signal according to the audio maximum value and the audio minimum value.

[0061] And, based on the audio sampling period, Fourier transform is performed on the scene audio information to output an audio frequency domain signal.

[0062] In some embodiments, before audio processing is performed, an audio sampling period is pre-set to ensure the stability and consistency of the audio data; the audio sampling period determines the frequency and interval of audio signal sampling, and technical personnel in this field can choose it according to actual application requirements and equipment performance, and it is not limited here.

[0063] Within the set audio sampling period, the system samples the audio signal point by point and records the maximum value (audio maximum) and minimum value (audio minimum) within each period. These values ​​reflect the intensity fluctuation of the audio signal within the period. Based on the extracted audio maximum and minimum values, the system generates corresponding time domain signals. These signals include the changes in audio intensity on the time axis, which are used for subsequent real-time audio feedback.

[0064] The sampled scene audio information is Fourier transformed and converted to the frequency domain; this transformation can reveal the distribution of audio signals in different frequency components and provide more detailed and multi-dimensional audio features. Through the results of Fourier transform, the system generates audio frequency domain signals, which show the energy distribution of audio in different frequency bands and provide the basis for subsequent advanced audio processing and feedback.

[0065] Furthermore, after the vehicle-mounted device receives the target feedback signal, the target feedback signal is converted into a feedback instruction and executed by the vehicle-mounted device; wherein the vehicle-mounted device includes at least a cockpit device and a handle device, and the feedback instruction includes at least a visual instruction, an auditory instruction, a tactile instruction and an olfactory instruction.

[0066] It should be noted that the cockpit equipment includes at least a steering wheel, seats, air conditioning, ambient lights, etc. Those skilled in the art may expand to more cockpit equipment according to actual application requirements and are not limited to this.

[0067] In addition, the components in the handle device that execute the feedback instructions include at least buttons, joysticks, motors, triggers, etc.

[0068] Furthermore, converting the target feedback signal into a feedback instruction includes: According to the image feedback signal, the feedback action rule is configured to output a corresponding feedback instruction.

[0069] And, according to the audio time domain signal and the audio frequency domain signal, the feedback action rule is configured to output a corresponding feedback instruction.

[0070] In some embodiments, the feedback action rules set a series of mapping relationships between image feedback signals, audio feedback signals and vehicle equipment feedback behaviors. For example, the rules may be defined as: when a specific image change is detected (such as an explosion scene), the vehicle's ambient lights should flash and special effects should be presented on the display screen.

[0071] The system interacts with the feedback action rules based on the image feedback signal and configures specific feedback instructions, including visual instructions (such as flashing lights), display instructions (such as dynamic interface changes), etc., to ensure that the feedback instructions can accurately reflect the current scene status.

[0072] Furthermore, the system analyzes the audio time domain and frequency domain signals, and determines the feedback instructions corresponding to the current audio effect through the set feedback action rules. For example, if a gunshot is detected, the rules may instruct the sound system to increase the volume of a specific frequency band and generate a vibration effect in the cockpit to enhance the sense of immersion; finally, the system will combine the audio time domain and frequency domain signals to generate corresponding feedback instructions, clarify the playback intensity, delay and other characteristics of the sound effect, so as to enhance the user's auditory experience.

[0073] Among them, by configuring the target feedback signal with the feedback action rules, the user can obtain a personalized feedback experience, and the system can generate corresponding feedback instructions according to different scene states and user preferences to achieve efficient interaction. In addition, the feedback mechanism of this application has good scalability, and can easily adjust the feedback action rules according to the new game state or device function to adapt to different application scenarios and entertainment needs.

[0074] In some embodiments, when the scene state feedback method is applied to game content, the vehicle system monitors the target scene interface of the terminal device, that is, the game interface, in real time; then extracts the corresponding images and audio based on the game interface; and then configures it based on the feedback action rules, and finally outputs the corresponding feedback instructions for execution by the cockpit device and the handle device.

[0075] Assume that a user is experiencing a racing-themed cloud game, where the in-vehicle game controller device, terminal device (displaying the game interface), cockpit device and vehicle system are tightly integrated; the vehicle system is integrated into the terminal device.

[0076] The user starts the terminal device, enters the in-car entertainment interface, and selects a racing game in the cloud game; the handle device is automatically connected to the car system via, for example, Bluetooth, that is, the handle device is connected to the terminal device.

[0077] In one embodiment, in the game, when the user's racing character encounters a collision or is attacked, the car system detects the change in the character's health value in real time based on the scene status information of the cloud game. After the handle device receives the corresponding feedback command, the motor simulates the impact of a vehicle collision in the real world to remind the user to pay attention to the character's status; at the same time, the seat receives the feedback command, and the seat produces a vibration synchronized with the bumpy road surface during the collision, allowing the user to experience a more realistic bumpy road surface in the real cockpit; and the sound system can play the impact sound effect, providing a more directional and surround sound effect. And the impact intensity can be vividly reflected through the change in vibration intensity.

[0078] In another embodiment, when the game progresses to the ice and snow scene, the vehicle system receives the real-time scene status information of the cloud game, and the ambient light of the cabin equipment is adjusted to a cool blue to enhance the immersive feeling of the ice and snow environment; at the same time, the temperature of the air conditioner in the car can also be adaptively adjusted to provide a more immersive gaming experience.

[0079] In another embodiment, the vehicle fragrance system is configured to simulate the smell of the gaming environment, such as the grassy scent of the forest, the dry breath of the desert, and the smell of fireworks in the city. These fragrance types correspond to the gaming scenes one by one.

[0080] For example, when the game character enters a dense forest, the car system recognizes the changes in the game scene and immediately sends corresponding feedback instructions to the fragrance system. The fragrance system begins to release the scent of plants and trees, simulating the natural atmosphere in the forest. At the same time, the sound effects of birdsong can be played through the audio system, greatly enhancing the sense of immersion.

[0081] For example, in the game, if the character moves quickly in the forest, the status signal of the cloud game will instruct the fragrance system to dynamically adjust the intensity of the smell; for example, the faster the character runs, the stronger the smell of grass and trees, simulating the changes in smell brought by natural wind.

[0082] For example, in some complex scenes, such as when a game character passes the edge of a forest and approaches a city, the fragrance system can intelligently mix multiple scents, allowing the user to smell the grassy scent of the forest and the fireworks of the city at the same time.

[0083] Furthermore, the target feedback signal is transmitted to the vehicle-mounted device via the vehicle-mounted system, and further includes: The operation signal of the handle device is monitored, and the operation signal is transmitted to the terminal device through the vehicle system, so as to display the scene state corresponding to the operation signal through the target scene interface of the terminal device.

[0084] In some embodiments, users control related operations in the game through a handle device such as a joystick, and the operation signal of the handle device is transmitted to the terminal device in real time through the vehicle system middleware, ensuring the immediacy and accuracy of the operation and realizing two-way interaction between the handle device and the terminal device.

[0085] For further information, see Figure 2 The present application also provides a vehicle system, the vehicle system is used to execute the scene state feedback method, and the vehicle system includes: The monitoring module is used to monitor the target scene interface to obtain scene status information.

[0086] The feedback module is used to generate a corresponding target feedback signal according to the scene state information, and send the target feedback signal to the vehicle machine device, so that the target feedback signal is converted into a feedback instruction and executed by the vehicle machine device.

[0087] The vehicle-mounted device at least includes a cockpit device and a handle device, and the feedback instructions at least include visual instructions, auditory instructions, tactile instructions and olfactory instructions.

[0088] In the above technical solution, the vehicle system extracts scene status information from the target scene interface and generates feedback, thereby improving the linkage effect between the handle device and the target scene; and the present application expands the feedback range of the handle device, so that users can not only feel the feedback in the game from the handle device, but also get a more immersive interactive experience through the cockpit device.

[0089] Furthermore, the monitoring module is also used to monitor the operation signal of the handle device; the feedback module is also used to transmit the operation signal to the terminal device so as to display the scene state corresponding to the operation signal through the target scene interface of the terminal device.

[0090] In the above technical solution, the vehicle-computer system realizes two-way interaction, and the operation signals of the handle device can be quickly transmitted to the terminal device, thereby improving the smoothness of control.

[0091] Furthermore, the present application also provides a readable storage medium having a computer program stored thereon, and the computer program implements the scene state feedback method when executed by a processor.

[0092] In some embodiments, the storage medium stores several computer programs for causing a device to execute all or part of the steps of the method described in various embodiments of the present application.

[0093] The medium may include a U disk, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and other media that can store program codes.

[0094] In summary, the present application provides a scene state feedback method, a vehicle system and a storage medium; first, the target scene interface is monitored to obtain scene state information; then, a corresponding target feedback signal is generated according to the scene state information, and the target feedback signal is sent to the vehicle device; and then, the target feedback signal is further converted into a feedback instruction by the vehicle device and executed; wherein, the vehicle device includes at least a cockpit device and a handle device, and the feedback instruction includes at least a visual instruction, an auditory instruction, a tactile instruction and an olfactory instruction. The present application can provide a richer and more immersive interactive experience through multi-dimensional feedback; and can realize the expansion of handle operation to the entire cockpit environment, providing users with a more comprehensive interactive experience.

[0095] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present application to this. Those of ordinary skill in the art may make various changes and modifications therein without departing from the scope and spirit of the present application. All these changes and modifications are intended to be included within the scope of the present application as required by the appended claims.

[0096] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0097] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0098] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some modules according to the embodiments of the present application. The application can also be implemented as a device program (e.g., computer program and computer program product) for executing a part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0099] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0100] Although the description of the present application is carried out in conjunction with the above specific embodiments, it is obvious that those skilled in the art can make many substitutions, modifications and changes based on the above content. Therefore, all such substitutions, improvements and changes are included in the spirit and scope of the appended claims.

Claims

1. A scene state feedback method, characterized in that: The following steps are involved: Monitor the target scene interface to obtain scene status information; Generate a corresponding target feedback signal according to the scene state information, and send the target feedback signal to the vehicle machine device; And, the target feedback signal is converted into a feedback instruction and executed through the vehicle-mounted device; wherein the vehicle-mounted device includes at least a cockpit device and a handle device, and the feedback instruction includes at least a visual instruction, an auditory instruction, a tactile instruction and an olfactory instruction.

2. The scene state feedback method according to claim 1, characterized in that: The scene state information includes at least scene image information and scene audio information; The target feedback signal at least includes an image feedback signal and an audio feedback signal.

3. The scene state feedback method according to claim 2, characterized in that: The generating of the target feedback signal comprises: Mobilizing an image processing model to understand the image based on the scene image information to generate an image feedback signal; And, based on the scene audio information, an audio processing model is mobilized to perform audio processing to generate an audio feedback signal; wherein the audio feedback signal at least includes an audio time domain signal and an audio frequency domain signal.

4. The scene state feedback method according to claim 3, characterized in that: The converting the target feedback signal into a feedback instruction comprises: According to the image feedback signal, configure with the feedback action rule and output the corresponding feedback instruction; And, according to the audio time domain signal and the audio frequency domain signal, the feedback action rule is configured to output a corresponding feedback instruction.

5. The scene state feedback method according to claim 4, characterized in that: Before image understanding, the scene image information is converted into a preset format in advance to obtain first scene information; the image processing model at least performs: Perform feature extraction based on the first scene information to obtain second scene information; Generate multiple groups of candidate scene areas according to the second scene information, perform target detection and target classification based on the candidate scene areas, and obtain target positions and target categories; And, the target position and target category are screened, and an image feedback signal is output.

6. The scene state feedback method according to claim 4, characterized in that: Before performing audio processing, the audio sampling period is predetermined; the audio processing module at least performs: Obtaining an audio maximum value and an audio minimum value in the scene audio information based on the audio sampling period, so as to output an audio time domain signal according to the audio maximum value and the audio minimum value; And, based on the audio sampling period, Fourier transform is performed on the scene audio information to output an audio frequency domain signal.

7. The scene state feedback method according to claim 1, characterized in that: The target feedback signal is transmitted to the vehicle machine device through the vehicle machine system, and further includes: The operation signal of the handle device is monitored, and the operation signal is transmitted to the terminal device through the vehicle system, so as to display the scene state corresponding to the operation signal through the target scene interface of the terminal device.

8. A vehicle computer system, characterized in that: The vehicle system is used to execute the scene state feedback method according to any one of claims 1 to 7, and the vehicle system includes: A monitoring module is used to monitor the target scene interface to obtain scene status information; A feedback module, used for generating a corresponding target feedback signal according to the scene state information, and sending the target feedback signal to the vehicle machine device, so that the target feedback signal is converted into a feedback instruction and executed by the vehicle machine device; The vehicle-mounted device at least includes a cockpit device and a handle device, and the feedback instructions at least include visual instructions, auditory instructions, tactile instructions and olfactory instructions.

9. The vehicle system according to claim 8, characterized in that: The monitoring module is also used to monitor the operation signal of the handle device; The feedback module is further used to transmit the operation signal to the terminal device so as to display the scene state corresponding to the operation signal through the target scene interface of the terminal device.

10. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the scene state feedback method according to any one of claims 1 to 7 is implemented.

Images