A method for transforming audiovisual data into an olfactory experience based on distributed processing

Abstract

A system and method for transforming audiovisual data into an olfactory experience based on distributed processing is provided. [Solution] This system can be widely used in scenes such as games, movies, and media. It includes a data center, cloud server, dataset, media content, base station, game computer, headphones, and an odor generator. It uses artificial intelligence to identify sounds or images in the scene and obtain corresponding odor data. For example, when a gunshot or firearm is fired in a movie scene, the odor generator mixes corresponding aromatic compounds based on the gunpowder odor data obtained through artificial intelligence analysis, generates and releases the corresponding odor, allowing the user to experience the sense of smell in addition to sight and hearing, thereby improving the on-site experience.

Description

[Technical Field]

[0001] The present invention relates to the field of computers, and more particularly to a system and method for rapidly converting video or audio data into an olfactory experience through artificial intelligence techniques. [Background technology]

[0002] With the advancement of science and technology, people's demands for entertainment experiences are increasing. Hearing and vision are important elements in entertainment. Various entertainments, such as movies, television programs, and video games, require high-quality sound effects to enhance the viewer's entertainment experience. With the development of science and technology, high-resolution displays enhance the sense of vision, surround sound systems enhance the sense of hearing, and even bass systems can cause the body to feel vibrations. In addition to visual and auditory experiences, olfactory experiences are beginning to attract attention, although the perception of smell is rarely realized in current entertainment systems.

[0003] However, the application of olfactory entertainment still faces several challenges, such as how to accurately control and deliver scents and odors, how to maintain the stability and consistency of the scent, etc. Therefore, olfactory entertainment needs more research and development to achieve a better user experience.

[0004] Currently, entertainment systems with olfactory information use predefined media data or metadata to trigger the mechanism for generating odors, and the generated odors are relatively fixed. When the entertainment system plays a specific scene, the odor generator generates the odor of that scene. However, systems that generate odors by analyzing and classifying real-time audio data require high data processing speeds, making it difficult to process odor information using audio data in real time.

[0005] Existing voice classification technologies typically need to be run on a local computer, which has drawbacks such as slow processing speed, high latency, and heavy computational load, which places a heavy burden on the local computer and slows down the computer, especially when large amounts of data need to be processed.

[0006] To solve this problem, cloud computing and distributed systems technologies have been widely used in recent years. Cloud computing technology centralizes computing and storage resources in the cloud, providing powerful computing and storage capabilities and effectively resolving the burden and latency issues of local computers. Distributed systems technology can allocate data and computing tasks to multiple computing nodes for processing, while providing high fault tolerance and scalability, better supporting large-scale data processing.

[0007] Therefore, by allocating and processing audio and image classification calculations on a distributed platform, computing efficiency and processing speed can be greatly improved, latency can be reduced, and large amounts of media content and data sets can be better managed and stored. The application of such technology will have a great impact and application value in fields such as voice processing, speech recognition, and entertainment. Summary of the Invention [Problem to be solved by the invention]

[0008] To solve the above-mentioned shortcomings of the prior art, the present invention provides a method for converting audiovisual data into an olfactory experience based on distributed processing, which is a method for realizing high-speed and low-latency audio classification using distributed processing. The system includes a data center, a cloud server, a dataset, media content, a base station, a gaming computer, headphones, and an odor generator. [Means for solving the problem]

[0009] The present invention is achieved by the following technical means.

[0010] A method for converting audiovisual data into an olfactory experience based on distributed processing, comprising: a cloud server, a distributed processing base station, an audiovisual media content storage center, an audiovisual playback unit, and an odor generating device, wherein the cloud server is installed with an artificial intelligence model for converting audiovisual data into odor data, and the distributed processing base station is installed with an intelligent analysis model for an odor feature dataset, and awakening The raw unit reads and plays audiovisual media content from the audiovisual media content storage center, and simultaneously transmits the audiovisual media content to a distributed processing base station. The intelligent analysis model of the distributed processing base station analyzes the images and / or sounds in the audiovisual media content to obtain olfactory data corresponding to the images and / or sounds, and transmits it to an odor generating device to release an olfactory experience. If the corresponding olfactory data cannot be obtained by analyzing the odor feature dataset of the intelligent analysis model, the related audiovisual media content is transferred to a cloud server, and the artificial intelligence model of the cloud server extracts odor feature datasets related to the images and / or sounds in the audiovisual media content and updates the related odor feature dataset to the intelligent analysis model of the distributed processing base station. This method converts audiovisual data into an olfactory experience based on distributed processing.

[0011] The step of converting an audio signal from audiovisual media into olfactory data includes: A1. Capturing audio signals from audiovisual media and A2. Normalizing the audio signal; A3. Performing a Fourier transform on the audio signal to obtain a time-frequency domain signal; A4. Inputting the time-frequency domain signal into an intelligent analysis model or an artificial intelligence model to obtain corresponding voice classification data; A5. Reading olfactory data corresponding to audio classification data; Includes:

[0012] The step of converting an image signal from an audiovisual medium into olfactory data includes: B1. Capturing image signals from audiovisual media and B2. Preprocessing the image signal to form an image matrix relating pixel intensities within the image signal; B3. Inputting the image matrix into an intelligent analysis model or an artificial intelligence model to obtain image classification data; B4. Reading olfactory data corresponding to image classification data; Includes:

[0013] The distributed processing base station, audiovisual playback unit, and odor generating device are installed locally, the cloud server is installed remotely, and the audiovisual media content storage center is installed locally or remotely.

[0014] The locally installed devices are connected by wireless data transmission, which includes Wifi, Bluetooth, mobile network, and radio frequency transmission. [Effects of the Invention]

[0015] The present invention has the following beneficial effects: A cloud server uses artificial intelligence to learn, process, and analyze a large number of media contents, extracting and storing a feature dataset related to odors. The distributed processing base station periodically updates a simple index of the odor feature dataset from the cloud server, allowing a specified number of features to be quickly and efficiently extracted and stored in the distributed processing base station. When a game computer streams and downloads videos for playback or gaming, the associated audio and / or image signals are first transmitted to a local base station for analysis. The base station utilizes a low-cost processor (GPU) built into the base station to process the input audio and / or image signals in a parallel / streaming manner. Because the base station is generally installed locally, i.e., connected to the same domain network as devices such as the game computer, headphones, and odor generator, the latency / delay of trigger signals for analysis can be reduced. If there are audio and / or image signals that are not present in the base station, the associated audio and / or image signals are transmitted to the cloud server for collaborative computation to extract feature datasets for the audio and / or image signals, and the associated feature datasets are then transmitted to the base station. At the same time, the cloud server periodically updates the calculated feature dataset to all base stations. This enables distributed processing, significantly improving processing speed and efficiency, reducing the computational load and latency of local base stations, and achieving high-speed, low-latency voice classification. Furthermore, the use of data centers and cloud data centers allows for the effective storage and management of large amounts of media content and datasets, which is expected to have a wide range of applications. [Brief explanation of the drawings]

[0016] The invention will now be further described with reference to the accompanying drawings. [Figure 1] 1 is a schematic structural diagram of the present invention; [Figure 2] 1 is a flowchart for converting an audio signal into olfactory data according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0017] See Figure 1. The method for converting audiovisual data into an olfactory experience based on distributed processing includes a cloud server, a distributed processing base station, an audiovisual media content storage center, an audiovisual playback unit, and an odor generating device. The cloud server is equipped with a GPU server with high computing power, which trains and calculates an artificial intelligence model that converts audiovisual data into odor data to obtain an odor feature dataset corresponding to the images and / or sounds in the audiovisual media content. The same model and algorithm as the base station are then executed to train new data, and more complex models, such as multi-layer DL models (CNN, RNN type) for dataset training and complex data prediction, are executed and calculated. The odor feature dataset corresponding to the images and / or sounds executed or calculated on the cloud server is updated to the distributed processing base station.

[0018] The distributed processing base station, the audiovisual playback unit, and the odor generating device are arranged on a local network. The video playback unit retrieves content to be played from an audiovisual media content storage center, including image data and / or audio data, and the image data and audio data are simultaneously transmitted to the video playback unit and the distributed processing base station on the local network. The fast transmission speed of the local network improves system response, while preventing network congestion caused by excessive access requests to the cloud server and reducing the computational load on the cloud server. The distributed processing base station also has an intelligent analysis model for the odor feature dataset. The intelligent analysis model calculates the image data and audio data played by the audiovisual playback unit in real time, obtains the corresponding odor data, and then transmits it to the odor generating device to unlock the olfactory experience, thereby improving the user's on-site experience.

[0019] The method by which the distributed processing base station and the cloud server convert audio signals in audiovisual media into olfactory data is described in detail below. Referring to Figure 2, the specific steps are as follows:

[0020] A1. The audiovisual media content storage center transmits audiovisual media in response to a request from the audiovisual playback unit, and simultaneously transmits the audio signal contained in the audiovisual media to a distributed processing base station or cloud server. The audio signal is typically transmitted to the distributed processing base station or cloud server in real time as a file such as a .wav file or an analog signal.

[0021] A2. The distributed processing base station or cloud server samples the audio signal and normalizes the audio signal, including intercepting the audio signal of a specified length.

[0022] A3. Perform a Fourier transform (short-time Fourier transform) on the audio signal to convert the time-domain representation of the audio into a time-frequency domain signal.

[0023] A4. The time-frequency domain signal is input into an intelligent analysis model or an artificial intelligence model. In the intelligent analysis model of the distributed processing base station and the artificial intelligence model of the cloud server, the time-frequency domain signal is input into a convolutional neural network (CNN) for calculations to obtain feature maps, and then a classifier predicts the feature category to obtain classification data for the sound corresponding to the smell.

[0024] A5. Based on the classification data, the olfactory data is read and sent to the odor generating device, which mixes different odor generating units according to the corresponding ratio through the olfactory data to obtain and emit the odor corresponding to the olfactory data.

[0025] The method by which the base station converts the image signal of the audiovisual media into olfactory data is described in detail below. The specific steps are as follows:

[0026] B1. The audiovisual media content storage center transmits the audiovisual media according to the request of the audiovisual playback unit, and also transmits the image signal in the audiovisual media to the distributed processing base station or cloud server.

[0027] B2. The distributed processing base station or cloud server pre-processes the image signal to form an image matrix relating pixel intensities in the image signal.

[0028] B3. The image matrix is ​​input into an intelligent analysis model or an artificial intelligence model, such as a convolutional neural network (CNN), and calculated to obtain feature maps. Then, a classifier predicts the feature category to obtain classification data of the image corresponding to the smell.

[0029] B4. Based on the classification data, read the olfactory data and send it to the odor generating device, which mixes different odor generating units according to the corresponding ratio through the olfactory data to obtain and release the odor corresponding to the olfactory data.

[0030] The present invention can be widely used in scenes such as games, movies, and media, by using artificial intelligence to identify sounds or images in the scene and obtain corresponding scent data. For example, when a gunshot or a firearm is fired in a movie scene, the scent generator will mix the corresponding aromatic compounds based on the gunpowder scent data obtained through artificial intelligence analysis, generate and release the corresponding scent, allowing the user to experience the sense of smell in addition to the sense of sight and hearing, thereby improving the on-site experience.

[0031] The above are merely preferred embodiments of the present invention, and the present invention is not limited to the above-mentioned embodiments. As long as the technical effects of the present invention are achieved by similar means, any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure, and all shall belong to the protection scope of the present invention. Various modifications and changes can be made to the technical means and / or embodiments within the protection scope of the present invention.

Claims

1. 1. A method for transforming audiovisual data into an olfactory experience based on distributed processing, comprising: The present invention includes a cloud server, a distributed processing base station, an audiovisual media content storage center, an audiovisual playback unit, and an odor generating device, The cloud server is installed with an artificial intelligence model that converts audiovisual data into odor data, and the distributed processing base station is installed with an intelligent analysis model of the odor feature dataset; The audiovisual playback unit reads and plays the audiovisual media content from the audiovisual media content storage center, and simultaneously transmits the audiovisual media content to the distributed processing base station. The intelligent analysis model of the distributed processing base station analyzes the images and / or sounds in the audiovisual media content to obtain olfactory data corresponding to the images and / or sounds, and transmits the olfactory data to the odor generating device to release the olfactory experience. If the corresponding olfactory data cannot be obtained from the smell feature dataset of the intelligent analysis model, the related audiovisual media content is transferred to a cloud server, and the artificial intelligence model of the cloud server extracts the smell feature dataset related to the image and / or sound in the audiovisual media content, and sends the related smell feature dataset to the distributed processing base station to update the smell feature dataset of the intelligent analysis model. A method for transforming audiovisual data into an olfactory experience based on distributed processing.

2. The step of converting an audio signal from audiovisual media into olfactory data includes: A1. Capturing an audio signal from an audiovisual medium; A2. Normalizing the audio signal; A3. Performing a Fourier transform on the audio signal to obtain a time-frequency domain signal; A4. Inputting the time-frequency domain signal into an intelligent analysis model or an artificial intelligence model to obtain corresponding voice classification data; A5. Reading olfactory data corresponding to audio classification data; 2. The method of transforming audiovisual data into an olfactory experience based on distributed processing according to claim 1, comprising:

3. The step of converting an image signal from an audiovisual medium into olfactory data includes: B1. Capturing an image signal from an audiovisual medium; B2. Pre-processing the image signal to form an image matrix relating pixel intensities within the image signal; B3. Inputting the image matrix into an intelligent analysis model or an artificial intelligence model to obtain image classification data; B4. Reading olfactory data corresponding to the image classification data; 2. The method of transforming audiovisual data into an olfactory experience based on distributed processing according to claim 1, comprising:

4. The method for converting audiovisual data into an olfactory experience based on distributed processing as described in claim 1, characterized in that the distributed processing base station, audiovisual playback unit, and odor generating device are installed locally, the cloud server is installed remotely, and the audiovisual media content storage center is installed locally or remotely.

5. The method for converting audiovisual data into an olfactory experience based on distributed processing as described in claim 4, characterized in that the locally installed devices are connected by wireless data transmission, and the wireless data transmission includes Wi-Fi, Bluetooth (registered trademark), mobile network, and radio frequency transmission.

Images