A method for transforming audiovisual data into an olfactory experience based on distributed processing.

Abstract

This invention provides a system and method for converting audiovisual data into olfactory experiences based on distributed processing. [Solution] A system that can be widely used in scenes such as games, movies, and media, comprising a data center, cloud server, dataset, media content, base station, game computer, headphones, and scent generator, which uses artificial intelligence to identify sounds or images in a scene and acquire corresponding scent data. For example, when gunshots or firearms are fired in a movie scene, the scent generator mixes corresponding aromatic compounds based on the gunpowder scent data obtained by artificial intelligence analysis, generating and releasing the corresponding scent, allowing the user to experience smell in addition to sight and hearing, thus realizing a better on-site experience.

Description

Technical Field

[0001] The present invention relates to the field of computers, and particularly to a system and method for rapidly converting video or audio data into a smell experience through artificial intelligence technology.

Background Art

[0002] With the progress of science and technology, people's requirements for entertainment experiences are increasing. In entertainment, hearing and vision are important elements. In various entertainments such as movies, TV shows, and video games, high-quality sound effects are required to improve the entertainment experience of viewers. With the development of science and technology, the high resolution of displays enhances the visual sense, the surround sound system enhances hearing, and even the bass system makes the body feel vibrations. In addition to visual and auditory experiences, olfactory experiences have also begun to attract attention, but the perception of smell has hardly been realized in current entertainment systems.

[0003] However, there are still several issues in the application of olfactory entertainment, such as how to accurately control and provide scents or smells, and how to maintain the stability and consistency of smells. Therefore, olfactory entertainment requires more research and development to achieve a better user experience.

[0004] Currently, entertainment systems with olfactory information use pre-defined media data or metadata to trigger a mechanism for generating smells, and the generated smells are relatively fixed. When the entertainment system plays a predetermined scene, the smell generation device generates the smell of that scene. However, in a system that uses real-time audio data to analyze and classify to generate smells, a high data calculation processing speed is required, so it is difficult to process smell information in real time with audio data.

[0005] Existing speech classification technologies typically need to run on a local computer, which has drawbacks such as slow processing speed, high latency, and heavy computational load. This puts a heavy burden on the local computer, especially when processing large amounts of data, and slows down the computer's performance.

[0006] To address this problem, cloud computing and distributed systems technologies have been widely used in recent years. Cloud computing technologies centralize resources such as computing and storage in the cloud, providing powerful computing and storage capabilities and effectively solving the problems of local computer burden and latency. Distributed systems technologies can allocate tasks such as data and computing to multiple computing nodes for processing, while simultaneously offering high fault tolerance and scalability, better supporting large-scale data processing.

[0007] Therefore, by allocating and processing audio and image classification calculations on a distributed platform, computing efficiency and processing speed can be significantly improved, latency can be reduced, and large amounts of media content and datasets can be better managed and stored. The application of such technologies will have a significant impact and application value in fields such as speech processing, speech recognition, and entertainment. [Overview of the project] [Problems that the invention aims to solve]

[0008] To address the shortcomings of the prior art described above, the present invention provides a method for converting audiovisual data into an olfactory experience based on distributed processing, which is a method for achieving high-speed and low-latency speech classification using distributed processing. The system includes a data center, cloud servers, datasets, media content, base stations, gaming computers, headphones, and odor generators. [Means for solving the problem]

[0009] This invention is achieved by the following technical means.

[0010] A method for converting audiovisual data into an olfactory experience based on distributed processing, comprising a cloud server, a distributed processing base station, an audiovisual media content storage center, an audiovisual playback unit, and an odor generator, wherein the cloud server is equipped with an artificial intelligence model for converting audiovisual data into odor data, and the distributed processing base station is equipped with an intelligent analysis model for odor feature datasets, and viewing awakening A live unit reads and plays audiovisual media content from an audiovisual media content memory center, while simultaneously transmitting the audiovisual media content to a distributed processing base station. The intelligent analysis model at the distributed processing base station analyzes the images and / or sounds within the audiovisual media content to obtain olfactory data corresponding to the images and / or sounds, and transmits it to a scent generator to release the olfactory experience. If the intelligent analysis model cannot obtain corresponding olfactory data by analyzing the scent feature dataset, the relevant audiovisual media content is transferred to a cloud server. The artificial intelligence model at the cloud server extracts the scent feature dataset associated with the images and / or sounds within the audiovisual media content and updates the intelligent analysis model at the distributed processing base station with the relevant scent feature dataset. This is a method for transforming audiovisual data into an olfactory experience based on distributed processing.

[0011] The step of converting audio signals from audiovisual media into olfactory data is, A1. To capture audio signals from audiovisual media, A2. Normalizing the audio signal, A3. Performing a Fourier transform on the audio signal to obtain a time-frequency domain signal, A4. Inputting a time-frequency domain signal into an intelligent analysis model or artificial intelligence model to obtain corresponding speech classification data, A5. Reading olfactory data corresponding to voice classification data, Includes.

[0012] The step of converting image signals from audiovisual media into olfactory data is, B1. Acquiring image signals from audiovisual media, B2. Preprocessing the image signal to form an image matrix related to the pixel intensity within the image signal, B3. Inputting an image matrix into an intelligent analysis model or artificial intelligence model to obtain image classification data, B4. Reading olfactory data corresponding to image classification data, Includes.

[0013] The distributed processing base station, audiovisual playback unit, and odor generator are installed locally, the cloud server is installed remotely, and the audiovisual media content storage center is installed locally or remotely.

[0014] The locally installed devices are connected by wireless data transmission, which includes Wi-Fi, Bluetooth®, mobile networks, and radio frequency transmission. [Effects of the Invention]

[0015] The present invention has the following beneficial effects. A cloud server learns, processes, and analyzes a large amount of media content using artificial intelligence, extracts a feature dataset related to odors, and stores it on the cloud server. The distributed processing base station periodically updates a simplified index of the odor feature dataset from the cloud server, and can quickly and efficiently extract a specified number of features and store them on the distributed processing base station. When a game computer streams and downloads video for playback or gaming, the associated audio and / or image signals are first sent to a local base station for analysis. The base station uses a low-cost processor (GPU) built into the base station to process the input audio and / or image signals in a parallel / streaming manner. Since the base station is generally installed locally, that is, connected to the same domain network as devices such as game computers, headphones, and odor generators, the latency / delay of trigger signals for analysis can be reduced. If there are audio and / or image signals that are not present at the base station, the associated audio and / or image signals are sent to the cloud server for co-computation, which extracts a feature dataset of the audio and / or image signals and sends the associated feature dataset to the base station. Simultaneously, the cloud server periodically updates all base stations with the feature dataset obtained through computation. This enables distributed processing, significantly improving processing speed and efficiency, reducing the computational load and latency of local base stations, and achieving high-speed, low-latency speech classification. Furthermore, by utilizing data centers and cloud data centers, large amounts of media content and datasets can be effectively stored and managed, opening up a wide range of potential applications. [Brief explanation of the drawing]

[0016] The present invention will be further described below with reference to the attached drawings. [Figure 1] This is a schematic diagram of the present invention. [Figure 2] This is a flowchart for converting audio signals into olfactory data according to the present invention. [Modes for carrying out the invention]

[0017] Refer to Figure 1. The method for converting audiovisual data into an olfactory experience based on distributed processing includes a cloud server, a distributed processing base station, an audiovisual media content storage center, an audiovisual playback unit, and a scent generator. The cloud server is equipped with high-powered GPU servers and trains and computes an artificial intelligence model that converts audiovisual data into scent data to obtain a scent feature dataset corresponding to images and / or sounds in the audiovisual media content. It runs the same model and algorithms as the base station to train new data and runs and computes more complex models, such as multi-layer DL models (CNN, RNN types), for training the dataset and complex data prediction. The scent feature dataset corresponding to images and / or sounds, executed or computed on the cloud server, is updated at the distributed processing base station.

[0018] The distributed processing base station, audiovisual playback unit, and odor generator are located on the local network. The video playback unit reads the content to be played from the audiovisual media content storage center, and this content includes image data and / or audio data. The image data and audio data are simultaneously transmitted to the video playback unit and the distributed processing base station on the local network. Due to the high transmission speed of the local network, system response can be improved, and network congestion caused by excessive access requests to the cloud server is prevented, reducing the computational load on the cloud server. The distributed processing base station also houses an intelligent analysis model for the odor feature dataset. Therefore, the intelligent analysis model can compute the image data and audio data played by the audiovisual playback unit in real time, obtain the corresponding odor data, and then transmit it to the odor generator to release the olfactory experience, thereby improving the user's on-site experience.

[0019] A method for a distributed processing base station and a cloud server to convert an audio signal in a visual media into olfactory data will be described in detail below. Referring to FIG. 2, the specific procedure is as follows.

[0020] A1. The visual media content storage center transmits the visual media in response to the request of the visual media playback unit, and at the same time transmits the audio signal in the visual media to the distributed processing base station or the cloud server. Usually, the audio signal is transmitted to the distributed processing base station or the cloud server in real time in the form of a file such as.wav or an analog signal.

[0021] A2. The distributed processing base station or the cloud server normalizes the audio signal, including sampling the audio signal and intercepting an audio signal of a specified length.

[0022] A3. Perform a Fourier transform (short-time Fourier transform) on the audio signal to convert the time-domain representation of the audio into a time-frequency domain signal.

[0023] A4. Input the time-frequency domain signal into an intelligent analysis model or an artificial intelligence model. In the intelligent analysis model of the distributed processing base station and the artificial intelligence model of the cloud server, the time-frequency domain signal is input into a convolutional neural network (CNN) for calculation to obtain a feature map (Feature Maps), and then the classifier predicts the category of the feature to obtain the classification data of the audio corresponding to the smell.

[0024] A5. Read the olfactory data based on the above classification data and transmit it to the smell generating device. The smell generating device mixes different smell generating units according to the corresponding ratio through the olfactory data, and obtains and emits the smell corresponding to the olfactory data.

[0025] A method for the base station to convert the image signal of the visual media into olfactory data will be described in detail below. The specific procedure is as follows.

[0026] B1. The audiovisual media content storage center transmits audiovisual media in response to requests from audiovisual playback units, and also transmits image signals within the audiovisual media to distributed processing base stations or cloud servers.

[0027] B2. A distributed processing base station or cloud server preprocesses the image signal to form an image matrix related to the pixel intensity within the image signal.

[0028] B3. The image matrix is ​​input into an intelligent analysis model or artificial intelligence model, such as a convolutional neural network (CNN), and calculations are performed to obtain feature maps. Then, the category of the features is predicted by a classifier to obtain classification data for images corresponding to smells.

[0029] B4. Based on the classification data above, olfactory data is read and transmitted to the odor generator. The odor generator mixes different odor generating units according to the corresponding proportions based on the olfactory data, and acquires and releases the odor corresponding to the olfactory data.

[0030] This invention can be widely used in scenes such as games, movies, and media, and uses artificial intelligence to identify sounds or images within a scene and acquire corresponding scent data. For example, when gunshots or firearms are fired in a movie scene, the scent generator mixes corresponding aromatic compounds based on the gunpowder scent data obtained through AI analysis, generating and releasing a corresponding scent. This allows the user to experience smell in addition to sight and hearing, resulting in a better on-set experience.

[0031] The above are merely preferred embodiments of the present invention, and the present invention is not limited to the embodiments described above. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure, insofar as the technical effects of the present invention are achieved by similar means, are included in the scope of protection of this disclosure and shall all fall within the scope of protection of the present invention. Various modifications and changes can be made to the technical means and / or embodiments of the present invention within the scope of protection.

Claims

1. A method for converting audiovisual data into an olfactory experience based on distributed processing, It includes a cloud server, a distributed processing base station, an audiovisual media content storage center, an audiovisual playback unit, and a scent generator. An artificial intelligence model that converts audiovisual data into odor data is installed on the aforementioned cloud server, and an intelligent analysis model for odor feature datasets is installed on the aforementioned distributed processing base station. The audiovisual media playback unit reads and plays audiovisual media content from the audiovisual media content storage center, and simultaneously transmits the audiovisual media content to a distributed processing base station. The intelligent analysis model of the distributed processing base station analyzes the images and / or sounds within the audiovisual media content, acquires olfactory data corresponding to the images and / or sounds, and transmits it to a scent generator to release the olfactory experience. If the intelligent analysis model cannot obtain corresponding olfactory data by analyzing the odor feature dataset, the relevant audiovisual media content is transferred to a cloud server, the artificial intelligence model on the cloud server extracts odor feature datasets related to images and / or sounds within the audiovisual media content, and updates the intelligent analysis model on the distributed processing base station with the relevant image and / or sound feature datasets. A method for converting audiovisual data into an olfactory experience based on distributed processing.

2. The step of converting audio signals from audiovisual media into olfactory data is, A1. To capture audio signals from audiovisual media, A2. Normalizing the audio signal, A3. Performing a Fourier transform on the audio signal to obtain a time-frequency domain signal, A4. Inputting a time-frequency domain signal into an intelligent analysis model or artificial intelligence model to obtain corresponding speech classification data, A5. Reading olfactory data corresponding to voice classification data, A method for converting audiovisual data into an olfactory experience based on the distributed processing described in claim 1, characterized by including the above.

3. The step of converting image signals from audiovisual media into olfactory data is, B1. Acquiring image signals from audiovisual media, B2. Preprocessing the image signal to form an image matrix related to the pixel intensity within the image signal, B3. Inputting an image matrix into an intelligent analysis model or artificial intelligence model to obtain image classification data, B4. Reading olfactory data corresponding to image classification data, A method for converting audiovisual data into an olfactory experience based on the distributed processing described in claim 1, characterized by including the above.

4. A method for converting audiovisual data into an olfactory experience based on distributed processing according to claim 1, characterized in that the distributed processing base station, audiovisual playback unit, and odor generator are installed locally, the cloud server is installed remotely, and the audiovisual media content storage center is installed locally or remotely.

5. A method for converting audiovisual data into an olfactory experience based on distributed processing according to claim 4, characterized in that the locally installed devices are connected by wireless data transmission, and the wireless data transmission includes Wi-Fi, Bluetooth®, mobile networks, and radio frequency transmission.

Images