Program, information processing method, and information processing device

Abstract

Generating knitting based on sound data [Solution] A program that causes an information processing device to acquire sound data, perform frequency conversion on the sound data, generate image data based on the power spectrum of the sound data after frequency conversion, and generate knitting data that can be read by a knitting machine based on the image data.

Description

[Technical Field]

[0001] The present invention relates to a program, an information processing method, and an information processing device. [Background technology]

[0002] 2. Description of the Related Art Conventionally, devices for visualizing sound have been known. For example, Patent Document 1 discloses a device that converts a mixed sound, which is a mixture of high-pitched and low-pitched sounds, into visible light and displays it. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-232466 Summary of the Invention [Problem to be solved by the invention]

[0004] However, a system for visualizing sound as knitting has not yet been realized.

[0005] The disclosed technology has been made in consideration of the above circumstances, and aims to provide a technology for generating knitted items based on sound data. [Means for solving the problem]

[0006] A program that is one aspect of the disclosed technology causes an information processing device to acquire sound data, perform frequency conversion on the sound data, generate image data based on the power spectrum of the sound data after frequency conversion, and generate knitting data that can be read by a knitting machine based on the image data. [Effects of the Invention]

[0007] According to the present invention, it is possible to generate knitted items based on sound data. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a block diagram illustrating an example of a configuration of a user terminal according to an embodiment. [Figure 3] FIG. 1 is a diagram illustrating an example of frequency conversion processing according to an embodiment; [Figure 4] FIG. 1 is a diagram illustrating an example of image data according to an embodiment. [Figure 5] FIG. 1 is a diagram illustrating an example of a knitted product according to an embodiment. [Figure 6] A flowchart illustrating an example of processing of a user terminal according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. The following embodiments are merely examples for explaining the present invention, and are not intended to limit the present invention to the embodiments. Furthermore, the present invention can be modified in various ways without departing from the spirit of the present invention. Furthermore, the same components in each drawing will be designated by the same reference numerals whenever possible, and redundant explanations will be omitted whenever possible.

[0010] <System Overview> Fig. 1 is a diagram illustrating an example of the configuration of an information processing system 1 according to an embodiment of the present disclosure. As shown in Fig. 1, the information processing system 1 includes a user terminal 10 and a knitting machine 20. The user terminal 10 and the knitting machine 20 can transmit and receive data to and from each other via a network N. Furthermore, the number of user terminals 10 and the number of knitting machines 20 may be any number.

[0011] The information processing system 1 generates a predetermined knitted item based on the sound data. By way of example, but not limitation, the knitted item may include clothes, hats, socks, handkerchiefs, stuffed toys, bags, and the like.

[0012] The user terminal 10 generates knitting data that can be read by the knitting machine 20. The user terminal 10 is, for example, a personal computer, a smartphone, or a tablet terminal. The user terminal 10 generates the knitting data by executing the process described below. The user terminal 10 outputs the generated knitting data to the knitting machine 20 via the network N.

[0013] The knitting machine 20 is a knitting machine that generates a predetermined knitted item based on knitting data. The knitting machine 20 acquires the knitting data from the user terminal 10, for example, via the network N. The knitting machine 20 may also acquire the knitting data from a storage medium, such as a USB (Universal Serial Bus), connected to the knitting machine 20, that stores the knitting data generated by the user terminal 10. The knitting machine 20 generates a knitted item based on the acquired knitting data.

[0014] The network N may be realized by, for example, a network such as the Internet or a mobile phone network, a LAN (Local Area Network), or a combination of these. The following describes in detail each component of the information processing system 1 that enables the present service to be executed.

[0015] <Server configuration> 2 is a block diagram illustrating an example of a user terminal 10 according to one embodiment. The user terminal 10 includes one or more processors (e.g., CPUs) 110, one or more network communication interfaces 120, a storage device (storage unit) 130, and one or more communication buses 150 for interconnecting these components.

[0016] The user terminal 10 may optionally include a user interface 140. The user interface 140 includes a display and / or an input device (such as a keyboard and / or a mouse or some other pointing device).

[0017] The storage device 130 may be, for example, a high-speed random access memory (main storage device) such as a DRAM, an SRAM, or other random access solid-state storage device. Alternatively, the storage device 130 may be a non-volatile memory (auxiliary storage device) such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. Alternatively, the storage device 130 may be a non-transitory computer-readable recording medium that stores programs and the like. Alternatively, the storage device 130 may be either a main storage device (memory) or an auxiliary storage device (storage), or may include both.

[0018] The storage device 130 stores data, programs, etc. used by the information processing system 1. For example, the storage device 130 stores sound data, image data, knitting data, type information of the knitting machine 20, etc., which will be described later.

[0019] Another example of storage device 130 may be one or more storage devices located remotely from processor 110. In some embodiments, storage device 130 stores programs, modules, and data structures, or a subset thereof, that are executed by processor 110.

[0020] The processor 110 executes a program stored in the storage device 130 to control, for example, the processing executed by the information processing unit 111.

[0021] The information processing unit 111 includes, for example, an acquisition unit 112, a conversion unit 113, an identification unit 114, a generation unit 115, an extraction unit 116, and an output unit 117.

[0022] The acquisition unit 112 acquires sound data. For example, the acquisition unit 112 acquires sound data by reading out sound data stored in the storage device 130. The acquisition unit 112 may also acquire sound data from another user terminal (not shown) and / or the Internet via the network N.

[0023] Sound data is data in which a predetermined sound is recorded for a predetermined playback time. Examples of sound data include, but are not limited to, data in which a predetermined song is recorded, data in which the cry of a predetermined living thing is recorded, data in which a predetermined voice is recorded, and data in which a predetermined everyday sound is recorded. The sound data may be in any data format. Examples of sound data formats include MP3, WAV, and WMA (Windows Media Audio).

[0024] The conversion unit 113 performs frequency conversion on the sound data acquired by the acquisition unit 112. For example, the conversion unit 113 applies FFT (Fast Fourier Transform) processing to the sound data to calculate frequency components (for example, a power spectrum).

[0025] The conversion unit 113 may divide the sound data into predetermined time intervals and perform frequency conversion on each divided sound data. For example, if the playback time of the sound data is 3 minutes, the conversion unit 113 divides it into 36 pieces of 5-second sound data. The conversion unit 113 may perform frequency conversion on each of the 36 divided sound data. Note that when the sound data is divided into n pieces of sound data (n is a natural number greater than or equal to 2), the kth divided sound data (k is a natural number greater than or equal to 1 and less than or equal to n) is also referred to as the "kth sound data."

[0026] Fig. 3 is a diagram for explaining an example of frequency conversion processing. In the example shown in Fig. 3, the conversion unit 113 performs frequency conversion on predetermined divided sound data, and generates a graph showing a power spectrum for each frequency band. The conversion unit 113 performs frequency conversion as many times as the number of divided sound data. That is, if there are 36 pieces of divided sound data, 36 graphs such as those shown in Fig. 3 are generated.

[0027] The generating unit 115 generates image data based on the power spectrum of the sound data after frequency conversion. The generating unit 115 generates image data based on, for example, the power spectrum of each of the divided sound data after frequency conversion.

[0028] The generation unit 115 sets one or more predetermined thresholds for the power spectrum as preprocessing for generating image data. Note that the predetermined thresholds may be fixed values ​​or may be values ​​that are changed appropriately depending on the sound data.

[0029] Dashed lines B1 to B3 parallel to the horizontal axis in FIG. 3 represent boundary lines based on thresholds set by the generation unit 115. For example, the generation unit 115 associates a predetermined RGB value (hereinafter also referred to as a "first RGB value") with a frequency band corresponding to a power spectrum located below the boundary line indicated by dashed line B1. For example, the generation unit 115 associates a predetermined RGB value (hereinafter also referred to as a "second RGB value") different from the first RGB value with a frequency band corresponding to a power spectrum located between the boundary line indicated by dashed line B1 and the boundary line indicated by dashed line B2. For example, the generation unit 115 associates a predetermined RGB value (hereinafter also referred to as a "third RGB value") different from the first RGB value and the second RGB value with a frequency band corresponding to a power spectrum located between the boundary line indicated by dashed line B2 and the boundary line indicated by dashed line B3. For example, the generation unit 115 associates a predetermined RGB value different from the first RGB value, the second RGB value, and the third RGB value with a frequency band corresponding to a power spectrum located above the boundary line indicated by dashed line B3. The number of threshold values ​​may be set according to the number of colors that can be knitted by the knitting machine 20.

[0030] The generation unit 115 generates image data including pixels of RGB values ​​associated with each frequency band of each of the divided sound data.

[0031] FIG. 4 is a diagram showing an example of image data according to an embodiment. In the example shown in FIG. 4, the horizontal axis represents each RGB value represented by the power spectrum of the kth sound data (see, for example, FIG. 3), the vertical axis represents the time axis, and the RGB values ​​of the kth sound data are arranged in order from the first sound data downward. For example, pixels with RGB values ​​associated with each frequency band of the first sound data are arranged horizontally from the upper left pixel to the right edge of the image data. Similarly, pixels adjacent to the pixel in the horizontal row based on the first sound data in the row below and having RGB values ​​associated with each frequency band of the second sound data are arranged horizontally from the left edge to the right edge of the image data. That is, in the example shown in FIG. 4, pixels adjacent to the pixel in the horizontal row based on the kth sound data in the image data in the row below and having RGB values ​​associated with each frequency band of the k+1th sound data are arranged horizontally from the left edge to the right edge of the image data.

[0032] The generation unit 115 may generate image data by applying the power spectrum of each frequency band of the divided sound data after frequency conversion to parameters of the coordinates of a 3D mesh. For example, the generation unit 115 divides the frequency bands of each divided sound data after frequency conversion into predetermined values ​​(hereinafter, described as 512). The generation unit 115 applies the power spectrum for each of the 512 frequency bands, i.e., the 512 power spectra of each sound data, to parameters of the coordinates of each vertex of the 512 3D meshes. The coordinates of each vertex of the 3D mesh correspond to each coordinate value in the color coordinate space. The generation unit 115 generates an image capturing mesh fluctuations according to the applied power spectrum corresponding to the coordinates of each vertex of the 3D mesh, thereby generating image data in which the color of each vertex of the 3D mesh changes over time. The generation unit 115 generates image data using a known slit scan for the image data.

[0033] Furthermore, based on the image data generated by the generation unit 115, knitting data that can be read by the knitting machine is generated. For example, based on the generated image data, the generation unit 115 generates knitting data in which each pixel included in the image data corresponds to a knitted portion of a knitted item. As an example, if the knitting machine 20 is an existing electronic knitting machine, the generation unit 115 may generate the knitting data from the image data using software compatible with this electronic knitting machine.

[0034] As described above, the user terminal 10 can generate knitting data that can be read by the knitting machine 20 based on the sound data. This allows the knitting machine 20 to generate a knitted item in which the sound data is visualized based on the knitting data. As a result, for example, a user using the user terminal 10 can obtain a knitted item in which the sound of a favorite song or the like is visualized.

[0035] The output unit 117 outputs the knitting data to the knitting machine 20. The output unit 117 controls the network communication interface 120, for example, to output the knitting data to the knitting machine 20.

[0036] The identifying unit 114 identifies a predetermined portion of the sound data acquired by the acquiring unit 112 based on information input by the user or characteristics of the sound.

[0037] The identification unit 114 identifies a predetermined portion of the sound data acquired by the acquisition unit 112, for example, based on information input by the user using the user terminal 10. More specifically, the user operates the user terminal 10 to specify a predetermined portion of the sound data. For example, the user inputs "1:25 to 2:10" into an input field displayed on the screen of the user terminal 10. In this case, the identification unit 114 identifies the sound data from 1 minute 25 seconds to 2 minutes 10 seconds from the sound data acquired by the acquisition unit 112.

[0038] The identification unit 114 identifies a predetermined portion of the sound data acquired by the acquisition unit 112 based on at least one sound characteristic such as loudness, pitch, and timbre. For example, the identification unit 114 may score at least one of loudness, pitch, and timbre included in the sound data by the acquisition unit 112 in chronological order from the first sound of the sound data. The identification unit 114 may identify a portion of the sound data acquired by the acquisition unit 112 where the absolute value of the difference between the score at a predetermined time point and the score at a time point immediately before the predetermined time point is greater than a predetermined threshold.

[0039] The extraction unit 116 extracts sound data corresponding to the predetermined portion identified by the identification unit 114 .

[0040] The conversion unit 113 may perform frequency conversion on the sound data extracted by the extraction unit 116. For example, the conversion unit 113 applies FFT (Fast Fourier Transform) processing to the sound data extracted by the extraction unit 116 to calculate frequency components (for example, a power spectrum).

[0041] As described above, the user terminal 10 can generate knitting data based on a predetermined portion of the sound data (for example, the chorus portion if the sound data is a song). This allows the knitting machine 20 to generate a knitted item in which the predetermined portion is visualized based on the knitting data. As a result, for example, a user using the user terminal 10 can obtain a knitted item in which the chorus portion of a favorite song is visualized.

[0042] The identifying unit 114 may identify a frequency band in which a power spectrum that satisfies a predetermined condition for feature extraction is present, based on the power spectrum.

[0043] The identifying unit 114 identifies, for example, for each of the divided sound data (the first sound data to the n-th sound data), a frequency band that satisfies the condition that there is a power spectrum equal to or greater than a predetermined threshold.

[0044] The generating unit 115 may generate the image data based on the power spectrum corresponding to the frequency band identified by the identifying unit 114.

[0045] As described above, the user terminal 10 can generate knitting data based on characteristic parts of sound data. This allows the knitting machine 20 to generate a knitted item in which the characteristic parts are visualized based on the knitting data. As a result, for example, a user using the user terminal 10 can obtain a knitted item in which the characteristic parts of a favorite song or the like are visualized.

[0046] The specifying unit 114 may specify the number of colors based on the type of knitting machine 20 into which the knitting data is read.

[0047] Generally, the number of usable yarns may differ depending on the type of knitting machine. For example, the number of usable yarns may differ between a domestic knitting machine and an industrial knitting machine. That is, the number of colors included in a knitted item may differ depending on the number of usable yarns of the knitting machine.

[0048] The identification unit 114, for example, identifies the number of yarns usable by the knitting machine 20. In this case, the acquisition unit 112 may acquire type information relating to the type of the knitting machine 20 from the knitting machine 20. The identification unit 114 may identify the number of yarns usable by the knitting machine 20 based on the type information acquired by acquisition 112, using association information that associates type information of the knitting machine 20 that has already been registered with the number of usable yarns.

[0049] The generating unit 115 may generate image data according to the number of colors identified by the identifying unit 114. For example, as preprocessing for generating image data, the generating unit 115 sets one or more predetermined thresholds for the power spectrum according to the number of colors identified by the identifying unit 114, and executes processing to generate image data.

[0050] As described above, the user terminal 10 can generate knitting data including the number of colors according to the type of knitting machine 20. This enables the knitting machine 20 to generate a knitted item including the number of colors according to the type of knitting machine 20 based on the knitting data.

[0051] The identification unit 114 may identify the fineness of knitting of the knitted item based on the type of knitting machine 20 that reads the knitting data. The identification unit 114 identifies the fineness of knitting of the knitted item based on the type information acquired by acquisition 112, for example.

[0052] Generally, even when knitting the same knitted item, the number of knitted portions may differ depending on the type of knitting machine. For example, even when knitting the same knitted item between a domestic knitting machine and an industrial knitting machine, the fineness of the knitted portion may differ. In other words, the user terminal 10 needs to generate image data with a pixel count according to the knitted portion. Therefore, the identification unit 114 identifies the pixel size, etc. of the knitted portion of the knitted item based on the type information.

[0053] The generating unit 115 may generate image data according to the fineness specified by the specifying unit 114. The generating unit 115 generates image data with a number of pixels based on the number of braided portions, for example.

[0054] As described above, the user terminal 10 can generate knitting data according to the fineness of knitting of the knitted item based on the type of knitting machine 20. This enables the knitting machine 20 to generate a knitted item with a fineness of knitting according to the type of knitting machine 20 based on the knitting data.

[0055] <Knitting example> Fig. 5 is a diagram showing an example of a knitted product generated based on knitting data by the knitting machine 20 according to an embodiment. In the example shown in Fig. 5, a knitted product in which sound data is visualized is generated based on the knitting data by the knitting machine 20. The example shown in Fig. 5 is an example of a knitted product generated based on image data generated by applying the above-described method using 3D meshes to sound data related to the crowds at the east exit of Shinjuku Station.

[0056] <Example of operation> An example of the operation of the user terminal 10 according to one embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the flow of processing by the user terminal 10.

[0057] In step S11, the acquisition unit 112 acquires sound data. For example, the acquisition unit 112 acquires sound data by reading out sound data stored in the storage device 130. The acquisition unit 112 may also acquire sound data from another user terminal (not shown) and / or the Internet via the network N.

[0058] In step S12, the conversion unit 113 performs frequency conversion on the sound data acquired by the acquisition unit 112. The conversion unit 113, for example, applies FFT (Fast Fourier Transform) processing to the sound data to calculate frequency components (for example, a power spectrum).

[0059] In step S13, the generation unit 115 generates image data based on the power spectrum of the sound data after frequency conversion. For example, the generation unit 115 generates image data based on the power spectrum of each of the divided sound data after frequency conversion.

[0060] In step S14, the generation unit 115 generates, based on the image data, knitting data that can be read by the knitting machine 20. For example, based on the generated image data, the generation unit 115 generates knitting data in which each pixel included in the image data corresponds to a knitted portion of the knitted item.

[0061] In step S15, the output unit 117 outputs the knitting data. The output unit 117 outputs the knitting data to the knitting machine 20. The output unit 117 controls the network communication interface 120, for example, to output the knitting data to the knitting machine 20.

[0062] The above-described embodiments are merely examples for explaining the present invention, and are not intended to limit the present invention to these embodiments. Furthermore, the present invention can be modified in various ways without departing from the spirit of the invention. Furthermore, those skilled in the art can adopt embodiments in which the above-described elements are replaced with equivalents, and such embodiments are also within the scope of the present invention. [Explanation of symbols]

[0063] 1...information processing system, 10...user terminal, 20...knitting machine, 110...processor, 111...information processing unit, 112...acquisition unit, 113...conversion unit, 114...identification unit, 115...generation unit, 116...extraction unit, 117...output unit, 120...network communication interface, 130...storage device, 140...user interface, 150...communication bus, B1-3...dashed lines, N...network

Claims

1. In the information processing device, Acquiring sound data; performing a frequency conversion on the sound data; generating image data based on a power spectrum of the sound data after frequency conversion; generating knitting data that can be read by a knitting machine based on the image data; A program that executes the following.

2. identifying a predetermined portion of the acquired sound data based on user-entered information or sound characteristics; extracting the sound data corresponding to the identified predetermined portion; performing the frequency conversion includes performing a frequency conversion on the extracted sound data; The program according to claim 1.

3. further performing, based on the power spectrum, identifying a frequency band in which a power spectrum that satisfies a predetermined condition for feature extraction exists; generating the image data includes generating image data based on the power spectrum corresponding to the identified frequency band; The program according to claim 1.

4. further specifying the number of colors based on the type of knitting machine into which the knitting data is read; The generating step comprises: generating the image data according to the number of the identified colors; The program according to claim 1 , comprising:

5. Identifying the fineness of knitting of the knitted product based on the type of knitting machine that reads the knitting data; The generating step comprises: generating the image data according to the specified fineness; The program according to claim 1 , comprising:

6. An information processing method executed by an information processing device, Acquiring sound data; performing a frequency conversion on the sound data; generating image data based on a power spectrum of the sound data after frequency conversion; generating knitting data that can be read by a knitting machine based on the image data; An information processing method that performs the above.

7. Acquiring sound data; performing a frequency conversion on the sound data; generating image data based on a power spectrum of the sound data after frequency conversion; generating knitting data that can be read by a knitting machine based on the image data; An information processing device that executes the above.

Images