Control device, control method, and computer program product

Abstract

The invention provides a control device, a control method, and a computer program product. The present invention controls a vehicle function in conjunction with the playback of an arbitrary content. The control device (40) is provided with a control unit (43) for playing content and controlling vehicle functions in conjunction with the content. The control unit (43) is configured to generate control data for controlling the vehicle function on the basis of the feature amount of the content, to play the content in response to a play request for the content, and to control the vehicle function in conjunction with the play of the content on the basis of the control data when playing the content.

Description

Technical Field

[0001] This invention relates to control devices, control methods, and computer program products. Background Technology

[0002] Patent document 1 discloses a system that autonomously controls the operation of in-vehicle devices such as car audio systems and lighting devices based on the detection results of the user's status.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2006-282115

[0004] However, if the user's state is not in a pre-determined, limited state, the aforementioned system cannot activate the in-vehicle device. For example, if one wants to control vehicle functions and activate in-vehicle devices in conjunction with the playback of music, animation, or other content, the vehicle's marketability might be reduced if the device only activates when specific content is played. Therefore, it is desirable to control vehicle functions and activate in-vehicle devices in conjunction with the playback of a wide variety of arbitrary content. Summary of the Invention

[0005] This invention was developed with the aim of addressing such problems, and its purpose is to enable control of vehicle functions in conjunction with the playback of a wide variety of arbitrary content.

[0006] To address the aforementioned issues, a control device according to a certain embodiment of the present invention includes a control unit that plays content and controls vehicle functions in conjunction with the content. The control unit is configured to: generate control data for controlling vehicle functions based on feature values ​​of the content; play the content in response to a playback request; and, while playing the content, control vehicle functions in conjunction with the playback of the content based on the control data.

[0007] In addition, in a certain aspect of the control method of the present invention, the following are performed: playing content and controlling vehicle functions in conjunction with the content; generating control data for controlling vehicle functions based on feature values ​​of the content; playing the content in response to a playback request; and controlling vehicle functions in conjunction with the playback of the content based on the control data while the content is being played.

[0008] In addition, in one aspect of the present invention, the computer program causes the computer to perform the following processes: playing content and controlling vehicle functions in conjunction with the content; generating control data for controlling vehicle functions based on the feature values ​​of the content; playing the content in response to a playback request; and controlling vehicle functions in conjunction with the playback of the content based on the control data while the content is being played.

[0009] According to the above-described method of the present invention, since control data for controlling vehicle functions can be automatically generated based on the feature quantity of the content, it is possible to link with the playback of any content and control vehicle functions based on the generated control data. Attached Figure Description

[0010] Figure 1 This is a simplified structural diagram of the control system in the first embodiment of the present invention.

[0011] Figure 2 This diagram illustrates a method for parsing content sequentially and simultaneously generating control data for a content-linking device, and for causing the content-linking device to operate in conjunction with the playback of the content after a predetermined time has elapsed since the start of parsing.

[0012] Figure 3 This is a flowchart illustrating an example of content linkage processing in the second embodiment of the present invention.

[0013] Explanation of reference numerals in the attached figures:

[0014] 1…control system; 20…content linkage device; 40…control device. Detailed Implementation

[0015] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Furthermore, in the following description, the same constituent elements will be labeled with the same reference numerals as in the accompanying drawings.

[0016] (First Embodiment)

[0017] Figure 1 This is a simplified structural diagram of the control system 1 in the first embodiment of the present invention.

[0018] The control system 1 is configured to include a content playback device 10, a content linkage device 20, a wireless communication device 30, and a control device 40. When content is played through the content playback device 10, the content linkage device 20 can operate in conjunction with the content. In this embodiment, the content playback device 10, the content linkage device 20, the wireless communication device 30, and the control device 40 are mounted in a vehicle and are connected to communicate via an in-vehicle network 9 that follows a standard such as a controller area network.

[0019] The content playback device 10 is an in-vehicle device equipped with an input device 11 for input operations by vehicle passengers (hereinafter referred to as "users"), and capable of playing music, animation, and other content selected by the user through operations on the input device 11. The content playback device 10 may be, for example, a smart screen interconnection system (display audio) capable of playing both music and animation content selected by the user, and the input device 11 may be, for example, a touch panel or microphone of the smart screen interconnection system.

[0020] In addition to this smart screen interconnection system, other examples of content playback devices 10 include car audio systems that can play music content selected by the user.

[0021] The content linkage device 20 is an in-vehicle device that operates in conjunction with the content during playback. The operation of the content linkage device 20 is controlled by the control device 40. In this embodiment, the content linkage device 20 includes a lighting lamp 21 installed inside the vehicle. The lighting lamp 21 is configured to include a full-color LED obtained by sealing red, green, and blue LED chips with a light-transmitting resin, and the emission color of the full-color LED can be changed to any color by controlling each LED chip.

[0022] Furthermore, examples of content-linking devices 20, besides the lighting 21, include seats, air conditioners, and dimmable glass. When the content-linking device 20 is a seat, it can, for example, move or vibrate the seat in conjunction with the content. When the content-linking device 20 is an air conditioner, it can, for example, supply air or change the air supply intensity in conjunction with the content. When the content-linking device 20 is dimmable glass installed on doors, roofs, etc., it can, for example, change the color and transparency of the dimmable glass in conjunction with the content.

[0023] The wireless communication device 30 is a vehicle-mounted device that performs wireless communication processing in accordance with prescribed wireless communication standards. The wireless communication device 30 connects to an external server by accessing a wireless base station connected to an external server (not shown) via a gateway or the like.

[0024] The control device 40 is an ECU (Electronic Control Unit) that includes a communication unit 41, a storage unit 42, and a control unit 43.

[0025] The communication unit 41 has a communication interface circuit for connecting the control device 40 to the in-vehicle network. The communication unit 41 supplies received data to the control unit 43. In addition, the communication unit 41 outputs data supplied from the control unit 43 to the outside.

[0026] The storage unit 42 has storage media such as HDD (Hard Disk Drive), SSD (Solid Disk Drive), and semiconductor memory to store various computer programs and data used for processing in the control unit 43.

[0027] The control unit 43 has one or more CPUs (Central Processing Units) and their peripheral circuitry, and executes various computer programs stored in the storage unit 42. The control unit 43 is, for example, a processor. By executing processing according to the computer program, the control unit 43 functions as a content acquisition unit 51, a feature extraction unit 52, a control data generation unit 53, a content playback control unit 54, and a content linkage control unit 55, operating as a functional unit (module) that implements the specified functions. In the following description, when describing the processing with each functional unit 51-55 as the subject, the execution of the program implementing each functional unit 51-55 by the control unit 43 is shown.

[0028] The content acquisition unit 51 stores content acquired from external sources in the storage unit 42. For example, the content acquisition unit 51 can automatically acquire content distributed from an external server via the wireless communication device 30, or in response to an acquisition request from a user who has operated the input device 11 of the content playback device 10. Additionally, the content acquisition unit 51 can, for example, acquire content stored in a user's terminal (e.g., a smartphone) that is connected to the content playback device 10 via a wired or wireless connection.

[0029] The feature extraction unit 52 extracts the feature quantities of the content obtained by the content acquisition unit 51.

[0030] When the content is music, the volume, pitch, and other parameters of the music change over time. Therefore, when the content is music, the temporal data of at least one of the volume and pitch (frequency) can be extracted as feature quantities of the music content.

[0031] Furthermore, when the content is animated, the hue, saturation, and other properties of the image displayed on the monitor change over time. Therefore, when the content is animated, at least one of the temporal data of the average color of the entire image or a specific part (e.g., the central region of the image) and the temporal data of the hue of the part with the highest saturation on the image can be extracted as a feature of the animated content. Additionally, since sound in animated content also changes over time, the temporal data of the sound's volume and pitch can also be extracted as a feature of the animated content.

[0032] The extraction of features for each content can be performed, for example, by pre-learning a machine learning model that takes each content as input and extracts its features in a way that allows for the extraction of features for each content.

[0033] The control data generation unit 53 generates timing data (hereinafter referred to as "control data of the content linkage device 20") based on the feature values ​​of the content, which are used to control the content linkage device 20 when the content is played. In the case that the content linkage device 20 is a lighting lamp 21, for example, the number of lit full-color LEDs and the color of the full-color LEDs can be used as control values ​​of the lighting lamp 21.

[0034] For example, when the content is music and the content-linking device 20 is a lighting lamp 21, the control data generation unit 53 can generate timing data for the number of lit full-color LEDs based on the timing data of the music content's volume. Furthermore, when the content is animation and the content-linking device 20 is a lighting lamp 21, the control data generation unit 53 can generate timing data for the emission color of the full-color LEDs based on the timing data of the average color of the central region of the animation content's image.

[0035] When the content playback control unit 54 receives a playback request for content from a user who has operated the input device 11 of the content playback device 10, it controls the content playback device 10 to play the content.

[0036] When the content linkage control unit 55 receives a playback request for content from a user who has operated the input device 11 of the content playback device 10, it controls the content linkage device 20 in a way that is synchronized with the playback of the content based on the timing data of the control quantity of the content linkage device 20 involved in the content.

[0037] The control device 40 of the control system 1 described above includes a control unit 43 that plays content and controls the content linkage device 20 (vehicle function) in conjunction with the content. The control unit 43 is configured to play content in response to a content playback request, generate control data for controlling the content linkage device 20 based on the characteristics of the content, and control the content linkage device 20 in conjunction with the content playback based on the control data when the content is played.

[0038] Typically, in order for the content-linked device 20 to operate in conjunction with the playback of content, the developer needs to pre-create control data for the content-linked device 20, so that the content-linked device 20 operates according to the pre-created control data. Therefore, in the past, the content-linked device 20 could only operate when playing limited content for which the developer had pre-created control data.

[0039] In contrast, according to this embodiment, since the control unit 43 can automatically generate control data for the content linkage device 20 related to the content based on the feature values ​​of the content, the content linkage device 20 can be operated in conjunction with the playback of a variety of arbitrary content selected by the user. Furthermore, since it is not necessary to pre-generate large amounts of control data for the content linkage device 20 related to each piece of content and, for example, store them in the storage unit 42, the time spent generating control data can be reduced, and the insufficient storage capacity of the storage unit 42 and the need for further expansion can be suppressed.

[0040] Furthermore, in this embodiment, the control unit 43 of the control device 40 of the control system 1 is configured to acquire content from the outside and extract the feature values ​​of the acquired content.

[0041] Therefore, even without pre-prepared content features, the content linkage device 20 can be activated in conjunction with the playback of the content simply by obtaining the content from the outside.

[0042] (Second Implementation)

[0043] Next, the second embodiment of the present invention will be described. This embodiment differs from the first embodiment in that it extracts feature values ​​of the content while playing the content and generates timing data for the control values ​​of the content linkage device 20 based on the extracted feature values. The following description will focus on this difference.

[0044] For example, in cases where a playback request for the content is made immediately after the content acquisition unit 51 acquires the content, sometimes the feature data of the content is extracted after the content is acquired, and the playback request for the content is made before the generation of the control data of the content linkage device 20 is completely finished.

[0045] In this situation, if the content-linking device 20 needs to wait for the generation of its control data before starting content playback in order to activate it in conjunction with content playback, it may take too long. On the other hand, if the control data for the content-linking device 20 is not generated in advance, but the content-linking device 20 is activated in real time based on factors such as the volume of the content while it is playing, then the volume needs to be detected, and the content-linking device 20 needs to be activated based on the detection result. Therefore, asynchrony may occur due to communication delays, making it impossible to synchronize the operation of the content-linking device 20 with the content playback.

[0046] Therefore, in this embodiment, as Figure 2As shown, the content is analyzed sequentially according to a time interval ΔT [s]. After the analysis of the content of the first cycle (the content of the first cycle) is completed, the playback of the content begins with a delay of one cycle, and the content linkage device 20 is activated in conjunction with the playback of the content.

[0047] also, Figure 2 (A) in the diagram represents the timing data of the content’s characteristic quantities (e.g., volume) when the content is played. Figure 2 (B) in the text indicates that it is based on Figure 2 The graph of the time series data of the control quantity determined by the characteristic quantity of (A) in the content. Figure 2 (C) in the figure represents the situation where, when the content linkage device 20 is activated in conjunction with the playback of the content from time t2, there is no asynchrony, but the control quantity of the content linkage device 20 changes synchronously with the characteristic quantity of the content.

[0048] like Figure 2 As shown in (A) and (B) in this embodiment, during the period from time t1 to time t2, feature quantities of the quantity content of the first cycle are extracted, and control data for the quantity content linkage device 20 of the first cycle is generated based on the extracted feature quantities. Then, as... Figure 2 As shown in (C), playback of the content begins at time t2, which is delayed by one cycle, and the content linkage device 20 is activated based on the control data of the content linkage device 20 based on the amount of the first cycle.

[0049] Moreover, such as Figure 2 As shown in (A) and (B), during the playback of the content of the first cycle from time t2 to time t3, feature quantities of the content of the next cycle (the content of the second cycle) are extracted, and control data for the content linkage device 20 of the second cycle is generated based on the extracted feature quantities. Thus, as... Figure 2 As shown in (C), the content can be played continuously from time t3 after the content of the first cycle, so that the content linkage device 20 can be operated in conjunction with the playback of the content.

[0050] Similarly, such as Figure 2 As shown in (A) and (B), during the playback of the content of the second cycle from time t3 to time t4, the analysis of the content of the next cycle (the content of the third cycle) is performed. Therefore, as... Figure 2 As shown in (C), the content of the third cycle can be played continuously from time t4, and the content linkage device 20 is activated in conjunction with the playback of the content.

[0051] By repeatedly performing this process, according to this embodiment, even if the feature quantity of the content and the control data of the content linkage device 20 are not present, the content linkage device 20 can still be operated in conjunction with the playback of the content with a delay of only one cycle while generating the control data of the content linkage device 20.

[0052] Figure 3 This is a flowchart illustrating an example of content linkage processing in this embodiment.

[0053] In step S1, the control device 40 determines whether a playback request for content has been received. If the control device 40 receives a playback request, it proceeds to step S2. Conversely, if the control device 40 does not receive a playback request, it terminates the current processing.

[0054] In step S2, the control device 40 determines whether control data for the content linkage device 20 involved in the content for which a playback request has been made has been generated and exists. If control data for the content linkage device 20 involved in the content for which a playback request has been made exists, the control device 40 proceeds to step S3. On the other hand, if control data for the content linkage device 20 involved in the content for which a playback request has been made does not exist, the control device 40 proceeds to step S4.

[0055] In step S3, the control device 40 plays the content and, based on the control data of the existing content linkage device 20, causes the content linkage device 20 to operate in conjunction with the playback of the content.

[0056] In step S4, the control device 40 begins to analyze the content sequentially in units of a predetermined periodic time ΔT [s]. Specifically, the control device 40 begins to analyze the content sequentially in units of a predetermined periodic time ΔT to extract the feature quantities of the content, and generates control data for the content linkage device 20 involved in the content based on the sequentially extracted feature quantities.

[0057] In step S5, the control device 40 determines whether the analysis of the quantity content of the first cycle has been completed. That is, it determines whether the extraction of feature quantities of the first cycle's content and the generation of control data for the content linkage device 20 related to the first cycle's content have been completed. If the analysis of the quantity content of the first cycle has been completed, the control device 40 proceeds to step S6. On the other hand, if the analysis of the quantity content of the first cycle has not been completed, the control device 40 opens a predetermined interval and determines again whether the analysis of the quantity content of the first cycle has been completed.

[0058] In step S6, the control device 40 starts playing the content with a delay of one cycle, and, based on the control data of the content linkage device 20 generated sequentially in step S4 before the playback of the content, causes the content linkage device 20 to operate in conjunction with the playback of the content.

[0059] Furthermore, in this embodiment, after the analysis of the initial amount of content for one cycle is completed in step S5, the playback of the content begins with a delay of one cycle in step S6. However, this is not limited to this. For example, the playback of the content may begin with a delay of several cycles after the analysis of the initial amount of content for several cycles is completed.

[0060] The control unit 43 of the control device 40 of the control system 1 described above is configured such that, when playing content in response to a content playback request, if the control data of the content linkage device 20 involved in the content is not available, the content is analyzed sequentially in a predetermined periodic time component, and the control data of the content linkage device 20 involved in the content is generated sequentially. After generating at least the control data of the content linkage device 20 for the first periodic time, the playback of the content begins, and the content linkage device 20 (vehicle function) is controlled in conjunction with the playback of the content based on the control data of the content linkage device 20 generated sequentially before the playback of the content.

[0061] Therefore, the content linkage device 20 can be activated in conjunction with content playback at the point when a portion of the control data for the content linkage device 20 has been generated. Thus, the content linkage device 20 can be activated in conjunction with content playback within a short time after content acquisition. Furthermore, since the content linkage device 20 can be activated based on the generated control data, situations where the operation of the content linkage device 20 is out of sync with content playback can be suppressed.

[0062] The embodiments of the present invention have been described above. However, the above embodiments are only a part of the application examples of the present invention, and are not intended to limit the technical scope of the present invention to the specific structure of the above embodiments.

[0063] For example, in the above embodiment, the content acquisition unit 51 acquires content from an external server or the like. However, if it is possible to acquire the content's features and content together from an external server or the like, then the content's features and content together can also be acquired from an external server or the like. That is, the control unit 43 can also be configured to acquire the content and its features from an external source.

[0064] Additionally, for example, in the above embodiments, the computer program executed by the control device 40 may be provided in the form of a computer-readable portable recording medium such as a semiconductor memory, magnetic recording medium or optical recording medium, or it may be provided as a computer program article.

Claims

1. A control device comprising a control unit that plays content and controls vehicle functions in conjunction with the content, wherein... The control unit is configured as follows: Content-based feature values ​​are used to generate control data for controlling the vehicle functions. In response to a playback request for the content, the content is played, and When the content is played, the vehicle functions are controlled in conjunction with the playback of the content based on the control data.

2. The control device according to claim 1, wherein, The control unit is further configured as follows: When playing the content in response to a playback request, if the control data related to the content is not available, the content is analyzed sequentially in a time sequence of a predetermined period of time to generate the control data related to the content. After generating at least the control data related to the content for the first period of time, the playback of the content begins, and the vehicle functions are controlled in conjunction with the playback of the content based on the control data generated sequentially before the playback of the content.

3. The control device according to claim 1 or 2, wherein, The control unit is configured to acquire the content from the outside and extract the feature values ​​of the acquired content.

4. The control device according to claim 3, wherein, The control unit is configured to extract at least one of the timing data of the volume and pitch of the music content as a feature quantity of the music content when the content is music content.

5. The control device according to claim 3, wherein, The control unit is further configured to extract at least one of the following when the content is animation content: the time-series data of the average color of the whole or a specific part of the image of the animation content, the time-series data of the hue of the part with the highest saturation on the image of the animation content, and the time-series data of the volume and pitch of the sound of the animation content as feature quantities of the animation content.

6. The control device according to claim 1 or 2, wherein, The control unit is further configured to acquire the content and the feature quantity of the content from the outside.

7. The control device according to claim 6, wherein, When the content is music, the feature quantities of the content include at least one of the timing data of the volume and pitch of the music content. When the content is animated content, the feature quantities of the content include at least one of the following: time-series data of the average color of the whole or a specific part of the image of the animated content, time-series data of the hue of the part with the highest saturation on the image of the animated content, and time-series data of the volume and pitch of the sound of the animated content.

8. A control method, executed by a control device, wherein the control device plays content and controls vehicle functions in conjunction with the content, wherein... Execution in the control method: Content-based feature generation is used to generate control data for controlling the vehicle functions. The content is played in response to a playback request. as well as When the content is played, the vehicle functions are controlled in conjunction with the playback of the content based on the control data.

9. A computer program article, wherein, The computer program article includes a computer program that causes a computer to perform the following processes: Play content and control vehicle functions in conjunction with the content; Content-based feature generation is used to generate control data for controlling the vehicle functions. The content is played in response to a playback request. as well as When the content is played, the vehicle functions are controlled in conjunction with the playback of the content based on the control data.

Images