Signal processing device, signal processing method, recording medium, and signal processing system

Abstract

The present technology relates to a signal processing device, a signal processing method, a recording medium, and a signal processing system which make it possible to appropriately recognize operation using gesture. The signal processing device according to the present technology comprises a setting unit that sets a method of recognition of operation using gesture on the basis of first vibration data indicating a vibration predicted to occur in the future, and an operation recognition unit that recognizes the operation indicated by a gesture performed by a user in accordance with the method of recognition set by the setting unit. The operation recognition unit recognizes the operation indicated by a gesture performed by a user in a vehicle equipped with the signal processing device. The present technology is applicable, for example, to a signal processing device that controls a display device provided in a vehicle.

Description

[Technical Field]

[0001] The present technology relates to a signal processing device, a signal processing method, a recording medium, and a signal processing system, and in particular to a signal processing device, a signal processing method, a recording medium, and a signal processing system that are capable of appropriately recognizing operations using gestures. [Background technology]

[0002] When autonomous driving becomes a reality, occupants will no longer need to drive and will be able to spend their time freely in the vehicle. During this free time, for example, it is conceivable that occupants will use a display device to watch video content for long periods of time. When watching video content from a position some distance from the display device, it is expected that the display device will be operated using gestures rather than touching physical buttons.

[0003] For example, when operating a digital device by pointing a finger at a UI (User Interface) display, vehicle vibrations can cause the UI display or finger to shake, making it difficult to operate the device properly.

[0004] In response to this, Patent Document 1 describes a technique in which vibrations of a vehicle are detected by an acceleration sensor, and the size of a reaction area for touch operations is changed based on the detected vibrations. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-126556 Summary of the Invention [Problem to be solved by the invention]

[0006] In the technology described in Patent Document 1, the detected vibration status is fed back to the UI. However, when the technology of Patent Document 1 is applied to operations performed inside a vehicle, the vibration status is fed back to the UI after the vibration occurs, which causes a delay before the vibration status is reflected in the UI, and does not necessarily lead to an effective improvement.

[0007] The present technology has been made in view of such circumstances, and makes it possible to appropriately recognize operations using gestures. [Means for solving the problem]

[0008] A signal processing device according to a first aspect of the present technology includes: a setting unit that sets a recognition method for an operation using a gesture based on first vibration data that indicates vibrations that are predicted to occur in the future; and an operation recognition unit that recognizes the operation indicated by the gesture performed by a user in accordance with the recognition method set by the setting unit.

[0009] A signal processing system according to a second aspect of the present technology includes a signal processing device including: a setting unit that sets a recognition method for an operation using a gesture based on vibration data indicating vibrations that are predicted to occur in the future; and an operation recognition unit that recognizes the operation indicated by the gesture performed by a user in accordance with the recognition method set by the setting unit; and a display device that includes a display unit that displays an image that is the target of the operation using the gesture.

[0010] In a first aspect of the present technology, a recognition method for an operation using a gesture is set based on first vibration data indicating vibrations predicted to occur in the future, and the operation indicated by the gesture performed by the user is recognized according to the set recognition method.

[0011] In a second aspect of the present technology, a method for recognizing an operation using a gesture is set based on vibration data indicating vibrations predicted to occur in the future, and the operation indicated by the gesture performed by the user is recognized according to the set recognition method, and an image that is the target of the operation using the gesture is displayed. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a vibration data management system according to an embodiment of the present technology. [Figure 2] FIG. 2 is a diagram showing a data flow in a vibration data management system. [Figure 3] FIG. 10 is a diagram showing an operation using gestures by an occupant of a leading vehicle. [Figure 4] 10A and 10B are diagrams showing operations using gestures by occupants of a target vehicle and a following vehicle. [Figure 5] FIG. 2 is a block diagram illustrating an example of the configuration of a server. [Figure 6] FIG. 2 is a block diagram showing an example of the configuration of a leading vehicle. [Figure 7] FIG. 2 is a block diagram showing an example of the configuration of a target vehicle. [Figure 8] 4 is a block diagram showing a detailed configuration example of a vibration pattern analysis unit. FIG. [Figure 9] 10A and 10B are diagrams illustrating examples of vibration pattern classification and UI correction data. [Figure 10] 10A and 10B are diagrams illustrating an example of correcting a method for recognizing an operation. [Figure 11] 10A and 10B are diagrams illustrating another example of correcting the operation recognition method. [Figure 12] 10 is a block diagram showing a detailed configuration example of a UI display control unit. FIG. [Figure 13] 10 is a flowchart illustrating processing performed by a leading vehicle and a server. [Figure 14] 10 is a flowchart illustrating processing performed by a target vehicle and a server. [Figure 15] 10 is a flowchart illustrating a UI correction data generation process. [Figure 16] 10 is a flowchart illustrating an operation determination process. [Figure 17] FIG. 10 is a block diagram showing another example configuration of the UI display control unit. [Figure 18] FIG. 10 is a block diagram showing another example of the configuration of the vibration pattern analysis unit. [Figure 19] 10 is a flowchart illustrating processing performed by a target vehicle and a server. [Figure 20] 10 is a flowchart illustrating a success rate calculation process. [Figure 21] FIG. 2 is a block diagram illustrating an example of the hardware configuration of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present technology will be described in the following order. 1. Vibration Data Management System 2. Configuration of each device 3. Operation of each device 4. Variations

[0014] <<1. Vibration Data Management System>> FIG. 1 is a diagram illustrating an example of the configuration of a vibration data management system according to an embodiment of the present technology.

[0015] The vibration data management system shown in FIG. 1 is configured by connecting vehicles 2A to 2C to a server 1 via a network such as the Internet.

[0016] The server 1 manages vibration data representing vibrations detected in the vehicles 2A to 2C.

[0017] Vehicles 2A to 2C are equipped with a signal processing device, a display device, an acceleration sensor, etc. The signal processing device plays video content and the like and displays it on the display device. The display device displays the video content and a UI for operating the signal processing device and the like under the control of the signal processing device. The acceleration sensor detects vibrations of each of vehicles 2A to 2C.

[0018] Passengers (users) of the vehicles 2A to 2C can operate the display device and the signal processing device using gestures. Note that, hereinafter, when there is no need to distinguish between the vehicles 2A to 2C, they will be simply referred to as the vehicle 2.

[0019] In Figure 1, three vehicles 2A to 2C are connected to the server 1, but in reality, any number of vehicles 2 are connected to the server 1, and vibration data representing vibrations detected in each vehicle 2 is managed by the server 1.

[0020] FIG. 2 is a diagram showing the flow of data in the vibration data management system.

[0021] In the example of Fig. 2, vehicles 2A to 2C travel so that they pass the same position in the order of vehicle 2A, vehicle 2B, and vehicle 2C. Hereinafter, vehicle 2A will also be referred to as leading vehicle 2A, vehicle 2B as target vehicle 2B, and vehicle 2C as following vehicle 2C. Unless otherwise specified, the following will be described assuming that the vehicle types of leading vehicle 2A, target vehicle 2B, and following vehicle 2C are the same.

[0022] FIG. 3 is a diagram showing an operation using gestures by an occupant of the leading vehicle 2A.

[0023] 3, video content is displayed on a display device 11A provided in a leading vehicle 2A. An occupant U1 of the leading vehicle 2A can control the signal processing device within the leading vehicle 2A, for example, by pointing a fingertip at a predetermined position on the display device 11A.

[0024] Vibrations occur in the preceding vehicle 2A while it is traveling due to road irregularities and the like. The vibrations of the preceding vehicle 2A cause the display device 11A and the fingertip of the occupant U1 to shake, as shown in FIG. 3 . Even if the occupant tries to point his / her fingertip at a predetermined range in the video content, the shaking of the display device 11A and the fingertip causes the range toward which the fingertip is pointed to move, for example, as shown by the three dashed ellipses. Therefore, the range recognized as the pointing range of the fingertip is unstable, and the signal processing device may perform an operation different from the intention of the occupant U1.

[0025] In contrast, when the preceding vehicle 2A detects vibrations while traveling, it transmits vibration data indicating the detected vibrations to the server 1, along with vehicle information indicating the model of the preceding vehicle 2A and location information indicating the location where the preceding vehicle 2A is traveling, as shown by arrow A1 in Figure 2.

[0026] The server 1 stores the vibration data transmitted from the preceding vehicle 2A in association with the position of the map data indicated by the position information. The map data is data that the server 1 uses to collectively manage vibration data for multiple positions. In other words, the vibration data is stored as travel data for the position where the preceding vehicle 2A is traveling.

[0027] As indicated by arrow A2, the target vehicle 2B acquires vibration data linked to a predicted position indicating a future traveling position from the server 1. For example, vibration data acquired by a vehicle 2 of a model corresponding to the model of the target vehicle 2B is acquired from the server 1. The model corresponding to the model of the target vehicle 2B is, for example, the same model as the model of the target vehicle 2B or a model similar to the model of the target vehicle 2B. Here, vibration data acquired by the preceding vehicle 2A is acquired from the server 1.

[0028] Based on the vibration data acquired from the server 1, the target vehicle 2B analyzes the pattern and magnitude of vibrations predicted to occur at the predicted position, and changes the method of recognizing gesture-based operations in accordance with the vibration analysis results. As indicated by arrow A3, the target vehicle 2B transmits UI correction data used to change the method of recognizing operations to the server 1, along with vibration data indicating vibrations actually detected at the predicted position. Vehicle information indicating the model of the target vehicle 2B and location information indicating the location where the target vehicle 2B is traveling are transmitted to the server 1 along with this vibration data and UI correction data.

[0029] The server 1 stores the vibration data and UI correction data transmitted from the target vehicle 2B in association with the position in the map data indicated by the position information. As a result, the vibration data acquired by the leading vehicle 2A and the vibration data and UI correction data acquired by the target vehicle 2B are associated with one position.

[0030] As indicated by arrow A4, the following vehicle 2C acquires vibration data and UI correction data linked to the predicted position from the server 1. Here, the vibration data acquired by the leading vehicle 2A and the vibration data acquired by the target vehicle 2B are acquired from the server 1. In addition, the UI correction data used in the target vehicle 2B is acquired from the server 1.

[0031] The following vehicle 2C analyzes the pattern and magnitude of vibrations occurring at the predicted position based on the vibration data acquired from the server 1, and changes the method of recognizing gesture-based operations according to the vibration analysis results. Furthermore, the following vehicle 2C corrects the method of recognizing operations as needed based on the UI correction data acquired from the server 1.

[0032] FIG. 4 is a diagram showing operations using gestures by the occupants of the target vehicle 2B and the following vehicle 2C.

[0033] 4A, video content is displayed on a display device 11B provided in the target vehicle 2B. An occupant U2 of the target vehicle 2B, like an occupant U1 of the preceding vehicle 2A, can control the signal processing device by pointing a fingertip at a predetermined position on the display device 11B within the target vehicle 2B.

[0034] 4A, even if the display device 11B or the fingertip of the occupant U2 shakes due to vibration of the target vehicle 2B, the signal processing device recognizes one of the ranges indicated by two dashed ellipses as the range to which the fingertip is pointed, for example, by expanding the range recognized as an operation. This allows the target vehicle 2B to recognize the operation of pointing the fingertip toward the display device 11B more stably than the preceding vehicle 2A, and makes it possible to prevent the target vehicle 2B from recognizing an operation different from the operation intended by the occupant U2 when vibration occurs suddenly, for example.

[0035] In the example of B in Fig. 4, video content is displayed on a display device 11C provided in a following vehicle 2C. An occupant U3 of the following vehicle 2C can control the signal processing device by pointing a fingertip at a predetermined position on the display device 11C, just like an occupant U1 of the preceding vehicle 2A and an occupant U2 of the target vehicle 2B.

[0036] As shown in B of FIG. 4, even if the display device 11C or the occupant U3's fingertip shakes due to vibrations from the following vehicle 2C, the signal processing device recognizes the area indicated by a single dashed ellipse as the area where the fingertip is pointed, for example, by changing the recognition method for gesture-based operations to take into account the influence of the vibrations. By using UI correction data in addition to vibration data, the following vehicle 2C can recognize the operation of pointing the fingertip toward the display device 11C more reliably than the preceding vehicle 2A or the target vehicle 2B. This makes it possible for the following vehicle 2C to more reliably prevent recognition of an operation different from the operation intended by the occupant U3 when vibrations occur.

[0037] As described above, the target vehicle 2B and the following vehicle 2C can provide a stable operating environment while traveling, where the vibration amount changes dynamically. By dynamically changing the operation recognition method according to the vibration pattern and magnitude, the occupants can comfortably use the displayed content without having to manually change the setting of the operation recognition method according to the traveling environment, for example.

[0038] <<2. Configuration of each device>> <Server configuration> FIG. 5 is a block diagram showing an example of the configuration of the server 1.

[0039] As shown in FIG. 5, the server 1 includes a data receiving unit 21, a vibration data managing unit 22, and a data transmitting unit .

[0040] The data receiving unit 21 receives vibration data, vehicle information, position information, and UI correction data transmitted from the vehicle 2 , and supplies them to the vibration data management unit 22 .

[0041] The vibration data management unit 22 stores the vibration data and UI correction data supplied from the data receiving unit 21 in association with the position in the map data indicated by the position information. The vibration data management unit 22 labels the vibration data with vehicle information.

[0042] In response to a request from the vehicle 2, the data transmission unit 23 acquires the vibration data and UI correction data linked to the predicted position of the vehicle 2 from the vibration data management unit 22 and transmits them to the vehicle 2.

[0043] <Configuration of leading vehicle> FIG. 6 is a block diagram showing an example of the configuration of the leading vehicle 2A.

[0044] 6, the leading vehicle 2A is configured with a vibration data acquisition unit 31, an acceleration sensor 32, a GNSS (Global Navigation Satellite System) receiver 33, and a data transmission unit 34. Some of the components of the leading vehicle 2A are realized by a signal processing device provided in the leading vehicle 2A executing a predetermined program.

[0045] The vibration data acquisition unit 31 acquires vibration data from the acceleration sensor 32 and acquires position information from the GNSS receiver 33. The vibration data acquisition unit 31 links the position information and vehicle information to the vibration data and supplies the data to the data transmission unit .

[0046] The acceleration sensor 32 detects vibrations occurring in the leading vehicle 2A, acquires vibration data indicating the detected vibrations, and supplies the vibration data to the vibration data acquisition unit 31.

[0047] The GNSS receiver 33 detects the traveling position of the leading vehicle 2A based on the signals received from the GNSS satellites, acquires position information indicating the detection result of the traveling position, and supplies it to the vibration data acquisition unit 31.

[0048] The data transmission unit 34 transmits the vibration data supplied from the vibration data acquisition unit 31 to the server 1 together with the position information and vehicle information.

[0049] 6, only the configuration for acquiring vibration data and transmitting it to the server 1 has been described as the configuration of the preceding vehicle 2A, but it is also possible to provide the preceding vehicle 2A with a configuration for displaying a UI. In this case, a configuration similar to the configuration of the target vehicle 2B, which will be described later, is provided in the preceding vehicle 2A.

[0050] <Target vehicle configuration> Overall structure FIG. 7 is a block diagram showing an example of the configuration of the target vehicle 2B.

[0051] 7, the target vehicle 2B is configured with a data receiving unit 41, a vibration pattern analyzing unit 42, an acceleration sensor 43, a GNSS receiver 44, a UI display control unit 45, a camera 46, a data transmitting unit 47, and a display device 11B. Some of the components of the target vehicle 2B are realized by a signal processing device provided in the target vehicle 2B executing a predetermined program.

[0052] The data receiving unit 41 acquires vibration data linked to the predicted position of the target vehicle 2B from the server 1. Specifically, the data receiving unit 41 requests the server 1 for vibration data labeled with vehicle information indicating the vehicle model corresponding to the vehicle model of the target vehicle 2B, and acquires the vibration data from the server 1. The vibration data acquired from the server 1 is supplied to the vibration pattern analysis unit 42.

[0053] The vibration pattern analysis unit 42 analyzes the pattern and magnitude of vibration occurring at the predicted position based on the vibration data supplied from the data receiving unit 41. The vibration pattern analysis unit 42 generates UI correction data based on the analysis result and supplies it to the UI display control unit 45.

[0054] In addition, the vibration pattern analysis unit 42 acquires location information from the GNSS receiver 44, and based on the location information, determines the future traveling position of the target vehicle 2B as a predicted position, and controls the data receiving unit 41 to acquire vibration data linked to the predicted position from the server 1.

[0055] Furthermore, the vibration pattern analysis unit 42 acquires vibration data from the acceleration sensor 43, and supplies the vibration data, UI correction data, position information, and vehicle information to the data transmission unit 47 in association with each other.

[0056] The acceleration sensor 43 detects vibrations occurring in the target vehicle 2B, acquires vibration data indicating the detection results of the vibrations, and supplies the vibration pattern analysis unit 42 with the vibration data.

[0057] The GNSS receiver 44 detects the traveling position of the target vehicle 2B based on the signals received from the GNSS satellites, acquires position information indicating the detection result of the traveling position, and supplies it to the vibration pattern analysis unit 42.

[0058] The UI display control unit 45 acquires image data from the camera 46 and recognizes an operation using a gesture by the occupant based on the image data. At this time, the UI display control unit 45 corrects the operation recognition method based on the UI correction data supplied from the vibration pattern analysis unit 42. In other words, the UI display control unit 45 recognizes an operation using a gesture by the occupant in accordance with the operation recognition method set by the vibration pattern analysis unit 42.

[0059] The UI display control unit 45 executes an operation according to an operation using a gesture by the occupant. For example, the UI display control unit 45 causes the display device 11B to display information and video content required in the vehicle interior space.

[0060] The camera 46 is provided inside the target vehicle 2B. The camera 46 captures an image of the interior space of the target vehicle 2B, acquires image data showing a moving image of the captured occupants, and supplies the image data to the UI display control unit 45.

[0061] The data transmission unit 47 transmits the vibration data, UI correction data, position information, and vehicle information supplied from the vibration pattern analysis unit 42 to the server 1.

[0062] Vibration pattern analysis unit configuration FIG. 8 is a block diagram showing a detailed configuration example of the vibration pattern analysis unit 42. As shown in FIG.

[0063] As shown in FIG. 8, the vibration pattern analysis unit 42 includes a vibration amplitude detection unit 61, a vibration frequency detection unit 62, a vibration occurrence frequency detection unit 63, a vibration pattern classification unit 64, an operation method setting unit 65, a recognition range setting unit 66, and a recognition determination time setting unit 67.

[0064] The vibration amplitude detector 61, the vibration frequency detector 62, and the vibration occurrence frequency detector 63 are each supplied with the same vibration data from the data receiver 41.

[0065] The vibration amplitude detection unit 61 detects the amplitude of the vibration indicated by the vibration data. For example, the average value of the amplitude occurring for each occurrence frequency is detected as the amplitude. Information indicating the amplitude is supplied to the vibration pattern classification unit 64.

[0066] The vibration frequency detection unit 62 detects the frequency of the vibration indicated by the vibration data. For example, the frequency at which the vibration occurs is detected. Information indicating the frequency is supplied to the vibration pattern classification unit 64.

[0067] The vibration occurrence frequency detection unit 63 detects the continuity of vibration indicated by the vibration data. For example, the rate at which vibration occurs for each occurrence frequency within a unit time (for example, 1 second) is detected as the continuity. The vibration occurrence frequency detection unit 63 also detects the randomness of vibration indicated by the vibration data. For example, at least one of the number of pulses having an amplitude equal to or greater than the average amplitude within a unit time and the variance value of the amplitude is detected as the randomness. Information indicating the continuity and randomness is supplied to the vibration pattern classification unit 64.

[0068] As described above, the vibration amplitude detection unit 61, the vibration frequency detection unit 62, and the vibration occurrence frequency detection unit 63 digitize four characteristics that indicate the behavior of vibration based on the vibration data, and generate vibration characteristic data in the following format. Vibration characteristics data = (randomness, frequency, amplitude, continuity)

[0069] The vibration pattern classification unit 64 classifies the pattern of vibration occurring at the predicted position based on the vibration characteristic data generated by the vibration amplitude detection unit 61, the vibration frequency detection unit 62, and the vibration occurrence frequency detection unit 63. The vibration occurring at the predicted position is classified into patterns according to randomness, frequency, magnitude (amplitude), and continuity, as shown in the first row of Fig. 9.

[0070] Specifically, vibrations occurring at the predicted position are classified into one of patterns, such as a pattern that occurs suddenly and randomly, a pattern that occurs steadily, etc. Vibration classification information indicating the pattern of vibrations occurring at the predicted position is supplied to an operation method setting unit 65, a recognition range setting unit 66, and a recognition determination time setting unit 67.

[0071] The operation method setting unit 65 sets operation methods that can be used by the occupant, specifically, types of gestures that can be used by the occupant, according to the vibration classification information supplied from the vibration pattern classification unit 64. For example, the operation method setting unit 65 selects parts of the occupant that can be used for gestures, and sets gestures using the selected parts as recognition targets. As shown in the second row of Fig. 9, for example, gestures using at least any part of the arms, hands, fingers, body, head, face, and gaze are set as recognition targets.

[0072] The operation method setting unit 65 supplies the UI display control unit 45 with operation method parameters indicating whether or not a gesture using each part is a recognition target. The operation method parameters are defined, for example, in the following format: A value of 1 indicates that it is a recognition target, and a value of 0 indicates that it is not a recognition target. In the following example, gestures using the arms, hands, and fingers are set as recognition targets, and gestures using the body, head, face, and gaze are not set as recognition targets. (arm, hand, finger, body, head, face, gaze) = (1,1,1,0,0,0,0)

[0073] The recognition range setting unit 66 sets a recognition range in accordance with the vibration classification information supplied from the vibration pattern classification unit 64. The recognition range is a range within the display range of the UI in which a gesture by the occupant is recognized as an operation.

[0074] The fourth row of FIG. 9 shows an example of a method for setting the recognition range. The thick rectangle indicates the display device 11B, and the dashed rectangle indicates the recognition range. The gray rectangle indicates the display range of the UI displayed on the display device 11B. Images that can be operated, such as buttons and icons, are displayed as the UI. For example, if it is recognized that the occupant's fingertip is pointing within the recognition range, it is determined that an operation to select the UI has been performed.

[0075] The recognition range is set to, for example, one of a small range, a standard range, and a large range relative to the display range of the UI. For example, when the recognition range is set to a small range, the recognition range is set to a range smaller than the standard range and is set to approximately the same range as the display range. When the recognition range is set to a large range, the recognition range is set to a range larger than the standard range and is set to a range that is a significant enlargement of the display range.

[0076] Furthermore, as shown in the third row of Fig. 9, it is also possible to change the UI display range, thereby substantially changing the recognition range. The UI display range is set to one of a small range, a standard range, and a large range. When the UI display range is set to a small range, the UI display range is set to a range smaller than the standard range. When the UI display range is set to a large range, the UI display range is set to a range larger than the standard range. The change in the UI display range is also performed by, for example, the recognition range setting unit 66.

[0077] When changing the recognition range, both the recognition range and the display range may be changed, or only one of them may be changed.

[0078] The recognition range setting unit 66 supplies a recognition range parameter indicating the recognition range to the UI display control unit 45. The recognition range parameter is defined, for example, in the following format: (x, y) indicate the display coordinates of the image displayed as the UI. k indicates the display range of the UI image. For example, if the display range is a small range, k is set to 0; if the display range is a standard range, k is set to 1; and if the display range is a large range, k is set to 2. f indicates the recognition range. For example, if the recognition range is a small range, f is set to 0; if the recognition range is a standard range, f is set to 1; and if the recognition range is a large range, f is set to 2. (x,y,k,f)=(0,0,1,1)

[0079] Even if the display range setting is the same, the size of the UI image displayed changes depending on the resolution of the display device 11B.

[0080] The recognition determination time setting unit 67 sets the recognition determination time and whether or not to apply a recognition invalid period according to the vibration classification information supplied from the vibration pattern classification unit 64. The recognition determination time is the duration of a gesture until the gesture is recognized as an operation. For example, when a gesture of pointing a fingertip within the recognition range for the recognition determination time is recognized, it is recognized that an operation of selecting a UI has been performed. As shown in the fifth row of FIG. 9, for example, any of 200 ms (short), 500 ms (standard), or 1 s or more (long) is set as the recognition determination time.

[0081] The recognition invalid period is a period during which gesture recognition is invalid. For example, a period during which an event that reduces gesture recognition accuracy, such as vibrations with amplitudes greater than a predetermined threshold, occurs is set as the recognition invalid period. When the recognition invalid period is applied, the duration of the gesture is measured excluding the recognition invalid period. On the other hand, when the recognition invalid period is not applied, the duration of the gesture is measured excluding the recognition invalid period.

[0082] The recognition determination time setting unit 67 supplies the UI display control unit 45 with a recognition determination time parameter indicating whether or not a recognition determination time and a recognition invalid period are applied.

[0083] The above-described operation method parameter, recognition range parameter, and recognition determination time parameter indicating the operation recognition method are used as UI correction data for correcting the operation recognition method by the UI display control unit 45.

[0084] FIG. 10 is a diagram illustrating an example of correcting the operation recognition method.

[0085] 10, as shown in the first row, the occurrence of sudden vibration at the predicted position is detected by classifying the vibration pattern indicated by the vibration data. For example, if the vibration data shows a single or intermittent occurrence of an amplitude greater than a predetermined threshold, it is determined that sudden vibration will occur at the predicted position.

[0086] In this case, the operation method setting unit 65 generates operation method parameters that set a gesture using the occupant's fingers as a recognition target. Therefore, as shown in the second row of Fig. 10, if a gesture using the fingers is set as a recognition target before the correction of the operation recognition method, the recognition target will not be changed even after the correction of the operation recognition method.

[0087] Furthermore, the recognition range setting unit 66 generates recognition range parameters that set the display range and recognition range of the UI to a small range. Therefore, as shown in the third and fourth rows of Fig. 10, if the display range and recognition range of the UI are set to a small range before the correction of the operation recognition method, the display range and recognition range of the UI will not be changed even after the correction of the operation recognition method.

[0088] The recognition determination time setting unit 67 generates a recognition determination time parameter that includes the recognition determination time and applies the recognition invalid period. Therefore, as shown in the fifth row of Fig. 10, before the correction of the operation recognition method, if the duration of the state in which the occupant points their fingertip within the recognition range reaches the recognition determination time, it is determined (recognized) that an operation has been performed. On the other hand, after the correction of the operation recognition method, if the duration of the state in which the occupant points their fingertip within the recognition range reaches the recognition determination period, during the period excluding the recognition invalid period in which the recognition accuracy of the operation decreases due to the large amplitude of vibration, it is determined that an operation has been performed.

[0089] Before the operation recognition method is corrected, the gesture observation time until it is determined that an operation has been performed matches the recognition determination time.

[0090] On the other hand, after the operation recognition method is corrected, the gesture observation time until it is determined that an operation has been performed may be longer than the recognition determination time. In other words, if a recognition invalid period occurs during gesture observation, the gesture observation time until it is determined that an operation has been performed is the recognition determination time plus the length of the recognition invalid period.

[0091] For example, if the operation recognition method is not corrected, sudden vibrations may cause the fingertip or the display device 11B to shake, causing the direction of the fingertip to move out of the recognition range. This may result in unstable operation recognition results, such as the occupant's gesture not being recognized as an operation even though the occupant intends to perform the operation.

[0092] In contrast, by disabling gesture recognition during the recognition invalid period when sudden vibration occurs, even if the direction of the fingertip goes out of the recognition range due to sudden vibration, the occupant's gesture is recognized as an operation as long as the state in which the direction of the fingertip remains within the recognition range for the recognition determination time outside the recognition invalid period. Therefore, even if vibration occurs in the target vehicle 2B, it is possible to stably recognize operations using gestures.

[0093] FIG. 11 is a diagram showing another example of correcting the operation recognition method.

[0094] 11, as shown in the first row, the occurrence of stable vibration at the predicted position is detected by classifying the vibration pattern indicated by the vibration data. For example, if amplitudes greater than a predetermined threshold value are continuously occurring in the vibration data, it is determined that stable vibration will occur at the predicted position.

[0095] In this case, the operation method setting unit 65 generates operation method parameters that do not target gestures using the occupant's fingers but target gestures using the occupant's arms. In other words, when stable vibrations are detected at the predicted position, the operation method setting unit 65 limits the types of gestures that the occupant can use. Therefore, as shown in the second row of FIG. 11, if the recognition target is set to a gesture using the fingers before the correction of the operation recognition method, the recognition target is changed to a gesture using the arms after the correction of the operation recognition method.

[0096] Furthermore, the recognition range setting unit 66 generates recognition range parameters that set the UI display range to a small range and the recognition range to a standard range. Therefore, as shown in the third row of Fig. 11, if the UI display range is set to a small range before the correction of the operation recognition method, the UI display range will not be changed even after the correction of the operation recognition method. Also, as shown in the fourth row of Fig. 11, if the recognition range is set to a small range before the correction of the operation recognition method, the recognition range will be changed to the standard range after the correction of the operation recognition method.

[0097] The recognition determination time setting unit 67 generates a recognition determination time parameter that includes the recognition determination time and does not apply the recognition invalid period. Therefore, as shown in the fifth row of Fig. 11, before and after correction of the operation recognition method, if the duration of the state in which the occupant points their fingertip or arm within the recognition range reaches the recognition determination time, it is determined that an operation has been performed.

[0098] For example, if the operation recognition method is not corrected, when large vibrations continue to occur, the fingertip or display device 11B may shake, making it impossible to accurately point the fingertip within a recognition range of a predetermined size.

[0099] On the other hand, if the operation recognition method is corrected, the recognition target can be seamlessly changed to a gesture using the arm. Even if the fingertips shake due to the vibration during a period when large vibrations continue, the UI that the occupant wants to select can be accurately recognized based on the direction of the arm, which is less affected by the vibration.

[0100] Furthermore, by expanding the recognition range, the UI that the occupant wants to select can be accurately recognized even if the occupant's arm shakes due to vibration. Therefore, even if vibration occurs in the target vehicle 2B, the gesture performed by the occupant can be recognized as the operation that the occupant intended.

[0101] As described above, the recognition method for gesture-based operations is set based on vibration data by the operation method setting unit 65, the recognition range setting unit 66, and the recognition determination time setting unit 67. By combining the selection of the operation method, the adjustment of the recognition range, and the adjustment of the recognition determination time, the target vehicle 2B can reduce the effect of vibration on gestures and maintain the operation system using gestures.

[0102] UI display control section configuration FIG. 12 is a block diagram showing a detailed configuration example of the UI display control unit 45. As shown in FIG.

[0103] As shown in FIG. 12, the UI display control unit 45 includes an operation recognizer 81 and a UI command generation unit 82.

[0104] The operation recognizer 81 is supplied with image data acquired from the camera 46 and operation method parameters from the vibration pattern analysis unit 42. Based on the image data acquired from the camera 46, the operation recognizer 81 recognizes the gesture set as the recognition target by the operation method parameters.

[0105] Data indicating the recognition result of the gesture set as the recognition target is supplied to the UI command generating unit 82.

[0106] The UI command generation unit 82 is supplied with a recognition range parameter and a recognition determination time parameter from the vibration pattern analysis unit 42. The UI command generation unit 82 recognizes an operation based on the gesture recognition result by the operation recognizer 81, and generates UI information indicating the content of the operation.

[0107] For example, the UI command generation unit 82 determines whether the gesture satisfies the operation condition, such as whether the duration of the gesture being performed within the recognition range reaches the recognition determination time. For example, if the gesture satisfies the operation condition, the UI command generation unit 82 determines that the gesture is an operation for selecting a UI corresponding to the recognition range. Note that the recognition range is indicated by a recognition range parameter, and the recognition determination time is indicated by a recognition determination time parameter.

[0108] The UI command generating unit 82 supplies UI information to the display device 11B, thereby causing the display device 11B to perform an operation according to the content of the operation.

[0109] The following vehicle 2C is provided with the same configuration as the target vehicle 2B.

[0110] <<3. Operation of each device>> The operation of each device in the vibration data management system having the above configuration will be described below.

[0111] <Operation of the preceding vehicle and server> The process performed by the leading vehicle 2A and the server 1 will be described with reference to the flowchart of FIG.

[0112] In step S1, the vibration data acquisition unit 31 of the leading vehicle 2A acquires vibration data from the acceleration sensor 32, acquires position information from the GNSS receiver 33, and supplies the data to the data transmission unit .

[0113] In step S2, the vibration data acquisition unit 31 of the preceding vehicle 2A acquires vehicle information of the vehicle itself (the preceding vehicle 2A) and supplies it to the data transmission unit .

[0114] In step S3, the data transmission unit 34 of the leading vehicle 2A transmits the vibration data, the vehicle information, and the position information to the server 1.

[0115] After transmitting the vibration data and the like, the preceding vehicle 2A repeats the processing from step S1 onwards.

[0116] In step S11, the data receiving unit 21 of the server 1 receives the vibration data, vehicle information, and position information transmitted from the leading vehicle 2A, and supplies them to the vibration data management unit 22.

[0117] In step S12, the vibration data management unit 22 of the server 1 stores the vibration data labeled with the vehicle information in association with the position of the map data based on the position information.

[0118] In the server 1, the processes from step S11 onwards are carried out every time vibration data and the like are transmitted from the leading vehicle 2A.

[0119] <Target vehicle and server operation> Overall operation The process performed by the target vehicle 2B and the server 1 will be described with reference to the flowchart of FIG.

[0120] In step S21, the vibration pattern analysis unit 42 of the target vehicle 2B acquires position information of the target vehicle 2B from the GNSS receiver 44, determines a predicted position, and notifies the data receiving unit 41 of the predicted position.

[0121] In step S22, the data receiving unit 41 of the target vehicle 2B acquires vehicle information about its own vehicle (target vehicle 2B).

[0122] In step S23, the data receiving unit 41 of the target vehicle 2B requests the vibration data acquired at the predicted position from the server 1. Here, based on the vehicle information acquired in step S22, the vibration data acquired by a vehicle of a type corresponding to the type of the target vehicle 2B is requested.

[0123] In step S41, the data transmission unit 23 of the server 1 transmits vibration data associated with the position requested by the target vehicle 2B to the target vehicle 2B. In the server 1, the process of step S41 is performed every time vibration data is requested from the target vehicle 2B.

[0124] In step S24, the data receiving unit 41 of the target vehicle 2B receives the vibration data of the predicted position transmitted from the server 1, and supplies it to the vibration pattern analyzing unit .

[0125] In step S25, the vibration pattern analysis unit 42 of the target vehicle 2B performs a UI correction data generation process. By the UI correction data generation process, UI correction data is generated based on the vibration data acquired from the server 1. Details of the UI correction data generation process will be described later with reference to FIG.

[0126] When the target vehicle 2B reaches the predicted position, in step S26, the UI display control unit 45 of the target vehicle 2B performs an operation determination process. By the operation determination process, a gesture performed by the occupant is recognized, and UI information is generated based on the gesture recognition result. At this time, the operation recognition method is corrected using the UI correction data generated in step S25, as necessary. The operation determination process will be described in detail later with reference to FIG. 16.

[0127] In step S27, the vibration pattern analysis unit 42 of the target vehicle 2B acquires vibration data from the acceleration sensor 43 and acquires position information from the GNSS receiver 44, and supplies the same to the data transmission unit 47.

[0128] In step S28, the data transmission unit 47 of the target vehicle 2B transmits to the server 1 the UI correction data, as well as the vibration data, vehicle information, and position information of the target vehicle 2B.

[0129] After transmitting the UI correction data and the like, the target vehicle 2B repeats the processing from step S21 onwards.

[0130] In step S42, the data receiving unit 21 of the server 1 receives the UI correction data, vibration data, vehicle information, and position information transmitted from the target vehicle 2B, and supplies them to the vibration data management unit 22.

[0131] In step S43, the vibration data management unit 22 of the server 1 stores the vibration data labeled with the vehicle information and the UI correction data in association with the position of the map data based on the position information.

[0132] In the server 1, the processes from step S42 onwards are carried out every time vibration data and the like are transmitted from the target vehicle 2B.

[0133] UI correction data generation process The UI correction data generation process performed in step S25 of FIG. 14 will be described with reference to the flowchart of FIG.

[0134] In step S51, the vibration pattern analysis unit 42 acquires vibration data from the data receiving unit 41.

[0135] In step S52, the vibration pattern analysis unit 42 analyzes the vibration pattern based on the vibration data.

[0136] Specifically, the vibration amplitude detection unit 61, the vibration frequency detection unit 62, and the vibration occurrence frequency detection unit 63 of the vibration pattern analysis unit 42 generate vibration characteristic data based on the vibration data and supply it to the vibration pattern classification unit 64. The vibration pattern classification unit 64 classifies the vibration pattern indicated by the vibration data based on the vibration characteristic data and generates vibration classification information. The vibration pattern classification unit 64 supplies the vibration classification information to the operation method setting unit 65, the recognition range setting unit 66, and the recognition determination time setting unit 67.

[0137] In step S53, the operation method setting unit 65 sets an operation method based on the vibration classification information. That is, the operation method setting unit 65 sets gestures to be recognized (usable). The operation method setting unit 65 generates operation method parameters used to set the gestures to be recognized.

[0138] In step S54, the recognition range setting unit 66 sets the recognition range based on the vibration classification information, and generates a recognition range parameter used to set the recognition range.

[0139] In step S55, the recognition determination time setting unit 67 sets the recognition determination time based on the vibration classification information. Also, the recognition determination time setting unit 67 sets whether or not to apply a recognition invalid period based on the vibration classification information. The recognition determination time setting unit 67 generates a recognition determination time parameter used to set the recognition determination time and whether or not to apply a recognition invalid period.

[0140] In step S56, the vibration pattern analysis unit 42 supplies the operation method parameter, the recognition range parameter, and the recognition determination time parameter to the UI display control unit 45 as UI correction data.

[0141] In step S57, the vibration pattern analysis unit 42 supplies the UI correction data to the data transmission unit 47.

[0142] Thereafter, the process returns to step S25 in FIG. 14, and the subsequent processes are carried out.

[0143] Operation decision processing The operation determination process performed in step S26 in FIG. 14 will be described with reference to the flowchart in FIG.

[0144] In step S71, the operation recognizer 81 recognizes a gesture. That is, the operation recognizer 81 recognizes a gesture of a body part set as a recognition target in the operation method parameters, for example, based on image data. For example, the operation recognizer 81 recognizes a gesture of a hand of an occupant.

[0145] In step S72, the UI command generation unit 82 recognizes the type of operation, for example, based on the gesture recognized by the operation recognizer 81. For example, when the operation recognizer 81 recognizes that a gesture of pointing a fingertip toward the display device 11B is being performed, the UI command generation unit 82 recognizes that an operation is being performed to select a UI displayed on the display device 11B.

[0146] In step S73, the UI command generation unit 82 determines whether the gesture performed by the occupant satisfies the operation conditions. For example, the UI command generation unit 82 compares the duration of a state in which a part used in the gesture set as the recognition target is directed within the recognition range with the recognition determination time. If the duration of the gesture reaches the recognition determination time, the UI command generation unit 82 determines that the gesture satisfies the operation conditions, and the process proceeds to step S74. At this time, if a recognition invalid period is set to be applied, the duration of the gesture is measured during a period excluding the recognition invalid period. Note that the recognition range is indicated by a recognition range parameter, and the recognition determination time and whether or not the recognition invalid period is applied are indicated by a recognition determination time parameter.

[0147] In step S74, the UI command generation unit 82 transmits the operation recognition result to the display device 11B. That is, the UI command generation unit 82 generates UI information indicating the content of the recognized operation and transmits it to the display device 11B.

[0148] Thereafter, the process returns to step S26 in FIG. 14, and subsequent processes are performed.

[0149] On the other hand, in step S73, if it is determined that the gesture made by the occupant does not satisfy the operation condition, the process of step S74 is skipped, the process returns to step S26 in FIG. 14, and subsequent processes are performed.

[0150] As described above, even if vibrations occur during traveling, the target vehicle 2B can appropriately recognize an operation performed by an occupant using a gesture.

[0151] <<4. Modified Example>> <Example of Feedback on Validity of UI Correction Data> For example, since states such as the state of the road, the posture of the occupant, and the state of the vehicle change depending on the situation and the passage of time, the correction of the operation recognition method using the UI correction data is not always optimal. In contrast, whether the UI correction data is valid or not is calculated as the success rate of operation recognition (UI operation), and when re - recognizing an operation with a low success rate, the UI correction data may be corrected and optimized.

[0152] FIG. 17 is a block diagram showing another configuration example of the UI display control unit 45 in FIG. 7. In FIG. 17, the same components as those in FIG. 12 are denoted by the same reference numerals. Redundant descriptions will be omitted as appropriate.

[0153] The configuration of the UI display control unit 45 shown in FIG. 17 is different from the configuration of the UI display control unit 45 in FIG. 12 in that a success rate determination device 101 is provided at a subsequent stage of the UI command generation unit 82.

[0154] The success rate determination device 101 includes a success rate calculation unit 111 and a labeling unit 112.

[0155] The success rate calculation unit 111 is supplied with the determination result by the UI command generation unit 82. The success rate calculation unit 111 calculates the success rate of operation recognition based on the determination result by the UI command generation unit 82. For example, the success rate calculation unit 111 calculates the success rate by determining that a gesture is successful when the UI command generation unit 82 determines that the gesture satisfies the operation conditions, and by determining that a gesture is unsatisfied when the UI command generation unit 82 determines that the gesture does not satisfy the operation conditions.

[0156] The success rate calculation unit 111 supplies information indicating the calculated success rate to the labeling unit 112.

[0157] The labeling unit 112 receives UI correction data (operation method parameters, recognition range parameters, and recognition determination time parameters) from the vibration pattern analysis unit 42. The labeling unit 112 labels the information received from the success rate calculation unit 111 onto the UI correction data as effectiveness feedback information indicating the effectiveness of the UI correction data.

[0158] The labeling unit 112 supplies the validity feedback information to the vibration pattern analysis unit 24, and supplies the UI correction data labeled with the validity feedback information to the data transmission unit 47.

[0159] Fig. 18 is a block diagram showing another example of the configuration of the vibration pattern analysis unit 42 of Fig. 7. In Fig. 18, the same components as those in Fig. 8 are given the same reference numerals. Duplicate explanations will be omitted as appropriate.

[0160] The configuration of the vibration pattern analysis unit 42 shown in FIG. 18 differs from the configuration of the vibration pattern analysis unit 42 shown in FIG. 8 in that validity feedback information is supplied from the UI display control unit 45 to an operation method setting unit 65, a recognition range setting unit 66, and a recognition determination time setting unit 67.

[0161] The operation method setting unit 65 corrects the operation method parameters based on the validity feedback information so as to invalidate operation methods with low success rates. For example, the operation method setting unit 65 corrects the operation method parameters so as to invalidate (exclude from recognition) gestures using parts that are easily affected by vibration, such as hands and fingers.

[0162] The recognition range setting unit 66 corrects the recognition range parameters based on the validity feedback information so as to optimize the recognition range or display range of an operation with a low success rate. For example, the recognition range setting unit 66 corrects the recognition range parameters so as to expand the recognition range or display range.

[0163] The recognition determination time setting unit 67 corrects the recognition determination time parameter based on the validity feedback information so as to optimize the recognition determination time for an operation with a low success rate. For example, the recognition determination time setting unit 67 corrects the recognition determination time parameter so as to shorten the recognition determination time.

[0164] The processing performed by the target vehicle 2B and the server 1 having the above configuration will be described with reference to the flowchart in Fig. 19. The processing performed by the server 1 is the same as the processing performed by the server 1 in Fig. 14, and therefore the description will be omitted.

[0165] In steps S101 to S106, the same processes as those in steps S21 to S26 in Fig. 14 are performed. That is, vibration data of the predicted position is acquired from the server 1, and UI correction data is generated. Also, UI information indicating the content of the operation corresponding to the gesture performed by the occupant is generated and transmitted to the display device 11B.

[0166] In step S107, the success rate determiner 101 of the target vehicle 2B performs a success rate calculation process. Through the success rate calculation process, validity feedback information is labeled to the UI correction data, and the validity of the UI correction data is fed back to the UI correction data generation process. The success rate calculation process will be described later with reference to FIG. 20.

[0167] In step S108, the vibration pattern analysis unit 42 of the target vehicle 2B acquires vibration data from the acceleration sensor 43 and acquires position information from the GNSS receiver 44, and supplies the same to the data transmission unit 47.

[0168] In step S109, the data transmission unit 47 of the target vehicle 2B transmits to the server 1 the UI correction data labeled with the validity feedback information, as well as the vibration data, vehicle information, and position information of the target vehicle.

[0169] After transmitting the UI correction data, etc., the processes from step S101 onward are repeated. In the UI correction data generation process of step S105, which is repeated, UI correction data modified based on the validity feedback information is generated.

[0170] The success rate calculation process performed in step S107 of FIG. 19 will be described with reference to the flowchart of FIG.

[0171] In step S121, the success rate calculation unit 111 calculates the success rate of the operation recognition, and supplies the labeling unit 112 with information indicating the calculation result of the success rate.

[0172] In step S122, the labeling unit 112 labels the UI correction data with the information supplied from the success rate calculation unit 111 as validity feedback information indicating the validity of the UI correction data. The labeling unit 112 supplies the UI correction data labeled with the validity feedback information to the data transmission unit 47.

[0173] In step S123, the labeling unit 112 feeds back the validity feedback information to the vibration pattern analysis unit .

[0174] Thereafter, the process returns to step S107 in FIG. 19, and the subsequent processes are carried out.

[0175] As described above, by introducing a mechanism for feeding back the effectiveness of the UI correction data, it becomes possible to continuously set UI correction data that can reduce the effects of vibrations occurring in the target vehicle 2B, regardless of road conditions, conditions such as the posture of the occupants, the condition of the vehicle, etc. Furthermore, the recognition accuracy of operations can be improved as the number of times the same gesture is performed increases.

[0176] Using the UI correction data generated by the target vehicle 2B and labeled with validity feedback information, the following vehicle 2C can set an operation recognition method that takes into account road conditions, etc. This makes it possible to achieve more optimal operation recognition. Furthermore, the operation recognition accuracy can be improved as the number of times the system is used increases, such as when passing over the same road.

[0177] <Other> Although an example has been described in which vibration data of a predicted position labeled with vehicle information is acquired from the server 1, it is also possible to configure the target vehicle 2B to communicate with the preceding vehicle 2A and acquire the vibration data, position information, and vehicle information directly from the preceding vehicle 2A. It is also possible to configure the target vehicle 2B to communicate with the following vehicle 2C and transmit the vibration data, position information, vehicle information, and UI correction data to the following vehicle 2C.

[0178] It is also possible to configure the target vehicle 2B to acquire vibration data of the predicted position based on video images of the leading vehicle 2A. The video images of the leading vehicle 2A are acquired by, for example, capturing images using a front camera mounted on the target vehicle 2B.

[0179] Although an example in which the UI and video content are displayed on a display device has been described, the UI and video content may be projected into the vehicle using, for example, a high-brightness projector.

[0180] For example, if vibration data acquired by a preceding vehicle 2A of a model corresponding to the model of the target vehicle 2B is not stored in the server 1, the vibration data acquired by a preceding vehicle 2A of a model different from that of the target vehicle 2B may be corrected according to the model of the target vehicle 2B. Such correction is performed, for example, by a signal processing device of the target vehicle 2B or the server 1.

[0181] Although an example in which vibrations of the vehicle 2 are detected by an acceleration sensor has been described, vibrations of the vehicle 2 may be detected by other vibration detection units.

[0182] For example, each vehicle 2 may transmit occupant information indicating characteristics such as the physique of the occupant to the server 1, and the server 1 may store the occupant information received from each vehicle 2 in association with UI correction data. Each vehicle 2 may then transmit occupant information indicating the characteristics of the occupant to the server 1, and the server 1 may transmit UI correction data associated with occupant information similar to the occupant information received from each vehicle 2 to each vehicle 2. This enables each vehicle 2 to correct the operation recognition method based on the UI correction data indicating the operation recognition method set for occupants with similar characteristics.

[0183] The present technology can also be applied to, for example, recognizing operations based on gestures by a passenger in a moving body other than a vehicle.

[0184] <Example of computer configuration> The processes executed by the server 1 and the signal processing device described above can be executed by hardware or software. When a series of processes is executed by software, the program constituting the software is installed from a program recording medium into a computer incorporated in dedicated hardware or a general-purpose personal computer.

[0185] FIG. 21 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes using a program.

[0186] A CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, and a RAM (Random Access Memory) 203 are interconnected by a bus 204.

[0187] An input / output interface 205 is also connected to the bus 204. An input unit 206 including a keyboard, a mouse, etc., and an output unit 207 including a display, a speaker, etc. are connected to the input / output interface 205. In addition, a storage unit 208 including a hard disk, a nonvolatile memory, etc., a communication unit 209 including a network interface, etc., and a drive 210 that drives removable media 211 are also connected to the input / output interface 205.

[0188] In the computer configured as above, the CPU 201 loads a program stored in the storage unit 208 into the RAM 203 via the input / output interface 205 and the bus 204 and executes the program, thereby performing the above-described series of processes.

[0189] The program executed by the CPU 201 is installed in the storage unit 208 by being recorded on a removable medium 211, or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting.

[0190] The program executed by the computer may be a program that processes in chronological order according to the order described in this specification, or may be a program that processes in parallel or at the required timing, such as when called.

[0191] In this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0192] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0193] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible without departing from the spirit of the present technology.

[0194] For example, this technology can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices via a network.

[0195] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by multiple devices.

[0196] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0197] <Configuration combination example> The present technology can also be configured as follows.

[0198] (1) a setting unit that sets a recognition method for an operation using a gesture based on first vibration data that indicates vibrations that are predicted to occur in the future; an operation recognition unit that recognizes the operation indicated by the gesture performed by the user in accordance with the recognition method set by the setting unit; A signal processing device comprising: (2) The operation recognition unit recognizes the operation indicated by the gesture performed by the user in the vehicle in which the signal processing device is installed. The signal processing device according to (1) above. (3) The first vibration data indicates vibrations detected by a preceding vehicle that has traveled ahead of the vehicle through a predicted position where the vehicle is predicted to travel in the future. The signal processing device according to (2) above. (4) The setting unit corrects the recognition method based on correction data indicating the recognition method set in the preceding vehicle. The signal processing device according to (3) above. (5) The setting unit sets the recognition method based on the first vibration data indicating vibration detected by the preceding vehicle of a vehicle type corresponding to the vehicle type of the vehicle. The signal processing device according to (3) or (4). (6) a vibration detection unit that detects vibrations of the vehicle and generates second vibration data that indicates a detection result of the vibrations of the vehicle; a transmission unit that transmits the second vibration data to a following vehicle that travels at the predicted position after the vehicle, or to a server that transmits the second vibration data to the following vehicle; The signal processing device according to any one of (3) to (5) above, further comprising: (7) The transmission unit transmits vehicle information indicating the type of the vehicle together with the second vibration data to the following vehicle or the server. The signal processing device according to (6) above. (8) The transmission unit transmits correction data indicating the recognition method set by the setting unit together with the second vibration data to the following vehicle or the server. The signal processing device according to (6) or (7). (9) an acquisition unit that acquires the first vibration data based on a moving image of the preceding vehicle; The signal processing device according to any one of (3) to (8) further comprises: (10) The setting unit sets, as the recognition method, at least one of a type of the gesture to be recognized, a range in which the gesture is recognized as the operation, and a recognition determination time until the gesture is recognized as the operation. The signal processing device according to any one of (1) to (9). (11) The setting unit sets the recognition method based on at least one of randomness, frequency, amplitude, and continuity of the vibration predicted to occur in the future. The signal processing device according to (10) above. (12) The operation recognition unit recognizes the operation by comparing the duration of the gesture, excluding a period during which vibrations with an amplitude greater than a predetermined threshold occur, with the recognition determination time. The signal processing device according to (11) above. (13) The setting unit limits the types of gestures to be recognized when it is predicted that vibrations having amplitudes greater than a predetermined threshold will occur continuously. The signal processing device according to (11) or (12). (14) The setting unit expands a range in which the gesture is recognized as the operation when it is predicted that vibrations having amplitudes greater than a predetermined threshold will occur continuously. The signal processing device according to any one of (11) to (13). (15) The operation recognition unit recognizes the gesture performed by the user based on a moving image of the user. The signal processing device according to any one of (1) to (14). (16) a determination unit that determines the validity of the recognition method set by the setting unit, The setting unit modifies the recognition method based on a result of determining the validity of the recognition method. The signal processing device according to any one of (1) to (15). (17) The signal processing device setting a recognition method for an operation using a gesture based on first vibration data indicating vibrations predicted to occur in the future; The operation indicated by the gesture performed by the user is recognized according to the set recognition method. Signal processing methods. (18) setting a recognition method for an operation using a gesture based on first vibration data indicating vibrations predicted to occur in the future; The operation indicated by the gesture performed by the user is recognized according to the set recognition method. A computer-readable recording medium that records a program for executing processing. (19) a setting unit that sets a recognition method for an operation using a gesture based on vibration data that indicates vibrations that are predicted to occur in the future; an operation recognition unit that recognizes the operation indicated by the gesture performed by the user in accordance with the recognition method set by the setting unit; a signal processing device comprising: a display unit that displays an image that is the target of the operation using the gesture; A display device provided A signal processing system having: (20) The signal processing system includes: a management unit that manages the vibration data in association with a position where the vibration indicated by the vibration data was detected; a server including: The setting unit sets the recognition method based on the vibration data acquired from the server. The signal processing system according to (19) above. [Explanation of symbols]

[0199] 1 Server, 2A Leading vehicle, 2B Target vehicle, 2C Following vehicle, 11A to 11C Display device, 21 Data receiving unit, 22 Vibration data management unit, 23 Data transmission unit, 31 Vibration data acquisition unit, 32 Acceleration sensor, 33 GNSS receiver, 34 Data transmission unit, 41 Data receiving unit, 42 Vibration pattern analysis unit, 43 Acceleration sensor, 44 GNSS receiver, 45 UI display control unit, 46 Camera, 47 Data transmission unit, 61 Vibration amplitude detection unit, 62 Vibration frequency detection unit, 63 Vibration occurrence frequency detection unit, 64 Vibration pattern classification unit, 65 Operation method setting unit, 66 Recognition range setting unit, 67 Recognition judgment time setting unit, 81 Operation recognizer, 82 UI command generation unit, 101 Success rate determination unit, 111 Success rate calculation unit, 112 Labeling Department

Claims

1. A setting unit that sets a recognition method for an operation using a gesture based on first vibration data that indicates vibrations that occurred at a predicted position where a vehicle is predicted to travel in the future in a preceding vehicle that has traveled ahead of the vehicle; an operation recognition unit that recognizes the operation indicated by the gesture performed by the user inside the vehicle in accordance with the recognition method set by the setting unit; A signal processing device comprising:

2. The setting unit corrects the recognition method based on correction data indicating the recognition method set in the preceding vehicle. The signal processing device according to claim 1 .

3. The setting unit sets the recognition method based on the first vibration data indicating vibrations occurring at the predicted position in the preceding vehicle of a vehicle type corresponding to the vehicle type of the vehicle.

3. The signal processing device according to claim 1.

4. a vibration detection unit that detects vibrations of the vehicle and generates second vibration data indicating a detection result of the vibrations of the vehicle; a transmission unit that transmits the second vibration data to a following vehicle that travels at the predicted position after the vehicle, or to a server that transmits the second vibration data to the following vehicle; The signal processing device according to claim 1 , further comprising:

5. The transmission unit transmits vehicle information indicating a type of the vehicle together with the second vibration data to the following vehicle or the server. The signal processing device according to claim 4 .

6. The transmission unit transmits correction data indicating the recognition method set by the setting unit together with the second vibration data to the following vehicle or the server.

6. The signal processing device according to claim 4 or 5.

7. an acquisition unit that acquires the first vibration data based on a moving image of the preceding vehicle; The signal processing device according to claim 1 , further comprising:

8. The setting unit sets, as the recognition method, at least one of a type of the gesture to be recognized, a range in which the gesture is recognized as the operation, and a recognition determination time until the gesture is recognized as the operation.

8. A signal processing device according to claim 1.

9. The setting unit sets the recognition method based on at least one of randomness, frequency, amplitude, and continuity of vibration predicted to occur in the vehicle at the predicted position based on the first vibration data. The signal processing device according to claim 8 .

10. The operation recognition unit recognizes the operation by comparing the duration of the gesture, excluding a period during which vibrations with an amplitude greater than a predetermined threshold occur, with the recognition determination time. The signal processing device according to claim 9 .

11. The setting unit limits the types of gestures to be recognized when it is predicted that vibrations having amplitudes greater than a predetermined threshold will occur continuously in the vehicle. The signal processing device according to claim 9 or 10.

12. The setting unit expands a range in which the gesture is recognized as the operation when it is predicted that vibrations having an amplitude greater than a predetermined threshold will occur continuously in the vehicle.

12. A signal processing device according to claim 9.

13. The operation recognition unit recognizes the gesture performed by the user based on a moving image of the user.

13. A signal processing device according to any one of claims 1 to 12.

14. a determination unit that determines the validity of the recognition method set by the setting unit, The signal processing device according to claim 1 , wherein the setting unit modifies the recognition method based on a result of determining the effectiveness of the recognition method.

15. The signal processing device setting a recognition method for an operation using a gesture based on vibration data indicating vibrations generated at a predicted position where the vehicle is predicted to travel in the future by a preceding vehicle that has traveled ahead of the vehicle; The operation indicated by the gesture performed by the user inside the vehicle is recognized according to the set recognition method. Signal processing methods.

16. A method for recognizing an operation using a gesture is set based on vibration data indicating vibrations generated at a predicted position where a vehicle is predicted to travel in the future in a preceding vehicle that has traveled ahead of the vehicle; The operation indicated by the gesture performed by the user inside the vehicle is recognized according to the set recognition method. A computer-readable recording medium that records a program for executing processing.

17. A setting unit that sets a recognition method for an operation using a gesture based on vibration data that indicates vibrations that occurred at a predicted position where a vehicle is predicted to travel in the future, in a preceding vehicle that has traveled ahead of the vehicle; an operation recognition unit that recognizes the operation indicated by the gesture performed by the user inside the vehicle in accordance with the recognition method set by the setting unit; a signal processing device comprising: a display unit that displays an image that is the target of the operation using the gesture; A display device provided A signal processing system having:

18. The signal processing system includes: a management unit that manages the vibration data in association with a position where the vibration indicated by the vibration data was detected; a server including: The setting unit sets the recognition method based on the vibration data acquired from the server.

18. A signal processing system according to claim 17.

Images