Display apparatus and display method

Abstract

To provide a display apparatus that displays image information that enables a user to objectively recognize a state of a moving body.SOLUTION: A display apparatus is configured to: measure a physical quantity generated in the moving body by vibration in a direction intersecting a surface on which the moving body moves; and display, on a display unit, image information including information relating to a state of the moving body generated based on a feature quantity extracted from the measured measurement data, where the image information includes an object relating to the state of the moving body.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to a display device and a display method. [Background technology]

[0002] Patent document 1 describes an automobile control device that can appropriately suppress the up and down vibrations of an automobile by calculating a vibration suppression driving force and a vibration suppression braking force to suppress the up and down vibrations of the automobile in accordance with the state of the up and down vibrations of the automobile, and applying the vibration suppression driving force and the vibration suppression braking force to the front and rear wheels of the automobile.

[0003] Patent Document 2 also describes a vehicle control device for an electric vehicle in which the wheels are driven by separate independent electric motors. When disturbances such as pitching, bouncing, or vehicle body sway in the turning direction occur when the electric vehicle is traveling straight, the vehicle control device includes: a torque detection unit that detects the output torque of the electric motor; a pitching detection unit that detects vertical vibrations around the left and right axis of the electric vehicle centered on the center of gravity due to the load that each tire attached to each wheel applies to the tire's contact surface; a pitching moment calculation unit that calculates the pitching moment from the detected vertical vibrations; a yaw rate detection unit that detects the rate at which the rotation angle of the electric vehicle in the turning direction changes; a yaw moment calculation unit that calculates the yaw moment occurring in the electric vehicle from the detected yaw rate; and a drive control unit that controls an output command value for driving the electric motor so that the calculated pitching moment value and yaw moment value both become zero. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-173089 [Patent Document 2] Japanese Patent Application Laid-Open No. 2010-259294 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with the techniques described in Patent Documents 1 and 2, it is difficult to objectively recognize the state of a moving body such as an automobile that changes due to vibrations in the up-down direction perpendicular to the moving direction of the moving body. [Means for solving the problem]

[0006] One aspect of the display device according to the present invention is measuring a physical quantity generated in the moving body due to vibration in a direction intersecting a plane on which the moving body moves, and displaying image information including information relating to the state of the moving body, which is generated based on feature quantities extracted from the measured measurement data, on a display unit; The image information is The object includes an object related to the state of the moving body.

[0007] One aspect of the display method according to the present invention is to measuring a physical quantity generated in the moving body due to vibration in a direction intersecting a plane on which the moving body moves, and displaying image information including information relating to a state of the moving body, which is generated based on feature quantities extracted from the measured measurement data, on a display unit; The image information is The object includes an object related to the state of the moving body. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is an explanatory diagram illustrating an overview of a display device. [Figure 2] FIG. 10 is a diagram showing an example of vibration measurement of a moving object. [Figure 3] FIG. 10 is a diagram showing an example of vibration measurement of a moving object. [Figure 4] FIG. 10 is a diagram showing an example of vibration measurement of a moving object. [Figure 5] FIG. 10 is a diagram showing an example of vibration measurement of a moving object. [Figure 6] FIG. 10 is a diagram showing an example of vibration measurement of a moving object. [Figure 7] FIG. 10 is a diagram showing an example of vibration measurement of a moving object. [Figure 8] FIG. 1 illustrates an example of the configuration of a display device. [Figure 9] FIG. 2 is a diagram showing an example of the configuration of a feature extraction circuit. [Figure 10] FIG. 10 is a diagram showing the amplitude spectral density calculated for measurement data measured when no bouncing phenomenon occurs in the moving object. [Figure 11] FIG. 10 is a diagram showing the amplitude spectral density calculated from measurement data measured when a bouncing phenomenon occurs in a moving object. [Figure 12] A plot of health indicators. [Figure 13] FIG. 4 is a diagram showing an example of image information displayed on a display unit. [Figure 14] FIG. 4 is a diagram showing an example of image information displayed on a display unit. [Figure 15] FIG. 4 is a diagram showing an example of image information displayed on a display unit. [Figure 16] FIG. 4 is a diagram showing an example of image information displayed on a display unit. [Figure 17] FIG. 4 is a diagram showing an example of image information displayed on a display unit. [Figure 18] FIG. 4 is a diagram showing an example of image information displayed on a display unit. [Figure 19] FIG. 4 is a diagram showing an example of image information displayed on a display unit. [Figure 20] FIG. 4 is a diagram showing an example of image information displayed on a display unit. [Figure 21] FIG. 4 is a flowchart showing the procedure of the display method according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Note that the embodiments described below do not unduly limit the content of the present invention as defined in the claims. Furthermore, not all of the configurations described below are necessarily essential components of the present invention.

[0010] 1. Display device overview Fig. 1 is a diagram for explaining an overview of a display device of this embodiment. As shown in Fig. 1, the display device 1 of this embodiment acquires measurement data measured by sensors 3 attached to multiple parts of a moving object 2. Note that although there may be only one sensor 3, the following description will be given assuming that there are multiple sensors 3.

[0011] Each sensor 3 measures a physical quantity generated in each component of the moving object 2 due to the vertical vibration of the moving object 2, and outputs measurement data corresponding to the measured physical quantity. The vertical direction is a direction intersecting the plane on which the moving object 2 moves, for example, a direction perpendicular to the plane on which the moving object 2 moves. The physical quantity measured by each sensor 3 may be, for example, acceleration, angular velocity, velocity, displacement, pressure, current, voltage, etc. Each measurement data may be an analog signal or a digital signal. The display device 1 extracts feature quantities from the measurement data measured by each sensor 3, generates information related to the state of the moving object 2 based on the extracted feature quantities, and displays image information including the information related to the state of the moving object 2 on a display unit (not shown).

[0012] The image information may include an object relating to the state of the moving object 2. For example, the object may be a radar chart or a level meter showing the state of multiple parts of the moving object 2, or a graph including a normal region showing the normal state of the moving object 2 and a fault region showing the fault state of the moving object 2. The object may also be an object relating to the state of at least one of the center of gravity, suspension, tires, power unit, and driver seat of the moving object 2. It may also be a project.

[0013] The image information may also include a first object relating to the current state of the moving body 2 and a second object relating to the predicted future state of the moving body 2, and may further include information on the time required to move from the current state to the predicted future state.

[0014] The display device 1 may generate sound information including information relating to the state of the moving object 2, or information for vibrating a vibration unit (not shown). That is, the display device 1 may generate sound or vibration in addition to displaying an image.

[0015] In the following explanation, the moving body 2 will be described as a racing car as an example, but the moving body 2 may be an automobile other than a racing car, or may be a moving body other than an automobile. Examples of the moving body 2 include, in addition to automobiles, agricultural machinery such as tractors, construction machinery such as excavators, unmanned guided vehicles, robotic lawn mowers, robotic vacuum cleaners, aircraft such as jet planes and helicopters, ships, rockets, artificial satellites, and railroad cars.

[0016] 2. Example of vibration measurement on a moving object 2 to 7 are diagrams showing examples of vibration measurement of a moving object 2. In the examples of Fig. 2 to 7, the moving object 2 is a racing car, and has a right front tire WFR, a left front tire WFL, a right rear tire WRR, a left rear tire WRL, a front suspension FS connected to the right front tire WFR and the left front tire WFL, and a rear suspension RS connected to the right rear tire WRR and the left rear tire WRL. However, the moving object 2 is not limited to a racing car, and may be, for example, a regular passenger car.

[0017] 2, the display device 1 measures the vertical vibrations Vfs of the front suspension FS, the vertical vibrations Vrs of the rear suspension RS, and the vertical vibrations of the center of gravity CG of the moving object 2. To this end, a sensor 3 is attached to each of the front suspension FS and the rear suspension RS. In addition, the sensor 3 is attached to a position close to the center of gravity CG of the moving object 2, for example, on the back of the driver's seat.

[0018] FIG. 3 shows an example of the mounting positions of sensors 3 on the front suspension FS. FIG. 3 shows twelve sensors 3a to 3l mounted at different positions so as to be oriented in such a way that they can detect vertical vibrations. The sensor 3 mounted on the front suspension FS may be any of sensors 3a to 3l, such as sensors 3b and 3c located near the two ends of the right upper arm, sensors 3e and 3f located near the two ends of the right lower arm, sensors 3h and 3i located near the two ends of the left upper arm, or sensors 3k and 3l located near the two ends of the left lower arm. Note that multiple sensors 3 may be mounted on the front suspension FS.

[0019] FIG. 4 shows an example of how sensors 3 are attached to the rear suspension RS. FIG. 4 shows twelve sensors 3m to 3x attached at different positions so as to be oriented to detect vertical vibrations. The sensors 3 attached to the rear suspension RS may be any of sensors 3m to 3x, such as sensors 3n and 3o located near the two ends of the right upper arm, sensors 3q and 3r located near the two ends of the right lower arm, sensors 3w and 3x located near the two ends of the left upper arm, and sensors 3t and 3u located near the two ends of the left lower arm. Note that multiple sensors 3 may be attached to the rear suspension RS.

[0020] In the example of FIG. 5, the display device 1 displays the vertical vibrations Vwfr of the right front tire WFR, the vertical vibrations Vwfl of the left front tire WFL, and the vertical vibrations Vwfl of the right rear tire WRR. To measure the vertical vibration Vwrl of the right front tire WFR, the left front tire WFL, the right rear tire WRR, and the left rear tire WRL, sensors 3 are attached to positions close to the right front tire WFR, the left front tire WFL, the right rear tire WRR, and the left rear tire WRL, respectively.

[0021] The sensor 3 that detects the vertical vibrations Vwfr of the right front tire WFR may be, for example, either the sensor 3a located near the base of the right upper arm or the sensor 3d located near the base of the right lower arm in the front suspension FS shown in Fig. 3. Note that a plurality of sensors 3 that detect the vertical vibrations Vwfr of the right front tire WFR may be attached to the front suspension FS.

[0022] The sensor 3 that detects the vertical vibrations Vwfl of the left front tire WFL may be, for example, either a sensor 3g located near the base of the left upper arm or a sensor 3j located near the base of the left lower arm in the front suspension FS shown in Fig. 3. Note that a plurality of sensors 3 that detect the vertical vibrations Vwfl of the left front tire WFL may be attached to the front suspension FS.

[0023] The sensor 3 that detects the vertical vibration Vwrr of the right rear tire WRR may be, for example, either the sensor 3p located near the base of the right upper arm or the sensor 3m located near the base of the right lower arm in the rear suspension RS shown in Fig. 3. Note that a plurality of sensors 3 may be attached to the rear suspension RS as the sensor 3 that detects the vertical vibration Vwrr of the right rear tire WRR.

[0024] The sensor 3 that detects the vertical vibrations Vwrl of the left rear tire WRL may be, for example, either the sensor 3v located near the base of the left upper arm or the sensor 3s located near the base of the left lower arm in the rear suspension RS shown in Fig. 3. Note that a plurality of sensors 3 that detect the vertical vibrations Vwrl of the left rear tire WRL may be attached to the rear suspension RS.

[0025] 6, the display device 1 measures the vertical vibration Vst of the driver's seat and the vertical vibration Vpu of the power unit. To this end, for example, sensors 3 are attached to the seating surface of the driver's seat and the power unit, respectively.

[0026] 7, the display device 1 measures the vertical vibration Vgb of the gearbox GB, the vertical vibration Vice of the engine ICE, the vertical vibration Vmguk of the kinetic energy recovery system MGU-K, and the vibration Vmguh of the thermal energy recovery system MGU-H. To this end, for example, sensors 3 are attached to each of the gearbox GB, the engine ICE, the kinetic energy recovery system MGU-K, and the thermal energy recovery system MGU-H.

[0027] 2 to 7, each sensor 3 is an acceleration sensor, and may be, for example, a one-axis acceleration sensor that detects vibrations in the up and down direction, or may be a three-axis acceleration sensor.

[0028] 3. Display Device Configuration FIG. 8 is a diagram showing an example of the configuration of the display device 1. As shown in FIG. 8, the display device 1 includes a plurality of sensors 3, a processing circuit 100, a memory circuit 110, an operation unit 120, a display unit 130, a sound output unit 140, a vibration unit 150, and a communication unit 160. Note that the display device 1 may be configured by omitting or changing some of the components shown in FIG. 8, or by adding other components. For example, the plurality of sensors 3 do not have to be components of the display device 1. The display device 1 may be provided inside the moving object 2 or outside the moving object 2.

[0029] As described above, each sensor 3 measures a physical quantity generated in each component of the moving object 2 due to the vertical vibration of the moving object 2, and outputs measurement data corresponding to the measured physical quantity. The measurement data output by each sensor 3 is input to the processing circuit 100.

[0030] The processing circuit 100 acquires the measurement data output by each sensor 3 and performs predetermined calculations. If the measurement data is an analog signal, an analog front end (not shown) amplifies and converts the measurement data into a digital signal, and the processing circuit 100 then performs predetermined calculations on the measurement data converted into a digital signal. Specifically, the processing circuit 100 executes a display processing program 111 stored in the storage circuit 110 to perform various calculations on the measurement data. The processing circuit 100 also performs various processes in response to operation signals from the operation unit 120, sending display signals to display various images on the display unit 130, sending sound signals to the sound output unit 140 to generate various sounds, sending vibration signals to the vibration unit 150 to generate various vibrations, and controlling the communication unit 160 to communicate data with an external device (not shown). The processing circuit 100 is implemented, for example, by a CPU or a DSP. CPU is an abbreviation for Central Processing Unit, and DSP is an abbreviation for Digital Signal Processor.

[0031] By executing the display processing program 111, the processing circuit 100 functions as a measurement data acquisition circuit 101, a feature extraction circuit 102, a status information generation circuit 103, and an image information output circuit 104. In other words, the display device 1 includes the measurement data acquisition circuit 101, the feature extraction circuit 102, the status information generation circuit 103, and the image information output circuit 104.

[0032] The measurement data acquisition circuit 101 acquires the measurement data measured by each sensor 3. The measurement data acquired by the measurement data acquisition circuit 101 is stored in the memory circuit 110.

[0033] The feature extraction circuit 102 extracts features from the measurement data acquired by the measurement data acquisition circuit 101. The feature extraction circuit 102 is realized by an AI model that learns to predict future values ​​of the measurement data. AI is an abbreviation for Artificial Intelligence. The AI ​​model is a machine learning model that automatically learns patterns and rules from the measurement data and classifies and predicts the measurement data. Features are quantitative numerical expressions of the measurement data and represent the essential attributes and patterns of the measurement data. The AI ​​model performs data analysis using the features and predicts future values ​​of the measurement data.

[0034] 9 is a diagram showing an example of the configuration of the feature extraction circuit 102. As shown in FIG. 9, the feature extraction circuit 102 is an AI model including an encoder 201 and a decoder 202.

[0035] The encoder 201 is realized by using, for example, a recurrent neural network, and converts p measurement values ​​x1 to x2 of measurement data acquired in time series. p is input, and r features z1 to z r Outputs the measured values ​​x1 to x p are m-dimensional vectors consisting of m elements. For example, if the measurement data is triaxial acceleration data, the measurement values ​​x1 to x p are three-dimensional vectors. r are n-dimensional vectors consisting of n elements. The total number of measurements, p, and the integer r are set to appropriate values ​​by the creator of the AI ​​model.

[0036] The decoder 202 is realized by using, for example, a recurrent neural network, and converts r feature quantities z1 to z2 output from the encoder 201 into r is input, and q predicted values ​​y1 to y q The predicted values ​​y1~y q are m-dimensional vectors each consisting of m elements.

[0037] This AI model is a trained model, and the decoder 202 generates p measurement values ​​x1 to x2 of the measurement data. p The following q predicted values ​​y1 to y q Alternatively, the decoder 202 may learn to output p measurement values ​​x1 to x2 of the measurement data, where p=q. p is restored as it is and the q predicted values ​​y1 to y q The AI ​​model can be trained to output the following. The training of the AI ​​model does not require abnormal data and can be performed without a teacher using only normal data. If abnormal data is present, the AI ​​model can also be trained in a supervised manner using both normal and abnormal data. By realizing the encoder 201 and the decoder 202 using a recurrent neural network, it is possible to input measurement data, which is time waveform data, directly to the AI ​​model. As a result of the training, the feature quantities z1 to z are passed from the encoder 201 to the decoder 202. r The information representing the state of the moving object 2 is collected in

[0038] The state information generation circuit 103 generates information related to the state of the mobile object 2 based on the feature amounts extracted from the measurement data by the feature amount extraction circuit 102. The information related to the state of the mobile object 2 may be information indicating whether the state of the mobile object 2 is normal or abnormal, information indicating whether the state of each of multiple parts constituting the mobile object 2 is normal or abnormal, or information indicating the degree of deterioration and remaining life of each part. For example, the state information generation circuit 103 classifies the state of the mobile object 2 using principal component analysis or a support vector machine (SVM) based on the feature amounts, and generates information related to the state of the mobile object 2.

[0039] For example, if the AI ​​model, which is the feature extraction circuit 102, has been trained using only normal data, the state information generation circuit 103 may calculate the Mahalanobis distance from the center point of the normal region in the feature space after transformation by principal component analysis as the degradation level. Alternatively, if the AI ​​model has been trained using normal data and abnormal data, the state information generation circuit 103 may calculate both the Mahalanobis distance from the center point of the normal region and the Mahalanobis distance from the center point of the abnormal region in the feature space after transformation by principal component analysis, and calculate the ratio of the two as the degradation level. The information generated by the state information generation circuit 103 is stored in the storage circuit 110.

[0040] The image information output circuit 104 outputs image information including information relating to the state of the moving object 2 generated by the state information generation circuit 103. The image information may include a threshold value for determining whether or not there is an abnormality in the moving object 2.

[0041] The storage circuitry 110 has a ROM and a RAM (not shown). ROM is an abbreviation for Read Only Memory, and RAM is an abbreviation for Random Access Memory. The ROM stores various programs such as the display processing program 111 and predetermined data, and the RAM stores data generated by the processing circuitry 100. The RAM is also used as a working area for the processing circuitry 100, and stores programs and data read from the ROM, data input from the operation unit 120, and data temporarily generated by the processing circuitry 100.

[0042] The operation unit 120 is an input device configured with operation keys, button switches, etc., and outputs an operation signal to the processing circuit 100 in response to an operation by a user.

[0043] The display unit 130 is a display device configured with an LCD or the like, and displays various images based on a display signal output from the processing circuit 100. LCD is an abbreviation for Liquid Crystal Display. The display unit 130 may be provided with a touch panel that functions as the operation unit 120. For example, when the condition of the moving object 2 is abnormal, the processing circuit 100 may output a display signal to the display unit 130 and cause the display unit 130 to display a warning image. Furthermore, for example, the processing circuit 100 may cause the display unit 130 to display an image including the degree of deterioration of the moving object 2. For example, the display unit 130 may be a display installed on the moving object 2. The display unit 130 may be a monitor of a personal computer or a smartphone. It may also be the display unit of a tablet terminal.

[0044] The sound output unit 140 is configured by a speaker or the like, and generates various sounds based on the sound signal output from the processing circuit 100. For example, when the state of the moving object 2 is abnormal, the processing circuit 100 may output a sound signal to the sound output unit 140, causing the sound output unit 140 to generate a warning sound (warning sound) or voice. For example, the sound output unit 140 may be a speaker installed in the moving object 2.

[0045] The vibration unit 150 generates various vibrations based on the vibration signal output from the processing circuit 100. For example, when the state of the moving object 2 is abnormal, the processing circuit 100 may output a vibration signal to the vibration unit 150 to vibrate the vibration unit 150. For example, the vibration unit 150 may be a vibration device incorporated in the handle of the moving object 2.

[0046] The communication unit 160 performs various controls to establish data communication between the processing circuit 100 and an external device. For example, the communication unit 160 transmits image information to the external device. The external device may display at least a part of the received image information on a display unit (not shown).

[0047] At least some of the measurement data acquisition circuit 101, the feature extraction circuit 102, the status information generation circuit 103, and the image information output circuit 104 may be implemented by dedicated hardware. The display device 1 may be a single device or may be configured with multiple devices. For example, the processing circuit 100 and the memory circuit 110 may be implemented by a cloud server or the like, which may generate image information and transmit the generated image information to the display unit 130 via a communication line. Alternatively, the display device 1 may not include the measurement data acquisition circuit 101, the feature extraction circuit 102, and the status information generation circuit 103, but may include the image information output circuit 104, and an external device may include the measurement data acquisition circuit 101, the feature extraction circuit 102, and the status information generation circuit 103. In this case, the external device may transmit image information to the display device 1, and the display device 1 may receive the image information and output an image based on the image information to the display unit 130.

[0048] 4. Example of analysis of the state of a moving object As mentioned above, each sensor 3 is attached to each part of the moving object 2, measures the physical quantity that occurs in each part due to the vertical vibration of the moving object 2, and outputs the measurement data. For example, vertical vibration occurs due to the bouncing phenomenon of the moving object 2. The bouncing phenomenon is a phenomenon in which the moving object 2 vibrates up and down as a whole due to the unevenness of the surface on which the moving object 2 moves. If the moving object 2 is a car, the unevenness of the road surface will cause the moving object 2 to vibrate up and down as a whole.

[0049] FIG. 10 is a diagram showing the amplitude spectrum density calculated for measurement data measured when the moving object 2 is not experiencing a bouncing phenomenon. On the other hand, FIG. 11 is a diagram showing the amplitude spectrum density calculated for measurement data measured when the moving object 2 is experiencing a bouncing phenomenon. Comparing FIG. 10 with FIG. 11, it can be seen that when a bouncing phenomenon occurs, the level of the signal component in the 0 to 20 Hz band increases. Therefore, the display device 1 can calculate the level of the signal component in the 0 to 20 Hz band and the integrated value of that level based on the measurement data, and analyze whether or not a bouncing phenomenon is occurring based on the calculation result.

[0050] Furthermore, for example, if the moving body 2 is a racing car, vertical vibrations will occur due to porpoising. Porpoising is a phenomenon that occurs due to the balance between lift and reverse lift acting on the moving body 2. Specifically, reverse lift is generated when the air between the car body and the road surface is sucked out, pushing the car body downward, but if the car body gets too close to the road surface, there is no more air to be sucked out and reverse lift no longer occurs. As a result, the car body rises upward due to the lift, and when the car body leaves the road surface, the car body and the road surface are again pressed down. The air between the moving object and the moving body 2 is sucked out, generating a reverse lift. This phenomenon is repeated, causing the moving object 2 to vibrate in the vertical direction. When this porpoising phenomenon occurs, the level of the signal component in a predetermined frequency band increases, just like the bouncing phenomenon. Therefore, the display device 1 calculates the level of the signal component in the predetermined frequency band and the integrated value of this level based on the measurement data, and can analyze whether or not the porpoising phenomenon is occurring based on the calculation results.

[0051] The display device 1 may also analyze the degree of deterioration and remaining life of each component based on the vertical vibration of the moving object 2. For example, the display device 1 may perform principal component analysis on measurement data measured by a sensor 3 attached to a specific component while the moving object 2 is traveling, plot the data on a graph, calculate the Mahalanobis distance, and calculate the degree of deterioration of the component from the Mahalanobis distance. The display device 1 may also fit the inverse of the Mahalanobis distance to an exponential deterioration model to estimate the remaining life of the component. For example, as an exponential deterioration model, a health indicator f(t) of a specific component at a given time t is defined as shown in Equation (1). In Equation (1), a is the initial health indicator value, and b is a parameter indicating the deterioration rate. The health indicator f(t) is the inverse of the Mahalanobis distance calculated from the measurement data at time t.

[0052]

number

[0053] FIG. 12 is a graph plotting 300 minutes of measurement data measured by a sensor 3 attached to the power unit, with the horizontal axis representing time and the vertical axis representing the health indicator (the inverse of the Mahalanobis distance). The curve shown in FIG. 12 represents an exponential deterioration model obtained by calculating the optimal values ​​of a and b in Equation (1) using the least squares method for the plot. When this exponential deterioration model falls below a predetermined threshold, it is assumed that the power unit has reached its end of life, and the remaining life can be estimated. The threshold is set, for example, to the minimum health indicator value that allows safe operation of the power unit. For example, if the threshold is set to 0.3, it is predicted that the health indicator value will reach 0.3 in 820 minutes, and therefore the remaining life of the power unit is estimated to be 520 minutes (= 820 minutes - 300 minutes).

[0054] 5. Example of image information display 13 to 20 show examples of image information displayed on the display unit 130. The image information shown in Fig. 13 includes an object OB1 indicating the vibration level of the front suspension, an object OB2 indicating the vibration level of the rear suspension, and an object OB3 indicating the vibration level of the center of gravity as objects relating to the state of the moving object 2. For example, the display device 1 can generate the objects OB1, OB2, and OB3 by acquiring measurement data from three sensors 3 attached to the front suspension, rear suspension, and center of gravity of the moving object 2, respectively.

[0055] The image information shown in FIG. 14 includes an object OB11 that resembles a power unit and four level meter objects OB12, OB13, OB14, and OB15 that are objects relating to the state of the moving object 2. The object OB12 indicates the state of the engine. The object OB13 indicates the state of the kinetic energy regeneration system. The object OB14 indicates the state of the gearbox. The object OB15 indicates the state of the thermal energy regeneration system. For example, the state of each part is a deterioration degree, and the higher the level in the objects OB12, OB13, OB14, and OB15, the greater the degree of deterioration. For example, the state of each part is normal if the level is equal to or lower than a threshold TH1, abnormal if the level is equal to or higher than a threshold TH2, and abnormal if the level is higher than a threshold TH3. The area between H1 and H2 is a normal state, but is approaching an abnormal state. For example, in the objects OB12, OB13, OB14, and OB15, the parts where the level is below the threshold TH1 may be displayed in green, the parts where the level is between the threshold TH1 and threshold TH2 may be displayed in yellow, and the parts where the level is above the threshold TH2 may be displayed in red. In other words, the objects OB12, OB13, OB14, and OB15 may be objects that resemble patrol lights.

[0056] The image information shown in FIG. 15 includes a radar chart object OB21 as an object related to the state of the moving object 2. The object OB21 indicates the degree of deterioration of each component of the moving object 2's center of gravity, engine, battery, gearbox, kinetic energy recovery system, and thermal energy recovery system. For example, the state of each component is normal if the degree of deterioration is equal to or less than a threshold value TH1, abnormal if the degree of deterioration is equal to or greater than a threshold value TH2, and normal but approaching an abnormal state if the degree of deterioration is between the threshold values ​​TH1 and TH2. The image information shown in FIG. 15 also includes a video VD captured by a camera (not shown) mounted on the moving object 2. The video VD includes an object OB22 indicating the degree of wear of the tires, which are objects related to the state of the moving object 2, and an object OB23 indicating the degree of wear of the tires of other moving objects included in the video VD. For example, the object OB22 may be color-coded according to the degree of wear of the four tires of the moving object 2, with tires with greater degrees of wear being displayed in darker colors. Similarly, the object OB23 may be displayed with the four tires of another moving object color-coded according to their degree of wear, with tires with greater degrees of wear being displayed in darker colors. For example, the other moving object may also be equipped with a display device 1, and the display device 1 equipped on the moving object 2 may acquire tire wear information from the display device 1 equipped on the other moving object. Furthermore, the image information shown in FIG. 15 includes an object OB24 of the course on which the moving object 2 is traveling. The object OB24 includes a pointer PT indicating the current position of the moving object 2 traveling on a road such as a circuit. The position of the pointer PT changes in conjunction with the moving object 2's position, and the image VD and objects OB21, OB22, and OB23 also change in conjunction with the moving object 2's position. Note that when the user moves the pointer PT, the image VD and objects OB21, OB22, and OB23 may display information about when the moving object 2 previously traveled through the position indicated by the pointer PT.

[0057] The image shown in FIG. 16 includes radar chart objects OB31 and OB32 as objects relating to the state of the moving object 2. Object OB31 is a first object relating to the current state of the moving object 2, and object OB32 is a second object relating to the predicted future state of the moving object 2. In the example of FIG. 16, object OB31 is similar to object OB21 in FIG. 15 and indicates the degree of deterioration as the current state of each part. Object OB32 indicates the degree of deterioration as the predicted future state of each part. In the example of FIG. 16, the degree of deterioration of each part is currently normal and is below threshold TH1, but it is predicted that the deterioration of the thermal energy recovery system will progress over time and the degree of deterioration will exceed threshold TH1.

[0058] The image shown in FIG. 17 includes radar chart objects OB33 and OB34 as objects relating to the state of the moving object 2. Object OB33 is a first object relating to the current state of the moving object 2, and object OB34 is a second object relating to the future predicted state of the moving object 2. In the example of FIG. 17, objects OB33 and OB34 indicate the deterioration levels of the respective parts of the center of gravity, right front tire, left front tire, right rear tire, left rear tire, and gearbox of the moving object 2. For example, the state of each part is normal if the deterioration level is equal to or lower than a threshold value TH1, abnormal if the deterioration level is equal to or higher than a threshold value TH2, and normal but approaching an abnormal state if the deterioration level is between the threshold values ​​TH1 and TH2. In the example of FIG. 17, the deterioration levels of each part are currently normal and equal to or lower than the threshold value TH1, but deterioration of the left rear tire progresses over time, and its deterioration level reaches the threshold value TH2. It is predicted to exceed TH1 and reach a threshold of TH2. It is possible to predict the state of tire wear, flat spots on the tire tread, blisters, which are caused when the tire temperature rises too high, causing the inside to boil or vaporize, and the centrifugal force of the tire rotating at high speed causes the tread to swell or produce bubbles, and the degree of deterioration in tire performance, or the progression of degradation. It will also be possible to estimate the displacement of the "friction circle" to determine a decrease in tire grip.

[0059] The image shown in FIG. 18 includes radar chart objects OB35 and OB36 as objects relating to the state of the moving object 2. Object OB35 is a first object relating to the current state of the moving object 2, and object OB36 is a second object relating to a predicted future state of the moving object 2. In the example of FIG. 18, objects OB35 and OB36 indicate the deterioration levels of the respective components of the moving object 2's center of gravity, right front suspension, left front suspension, right rear suspension, left rear suspension, and gearbox. For example, the state of each component is normal if the deterioration level is equal to or less than a threshold value TH1, abnormal if the deterioration level is equal to or greater than a threshold value TH2, and normal but approaching an abnormal state if the deterioration level is between the threshold values ​​TH1 and TH2. In the example of FIG. 18, the deterioration levels of each component are currently normal and equal to or less than the threshold value TH1, but the deterioration of the left rear suspension is predicted to progress over time and exceed the threshold value TH1.

[0060] The image information shown in FIG. 19 includes a level meter object OB41, which is an object relating to the status of the moving object 2, and level meter objects OB42 to OB46, which are objects relating to the status of the other five moving objects. The object OB41 indicates the deterioration level of the power unit of the moving object 2, and the objects OB42 to OB46 indicate the deterioration levels of the power units of the other five moving objects. For example, the status of each power unit is normal if the level is equal to or lower than a threshold TH1, abnormal if the level is equal to or higher than a threshold TH2, and normal but approaching an abnormal state if the level is between the thresholds TH1 and TH2. For example, in the objects OB41 to OB46, portions of the objects whose levels are equal to or lower than the threshold TH1 may be displayed in green, portions whose levels are between the thresholds TH1 and TH2 may be displayed in yellow, and portions whose levels are equal to or higher than the threshold TH2 may be displayed in red. That is, the objects OB41 to OB46 may be objects resembling patrol lights. For example, the other five mobile bodies may each be equipped with a display device 1, and the display device 1 equipped on mobile body 2 may acquire information on the degree of deterioration of the power unit from the display devices 1 equipped on the other five mobile bodies.

[0061] The image shown in FIG. 20 includes a graph object OB51 that includes a normal region A1 and a fault region A2 as an object related to the state of the mobile unit 2. In the example of FIG. 20, the graph is similar to the graph shown in FIG. 12 and shows the curve of an exponential deterioration model. The region where the health indicator value is greater than 0.3 corresponds to the normal region A1 where the mobile unit 2 is normal, and the region where the health indicator value is 0.3 or less corresponds to the fault region A2 where the mobile unit 2 is faulty. The example of FIG. 20 shows that if the current time is 300 minutes, the remaining life is estimated to be 520 minutes (= 820 minutes - 300 minutes).

[0062] If the mobile object 2 is a connected car, an autonomous vehicle, or an equivalent vehicle, the mobile object 2 comprises a control unit, a memory unit, a communication unit, and an operating unit. The control unit is a control device that controls the mobile object 2. The memory unit is a storage device that stores various information, and stores information for connecting to a server system on the cloud, information about the mobile object 2, circuit information, map information, etc. The communication unit transmits and receives various information based on the control of the control unit, and comprises a transmitting unit and a receiving unit. The communication unit transmits and receives various information based on the control of the control unit, and is configured to transmit and receive information using a remote measurement method ( Using telemetry (also called telemetering), measurement data is transmitted to facilities such as pit wall stands, pit garages, factories, or laboratories. For example, when a warning light comes on on the mobile unit 2, the transmitting unit transmits information related to the malfunction or abnormality of the mobile unit 2, and the receiving unit receives diagnostic information and countermeasure information for the malfunction information transmitted from the server system. The server system notifies the facility of the malfunction information, and engineers and supervisors perform various analyses as mentioned above based on the measurement data transmitted along with the malfunction information, and give appropriate advice such as whether to continue or stop the drive, or whether to take refuge on the shoulder of the road. Communication between the communication unit and the server system is carried out using wireless access technology. This is done using Radio Access Technology (RAT).

[0063] RATs include WiMAX (Worldwide Interoperability for Microwave Access), GSM (Global System for Mobile Communications) (2G), GSM EDGE Radio Access Network (GERAN), General Packet Radio Service (GPRS), Universal Mobile Telecommunications System (UMTS) (3G) based on basic Wideband-Code Division Multiple Access (W-CDMA), High-Speed ​​Packet Access (HSPA), Long Term Evolution (LTE), LTE-Advanced, and enhanced LTE (eLTE), which is an evolution of LTE that connects to a 5G core. LTE is also known as evolved UMTS Terrestrial Radio Access (EUTRA) or evolved UMTS Terrestrial Radio Access Network (EUTRAN).

[0064] 6.Display method 21 is a flowchart showing the steps of the display method of this embodiment. The display method of this embodiment is executed, for example, by the processing circuit 100 of the display device 1 operating in accordance with a display processing program 111. In other words, the display processing program 111 is a program that causes the display device 1, which is a computer, to execute each step of the flowchart shown in FIG.

[0065] As shown in FIG. 21, first, in a measurement data acquisition step S10, the processing circuit 100 functions as a measurement data acquisition circuit 101 and acquires the measurement data measured by each sensor 3.

[0066] Next, in a feature extraction step S20, the processing circuit 100 functions as a feature extraction circuit 102 to extract features from each measurement data acquired in step S10.

[0067] Next, in a state information generating step S30, the processing circuit 100 functions as the state information generating circuit 103, and generates information relating to the state of the moving object 2 based on the feature amounts extracted from each measurement data in step S20.

[0068] Next, in an image information output step S40, the processing circuit 100 functions as the image information output circuit 104, outputs image information including information relating to the state of the moving object 2 generated in step S30, and displays it on the display unit .

[0069] Then, the processing circuit 100 repeats steps S10 to S40 until the display process ends in step S50.

[0070] 7. Action and Effects As described above, the display device 1 of this embodiment allows a user to objectively recognize the state of the moving object 2, which changes due to vertical vibrations caused by bouncing, porpoising, and the like, based on the object related to the state of the moving object 2 included in the image information. In particular, the AI ​​model predicts future values ​​of the measurement data and extracts feature quantities, allowing the user to objectively recognize the state of the moving object 2 based on the image information generated based on the feature quantities. For example, if the object is a radar chart showing the states of multiple parts of the moving object 2, the user can easily visually recognize the states of the multiple parts. Furthermore, the image information includes thresholds TH1 and TH2 for determining whether or not there is an abnormality in the moving object 2, allowing the user to easily determine whether or not there is an abnormality in the moving object 2.

[0071] Furthermore, according to the display device 1 of this embodiment, the image information includes a first object related to the current state of the moving object 2 and a second object related to the predicted future state of the moving object 2, thereby allowing the user to objectively recognize the predicted future state of the moving object 2. Furthermore, the image information includes information on the time required from the current state of the moving object 2 to the predicted future state of the moving object 2, thereby allowing the user to objectively recognize the time required to reach the predicted future state of the moving object 2. Furthermore, if the object related to the state of the moving object 2 is a graph including a normal region indicating the normal state of the moving object 2 and a fault region indicating the fault state of the moving object 2, the user can objectively recognize whether or not the moving object 2 is faulty.

[0072] The present invention is not limited to the present embodiment, and various modifications are possible within the scope of the present invention.

[0073] The above-described embodiment and modifications are merely examples, and the present invention is not limited to these. For example, the embodiments and modifications can be combined as appropriate.

[0074] The present invention includes configurations that are substantially the same as the configurations described in the embodiments, for example, configurations with the same functions, methods, and results, or configurations with the same purpose and effects. The present invention also includes configurations that replace non-essential parts of the configurations described in the embodiments. The present invention also includes configurations that achieve the same effects or purposes as the configurations described in the embodiments. The present invention also includes configurations that add publicly known technology to the configurations described in the embodiments.

[0075] The following can be derived from the above-described embodiment and modifications.

[0076] One aspect of the display device is measuring a physical quantity generated in the moving body due to vibration in a direction intersecting a plane on which the moving body moves, and displaying image information including information relating to the state of the moving body, which is generated based on feature quantities extracted from the measured measurement data, on a display unit; The image information is The object includes an object related to the state of the moving body.

[0077] With this display device, the user can objectively recognize the state of the moving body that changes due to vibrations in a direction that intersects with the plane on which the moving body moves, based on objects related to the state of the moving body contained in the image information.

[0078] In one aspect of the display device, The feature amount is The AI ​​model that learns to predict future values ​​of the measurement data may be extracted from the measurement data.

[0079] With this display device, the AI ​​model predicts future values ​​of the measurement data and extracts features, allowing the user to objectively recognize the condition of the moving object based on the image information generated based on the features.

[0080] In one aspect of the display device, The image information may include a threshold value for determining whether or not there is an abnormality in the moving object.

[0081] This display device allows the user to easily determine whether or not there is an abnormality in the moving object.

[0082] In one aspect of the display device, the object is a first object related to a current state of the moving object, The image information may further include a second object relating to a predicted future state of the moving object.

[0083] This display device allows the user to objectively recognize the predicted future state of the moving object.

[0084] In one aspect of the display device, The image information may include information about the time it takes from the current state to the predicted future state.

[0085] This display device allows the user to objectively recognize the time remaining until the mobile object reaches a predicted future state.

[0086] In one aspect of the display device, The object may be a graph including a normal region indicating a normal state of the mobile object and a fault region indicating a fault state of the mobile object.

[0087] This display device allows the user to objectively recognize whether or not there is a malfunction in the mobile object.

[0088] In one aspect of the display device, The object may be a radar chart showing the states of a plurality of parts of the moving object.

[0089] This display device allows the user to easily recognize the states of multiple parts of the moving object.

[0090] In one aspect of the display device, The vibration in a direction intersecting the plane of movement of the moving body may be vibration caused by a porpoise phenomenon resulting from the balance between lift and counter-lift acting on the moving body.

[0091] This display device allows the user to objectively recognize the state of the moving object that changes due to vibrations caused by the porpoise phenomenon.

[0092] In one aspect of the display device, The vibration in a direction intersecting the plane on which the moving body moves may be vibration caused by a bouncing phenomenon of the moving body.

[0093] This display device allows the user to objectively recognize the state of the moving object that changes due to vibrations caused by the bouncing phenomenon.

[0094] In one aspect of the display device, The object is The object may be related to at least one of the state of the center of gravity, suspension, tires, power unit, and driver seat of the moving object.

[0095] This display device allows the user to easily recognize the state of at least one of the center of gravity, suspension, tires, power unit, and driver seat of the moving body.

[0096] One aspect of the display method is measuring a physical quantity generated in the moving body due to vibration in a direction intersecting a plane on which the moving body moves, and displaying image information including information relating to a state of the moving body, which is generated based on feature quantities extracted from the measured measurement data, on a display unit; The image information is The object includes an object related to the state of the moving body.

[0097] According to this display method, the user can objectively recognize the state of the moving body that changes due to vibrations in a direction that intersects with the plane on which the moving body moves, based on the objects related to the state of the moving body contained in the image information. [Explanation of symbols]

[0098] 1...display device, 2...mobile body, 3, 3a to 3x...sensors, 100...processing circuit, 101...measurement data acquisition circuit, 102...feature extraction circuit, 103...status information generation circuit, 104...image information output circuit, 110...memory circuit, 111...display processing program, 120...operation unit, 130...display unit, 140...sound output unit, 150...vibration unit, 160...communication unit, 201...encoder, 202...decoder

Claims

1. measuring a physical quantity generated in the moving body due to vibration in a direction intersecting a plane on which the moving body moves, and displaying image information including information relating to the state of the moving body, which is generated based on feature quantities extracted from the measured measurement data, on a display unit; The image information is A display device including an object relating to the state of the moving body.

2. In claim 1, The feature amount is A display device in which an AI model that learns to predict future values ​​of the measurement data is extracted from the measurement data.

3. In claim 1, A display device, wherein the image information includes a threshold value for determining whether or not there is an abnormality in the moving object.

4. In claim 1, the object is a first object related to a current state of the moving object, A display device, wherein the image information further includes a second object relating to a predicted future state of the moving object.

5. In claim 4, A display device, wherein the image information includes information about the time it takes from the current state to the predicted future state.

6. In claim 1, A display device, wherein the object is a graph including a normal region indicating a normal state of the mobile object and a fault region indicating a fault state of the mobile object.

7. In claim 1, A display device, wherein the object is a radar chart showing the states of a plurality of parts of the moving object.

8. In any one of claims 1 to 7, A display device, wherein the vibration in a direction intersecting the plane of movement of the moving body is vibration caused by a porpoise phenomenon resulting from a balance between lift and counter-lift acting on the moving body.

9. In any one of claims 1 to 7, A display device, wherein the vibration in a direction intersecting a plane on which the moving body moves is vibration caused by a bouncing phenomenon of the moving body.

10. In any one of claims 1 to 7, The object is A display device that is an object related to at least one state of the center of gravity, suspension, tires, power unit, and driver seat of the moving body.

11. measuring a physical quantity generated in the moving body due to vibration in a direction intersecting a plane on which the moving body moves, and displaying image information including information relating to a state of the moving body, which is generated based on feature quantities extracted from the measured measurement data, on a display unit; The image information is A display method including an object relating to the state of the moving object.

Images