Visibility evaluation system
The integration of a biomechanical model with a visual field judgment model in 3D CAD systems addresses subjective evaluations and high computational loads by optimizing eye-head coordinated movements, enhancing visibility evaluation quality and adaptability.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- SUZUKI MOTOR CORP
- Filing Date
- 2022-07-26
- Publication Date
- 2026-07-01
Smart Images
Figure 0007883231000015 
Figure 0007883231000016 
Figure 0007883231000017
Abstract
Description
Technical Field
[0001] The present invention relates to a visibility evaluation system for 3D CAD using a biomechanical model.
Background Art
[0002] Conventionally, in product design using 3D CAD, evaluations regarding visual effects such as visibility and evaluations regarding practical effects such as operability and ease of handling have been performed separately. Moreover, evaluations by testers could not exclude the influence of subjectivity and it was difficult to obtain a quantitative evaluation.
[0003] Therefore, the present inventors performed a simulation of a confirmation operation for a visibility target point defined in a virtual 3D space using a biomechanical model, reflected the posture of the biomechanical model in a human model, calculated the visibility of the visibility target point from a field-of-view image of the human model corresponding to the line of sight of the biomechanical model, calculated the burden level of the biomechanical model required for the confirmation operation, and created a visibility evaluation system for evaluating easy visibility from the integrated value of the burden level and the final value of the visibility (see Patent Documents 1 to 3).
Prior Art Documents
Patent Documents
[0004]
Patent Document 1
Patent Document 2
Patent Document 3
Summary of the Invention
Problems to be Solved by the Invention
[0005] In this visibility evaluation system, based on a configuration that autonomously simulates the confirmation operation performed by a human by applying perturbations to a biomechanical model and generating a posture with the minimum burden and maximum visibility, the operation generates a posture. This biomechanical model includes an eye movement model that determines the eye posture in conjunction with the head movement of the skeletal model, and by adding the angular velocity output from the eye movement model to the body movement load as a movement load, a coupled movement of eye movement and body movement is generated.
[0006] By the way, the eyeball realizes eye movement by the contraction of the extraocular muscles, and as the movement increases, the burden also increases, so head movement is induced. Such eye-head coordinated movement may include a state that is actually impossible in the perturbation stage only by adding the eye movement load to the body movement load, and it has been found that there is room for further improvement in reproducibility and reduction of computational load.
[0007] The present invention has been made in view of the above actual situation, and its object is to enhance the compatibility of the simulation of the confirmation operation for design elements with the situation in the visibility evaluation in product design by 3DCAD, and to improve the quality of the visibility evaluation.
Means for Solving the Problem
[0008] In order to solve the above problems, the present invention is a visibility evaluation system for an object by 3DCAD, a biomechanical model for simulating a confirmation operation for a visual recognition target point in a virtual 3D space in which the object and its visual recognition target point are defined, a visual field judgment model for calculating the visibility for the visual recognition target point in the visual field image in the line of sight of the biomechanical model, a general control unit that couples the biomechanical model and the visual field judgment model under the condition that the burden of the biomechanical model is minimized and the visibility is maximized, and is configured to calculate easy visibility based on the burden and the visibility. The biomechanical model includes a musculoskeletal model and an eye movement model that determines eye position in conjunction with head movement in the musculoskeletal model, and the load is calculated from the physical load in the musculoskeletal model and the eye load in the eye movement model. The aforementioned eye load includes a load component corresponding to the eye angle representing the eye posture, and the load component is configured to rapidly increase in load characteristics from around a predetermined transition angle, and the system is configured such that eye movement or eye-head coordinated movement occurs when the eye angle is given by the visual target point. [Effects of the Invention]
[0009] As described above, the visibility evaluation system according to the present invention includes a load component corresponding to the eyeball angle in the eye load, and the load component is configured to have a load characteristic that increases sharply from around a predetermined transition angle. This configuration allows for: (i) for visual targets that can be perceived with a small eyeball angle within the transition angle, head movement is suppressed under conditions that minimize the burden and maximize visibility, generating visual actions primarily involving eye movement; and (ii) for visual targets that require an eyeball angle larger than the transition angle, a transition to visual actions involving head movement (and even body movement) occurs autonomously under conditions that minimize the burden and maximize visibility. This configuration is advantageous for improving the adaptability of the simulation of confirmation actions to the situation, from confirmation actions involving only eye movement or eye-head coordination to confirmation actions involving body movement, and for improving the quality of visibility evaluation. [Brief explanation of the drawing]
[0010] [Figure 1] This is a block diagram showing a visibility evaluation system according to an embodiment of the present invention. [Figure 2] This is a schematic side view showing a simulation using the visibility evaluation system according to an embodiment of the present invention. [Figure 3] This is a schematic plan view showing the coupling of a biomechanical model and a field of view judgment model through data transfer in a visibility evaluation system according to an embodiment of the present invention. [Figure 4]This is an ER diagram showing the data structure used in the visibility evaluation system according to the present invention. [Figure 5] This is a flowchart showing the processing of the visibility evaluation system according to an embodiment of the present invention. [Figure 6] This flowchart describes the processing of the visibility evaluation system according to the embodiment of the present invention, divided into a biomechanical model and a field of view judgment model. [Modes for carrying out the invention]
[0011] Embodiments of the present invention will be described in detail below with reference to the drawings.
[0012] 1. Overview of the Evaluation System Figure 1 is a block diagram showing a visibility evaluation system 1 according to an embodiment of the present invention. In this embodiment, the evaluation system 1 uses 3D CAD data of an instrument panel located in the front of the vehicle cockpit as evaluation target data 2, and as shown in Figures 2 and 3, it uses a biomechanical model 10 to simulate the confirmation action for visualizing a target point 20, and evaluates the visibility, including the ease of doing so.
[0013] Evaluation System 1 includes a biomechanical model 10 and a visual field judgment model 50, and separate programs exist for the operation of each. Each program and data, as well as the operation program that comprehensively operates each program, are stored in an external storage device of a computer (not shown), and are loaded into the main memory (RAM), where the CPU performs calculations to enable the evaluation system to function.
[0014] The biomechanical model 10 consists of a musculoskeletal model 3 and an eye movement model 4. By integrating these, it generates body movements and eye movements for visual field judgment, calculates the burden 13 required for these movements, and reflects the posture data of the eyeballs 5 obtained as a result of each movement in the camera 55 of the visual field judgment model 50 to calculate the visibility 15 for the target point 20. Finally, the evaluation unit 17 evaluates the ease of viewing (ease of recognition), including the ease of confirmation, based on the burden 13 and visibility 15.
[0015] The visibility score 15 calculated by the visual field judgment model 50 is fed back to the central control unit 11 of the biomechanical model 10, and together with the burden score 13, becomes part of the motion norm potential that optimizes the confirmation action of the biomechanical model 10 through the minimization calculation of the visibility objective function 16. In this way, the biomechanical model 10 and the visual field judgment model 50 operate independently but are coupled by sharing or exchanging data with each other, enabling simulations that simultaneously achieve minimization of the burden score 13 in the biomechanical model 10 and maximization of the visibility score 15 in the visual field judgment model 50.
[0016] In other words, the two models are coupled in such a way that the optimal operation to minimize the burden 13 in the biomechanical model 10 leads to an improvement in visibility 15 in the visual field judgment model 50, and the improvement in visibility 15 in the visual field judgment model 50 leads to the optimal operation to minimize the burden 13 in the biomechanical model 10.
[0017] In particular, the eye load in eye movement model 4 includes a load component corresponding to the eye angle representing eye posture, and this load component is set to rapidly increase from around a predetermined transition angle. This load characteristic enables autonomous simulation of confirmation actions that induce eye movements or eye-head coordinated movements.
[0018] Furthermore, by formalizing the transition from eye movements to coordinated eye-head movements, unrealistic movement states can be excluded in advance, thereby reducing the computational load.
[0019] The sharing or exchange of data between the two models is done by accessing database 8 as shown in Figure 4, but it can also be done in any other way, such as data communication or the transfer of data files through file operations.
[0020] Database 8 includes simulation data 80 that stores conditions and results for each simulation (trial), behavior data 81 that stores data for each behavior process in each simulation (trial), and perturbation data 82 that stores data for perturbation processes that were virtually performed to obtain each behavior process. It is configured as a relational database in which a one-to-many relationship is set between the primary key (PK) of each data table and the foreign key (FK) of the corresponding data table.
[0021] Simulation data 80 stores input conditions for each simulation (trial). For example, in the illustrated example, it stores support structure information corresponding to the CAD data of the cockpit in which the simulation is performed (first support point (hip point; HP) on the seat 6, second support points (left and right gripping points; coordinates (x, y, z)) on the handle 7, body size setting information of the person to be simulated (height, weight), CAD data to be evaluated (name, identification code), and coordinates of the visual target point (coordinates (x, y, z)). It also stores visibility evaluation and rendering information as a result of the simulation process.
[0022] Although not shown in the diagram, the basic information such as support structure information (cockpit CAD data), body size setting information, and evaluation target CAD data, as well as rendering data, are basically registered in separate data tables and referenced by relationships set in identification codes (foreign keys). Furthermore, when performing simulations by transferring data files, data files in an appropriate format corresponding to simulation data 80, behavior data 81, and perturbation data 82 are constructed, and data is added and updated. It is also possible to set the system so that the perturbation data 82 required for each behavior process is not retained after the behavior data 81 is acquired.
[0023] As will be described in detail later, the behavior data 81 and perturbation data 82 are performed by repeatedly executing a perturbation process, which involves virtually operating the models a number of times equal to the number of state variables to determine the optimal behavior and calculate the burden B, deviation D, visibility R, and visibility objective function Fo, and then actually operating the models for a unit of time under conditions that minimize the visibility objective function Fo.
[0024] Therefore, first, a behavior No. (ID) is assigned to the behavior data 81 by automatic numbering or the like, and then, for each behavior No., a perturbation No. for the state variables is assigned to the perturbation data 82, and perturbations are performed on the musculoskeletal model 3 and the eye movement model 4 for each state variable. The skeletal link posture, eye posture, burden B, and deviation D obtained as a result of each perturbation are stored in the perturbation data 82, the eye posture is reflected in the camera 55 of the visual field judgment model 50, and the visibility R calculated from the visual field image is stored in the perturbation data 82.
[0025] Next, the conditions for minimizing the visibility objective function Fo, calculated based on the burden level B, deviation level D, and visibility level R, are stored in the behavior data 81. The behavior process is then executed accordingly, and the skeletal link posture and eyeball posture are registered as keyframe information. By accumulating this keyframe information, it becomes possible to generate a 3DCG video of the field of view image corresponding to the confirmation action (and a 3DCG video viewed from an arbitrarily set viewpoint). In addition, along with the execution of the behavior process, the visibility objective function Fo obtained in the previous behavior process is updated and stored in the behavior data 81. This visibility objective function Fo becomes the basic information for the final visibility evaluation in the simulation.
[0026] 2. Details of the visual field judgment model The field of view judgment model 50 defines evaluation target data 2 having a visual target point 20, a human body model 53 corresponding to a biomechanical model (musculoskeletal model 3, eye movement model 4), and a camera 55 in a 3D coordinate system (virtual data space). It reflects the body movements and eye movements (body posture and eye posture) generated by the biomechanical model into the human body model 53 and the camera 55, and obtains the degree of visibility of the visual target point 20 in that state.
[0027] This camera 55 is a viewpoint set in a 3D coordinate system (equivalent to a human eye), and the camera image becomes the field of view of the human (human body model 53). By registering the posture of the human body model 53 as keyframes moment by moment, a video can ultimately be created. By setting an external viewpoint, it is also possible to visually observe the body's movements.
[0028] The following describes the visibility calculation process using the field of view judgment model 50. The entire process is automated by the program. Since it is based on image processing in a 3D data space, general-purpose 3DCG applications can also be used.
[0029] As described above, the evaluation system 1 of this embodiment uses 3D CAD data of the instrument panel located in the front of the vehicle's cockpit as evaluation target data 2, and as shown in Figures 2 and 3, it uses a biomechanical model 3 corresponding to the driver to simulate the confirmation action for visualizing the target point 20, and evaluates the visibility, including the ease of doing so. The field of view judgment model 50 calculates the degree of visibility of the target point 20 at each point in time of the confirmation action (perturbation and behavior).
[0030] Humans have a horizontal field of view of approximately 200°, but the range of the field of view that has a certain influence on cognitive behavior and allows for effective use of information is only about 2° from the central field of vision. Therefore, it is necessary to capture the target point 20 within the field of view through eye movements. Furthermore, although the maximum range of eye movement is about 55°, large eye movements are strenuous, so the reflex eye movements called saccades are 15° or less. If a greater change in viewpoint is required, not only the eyes but also the head actively moves.
[0031] Therefore, even when the visual target point 20 is set within the field of view, confirmation actions involving bodily movements, including eye movements and eye-head coordination, will be triggered. Even when the visual target point 20 is outside the field of view, if prior knowledge of the location of the visual target point 20 is obtained, confirmation actions will be triggered in that direction.
[0032] In the illustrated example, the target point 20 is assumed to be a switch located on the lower right side of the instrument panel 2. This switch is hidden by the steering wheel 7 and is difficult to see directly from the driver's (3) viewpoint (5) while seated in the seat 6. As shown in Figure 3, it becomes gradually visible through a checking action involving upper body movement to the right and head rotation.
[0033] In order to determine the visibility of the target point 20, including cases where it cannot be directly seen at its initial position, a gradient sphere 21 is defined as a three-dimensional object obtained by radially expanding the target point 20. This makes it possible to calculate the visibility if at least a portion of the gradient sphere 21 is included in the field of view image.
[0034] Furthermore, in order to weight the visibility between the center of the gradient sphere 21, i.e., the vicinity of the viewing target point 20, and the periphery, specific pixel information is gradually reduced according to the distance from the center of the gradient sphere 21. In this embodiment, the R value, which indicates the intensity (luminance) of red, is used as specific pixel information, and the gradient sphere 21 has a radial gradient such that the red color becomes darker towards the center and lighter towards the periphery.
[0035] The pixel information used in the field of view judgment model 50 is based on the "RGBA" color model, which adds an alpha channel (A) for transparency to the three primary colors of red (R), green (G), and blue (B). RBGA values are each between 0 and 1.0 and are represented as [R, B, G, A]. The gradient sphere 21 placed at the visual target point 20 has an R value that gradually decreases from the center R value = 1.0 (maximum) to the surface R value = 0 (minimum). The G and B values are 0, and since the entire sphere is transparent, the A value is 1.0. Therefore, the RGBA of the gradient sphere 21 is: [R,G,B,A]=[0~1.0,0,0,1.0] It will be expressed within the range of [this range].
[0036] In calculating visibility in the field of view judgment model 50, all solids other than the gradient sphere 21 (evaluation target data 2, handle 7, sheet 6, person model 53, etc.) are set to black and opaque (not reflecting light sources, [R,G,B,A]=[0,0,0,0]), so that only the gradient sphere 21, partially obscured by black solids (handle 7, etc.), is visible in the field of view image of the camera 55.
[0037] In this state, by summing the R values of all pixels in the field of view image of camera 55, it is possible to quantify how much of the red gradient sphere 21 is visible, and based on this, the visibility of the target point 20 can be determined.
[0038] Because this visibility (total R value) uses a gradient sphere 21, it can be quantified even when the target point 20 is not directly visible. Since it can capture the improvement in visibility as the state moves from being directly invisible to being visible, it is ideal for evaluating ease of viewing (ease of recognition), including confirmation actions. Furthermore, as will be detailed later, it is significant because it is fed back into the biomechanical model 10 and reflected in the motion norm potential that directs the motion (confirmation actions) of the biomechanical model 3.
[0039] Furthermore, the specific pixel information of the gradient sphere 21 used in the field of view judgment model 50 can also use the brightness of green or blue colors other than red, and it is also possible to use information other than brightness. For example, red dots can be defined inside a transparent gradient sphere at a density corresponding to the distance from the center, and the sum of the R values can be taken.
[0040] 3. Details of the biomechanical model The biomechanical model 10 consists of a musculoskeletal model 3 and an eye movement model 4. Basically, body movements are generated in the musculoskeletal model 3, and eye movements are triggered in conjunction with these body movements. First, we will explain the process by which the musculoskeletal model 3 generates body movements for the action of confirming the visual target point 20.
[0041] 3.1 Musculoskeletal Model (Body Mechanics Model) Musculoskeletal Model 3 is a rigid link model that corresponds human body segments to three-dimensional rigid links, with 17-20 links and 39-43 degrees of freedom in the entire body.
[0042] In other words, the musculoskeletal model 3 consists of the head 31, cervical vertebrae 32, chest (thoracic vertebrae) 33, lumbar vertebrae, pelvis 35, left and right legs 36 (feet, lower legs, and feet 36c), left and right clavicles, and left and right arms 37 (upper arms, forearms, and hands). For example, the elbow joint between the upper arm and forearm has one degree of freedom for flexion, and the wrist joint between the forearm and hand 37d has three degrees of freedom for flexion (forward and backward, inward and outward) and twisting.
[0043] In the embodiment, the values of body parameters such as the mass and moment of inertia of each body segment are standardized to those of a typical adult male, and the body parameters are scaled based on data such as height and weight in the input data (simulation data). In addition, passive resistance acts on each joint due to the joint structure, such as the joint soft tissue.
[0044] 3.2 Definition of a Point of Representative Motion The musculoskeletal model 3 is preferably biomechanically valid for reproducing various body movements, and is also simple and computationally intensive. Therefore, representative movement points are set for each part of the body, and the movement trajectory of each representative movement point can be defined. In this embodiment, since the body movement is for confirming a visual target point 20 in a seated position, two points, the eye coordinates of the head 31 (which may be the midpoint coordinate of the left and right eyeballs 5 or the center coordinate of the head 31) and the chest 33 (which is also the reference area), are set as the representative movement points p rep Defined as (3 degrees of freedom), the representative angle of motion q of the neck 32. rep By defining it as (3 degrees of freedom), we defined body movement using a total of 9 variables.
[0045] The three-dimensional position of the motion representative point is described by a spatial coordinate system, and body motion is defined as movement from the initial position of the motion representative point to the target (endpoint) position. This coordinate position can also be defined as a relative coordinate value with respect to other motion representative points. For example, if the center coordinate of the head 31 is used as the motion representative point, the confirmation action can be defined as the movement of this point, and the eyeball coordinates can also be defined with respect to this motion representative point.
[0046] 3.3 Posture generation using quasi-static motion normative potential For the nine variables of the motion representative points and motion representative angles that define body movement, the joint degrees of freedom of the skeletal model 3 are redundant. To determine the joint angles existing in this redundancy in an autonomous and physiologically appropriate manner, the following inverse model-based mechanical calculation algorithm using a quasi-static motion norm potential is used.
[0047] First, regarding the joint angle q of each joint at a certain point in discrete time as a state variable, based on angular velocity, angular acceleration, and external force, through inverse dynamics calculation for the equation of motion of the body's skeletal model 3, the virtual joint drive torque n that realizes its posture and movement v is defined as follows.
Equation
[0048] Next, if the joint angle q at the current time is determined, the position p of the motion representative point defined on the skeletal model 3 determined thereby rep (q), the difference between the visually recognized target point position obtained as setting information and the target motion representative point position p determined by the following optimization calculation ser is defined as the motion norm potential UΔp for the motion representative point. Similarly, for the motion representative angle q rep (q) on the skeletal model 3 at the current time and the target motion representative angle q ser the difference is defined as the motion norm potential UΔq for the motion representative angle. Then, the motion norm potential UΔp and the motion norm potential UΔq are as follows.
Equation
[0049] In addition to these motion norm potentials, the virtual joint torque n from the inverse dynamics calculation of Equation (3.1) vWe consider the motion norm potential for physical exercise loads defined by the sum of the squares of these factors, and define the overall motion norm potential U(q) as a weighted linear sum of these factors.
number
number
[0050] The virtual joint drive torque n calculated using the inverse dynamics method described above. v By adding the effect of the gradient ∇U of the above-mentioned motion norm potential, the actual joint drive torque n real The equation is as follows:
number
[0051] 3.4 Generating bodily motion using forward dynamics calculations The joint drive torque n is determined by the motion norm potential as described above. real In addition, forward dynamics calculations are performed to generate body motion corresponding to the given representative point position of motion. Here, a state variable v corresponding to the joint drive torque is assumed, and the resulting motion reference potential U fwd We define (v) and define the dynamics that decrease it as follows:
number
[0052] Motion norm potential U fwd This is determined by forward dynamics calculation as follows. First, given a state variable v equivalent to the joint drive torque, the acceleration at the current time (time t) can be estimated as follows.
number
[0053] Here, M is the inertia matrix, h is the Coriolis force, and f ext This is an external force. From this acceleration, the displacement and velocity of the body at time t+Δt can be estimated by the time integral shown in the following equation.
number
[0054] From these, the velocity and position of the representative point of motion at time t+Δt can be estimated as follows:
number
[0055] The above f1 and f2 are functions that determine the position and velocity of the representative point of motion from joint displacement and velocity through forward kinematic calculations of the body structure. The motion reference potential U fwd This is the predicted representative motion point position P obtained by forward dynamics calculations. pre The difference between this and the target motion representative point position Pser is defined as follows: A2 and A3 are weight coefficient matrices.
number
[0056] The motion-referencing potential U defined by the above equation fwd This does not consider the coupling with eye movements. The coupling between body movements and eye movements is achieved by an optimization calculation that minimizes the visual objective function Fo, which will be described later.
[0057] 3.5 Eye Movement Models Eye movement is achieved through the contraction of six types of muscles called extraocular muscles. The greater the movement, the more fatigued the extraocular muscles become, and the more strain a person feels on their eyes. Therefore, the degree of strain on the eyeball is defined as follows, based on the angular velocity and posture of the eyeball during eye movement.
number
[0058] Eyeball posture θ eye is the angle of the dot product of the line of sight when the target point is viewed and the neutral position of the eyeball 5, and in this embodiment, the neutral position corresponds to the reference position. Therefore, when the simulation is started with the neutral position (reference position) as the initial position, the spatial coordinates of the viewing target point 20 are obtained, which causes eye movements to be generated in an attempt to direct the line of sight towards the viewing target point 20.
[0059] This is defined not as a reflex movement associated with the acquisition of a visual target within the field of vision, but as an active eye movement in which, when a person performs a visual action, if they have prior knowledge of the location of the visual target, the brain issues a command to direct their gaze in that direction, resulting in eye movement, and head movement (and even body movement) occurs to compensate for the strain and / or lack of visibility corresponding to the eye angle.
[0060] Furthermore, in equation (3.14) above, the dependent term for eye position (eye angle) is defined, and the eye load is equal to the eye angle θ. eye is coefficient k 13The load characteristics are set such that the increase is relatively gradual up to a predetermined transition angle given by [the function], but increases exponentially in the region greater than the transition angle.
[0061] Therefore, (i) when the target point is near the line of sight in the reference posture, eye movements (eye position θ) that do not involve body or head movements are considered. eye (ii) Maximum visibility R is obtained with only (ii) and the confirmation action is completed without movement of the representative point of motion. However, if (ii) the target point of visual recognition is in a region where the transition angle is greater than that of the line of sight in the reference posture, the increased eye load (L eye Since the sum of the eye load and head movement load (neck load) reduced by head rotation is smaller than the eye load, the system transitions to eye-head coordinated movement as shown in the minimization calculation below.
[0062] The load characteristics of eye movements involved in the transition to the eye-head coordinated movement described above differ between static environments, such as when the vehicle is stationary, and driving environments, such as when the vehicle is in motion. The degree of strain on the eyes also changes according to these environments. Specifically, in a static environment, when the horizontal angle of the target point for eye movement exceeds 10°, head movement occurs, and the head angle (angle of rotation of the neck) becomes larger than the eye angle (approximately twice the eye angle). In contrast, in a driving environment, eye movement takes precedence, and active head movement does not occur until the horizontal angle of the target point is around 30°.
[0063] Therefore, in equation (3.14) above, the coefficient k gives the load characteristics (transition angle) of eye movement. 13 In a static environment, θ eye =10° (0.17 rad), θ under driving conditions eye By setting different values such that the eye load increases sharply from 30° (0.56 rad), it becomes possible to perform simulations and visibility evaluations that take the viewing environment into account.
[0064] For example, design elements installed in a vehicle's cockpit include those used in a driving environment, such as rearview mirrors for checking the surroundings while driving, and those used only in a static environment, such as a start switch for starting the engine while parked. However, by simply switching coefficients (or the formula itself), visibility evaluations can be selectively performed in simulations assuming a static environment and simulations assuming a driving environment.
[0065] Furthermore, regarding the neutral position of the eyeball, in addition to the geometrically neutral position with zero elevation / depression angle, we made it possible to set the neutral position of the eyeball to a position where the eye load is minimal regardless of visual information, i.e., a position in which there is no contraction of the extraocular muscles and the eyeball is not moving. When we conducted verification experiments with subjects to see how much the neutral position of the eyeball is tilted vertically with respect to the head coordinate system (31a), it was confirmed that in a seated position in a static environment, it is tilted downward with an elevation angle of 5 to 10° with respect to the head coordinate system. Therefore, by considering the characteristics of the neutral position of the eyeball in the vertical direction, in addition to the horizontal angle characteristics of eye movement in the driving environment and static environment described above, it is possible to perform simulations and visibility evaluations that reflect each visual environment.
[0066] 3.6 Optimization of visual perception As mentioned earlier, the body motion of musculoskeletal model 3 (rigid link model) is defined by defining the confirmation motion and eye coordinates using the representative motion points of the head 31 and chest 33, which play a central role in the confirmation motion, and the representative motion angle of the neck 32 as variables, and by applying a perturbation Δq with the joint angle q as a state variable and calculating the joint drive torque, thereby generating a quasi-static motion posture, and further calculating the motion reference potential U using forward dynamics calculations. fwd This method calculates the position of the representative point of motion and the representative angle of motion after a unit time Δt, but it does not consider coupling with eye movements.
[0067] Therefore, in visual recognition actions, considering the degree of deviation D from the reference posture, a visual recognition objective function Fo is defined as shown in the following equation, which defines the relationship between the burden B of physical movement, including eye load and body load, and the visibility R output from the visual field judgment model. By coupling physical and eye movements through optimization calculations that minimize this function, the optimal visual recognition action is generated.
number
[0068] Eyeball load L eye This is given by the aforementioned equation (3.14), and the cervical load L neck and physical load L body It is given by the following equation.
number
[0069] Degree of deviation from the reference posture D(1+k) 14 C pos The reciprocal of this value is an index of maintaining the reference posture, and when targeting visual actions in a seated position, the chest 33 can be used as the reference area to suitably reflect the degree of maintenance of the reference posture. Displacement C of the reference area pos The current position p of the chest 33 is pre , initial position p int It is given by the following equation.
number
[0070] Displacement C of the reference area set on the chest 33 pos If it is zero, it means that the visual action is performed by eye movements alone, or by eye movements and head rotation, and if the head 31 is tilted, displacement C pos This is a significant value. Therefore, the reference site can also be set to the head 31.
[0071] Furthermore, even when considering visual actions in positions other than seated, such as standing, the degree of deviation D can be taken into account using the chest 33 or head 31 as reference points to reflect the degree of maintenance of the reference posture in the simulation.
[0072] In the aforementioned equation (3.14), the ocular load L eye The eyeball posture θ is eyeBy taking this into consideration, for example, even when a target point 20 is set within the field of view, if eye movement alone would require a large eye angle, the characteristics of eye-head coordination, which avoids increasing eye load and replaces it with head movement, are reproduced through minimizing the visual objective function Fo.
[0073] Furthermore, by considering the degree of deviation D from the basic posture in the aforementioned equation (3.15), eye movements are first induced to maintain the basic posture as much as possible, and a natural confirmation action is autonomously reproduced, transitioning from eye-head coordinated movement to body movement. This is expected to improve the suitability to the simulation situation and, consequently, the quality of the visibility evaluation.
[0074] In particular, the degree of deviation D is the displacement C of the reference part. pos Since it is adjusted so that it becomes 1 when it is the smallest of zero, it is advantageous for continuously evaluating ease of recognition from confirmation actions by eye movements or eye-head coordination movements that maintain the basic posture to confirmation actions that involve body movements.
[0075] On the other hand, as mentioned above, by setting load characteristics related to the transition from eye movement to eye-head coordinated movement, it is possible to reproduce the transition from eye movement to eye-head coordinated movement by simply minimizing the eye load, neck load (head movement load), and body load corresponding to the load B in equation (3.15), and to achieve confirmation movements that maintain the basic posture as much as possible.
[0076] Furthermore, in the initial stages of visual recognition, the transition from eye movements to coordinated eye-head movements, eye load and neck load associated with head rotation are the main factors. Body movement is triggered only when visual recognition does not improve with coordinated eye-head movements. Thus, a confirmation action can be defined that maintains the basic posture as much as possible.
[0077] For example, in Figure 3, the initial posture (reference posture, neutral posture) of the biomechanical model 10 is generated, and a motion norm potential is obtained from the deviation between its line of sight 5a and the target point 20, which generates eye movements or eye-head coordinated movements. Here, under the condition of minimizing the burden B, with zero movement of the representative point of movement (zero physical load), an eye movement as shown in 5b in the figure is generated as a confirmation action to maximize the visibility R. When the eye angle θb is greater than the transition angle, the eye load becomes excessive, and under the minimization condition, it transitions to an eye-head coordinated movement, generating a head rotation movement (neck rotation movement), and as shown in 5c in the figure, the burden B is minimized so that the eye load is replaced by a head movement load. However, if the visibility R still does not improve, a physical movement is generated under the condition that maximizes the visibility R and minimizes the burden B.
[0078] 4. Processing flow of the visibility evaluation system As described above, the evaluation system 1 according to the present invention uses a biomechanical model 10 to simulate the confirmation action of the visual target point 20 in the data to be evaluated 2, and evaluates the visibility considering the ease (burden) of the confirmation action and the degree to which the reference posture is maintained. Figure 5 is a flowchart of the processing flow of the evaluation system 1 according to the embodiment.
[0079] As mentioned above, the evaluation system 1 consists of a biomechanical model 10 and a field of view judgment model 50 that operate independently while being coupled. Figure 6 shows the processing flow of the evaluation system 1 according to the embodiment, divided into processing in the biomechanical model 10 (confirmation operation simulation) and processing in the field of view judgment model 50 (visibility judgment). Figure 3 also shows some of the reference numerals for the processing in Figure 6, illustrating the data transfer between the biomechanical model 10 and the field of view judgment model 50.
[0080] In Figure 5, the last two digits of the code for each step shown in the 100s correspond to the last two digits of the code for each step shown in the 200s and 300s in Figure 6. The processing flow of evaluation system 1 will be explained below using both Figures 5 and 6.
[0081] For the visibility evaluation, first, the computers constituting the evaluation system 1 are started, and with the main program (1) that oversees the simulation and database 8 running, the biomechanical model 10 and the field of view judgment model 50 are started (steps 100, 200, 300).
[0082] Next, according to the input information of the simulation data 80, the predetermined evaluation target data 2 is loaded into the 3D data space of the field of view judgment model 50, and the visual target points 20 of the evaluation target data 2 are acquired by the biomechanical model 10 (steps 101, 201, 301).
[0083] Next, in the visual field judgment model 50, a gradient sphere 21 is set at the visual target point 20 of the evaluation data 2 (steps 102, 302). The data for the gradient sphere 21 itself is prepared in advance in the visual field judgment model 50, and the center coordinates of the gradient sphere 21 are set at the visual target point 20.
[0084] Next, posture information is read from the simulation data 80, and an initial posture (reference posture) is generated in the musculoskeletal model 3 and eye movement model 4 of the biomechanical model 10. This initial posture is then reflected in the human body model 53 of the visual field judgment model 50, and the initial position of the camera 55 is obtained (steps 103, 203, 303). In this state, a visual field image is acquired from the camera 55, and the initial value R0 of visibility is calculated.
[0085] After the above preparations are completed, the perturbation / behavior process begins. First, the motion generation module 12 calculates the state variables for the inverse dynamics calculation routine and the forward dynamics calculation routine (steps 110, 210, 310).
[0086] Next, joint drive torques are calculated by inverse dynamics calculations, and then the posture after a unit of time is estimated by integrating the acceleration over time using forward dynamics calculations. In this way, perturbations are applied to the biomechanical model 10, and the obtained posture information (skeletal link posture, eyeball posture, both as quaternions), eyeball load, and joint drive torques acting on the biomechanical system (neck load, other body loads) are acquired as the load level B, the displacement of the reference part 33 is acquired as the deviation level D from the reference posture, and these are stored in the perturbation data 82 along with the state variables (steps 111, 211).
[0087] Furthermore, the posture information obtained by the above perturbation is reflected in the human body model 53 and camera 55 of the field of view judgment model 50 (Figure 6, step 311), and the visibility R is calculated as the sum of the R values of the field of view images acquired by the camera 55 (steps 112, 312), and stored in the perturbation data 82.
[0088] The perturbation process described above is repeated for each state variable, and once all state variables have been perturbed (steps 113, 213), the process transitions to the behavior process.
[0089] First, from the state variables obtained in the perturbation process, such as the degree of burden B, the degree of deviation D, and the degree of visibility R, the state variable that minimizes the visibility objective function Fo (minimizing the purposeful burden B and D and maximizing the degree of visibility R) is calculated, and the corresponding joint drive torque is determined (steps 120, 220).
[0090] Next, the biomechanical model 10 (musculoskeletal model 3 and eye movement model 4) is actually driven by the determined state variables and joint drive torques, a posture is generated by the behavior, and posture information (skeletal link posture, eye posture, both quaternions) is stored in the behavior data 81 (steps 121, 221).
[0091] Furthermore, the posture information resulting from the above behavior is reflected in the human body model 53 and camera 55 of the field of view judgment model 50 (Figure 6, step 321), and the visibility R is calculated as the sum of the R values of the field of view images acquired by camera 55 (steps 122, 322), and stored in the behavior data 81. In addition, the posture information of the human body model 53 is registered as a keyframe (Figure 6, step 324).
[0092] In parallel with this, the cumulative value of the burden B, including the behavior process, is calculated in database 8, and the ease of viewing is calculated considering the burden B (cumulative value) by referring to the visibility R mentioned above (step 222).
[0093] Next, the visibility R is compared to a pre-set threshold (steps 130, 230, 330), and if it is determined that the threshold has not been reached, the perturbation / behavior process described above is repeated.
[0094] If the visibility R exceeds a predetermined threshold, the simulation by the biomechanical model 10 is terminated, and the ease of visibility evaluation, which takes into account the visibility R and the burden B (cumulative value) calculated in the final behavior process, is recorded in the simulation data 80 as the result of that simulation (trial No.) (Figure 6, step 231).
[0095] Simultaneously, the field of view judgment model 50 performs rendering based on the keyframe data registered up to the final behavior process (in the example, at 0.1-second intervals and 10 fps), and an evaluation video is created (Figure 6, step 331).
[0096] During rendering, body movements between keyframes are interpolated using the video creation function. Furthermore, the surface information of the evaluation target data 2 (instrument panel) and the human body model 53 may be reflected, or it may be made grayscale or similar to clearly indicate the position of the visual target point 20. In addition to the field of view from camera 55 (human body model 53), images can be created from external viewpoints set at appropriate positions, such as diagonally above and behind the human body model 53.
[0097] In the perturbation process of the above embodiment (steps 111-112, 211, 311-312), instead of reflecting the posture information obtained in the biomechanical model 10 to the human body model 53 and camera 55 of the field of view judgment model 50 for each perturbation and calculating the visibility R from the field of view image, it is also possible to acquire posture information for all (or some) state variables in the biomechanical model 10 and then transfer it all at once to the field of view judgment model 50 to acquire the field of view image and calculate the visibility R.
[0098] Furthermore, while the perturbation process in the above embodiment (steps 111-112, 211, 311-312) describes the case where perturbations are applied to the biomechanical model 10 as many times as there are state variables, it is also possible to estimate the state variables that minimize the visual objective function Fo and determine the joint drive torque for the behavior by applying multiple perturbations, fewer than the number of state variables.
[0099] Furthermore, in the above embodiment, we described a case where the perturbation / behavior process is repeated until the visibility R exceeds a predetermined threshold. However, it is also possible to set an upper limit on the number of iterations (or iteration time) of the perturbation / behavior process, so that the simulation by the biomechanical model 10 ends when a predetermined number of iterations (or predetermined time) is reached.
[0100] On the other hand, as the visibility R improves due to the repetition of the perturbation / behavior process, the joint drive torque calculated by the motion generation module 12 gradually decreases, and the acceleration gradually decreases. When any state value (or its rate of change) falls below a set value, it may be considered that the confirmation action by the biomechanical model 10 has autonomously converged.
[0101] Furthermore, in step 100(200) above, multiple biomechanical models 10 may be launched (multiple launches) for a single visual field judgment model 50, and in step 111(211), state variables may be assigned to each biomechanical model 10 to calculate perturbation and burden B (and the degree of deviation D from the reference posture) using parallel computing. Since the biomechanical models 10 have a higher computational load than the visual field judgment model 50, parallel computing is advantageous for shortening processing time and leveling resources.
[0102] In the above embodiment, we showed a case where the biomechanical model 10 simulates the action of confirming one visual target point 20 from an initial posture. However, it is also possible to simulate the action of sequentially confirming multiple visual target points and evaluate overall visibility.
[0103] Although embodiments of the present invention have been described above, the present invention is not limited to the above embodiments, and various further modifications and changes are possible based on the technical concept of the present invention.
[0104] For example, in the above embodiment, the instrument panel of a vehicle was used as the evaluation target data 2, and the switch located on the lower right side of it was used as the visual target point 20. However, the present invention is not limited to this, and the visual target point may be in a different location, or something other than the instrument panel may be used as the evaluation target data. Furthermore, it can be applied to the evaluation of the ease of visibility and design support for various items that are expected to be visually inspected with a confirmation action, such as standing postures other than seated postures, walking postures, and the action of confirming the visual target point when getting in and out of a vehicle. [Explanation of Symbols]
[0105] 1. Evaluation System 2. Data to be evaluated (instrument panel) 3. Musculoskeletal Model (Body Model) 4. Eye movement models 5 Eyeball 6. Support structure (sheet) 7. Support structure (handle) 8 Databases 10 Biomechanical Models 11. Control Unit 12. Motion generation module 13. Degree of burden 14. Degree of deviation 15. Visibility 16. Visual Perception Objective Function 17. Comprehensive Evaluation Department for Visibility 20 Visual Target Points 21 Gradient Sphere 30 body 31 Head 32 Neck 33 Chest (reference site) 50 Field of View Judgment Models 53 Human Body Models 55 Camera (perspective)
Claims
1. A visibility evaluation system for objects created using 3D CAD, A biomechanical model for simulating a confirmation action toward the aforementioned target point in a virtual 3D space in which the aforementioned object and its target point are defined, A field of view judgment model for calculating the degree of visibility for the target point in the field of view image at the line of sight of the biomechanical model, A control unit that couples the biomechanical model and the field of view judgment model under conditions that minimize the burden on the biomechanical model and maximize the visibility, It is equipped with a system configured to calculate visibility based on the burden and visibility, The biomechanical model includes a musculoskeletal model and an eye movement model that determines eye position in conjunction with head movement in the musculoskeletal model, and the load is calculated from the physical load in the musculoskeletal model and the eye load in the eye movement model. The aforementioned eye load includes a load component corresponding to the eye angle representing the eye posture, and the load component is configured to rapidly increase in load characteristics from around a predetermined transition angle, and the system is configured such that eye movement or eye-head coordinated movement occurs when the eye angle is given by the visual target point.
2. The visibility evaluation system according to claim 1, configured to allow selective setting of the coefficient that gives the transition angle.
3. The eyeball load (L eye ) represents the eyeball angle θ which indicates the eyeball posture. eye , the angular velocity vector ω of the aforementioned eye movement eye coefficient k 11 ,k 12 ,k 13 as, Formula: L eye = k 11 ||ω eye || + exp(k 12 (θ eye - k 13 )) The coefficient k is calculated by 13 The visibility evaluation system according to claim 1, configured such that the transition angle is given by selectively setting the following.
4. The visibility evaluation system according to claims 1 to 3, wherein the central control unit calculates the degree of burden along with the degree of deviation from a reference posture set in the biomechanical model, and couples the biomechanical model and the field of view judgment model under conditions where the purposeful burden obtained by multiplying the degree of burden by the degree of deviation is minimized and the visibility is maximized.
5. The visibility evaluation system according to claim 4, wherein the object is 3D CAD data of an instrument panel disposed in front of the driver's seat of a vehicle defined as the virtual 3D space, the biomechanical model is defined as a seating posture in which the driver is seated in the driver's seat and looking straight ahead as the reference posture, the reference part that serves as the basis for calculating the degree of deviation is set to the chest or head of the biomechanical model, and the biomechanical model and the field of view judgment model are adapted to perform a simulation of confirmation action toward the visual target point installed on the instrument panel.