Installation position information providing method, installation position information providing device, and installation position information providing program

Abstract

To accurately identify an installation position of a target used for aiming a vehicle.SOLUTION: A device captures a video including a vehicle by a camera, identifies a predetermined portion in a 3D object of the vehicle, irradiates the 3D object of the vehicle with a measuring virtual object group from above toward below, adjusts directions of a straight line and a measuring virtual object group by using the straight line when the adjustment is completed as a center line of the vehicle so that a difference between the number of measuring virtual objects that collided with the 3D object of the vehicle in a left region and the number of measuring virtual objects that collided with the 3D object of the vehicle in a right region is less than a predetermined threshold among the left region and the right region obtained by dividing the measuring virtual object group by the straight line passing through a position of the predetermined portion, calculates an installation position of a target on the basis of the center line and preliminarily defined height of the predetermined portion, and displays information indicating the installation position of the target by superimposing the information on a field of view of an operator.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present disclosure relates to an installation location information providing method, an installation location information providing device, and an installation location information providing program. [Background technology]

[0002] In order to perform the adjustment work (aiming) of the sensor mounted on the vehicle with high accuracy, it is necessary to install the adjustment device (target) required for the adjustment work in an appropriate position. The target installation work is generally carried out by multiple workers while performing various surveys, and requires a lot of man-hours.

[0003] Patent document 1 discloses that an Augmented Reality (AR) device is equipped with a camera and a display unit, displays an image captured by the camera on the display unit, recognizes two specific parts and a floor surface of the vehicle or in the vicinity of the vehicle contained in the image captured by the camera, calculates an installation position of adjustment equipment used in adjustment work of an in-vehicle sensor based on the recognition results, and superimposes an installation position object indicating the installation position on the calculated installation position on the image displayed on the display unit. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-121322 A Summary of the Invention [Problem to be solved by the invention]

[0005] In Patent Document 1, the AR device recognizes two specific parts of the vehicle or in the vicinity of the vehicle included in the image captured by the camera, but does not consider the positional accuracy of the recognized parts, and if the distance between the camera and the two specific parts is large, the positioning cannot be performed with high accuracy, and the relative positional accuracy of the two parts becomes low. As a result, the accuracy of the calculated installation position also becomes low.

[0006] An object of the present disclosure is to more accurately identify the installation position of adjustment equipment used to adjust a sensor mounted on a vehicle. [Means for solving the problem]

[0007] According to one embodiment, a method for providing installation position information includes capturing an image including a vehicle with a camera, identifying a predetermined portion of a 3D object of the vehicle, projecting a group of measurement virtual objects consisting of a plurality of measurement virtual objects from above downward onto at least a part of the 3D object of the vehicle, dividing the group of measurement virtual objects into a left region and a right region by a straight line passing through the position of the predetermined portion, adjusting the orientation of the straight line and the group of measurement virtual objects so that a difference between the number of the measurement virtual objects that collide with the 3D object of the vehicle in the left region and the number of the measurement virtual objects that collide with the 3D object of the vehicle in the right region is less than a predetermined threshold, determining the straight line when the adjustment is completed as a center line of the vehicle, calculating an installation position of a target used to adjust a sensor mounted on the vehicle based on the center line and a height of the predetermined portion that is defined in advance, and displaying information indicating the installation position of the target by superimposing it on the image captured by the camera or on the field of view of an operator.

[0008] According to one embodiment, a method for providing installation position information includes capturing an image including a vehicle with a camera, identifying a predetermined portion of a 3D object of the vehicle, projecting a group of measurement virtual objects consisting of a plurality of measurement virtual objects from above downward onto at least a part of the 3D object of the vehicle, dividing the group of measurement virtual objects into a left region and a right region by a straight line passing through the position of the predetermined portion and parallel to a floor surface, adjusting orientations of the straight line and the group of measurement virtual objects so that a difference between the number of the measurement virtual objects that collide with the 3D object of the vehicle in the left region and the number of the measurement virtual objects that collide with the 3D object of the vehicle in the right region is less than a predetermined threshold, determining the straight line upon completion of the adjustment as a center line of the vehicle, calculating an installation position of a target used to adjust a sensor mounted on the vehicle based on the center line and the floor surface, and displaying information indicating the installation position of the target by superimposing it on the image captured by the camera or on an operator's field of view.

[0009] According to one embodiment, an installation location information providing device includes a camera, a display unit, and a control unit. The camera captures an image including a vehicle. The control unit identifies a predetermined portion of a 3D object of the vehicle, projects a group of measurement virtual objects consisting of a plurality of measurement virtual objects from above downward onto at least a part of the 3D object of the vehicle, and adjusts the orientation of the line and the group of measurement virtual objects so that a difference between the number of the measurement virtual objects that collide with the 3D object of the vehicle in the left region and the number of the measurement virtual objects that collide with the 3D object of the vehicle in the right region is less than a predetermined threshold value, the line passing through the position of the predetermined portion is set as the center line of the vehicle, and the installation position of a target used to adjust a sensor mounted on the vehicle is calculated based on the center line and a height of the predetermined portion that is defined in advance. Information indicating the installation position of the target is displayed on the display unit by superimposing it on the image captured by the camera or on the field of view of an operator.

[0010] According to one embodiment, an installation location information providing device includes a camera, a display unit, and a control unit. The camera captures an image including a vehicle. The control unit identifies a predetermined portion of a 3D object of the vehicle, projects a group of measurement virtual objects consisting of a plurality of measurement virtual objects from above downward onto at least a part of the 3D object of the vehicle, and divides the group of measurement virtual objects into a left region and a right region by a straight line passing through the position of the predetermined portion and parallel to a floor surface. The control unit adjusts the orientation of the straight line and the group of measurement virtual objects so that a difference between the number of the measurement virtual objects that collide with the 3D object of the vehicle in the left region and the number of the measurement virtual objects that collide with the 3D object of the vehicle in the right region is less than a predetermined threshold. The straight line when the adjustment is completed is set as a center line of the vehicle. The control unit calculates an installation position of a target used to adjust a sensor mounted on the vehicle based on the center line and the floor surface, and displays information indicating the installation position of the target on the display unit by superimposing the information on the image captured by the camera or on the field of view of an operator.

[0011] According to one embodiment, an installation position information providing program causes a computer to execute the following steps: capture an image including a vehicle with a camera; identify a predetermined portion of a 3D object of the vehicle; project a group of measurement virtual objects consisting of a plurality of measurement virtual objects from above downward onto at least a part of the 3D object of the vehicle; divide the group of measurement virtual objects into a left region and a right region by a straight line passing through the position of the predetermined portion; adjust orientations of the straight line and the group of measurement virtual objects so that a difference between the number of the measurement virtual objects that collide with the 3D object of the vehicle in the left region and the number of the measurement virtual objects that collide with the 3D object of the vehicle in the right region is less than a predetermined threshold; set the straight line when the adjustment is completed as a center line of the vehicle; calculate an installation position of a target used to adjust a sensor mounted on the vehicle based on the center line and a height of the predetermined portion that is defined in advance; and display information indicating the installation position of the target by superimposing it on the image captured by the camera or on an operator's field of vision.

[0012] According to one embodiment, an installation position information providing program causes a computer to execute the following steps: capture an image including a vehicle with a camera; identify a predetermined portion of a 3D object of the vehicle; project a group of measurement virtual objects consisting of a plurality of measurement virtual objects from above downward onto at least a part of the 3D object of the vehicle; divide the group of measurement virtual objects into left and right regions by a straight line passing through the position of the predetermined portion and parallel to a floor surface; adjust orientations of the straight line and the group of measurement virtual objects so that a difference between the number of the measurement virtual objects that collide with the 3D object of the vehicle in the left region and the number of the measurement virtual objects that collide with the 3D object of the vehicle in the right region is less than a predetermined threshold; set the straight line when the adjustment is completed as a center line of the vehicle; calculate an installation position of a target used to adjust a sensor mounted on the vehicle based on the center line and the floor surface; and display information indicating the installation position of the target by superimposing it on the image captured by the camera or on an operator's field of view.

[0013] Furthermore, these comprehensive or specific aspects may be realized by a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized by any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. Effect of the Invention

[0014] According to the present disclosure, it is possible to more accurately identify the installation location of adjustment equipment used to adjust a sensor mounted on a vehicle. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 is a block diagram showing a configuration example of an XR device according to a first embodiment; [Diagram 2] FIG. 1 is a side view of an example in which a group of measurement virtual objects is projected onto a vehicle according to the first embodiment. [Diagram 3] FIG. 1 is a top view of an example in which a group of virtual objects for measurement is projected onto a vehicle according to the first embodiment, illustrating a situation in which the center line of the vehicle is being searched for; [Figure 4] FIG. 1 is a top view of an example of projecting a group of measurement virtual objects onto a vehicle according to the first embodiment, illustrating a state in which searching for the center line of the vehicle has been completed; [Diagram 5] 1 is a flowchart showing an example of a process in which the XR device according to the first embodiment displays the installation position of a target. [Figure 6] FIG. 1 is a diagram for explaining projection of a group of virtual objects for a target onto a target according to the first embodiment; [Figure 7] 1 is a flowchart showing an example of a process for confirming the installation position of a target according to the first embodiment. [Figure 8] FIG. 13 is a diagram for explaining a positioning error in an XR device according to a second embodiment; [Figure 9] FIG. 11 is a diagram for explaining a method for accurately measuring the position of an object that is located beyond the positioning limit distance in an XR device according to embodiment 2. [Figure 10]Flowchart showing an example of the process of setting the center line of a vehicle using the XR device according to Embodiment 2 [Figure 11] Schematic diagram for explaining a method of calculating the installation position of a target using the grid-like reference points according to Embodiment 2 [Figure 12] Schematic diagram for explaining a first example of a method of calculating the installation position of a target using a non-expandable jig according to Embodiment 2 [Figure 13] Schematic diagram for explaining a second example of a method of calculating the installation position of a target using a non-expandable jig according to Embodiment 2

Embodiments for Carrying Out the Invention

[0016] Hereinafter, embodiments of the present disclosure will be described in detail with appropriate reference to the drawings. However, a more detailed description may be omitted as necessary. For example, detailed descriptions of well-known matters and duplicate descriptions of substantially the same configurations may be omitted. This is to avoid making the following description unnecessarily redundant and to facilitate the understanding of those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0017] (Embodiment 1) Conventionally, when performing adjustment work (aiming work) on various sensors (such as cameras, radars, etc.) mounted on a vehicle, an operator manually specifies the installation position of an adjustment device (target) using a level, a plumb bob, a compass, a measuring tool, etc. However, this method requires a lot of man-hours and takes time. Therefore, hereinafter, a method of improving the efficiency of aiming work using an Extended Reality (XR) device will be described. Note that the XR device is an example, and any device may be used as long as it can superimpose virtual images, images, information, etc. on the image of the scenery that the operator is actually seeing.

[0018] <Configuration of XR Device> FIG. 1 is a block diagram showing an example of the configuration of an XR device 10 according to the first embodiment.

[0019] The XR device 10 is a device that can be worn by a worker 1 (see FIG. 2), and includes a camera 11, a spatial recognition sensor 12, a position recognition sensor 13, an input unit 14, a communication unit 15, a display unit 16, an audio output unit 17, a control unit 18, and a storage unit 19. These components can transmit and receive information via an internal bus.

[0020] The camera 11 captures at least the line of sight of the worker 1 wearing the XR device 10, and generates a captured image. The camera 11 may be interpreted as an imaging device.

[0021] The spatial recognition sensor 12 senses the space and objects in the direction of the line of sight of the worker 1 wearing at least the XR device 10, and generates 3D point cloud data. The spatial recognition sensor 12 may be, for example, a Time of Flight (ToF) sensor or a Light Detection and Ranging (LiDAR). Alternatively, the spatial recognition sensor 12 may be a stereo camera. The spatial recognition sensor 12 and the camera 11 may be an integrated device.

[0022] The position recognition sensor 13 is a sensor for recognizing the three-dimensional position and attitude (orientation) of the XR device 10. The position recognition sensor 13 is configured by, for example, a six-axis gyro, an electronic compass, and / or a Global Navigation Satellite Systems (GNSS) receiver.

[0023] The input unit 14 is an interface for the worker 1 to input instructions to the XR device 10. The input unit 14 may be a function that recognizes gestures of the worker 1 using the camera 11, rather than an actual device. Alternatively, the input unit 14 may be an actual device (e.g., a mouse or a controller) that accepts operations by the worker 1.

[0024] The communication unit 15 is a device for the XR device 10 to perform wireless communication with an external device (for example, the PC 30 or the server 31, etc.). Examples of wireless communication include wireless LAN, Bluetooth (registered trademark), and mobile communication (LTE, 4G, 5G, etc.). The communication unit 15 may also perform wired communication.

[0025] The display unit 16 is an interface for showing images and the like to the worker 1 (operator) wearing the XR device 10. Examples of the display unit 16 include a liquid crystal display and an organic EL display. The display unit 16 may be any of a transmissive type, a reflective type, and a semi-transmissive type.

[0026] The audio output unit 17 is an interface for letting the worker 1 wearing the XR device 10 hear audio and the like. Examples of the audio output unit 17 include a headphone speaker, a bone conduction speaker, and the like.

[0027] The storage unit 19 is a device that stores computer programs and data for implementing the functions of the XR device 10. The storage unit 19 may be configured with a volatile storage medium (e.g., Random Access Memory (RAM)) and / or a non-volatile storage medium (e.g., Read Only Memory (ROM) or flash memory).

[0028] The control unit 18 controls the camera 11, the spatial recognition sensor 12, the position recognition sensor 13, the input unit 14, the communication unit 15, the display unit 16, and the audio output unit 17, and executes a computer program in cooperation with the storage unit 19 to realize the functions of the XR device 10. The functions of the XR device 10 will be described in detail later.

[0029] The XR device 10 may be able to transmit and receive information to and from the PC 30 and / or the server 31 via the communication unit 15. The PC 30 and / or the server 31 may execute at least a part of the functions of the XR device 10. This can reduce the processing load on the control unit 18 of the XR device 10. In the following description, the XR device 10 will be the main subject. and The process may be interpreted as a process mainly performed by the control unit 18, or a process in which the control unit 18 cooperates with the PC 30 and / or the server 31. The XR device 10 may be interpreted as an installation position information providing device.

[0030] <Display target placement position> FIG. 2 is a side view of an example of projecting a measurement virtual object group 50 onto the vehicle 2 according to the first embodiment. FIG. 3 is a top view of an example of projecting a measurement virtual object group 50 onto the vehicle 2 according to the first embodiment, and is a diagram for explaining a state in which the center line 42 of the vehicle 2 is being searched for. FIG. 4 is a top view of an example of projecting a measurement virtual object group 50 onto the vehicle 2 according to the first embodiment, and is a diagram for explaining a state in which the search for the center line 42 of the vehicle 2 has been completed. FIG. 5 is a flowchart showing an example of a process in which the XR device 10 according to the first embodiment displays the installation position 45 of the target 5. Next, the process shown in FIG. 5 will be described with reference to FIGS. 2, 3, and 4.

[0031] For convenience of explanation, as shown in Figures 2, 3, and 4, the axis extending in the vehicle width direction of the vehicle 2 is the X-axis, the axis extending in the overall length direction of the vehicle 2 is the Y-axis, and the axis extending in the height direction of the vehicle 2 is the Z-axis. For convenience of explanation, the positive direction of the Z-axis may be referred to as "up", the negative direction of the Z-axis as "down", the positive direction of the X-axis as "right", the negative direction of the X-axis as "left", the positive direction of the Y-axis as "rear", and the negative direction of the Y-axis as "front". These expressions relating to directions are used for convenience of explanation and are not intended to limit the posture of the structure during actual use.

[0032] The worker 1 wearing the XR device 10 stands in a position where the front of the vehicle 2 can be seen (S101). Note that the position where the worker 1 stands does not necessarily have to be in front of the vehicle 2, but may be a position where at least the emblem 4A on the front of the vehicle 2 can be seen.

[0033] The XR device 10 captures an image of a predetermined range ahead with the camera 11, generates a captured image, and senses the predetermined range ahead with the spatial recognition sensor 12, generating 3D point cloud data (S102). An actual vehicle 2 exists within the predetermined range, the captured image includes the vehicle 2, and the 3D point cloud data includes the shape of the vehicle 2, etc.

[0034] The XR device 10 displays the captured image on the display unit 16 (S103). The worker 1 wearing the XR device 10 can visually recognize the scenery (vehicle 2, etc.) through the XR device 10 by looking at the captured image displayed on the display unit 16.

[0035] The XR device 10 generates 3D objects of the vehicle 2 and the floor surface 3 from the generated 3D point cloud data (S104). Note that the generation of the 3D objects may be performed by the PC 30, the server 31, or the like, instead of the XR device 10. Note that the 3D object of the floor surface 3 does not have to be generated.

[0036] While watching the captured image displayed on the display unit 16, the worker 1 specifies the position of the emblem 4A in front of the vehicle 2 in the captured image by a predetermined gesture (for example, an action of touching the emblem 4A) (S105). The XR device 10 sets the position (three-dimensional position) specified by the worker 1 as the emblem position (three-dimensional position). However, the method in which the XR device 10 recognizes the position of the emblem 4A is not limited to the method in which the worker 1 manually specifies it as described above. For example, the XR device 10 may automatically recognize the shape and pattern of the emblem 4A from the captured image and specify the recognized position as the position of the emblem 4A. In addition, when specifying the position of the emblem 4A, the XR device 10 may photograph and sense only the bonnet part of the vehicle 2. In addition, the specification of the position of the emblem 4A of the vehicle 2 is one example, and any specific part may be used as long as it is located on the center line 42 that divides the width direction of the vehicle 2 in half and extends in the overall length direction of the vehicle. In addition to emblems, examples of specific parts include license plates.

[0037] As shown in FIG. 2, FIG. 3, and FIG. 4, the XR device 10 virtually projects the measurement virtual object group 50 from above downward (i.e., along the Z axis) toward a predetermined range including the bonnet of the 3D object of the vehicle 2 (S106). However, the range to which the measurement virtual object group 50 is projected is not limited to the bonnet of the vehicle 2, and may be, for example, the entire vehicle 2. As shown in FIG. 3 or FIG. 4, the measurement virtual object group 50 may be a plurality of measurement virtual objects (points) arranged in a lattice shape at equal intervals within a rectangle of a predetermined size in the XY plane view. Furthermore, the XR device 10 may repeatedly project the measurement virtual object group 50 at regular intervals.

[0038] The XR device 10 calculates a straight line 41 that passes through the position of the emblem 4A, bisects the measurement virtual object group 50, and is parallel to the floor surface 3 (S107).

[0039] 3, the XR device 10 counts the number of the measurement virtual object group 50 that collided with the 3D object of the vehicle 2 in the left area of ​​the straight line 41 (hereinafter referred to as the number of left area collisions) and the number of the measurement virtual object group 50 that collided with the 3D object of the vehicle 2 in the right area of ​​the straight line 41 (hereinafter referred to as the number of right area collisions) (S108). Note that in FIGS. 3 and 4, black circles indicate those of the measurement virtual object group 50 that collided with the 3D object of the vehicle 2, and white circles indicate those that did not collide with the 3D object of the vehicle 2.

[0040] The XR device 10 determines whether the left area collision count and the right area collision count are approximately the same (S109). For example, the XR device 10 determines whether the difference between the left area collision count and the right area collision count is less than a predetermined threshold close to 0.

[0041] If the number of left area collisions and the number of right area collisions are not substantially the same (S109: NO), the XR device 10 rotates the line 41 and the measurement virtual object group 50 clockwise or counterclockwise by a predetermined rotation angle with the position of the emblem 4A as the rotation axis as illustrated in FIG. 3 (S110), and returns the process to step S106. If the number of right area collisions is greater than the number of left area collisions by a predetermined number or more, the XR device 10 may rotate the line 41 and the measurement virtual object group 50 clockwise by a predetermined angle. This increases the number of left area collisions and decreases the number of right area collisions. If the number of left area collisions is greater than the number of right area collisions by a predetermined number or more, the XR device 10 may rotate the line 41 and the measurement virtual object group 50 counterclockwise by a predetermined angle. This increases the number of right area collisions and decreases the number of left area collisions.

[0042] By repeating steps S106 to S110, the inclination of the straight line 41 as shown in FIG. 3 gradually approaches a center line 42 (center line of the vehicle) that bisects the width direction (vehicle width) of the vehicle 2 as shown in FIG.

[0043] In the process from step S106 to step S110, the XR device 10 may display the straight line 41 or the center line 42 and the measurement virtual object group 50 on the display unit 16 by superimposing them on the captured image or on the field of view of the worker 1 (operator) wearing the XR device 10. In addition, when displaying the measurement virtual object group 50, the XR device 10 may display the measurement virtual object group that has collided with the 3D object of the vehicle 2 (hereinafter referred to as a collision virtual object group) and the measurement virtual object group that has not collided with the 3D object of the vehicle 2 (hereinafter referred to as a non-collision virtual object group) in a distinguishable manner (for example, different colors). For example, the XR device 10 may display the collision virtual object group in green and the non-collision virtual object group in red. This allows the worker 1 to visually recognize the straight line 41 or the center line 42, the collision virtual object group, and the non-collision virtual object group through the XR device 10. However, this display is not essential, and the XR device 10 does not have to perform this display in the processes from step S106 to step S110.

[0044] If the number of left area collisions and the number of right area collisions are approximately the same (S109: YES), the XR device 10 sets the current straight line 41 to a center line 42 that bisects the width direction of the vehicle 2, as illustrated in FIG. 4, and displays the center line 42 on the display unit 16 by superimposing it on the captured image or on the field of view of the worker 1 (operator) wearing the XR device 10 (S111).

[0045] The XR device 10 may perform a process of finely adjusting the center line 42 set in step S111 (S112). This process of finely adjusting the center line 42 may be performed automatically by the XR device 10 or manually by the operator 1.

[0046] The XR device 10 calculates the installation position 45 of the target 5 using the center line 42 (S113). For example, the XR device 10 calculates a position on an extension of the center line 42 and a specified distance away from the position of the emblem 4A as the installation position 45 of the target 5. The specified distance may be predetermined for each type of vehicle 2. Therefore, when starting aiming of the vehicle 2, the worker 1 inputs the type (model) of the vehicle 2 into the XR device 10, and the XR device 10 may obtain and set the specified distance corresponding to the input type of vehicle 2 from the PC 30 or the server 31.

[0047] The XR device 10 displays information indicating the calculated installation position 45 of the target 5 (for example, a virtual image of the target 5) on the display unit 16 by superimposing it on the captured image or on the field of view of the worker 1 (operator) wearing the XR device 10 (S114). The XR device 10 may display information indicating the height position on the display unit 16 by superimposing it on the virtual image or on the field of view of the worker 1 (operator) wearing the XR device 10, regarding the position of a specified height h1 from the floor surface 3 converted into a 3D object as the height position of the main surface of the target 5. The specified height h1 is the height of the main surface of the target 5, and may be determined in advance depending on the type of the vehicle 2. Alternatively, the XR device 10 may calculate the height position of the main surface of the target 5 by the following method without converting the floor surface 3 into a 3D object. That is, the XR device 10 may determine the position of a specified height h3 from the height of the set center line 42 as the height position of the main surface of the target 5, and may display information indicating the height position on the display unit 16 by superimposing it on the virtual image or on the field of view of the worker 1 (operator) wearing the XR device 10. The specified height h3 may be a value obtained by subtracting the height h2 of the emblem 4A from the floor surface 3 from the above-mentioned specified height h1. The height h2 of the emblem 4A from the floor surface 3 may be determined in advance depending on the type of vehicle 2.

[0048] The worker 1 actually places the target 5 in accordance with the displayed information indicating the placement position of the target (S115).

[0049] Then, this process ends. Note that following this process, the XR device 10 may execute the process shown in FIG.

[0050] This allows the worker 1 to easily recognize the installation position 45 of the target 5 and install the target 5 in the correct position by wearing the XR device 10 and specifying the position of the emblem 4A.

[0051] Fig. 6 is a diagram for explaining projection of a target virtual object group 55 onto a target 5 according to embodiment 1. Fig. 7 is a flowchart showing an example of a process for confirming an installation position of the target 5 according to embodiment 1. Next, the process shown in Fig. 7 will be explained with reference to Fig. 6.

[0052] The worker 1 wearing the XR device 10 stands in a position where the front of the target 5 set in step S115 of FIG. 5 can be seen (S201). too It does not have to be in front of the target 5, but it is sufficient that the target 5 is at least visible.

[0053] The XR device 10 captures an image of a predetermined range in front of the vehicle with the camera 11 to generate a captured image, and also senses the predetermined range in front of the vehicle with the spatial recognition sensor 12 to generate 3D point cloud data (S202).

[0054] The XR device 10 displays the captured image on the display unit 16 (S203). The worker 1 can visually recognize the scenery (the target 5, etc.) through the XR device 10 by looking at the captured image displayed on the display unit 16.

[0055] The XR device 10 generates a 3D object of the target 5 from the generated 3D point cloud data (S204).

[0056] The XR device 10 virtually projects the target virtual object group 55 toward the 3D object of the target 5 along (for example, parallel to) the center line 42 set in the process of Fig. 5 (S205). As shown in Fig. 6, the rectangular shape and size of the target virtual object group 55 in the ZX plane may be set to match the rectangular shape and size of the main surface of the 3D object of the target 5 in the ZX plane.

[0057] The XR device 10 displays the target virtual object group 55 on the display unit 16 by superimposing the target virtual object group 55 on the captured image or on the field of view of the worker 1 (operator) wearing the XR device 10 (S206). At this time, the XR device 10 may display the target virtual object group 55 that collided with the main surface of the 3D object of the target 5 (hereinafter referred to as a target collision virtual object group) and the target virtual object group 55 that did not collide with the main surface of the 3D object of the target 5 (hereinafter referred to as a target non-collision virtual object group) in a distinguishable manner (for example, in different colors).

[0058] The XR device 10 counts the number of target non-collision virtual objects (hereinafter, referred to as the number of target non-collision objects) (S207).

[0059] The XR device 10 determines whether the target non-collision number is approximately 0 (S208). For example, the XR device 10 determines whether the target non-collision number is less than a predetermined threshold. As described above, the rectangular shape and size of the target virtual object group 55 on the ZX plane are set to match the rectangular shape and size of the main surface of the target 5 on the ZX plane, so that if the target 5 is not displaced in the X-axis direction or rotated around the Z axis, the target non-collision number is approximately 0. On the other hand, if the target 5 is displaced in the X-axis direction or rotated around the Z axis, the target non-collision number is equal to or greater than the predetermined threshold.

[0060] If the number of non-collisions of the target is not substantially 0 (S208: NO), the XR device 10 notifies the user that the target is not correctly set (S209). For example, the XR device 10 displays information that the target 5 is not correctly set on the display unit 16, or outputs a voice that the target 5 is not correctly set to the voice output unit 17. In this case, the operator 1 adjusts the actual setting position of the target 5 so that it is correctly set (S210). Then, the process returns to step S205.

[0061] By repeating steps S205 to S210, the operator 1 can adjust the target 5 so that it is placed in the correct position.

[0062] If the number of non-collisions of targets is approximately 0 (S208: YES), the XR device 10 notifies the user that the target 5 is correctly placed (S211) and ends this process. For example, the XR device 10 displays information that the target 5 is correctly placed on the display unit 16, or outputs a voice that the target 5 is correctly placed to the voice output unit 17. This allows the worker 1 to confirm that the target 5 is placed in the correct position.

[0063] (Summary of the first embodiment) The above description of the first embodiment discloses the following techniques.

[0064] <Technology A1> According to one embodiment, a method for providing installation location information includes capturing an image including a vehicle (2) using a camera (11), identifying a predetermined portion (e.g., an emblem 4A) in a 3D object of the vehicle, projecting a group of measurement virtual objects (50) consisting of a plurality of measurement virtual objects from above downward onto at least a part of the 3D object of the vehicle, dividing the group of measurement virtual objects into a left region and a right region by a straight line (41) passing through the position of the predetermined portion, and calculating the number of the measurement virtual objects that collide with the 3D object of the vehicle in the left region (number of left region collisions) and The orientation of the straight line and the group of measurement virtual objects is adjusted so that the difference between the number of the measurement virtual objects that collided with the 3D object of the vehicle in the right area (right area collision number) is less than a predetermined threshold, the straight line obtained when the adjustment is completed is set to be the center line (42) of the vehicle, and an installation position (45) of a target (5) used to adjust a sensor mounted on the vehicle is calculated based on the center line and the height of the predetermined part, and information indicating the installation position of the target is displayed by superimposing it on the image captured by the camera or on the operator's field of vision. This allows the center line 42 of the vehicle 2 to be calculated with high accuracy, and therefore the installation position 45 of the target 5 to be calculated with high accuracy. Therefore, the worker 1 can easily and accurately install the actual target in accordance with the installation position 45 of the displayed target 5.

[0065] <Technology A2> According to one embodiment, a method for providing installation location information includes capturing an image including a vehicle (2) using a camera (11), identifying a predetermined portion (e.g., an emblem 4A) in a 3D object of the vehicle, projecting a measurement virtual object group 50 consisting of a plurality of measurement virtual objects from above downward onto at least a part of the 3D object of the vehicle, dividing the measurement virtual object group into a left region and a right region by a straight line (41) that passes through the position of the predetermined portion and is parallel to a floor surface (3), and projecting a measurement virtual object group 50 including a plurality of measurement virtual objects that has collided with the 3D object of the vehicle in the left region. The orientation of the straight line and the group of measurement virtual objects is adjusted so that a difference between the number of the measurement virtual objects that collided with the 3D object of the vehicle in the right area (number of collisions in the left area) and the number of the measurement virtual objects that collided with the 3D object of the vehicle in the right area (number of collisions in the right area) is less than a predetermined threshold, the straight line upon completion of the adjustment is set as a center line (42) of the vehicle, and an installation position (45) of a target (5) used for adjusting a sensor mounted on the vehicle is calculated based on the center line and the floor surface, and information indicating the installation position of the target is displayed by superimposing it on the image captured by the camera or on the field of view of the operator. This allows the center line 42 of the vehicle 2 to be calculated with high accuracy, and therefore the installation position 45 of the target 5 to be calculated with high accuracy. Therefore, the worker 1 can easily and accurately install the actual target in accordance with the installation position 45 of the displayed target 5.

[0066] <Technology A3> In the installation position information providing method according to Technique A1 or A2, the center line is displayed by being superimposed on the image captured by the camera or on the operator's field of vision. This allows the worker (1) to visually confirm the set center line (42) of the vehicle (2).

[0067] <Technology A4> In a method for providing installation position information described in any one of Technologies A1 to A3, a group of target virtual objects (55) consisting of a plurality of target virtual objects is projected along the center line toward the target 3D object installed at the installation position, and it is determined whether the number of the target virtual objects that do not collide with the target 3D object (the number of non-collisions of the target) is less than a predetermined threshold value. If the number of the target virtual objects that do not collide with the target 3D object (the number of non-collisions of the target) is less than the predetermined threshold value, a notification to that effect is provided. This allows the worker (1) to easily recognize whether the target (5) is correctly set or not.

[0068] <Technology A5> An installation position information providing device (e.g., an XR device 10) according to one embodiment includes a camera (11), a display unit (16), and a control unit (18). The camera captures an image including a vehicle (2). The control unit identifies a predetermined portion (e.g., an emblem 4A) in a 3D object of the vehicle, projects a measurement virtual object group (50) consisting of a plurality of measurement virtual objects from above downward onto at least a part of the 3D object of the vehicle, and divides the measurement virtual object group into a left region and a right region by a straight line (41) passing through the position of the predetermined portion, and detects a measurement virtual object that has collided with the 3D object of the vehicle in the left region. The orientation of the straight line and the group of measurement virtual objects is adjusted so that the difference between the number of virtual objects (number of left area collisions) and the number of the measurement virtual objects that collided with the 3D object of the vehicle in the right area (number of right area collisions) is less than a predetermined threshold, the straight line when the adjustment is completed is set to be the center line (42) of the vehicle, and an installation position (45) of a target (5) used to adjust a sensor mounted on the vehicle is calculated based on the center line and the height of the predetermined portion that is defined in advance, and information indicating the installation position of the target is displayed on the display unit by superimposing it on the image captured by the camera or on the operator's field of vision. This allows the center line 42 of the vehicle 2 to be calculated with high accuracy, and therefore the installation position 45 of the target 5 to be calculated with high accuracy. Therefore, the worker 1 can easily and accurately install the actual target in accordance with the installation position 45 of the displayed target 5.

[0069] <Technology A6> An installation position information providing device (e.g., an XR device 10) according to one embodiment includes a camera (11), a display unit (16), and a control unit (18). The camera captures an image including a vehicle (2). The control unit identifies a predetermined portion (e.g., an emblem 4A) in a 3D object of the vehicle, projects a measurement virtual object group (50) consisting of a plurality of measurement virtual objects from above downward onto at least a part of the 3D object of the vehicle, and divides the measurement virtual object group into a left region and a right region by a straight line (41) that passes through the position of the predetermined portion and is parallel to a floor surface (3). The control unit projects the 3D object of the vehicle in the left region. The orientation of the straight line and the group of measurement virtual objects is adjusted so that a difference between the number of the measurement virtual objects that collided with a 3D object of the vehicle in the right area (number of collisions in the left area) and the number of the measurement virtual objects that collided with a 3D object of the vehicle in the right area (number of collisions in the right area) becomes less than a predetermined threshold value, the straight line when the adjustment is completed is set to a center line (42) of the vehicle, and an installation position (45) of a target (5) used to adjust a sensor mounted on the vehicle is calculated based on the center line and the floor surface, and information indicating the installation position of the target is displayed on the display unit by superimposing it on the image captured by the camera or on the field of view of the operator. This allows the center line 42 of the vehicle 2 to be calculated with high accuracy, and therefore the installation position 45 of the target 5 to be calculated with high accuracy. Therefore, the worker 1 can easily and accurately install the actual target in accordance with the installation position 45 of the displayed target 5.

[0070] <Technology A7> An installation position information providing program according to one embodiment captures an image including a vehicle (2) using a camera (11), identifies a predetermined portion (e.g., an emblem 4A) of a 3D object of the vehicle, projects a group of measurement virtual objects (50) consisting of a plurality of measurement virtual objects from above downward onto at least a part of the 3D object of the vehicle, and divides the group of measurement virtual objects into a left region and a right region by a straight line (41) passing through the position of the predetermined portion. The program calculates the number of the measurement virtual objects that collide with the 3D object of the vehicle in the left region (left region collision number) and the number of the measurement virtual objects that collide with the 3D object of the vehicle in the right region. The orientation of the straight line and the group of measurement virtual objects is adjusted so that the difference with the number of the measurement virtual objects that collided with the 3D object (number of right area collisions) is less than a predetermined threshold, the straight line when the adjustment is completed is set as the center line (42) of the vehicle, and based on the center line and the height of the predetermined specified part, an installation position (45) of a target (5) used to adjust a sensor mounted on the vehicle is calculated, and information indicating the installation position of the target is displayed by superimposing it on the image captured by the camera or on the operator's field of vision. This allows the center line 42 of the vehicle 2 to be calculated with high accuracy, and therefore the installation position 45 of the target 5 to be calculated with high accuracy. Therefore, the worker 1 can easily and accurately install the actual target in accordance with the installation position 45 of the displayed target 5.

[0071] <Technology A8> According to one embodiment, the installation position information providing program captures an image including a vehicle (2) using a camera (11), identifies a predetermined portion (e.g., an emblem 4A) of a 3D object of the vehicle, projects a measurement virtual object group (50) consisting of a plurality of measurement virtual objects from above downward onto at least a part of the 3D object of the vehicle, and divides the measurement virtual object group into a left region and a right region by a straight line (41) that passes through the position of the predetermined portion and is parallel to a floor surface (3). The program calculates the number of the measurement virtual objects that collide with the 3D object of the vehicle in the left region (left region collision number) and the number of the measurement virtual objects that collide with the 3D object of the vehicle in the left region (left region collision number). The orientation of the straight line and the group of measurement virtual objects is adjusted so that the difference between the number of the measurement virtual objects that collided with the 3D object of the vehicle in the right area (right area collision number) is less than a predetermined threshold, the straight line when the adjustment is completed is set as the center line (42) of the vehicle, and based on the center line and the floor surface, an installation position (45) of a target (5) used to adjust a sensor mounted on the vehicle is calculated, and information indicating the installation position of the target is displayed by superimposing it on the image captured by the camera or on the operator's field of vision. This allows the center line 42 of the vehicle 2 to be calculated with high accuracy, and therefore the installation position 45 of the target 5 to be calculated with high accuracy. Therefore, the worker 1 can easily and accurately install the actual target in accordance with the installation position 45 of the displayed target 5.

[0072] (Embodiment 2) In the second embodiment, a method for overcoming the positioning error in the XR device 10 and setting the center line 42 of the vehicle 2 will be described. Note that the configuration of the XR device 10 according to the second embodiment may be similar to the configuration shown in FIG.

[0073] FIG. 8 is a diagram for explaining the positioning error in the XR device 10 according to the second embodiment.

[0074] There is a limit to the spatial range in which the spatial recognition sensor 12 of the XR device 10 can accurately measure the position of an object. Hereinafter, the maximum spatial range in which the spatial recognition sensor 12 can accurately measure the position of an object is referred to as the positionable space 60. In this embodiment, the maximum distance of the positionable space 60 (hereinafter referred to as the positioning limit distance) is described as about 3 m, but the positioning limit distance varies depending on the performance of the XR device 10 and may be any number. The spatial recognition sensor 12 of the XR device 10 can accurately measure the position of an object that exists at a position within the positioning limit distance from its own position, but cannot accurately measure the position of an object that exists at a position beyond the positioning limit distance from its own position (i.e., a positioning error occurs).

[0075] Since the overall length of the vehicle 2 is often 3 m or more, the XR device 10, which has a positioning limit distance of about 3 m, cannot accurately position the position of the emblem 4B at the rear of the vehicle 2 and the emblem 4A at the front of the vehicle 2 at the same time, as shown in Fig. 8. Therefore, in this state, it is not possible to use the XR device 10 to set the center line 42 of the vehicle 2 by connecting the position of the emblem 4B at the rear of the vehicle 2 and the position of the emblem 4A at the front of the vehicle 2 with a straight line 41.

[0076] In addition, since the distance from the rear of the vehicle 2 to the target 5 is often 3 m or more, the XR device 10, which has a positioning limit distance of approximately 3 m, cannot accurately calculate the installation position of the target 5 on the extension of the center line 42.

[0077] In this embodiment, a method for accurately locating the position of an object located at a position beyond the positioning limit distance using an XR device 10 and a method for accurately determining the center line 42 of a vehicle using that method will be described.

[0078] Fig. 9 is a diagram for explaining a method for accurately locating an object existing at a position beyond the positioning limit distance in the XR device 10 according to the second embodiment. Fig. 10 is a flowchart showing an example of a process for setting the center line 42 of the vehicle 2 using the XR device 10 according to the second embodiment. Next, the process shown in Fig. 10 will be explained with reference to Fig. 9. At least a part of the process shown in Fig. 10 may be performed by the PC 30 or the server 31.

[0079] The worker 1 sets a first reference point 61 at an arbitrary position in the real space (S301). The first reference point 61 may be a predetermined AR marker (e.g., a two-dimensional code) that can be detected and identified by the XR device 10. The first reference point 61 may be set anywhere, but in FIG. 9, it is set at a position that is within the positioning limit distance (e.g., about 3 m) from both the position of the emblem 4A at the front of the vehicle 2 and the position of the emblem 4B at the rear of the vehicle 2.

[0080] The worker 1 sets the second reference point 62 at a position a predetermined distance from the first reference point 61 in the real space (S302). For example, the second reference point 62 is set at a position a predetermined distance from the first reference point 61 in the direction of the target 5. The second reference point 62 may also be a predetermined AR marker that can be detected and identified by the XR device 10. Hereinafter, the distance between the first reference point 61 and the second reference point 62 is referred to as the inter-reference point distance.

[0081] The XR device 10 pre-stores the accurately measured distance between the reference points (S303). The accurately measured distance between the reference points may be input to the XR device 10 by the worker 1, or may be pre-stored in the XR device 10.

[0082] The XR device 10 measures the position of the rear emblem 4B relative to the first reference point 61 in a virtual space α that includes both the first reference point 61 and the rear emblem 4B of the vehicle 2 within the positionable space 60 and has the first reference point 61 as its origin (S304). The XR device 10 can accurately measure the position of an object existing in the virtual space α.

[0083] The XR device 10 measures the position of the front emblem 4A relative to the first reference point 61 in a virtual space β with the first reference point 61 as the origin, the virtual space β including both the first reference point 61 and the front emblem 4A of the vehicle 2 within the positionable space 60 (S305). The XR device 10 can accurately measure the position of an object existing in the virtual space β.

[0084] The XR device 10 generates a virtual space γ with the second reference point 62 as the origin (S306).

[0085] The XR device 10 integrates the virtual space α, the virtual space β, and the virtual space γ into the virtual world space (S307). For example, the XR device 10 integrates the virtual space α and the virtual space β into the virtual world space so that the first reference point 61 in the virtual space α coincides with the first reference point 61 in the virtual space β, and further integrates the virtual space γ into the virtual world space using the accurately measured inter-reference point distance between the first reference point 61 and the second reference point 62. As a result, the positions measured in each of the virtual spaces α, β, and γ have an accurate positional relationship with each other in the virtual world space.

[0086] The XR device 10 connects the position of the rear emblem 4B and the position of the front emblem 4A with a straight line in the virtual world space, and sets the center line 42 (S308). Note that the emblems 4A and 4B are examples of specific parts. In addition to emblems, license plates are also examples of specific parts.

[0087] The XR device 10 may perform a process of finely adjusting the set center line 42 (S309). This process of finely adjusting the center line 42 may be performed automatically by the XR device 10 or manually by the operator 1.

[0088] The XR device 10 calculates the installation position of the target 5 on the extension of the center line 42 in the virtual world space (S310).

[0089] The XR device 10 converts the installation position of the target 5 calculated in step S310 into a position in the virtual space γ including both the second reference point 62 and the position where the target 5 is to be installed in the positionable space 60. Then, the XR device 10 displays an object of the target 5 (for example, a virtual image of the target 5) at that position on the display unit 16 by superimposing it on the captured image or on the field of view of the worker 1 (operator) wearing the XR device 10 (S311). At this time, the worker 1 may manually correct the position of the object of the target 5 through the XR device 10. Then, this process ends.

[0090] In this way, the XR device 10 can accurately measure the position of an object in the virtual world space beyond the positioning limit distance by integrating multiple virtual spaces into the virtual world space using multiple reference points whose accurate inter-reference point distances are known. In addition, using this method, the XR device 10 can accurately calculate the center line 42 of the vehicle 2 and accurately calculate the installation position of the target 5. Note that the number of virtual spaces that can be integrated into the virtual world space is not limited to two, and may be three or more.

[0091] FIG. 11 is a schematic diagram for explaining a method for calculating the installation position of the target 5 using the lattice-like reference points according to the second embodiment.

[0092] As shown in Fig. 11, a plurality of reference points 63 are arranged in advance on the floor surface at equal intervals in a lattice pattern. The XR device 10 may calculate the installation position of the target 5 as follows, using the plurality of reference points 63 arranged on the floor surface as shown in Fig. 11. Note that the XR device 10 is assumed to integrate the virtual spaces α, β, γ1, γ2, γ3, and γ4 in the following description into the virtual world space.

[0093] The XR device 10 identifies a reference point (hereinafter referred to as the shortest reference point) 63A that is closest to the rear emblem 4B among the multiple reference points 63, and measures the position of the rear emblem 4B in a virtual space α with the shortest reference point 63A as the origin. Similarly, the XR device 10 identifies a shortest reference point 63B that is closest to the front emblem 4A among the multiple reference points 63, and measures the position of the front emblem 4A in a virtual space β with the shortest reference point 63B as the origin. Then, the XR device 10 integrates the virtual space α and the virtual space β into a virtual world space, and in the virtual world space, connects the position of the rear emblem 4B and the position of the front emblem 4A with a straight line to set a center line (not shown). In this way, the XR device 10 can also identify the center line of the vehicle 2. Furthermore, by using the closest reference points 63A and 63B in this way, the XR device 10 can more accurately measure the positions of the emblems 4A and 4B.

[0094] Furthermore, when displaying the installation position of the target 5 calculated in the virtual world space, the XR device 10 may convert the installation position into an installation position in the virtual space γ1 with the nearest reference point 63C closest to the installation position of the target 5A as the origin, and display the target 5A superimposed on the captured image or superimposed on the field of view of the worker 1 (operator) wearing the XR device 10. Even in this way, the XR device 10 can accurately display the installation position of the target 5A.

[0095] Furthermore, as shown in FIG. 11, when targets 5B, 5C, and 5D are placed at the rear, right, and left sides of the vehicle 2, respectively, the XR device 10 may calculate and display the placement positions of the targets 5B, 5C, and 5D at the rear, right, and left sides of the vehicle 2, similar to the case where the target 5A is placed in front of the vehicle 2 described above.

[0096] For example, the XR device 10 calculates the installation position of the target 5B behind the vehicle 2 in the virtual world space, converts it to an installation position in the virtual space γ2 with the shortest reference point 63D closest to the target 5B behind the vehicle 2 as the origin, and displays it superimposed on the captured image or superimposed on the field of view of the worker 1 (operator) wearing the XR device 10.

[0097] For example, the XR device 10 calculates the installation position of the target 5C on the right side of the vehicle 2 in the virtual world space, converts it to an installation position in the virtual space γ3 with the shortest reference point 63E closest to the target 5C on the right side of the vehicle 2 as the origin, and displays it superimposed on the captured image or superimposed on the field of view of the worker 1 (operator) wearing the XR device 10.

[0098] For example, the XR device 10 calculates the installation position of the target 5D on the left side of the vehicle 2 in the virtual world space, converts it to an installation position in the virtual space γ4 with the shortest reference point 63F closest to the target 5D on the left side of the vehicle 2 as the origin, and displays it superimposed on the captured image or superimposed on the field of view of the worker 1 (operator) wearing the XR device 10.

[0099] FIG. 12 is a schematic diagram for explaining a first example of a method for calculating the installation position of the target 5 using the non-stretchable jig 70 according to the second embodiment.

[0100] 12, a non-stretchable rod-shaped jig (hereinafter simply referred to as jig) 70 on which a plurality of reference points 64 are arranged is installed next to the vehicle 2. The inter-reference point distance between the reference point 64A and the reference point 64B and the inter-reference point distance between the reference point 64B and the reference point 64C are each accurately measured in advance and stored in the XR device 10. In addition, the reference points 64 may be AR markers that can be detected and identified by the XR device 10.

[0101] For example, a jig 70 having three reference points 64A, 64B, and 64C as shown in FIG. 12 may be installed next to the vehicle 2, and the XR device 10 may calculate the installation position of the target 5 by the following method.

[0102] The XR device 10 measures the position of the emblem 4B at the rear of the vehicle 2 relative to the reference point 64A in a virtual space α with the reference point 64A of the jig 70 as the origin. The distance between the reference point 64A and the front emblem 4A is within the positioning limit distance of the XR device 10.

[0103] The XR device 10 measures the position of the emblem 4A in front of the vehicle 2 relative to the reference point 64B in a virtual space β with the reference point 64B of the jig 70 as the origin. The distance between the reference point 64B and the emblem 4A in front is within the positioning limit distance of the XR device 10.

[0104] The XR device 10 integrates the virtual space α, the virtual space β, and the virtual space γ whose origin is the reference point 64C of the jig 70 into a virtual world space.

[0105] The XR device 10 identifies the center line 42 of the vehicle 2 in the virtual world space and calculates the installation position of the target 5.

[0106] The XR device 10 converts the calculated installation position of the target 5 in the world space into an installation position in a virtual space γ with the reference point 64C of the jig 70 as the origin, and displays it superimposed on the captured image or superimposed on the field of view of the worker 1 (operator) wearing the XR device 10. The distance between the reference point 64C and the installation position of the target 5 is within the positioning limit distance of the XR device 10.

[0107] With this method, the XR device 10 can also accurately measure the installation position of the target 5. Furthermore, with this method, by simply installing the jig 70 beside the vehicle 2, the worker 1 can easily recognize the accurate installation position of the target 5 using the XR device 10.

[0108] FIG. 13 is a schematic diagram for explaining a second example of a method for calculating the installation position of the target 5 using the non-stretchable jig 70 according to the second embodiment.

[0109] 13, a non-stretchable rod-shaped jig 70 on which a plurality of reference points 65 are arranged is placed between the vehicle 2 and the position where the target 5 is to be placed. The distance between the reference point 65A and the reference point 65C (the distance between the reference points) is accurately measured in advance and stored in the XR device 10. The reference point 65 may be an AR marker that can be detected and identified by the XR device 10.

[0110] For example, a jig 70 having three reference points 65A, 65B, and 65C as shown in FIG. 13 may be placed between the vehicle 2 and the position where the target 5 is to be placed, and the XR device 10 may calculate the placement position of the target 5 by the following method.

[0111] The XR device 10 measures the position of the emblem 4A in front of the vehicle 2 relative to the reference point 65A (or the reference point 65B) in a virtual space β with the reference point 65A (or the reference point 65B) of the jig 70 as the origin. The distance between the reference point 65A (or the reference point 65B) and the emblem 4A in front is within the positioning limit distance of the XR device 10.

[0112] The XR device 10 integrates the virtual space β and the virtual space γ having the reference point 64C of the jig 70 as its origin into a virtual world space.

[0113] The XR device 10 calculates the center line 42 passing through the front emblem 4A by the method described in the first embodiment.

[0114] The XR device 10 calculates the installation position of the target 5 on an extension of the center line 42 in the virtual world space.

[0115] The XR device 10 converts the calculated installation position of the target 5 in the world space into an installation position in a virtual space γ with the reference point 65C of the jig 70 as the origin, and displays it superimposed on the captured image or superimposed on the field of view of the worker 1 (operator) wearing the XR device 10. The distance between the reference point 65C and the installation position of the target 5 is within the positioning limit distance of the XR device 10.

[0116] With this method as well, the XR device 10 can accurately calculate the installation position of the target 5. Furthermore, with this method, the worker 1 can easily recognize the accurate installation position of the target 5 using the XR device 10 by simply installing the jig 70 between the vehicle 2 and the position where the target 5 is to be installed.

[0117] (Summary of the second embodiment) The above description of the second embodiment discloses the following technology.

[0118] <Technology B1> In one embodiment, a method for providing installation position information includes the steps of: capturing an image including a vehicle (2) placed in the real space by a camera (11); setting a first virtual space having a first reference point among the plurality of reference points as an origin; measuring a position of a first predetermined part of the vehicle in the first virtual space; setting a second virtual space having a second reference point among the plurality of reference points as an origin; measuring a position of a second predetermined part of the vehicle in the second virtual space; integrating the first virtual space and the second virtual space into a virtual world space; setting a center line (42) of the vehicle in the virtual world space based on the positions of the first predetermined part and the second predetermined part; calculating an installation position (45) of a target (5) used to adjust a sensor mounted on the vehicle based on the center line in the virtual world space; and displaying information indicating the installation position of the target by superimposing it on the image captured by the camera or on an operator's field of view. As a result, even if the first predetermined portion and the second predetermined portion are located at positions separated from each other, the center line of the vehicle can be calculated with high accuracy.

[0119] <Technology B2> In the installation location information providing method described in Technology B1, the first reference point and the second reference point are the same point (e.g., reference point 61), and the first virtual space and the second virtual space are different virtual spaces (e.g., virtual space α and virtual space β). This makes it possible to calculate the center line of the vehicle with high accuracy even when the first reference point and the second reference point are the same.

[0120] <Technology B3> In the installation position information providing method described in Technology B1 or B2, a third virtual space (e.g., virtual space γ) is set with a third reference point among the multiple reference points as its origin, the third virtual space is further integrated into the virtual world space, the installation position of the target calculated in the virtual world space is converted into a position in the third virtual space, and the information indicating the installation position of the target includes a position in the third virtual space. This allows the target placement position to be calculated with high accuracy.

[0121] <Technology B4> In an installation location information providing method described in any one of Technologies B1 to B3, the first reference point is a reference point among the plurality of reference points that is closest to the first specified portion, and the second reference point is a reference point among the plurality of reference points that is closest to the second specified portion. This makes the distance between the first reference point and the first predetermined portion and the distance between the second reference point and the second predetermined portion relatively short, so that the center line of the vehicle can be calculated with higher accuracy.

[0122] <Technology B5> In the installation position information providing method described in Technology B3 or B4, the distance from the first reference point to the first specified portion in the real space, the distance from the second reference point to the second specified portion, and the distance from the third reference point to the installation position of the target calculated are each less than a predetermined distance. As a result, the distance from the first reference point to the first specified location, the distance from the second reference point to the second specified location, and the distance from the third reference point to the target installation position calculated are within a specified distance for which the accuracy of distance measurement by the camera is sufficient, so that the center line of the vehicle and the target installation position can be calculated with high accuracy.

[0123] <Technology B6> In the installation location information providing method described in Technology B5, at least one of the distance between the first reference point and the second reference point, the distance between the second reference point and the third reference point, or the distance between the first reference point and the third reference point is greater than the specified distance. In this way, even if the distance between the reference points is greater than a predetermined distance, the center line of the vehicle can be calculated with high accuracy.

[0124] <Technology B7> According to one embodiment, an installation position information providing device including a camera (11), a display unit (16), and a control unit (18) has a plurality of reference points arranged in a real space, and the distance between each of the reference points is determined in advance. The camera captures an image including a vehicle (2) arranged in the real space. The control unit sets a first virtual space having a first reference point among the plurality of reference points as an origin, measures the position of a first predetermined portion of the vehicle in the first virtual space, sets a second virtual space having a second reference point among the plurality of reference points as an origin, and calculates the position of a first predetermined portion of the vehicle in the first virtual space. a position of a second specified portion of the vehicle is determined, the first virtual space and the second virtual space are integrated into a virtual world space, a center line of the vehicle is set in the virtual world space based on the positions of the first specified portion and the second specified portion, an installation position of a target used to adjust a sensor mounted on the vehicle is calculated in the virtual world space based on the center line, and information indicating the installation position of the target is displayed on the display unit by superimposing it on the image captured by the camera or on the operator's field of view. As a result, even if the first predetermined portion and the second predetermined portion are located at positions separated from each other, the center line of the vehicle can be calculated with high accuracy.

[0125] <Technology B8> An installation position information providing program according to one embodiment causes a computer to execute the following steps: a plurality of reference points are arranged in a real space, the distance between each of the reference points is predetermined; a camera (11) is used to capture an image including a vehicle (2) arranged in the real space; a first virtual space having a first reference point among the plurality of reference points as an origin; a position of a first predetermined part of the vehicle in the first virtual space; a second virtual space having a second reference point among the plurality of reference points as an origin; a position of a second predetermined part of the vehicle in the second virtual space; an integration of the first virtual space and the second virtual space into a virtual world space; a center line of the vehicle is set in the virtual world space based on the positions of the first predetermined part and the second predetermined part; a calculation is made in the virtual world space based on the center line to calculate an installation position of a target used to adjust a sensor mounted on the vehicle; and information indicating the installation position of the target is superimposed on the image captured by the camera or on an operator's field of vision and displayed. As a result, even if the first predetermined portion and the second predetermined portion are located at positions separated from each other, the center line of the vehicle can be calculated with high accuracy.

[0126] Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, corrections, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also belong to the technical scope of the present disclosure. In addition, the components in the above-mentioned embodiments may be arbitrarily combined within the scope of the invention. [Industrial Applicability]

[0127] The technology disclosed herein is useful for improving the efficiency of vehicle aiming operations. [Explanation of symbols]

[0128] 1. Worker 2 Vehicles 3. Floor surface 4A, 4B Emblem 5. Target 10 XR Devices 11 Camera 12 Spatial recognition sensor 13 Position recognition sensor 14 Input section 15 Communications Department 16 Display section 17 Audio output section 18 Control Unit 19 Memory section 30 PC 31 Server 41 straight line 42 Center line 45 Installation position 50 Virtual objects for measurement 55 Virtual Objects for Targets 60 Positionable space 61 1st reference point 62 Second reference point 63 Reference point 63A, 63B, 63C, 63D, 63E, 63F Shortest reference point 64, 64A, 64B, 64C reference point 65, 65A, 65B, 65C reference point 70 Jig

Claims

1. The camera captures images including the vehicle, Identifying a predetermined portion of the 3D object of the vehicle; projecting a group of measurement virtual objects, which are made up of a plurality of measurement virtual objects, from above toward below at least a portion of the 3D object of the vehicle; the group of measurement virtual objects is divided into a left region and a right region by a straight line passing through the position of the predetermined portion, and an orientation of the straight line and the group of measurement virtual objects is adjusted so that a difference between the number of the measurement virtual objects that have collided with the 3D object of the vehicle in the left region and the number of the measurement virtual objects that have collided with the 3D object of the vehicle in the right region is less than a predetermined threshold value, and the straight line upon completion of the adjustment is set as a center line of the vehicle; calculating an installation position of a target used for adjusting a sensor mounted on the vehicle based on the center line and a height of the predetermined portion defined in advance; information indicating the installation position of the target is displayed by being superimposed on the image captured by the camera or on the field of view of the operator; Installation location information provision method.

2. The camera captures images including the vehicle, Identifying a predetermined portion of the 3D object of the vehicle; projecting a group of measurement virtual objects, which are made up of a plurality of measurement virtual objects, from above toward below at least a portion of the 3D object of the vehicle; dividing the group of measurement virtual objects into a left region and a right region by a straight line that passes through the position of the predetermined portion and is parallel to a floor surface, adjusting orientations of the straight line and the group of measurement virtual objects so that a difference between the number of the measurement virtual objects that have collided with the 3D object of the vehicle in the left region and the number of the measurement virtual objects that have collided with the 3D object of the vehicle in the right region is less than a predetermined threshold, and setting the straight line upon completion of the adjustment as a center line of the vehicle; calculating an installation position of a target used for adjusting a sensor mounted on the vehicle based on the center line and the floor surface; information indicating the installation position of the target is displayed by being superimposed on the image captured by the camera or on the field of view of the operator; Installation location information provision method.

3. The center line is displayed by being superimposed on the image captured by the camera or on the field of view of the operator. The method for providing installation location information according to claim 1 or 2.

4. projecting a group of target virtual objects, which are made up of a plurality of target virtual objects, along the center line toward the target 3D object installed at the installation position; determining whether the number of the target virtual objects that do not collide with the target 3D object is less than a predetermined threshold; If the number of the target virtual objects that do not collide with the target 3D object is less than a predetermined threshold, notify the user of the fact. The method for providing installation location information according to claim 1 or 2.

5. An installation location information providing device including a camera, a display unit, and a control unit, The camera captures an image including a vehicle, The control unit is Identifying a predetermined portion of the 3D object of the vehicle; projecting a group of measurement virtual objects, which are made up of a plurality of measurement virtual objects, from above toward below at least a portion of the 3D object of the vehicle; the group of measurement virtual objects is divided into a left region and a right region by a straight line passing through the position of the predetermined portion, and an orientation of the straight line and the group of measurement virtual objects is adjusted so that a difference between the number of the measurement virtual objects that have collided with the 3D object of the vehicle in the left region and the number of the measurement virtual objects that have collided with the 3D object of the vehicle in the right region is less than a predetermined threshold value, and the straight line upon completion of the adjustment is set as a center line of the vehicle; calculating an installation position of a target used for adjusting a sensor mounted on the vehicle based on the center line and a height of the predetermined portion defined in advance; information indicating the installation position of the target is displayed on the display unit by superimposing the information on the image captured by the camera or on the field of view of the operator; Installation location information providing device.

6. An installation location information providing device including a camera, a display unit, and a control unit, The camera captures an image including a vehicle, The control unit is Identifying a predetermined portion of the 3D object of the vehicle; projecting a group of measurement virtual objects, which are made up of a plurality of measurement virtual objects, from above toward below at least a portion of the 3D object of the vehicle; dividing the group of measurement virtual objects into a left region and a right region by a straight line that passes through the position of the predetermined portion and is parallel to a floor surface, adjusting orientations of the straight line and the group of measurement virtual objects so that a difference between the number of the measurement virtual objects that have collided with the 3D object of the vehicle in the left region and the number of the measurement virtual objects that have collided with the 3D object of the vehicle in the right region is less than a predetermined threshold, and setting the straight line upon completion of the adjustment as a center line of the vehicle; calculating an installation position of a target used for adjusting a sensor mounted on the vehicle based on the center line and the floor surface; information indicating the installation position of the target is displayed on the display unit by superimposing the information on the image captured by the camera or on the field of view of the operator; Installation location information providing device.

7. The camera captures images including the vehicle, Identifying a predetermined portion of the 3D object of the vehicle; projecting a group of measurement virtual objects, which are made up of a plurality of measurement virtual objects, from above toward below at least a portion of the 3D object of the vehicle; the group of measurement virtual objects is divided into a left region and a right region by a straight line passing through the position of the predetermined portion, and an orientation of the straight line and the group of measurement virtual objects is adjusted so that a difference between the number of the measurement virtual objects that have collided with the 3D object of the vehicle in the left region and the number of the measurement virtual objects that have collided with the 3D object of the vehicle in the right region is less than a predetermined threshold value, and the straight line upon completion of the adjustment is set as a center line of the vehicle; calculating an installation position of a target used for adjusting a sensor mounted on the vehicle based on the center line and a height of the predetermined portion defined in advance; information indicating the installation position of the target is displayed by being superimposed on the image captured by the camera or on the field of view of the operator; An installation location information providing program that causes a computer to execute the above.

8. The camera captures images including the vehicle, Identifying a predetermined portion of the 3D object of the vehicle; projecting a group of measurement virtual objects, which are made up of a plurality of measurement virtual objects, from above toward below at least a portion of the 3D object of the vehicle; dividing the group of measurement virtual objects into a left region and a right region by a straight line that passes through the position of the predetermined portion and is parallel to a floor surface, adjusting orientations of the straight line and the group of measurement virtual objects so that a difference between the number of the measurement virtual objects that have collided with the 3D object of the vehicle in the left region and the number of the measurement virtual objects that have collided with the 3D object of the vehicle in the right region is less than a predetermined threshold, and setting the straight line upon completion of the adjustment as a center line of the vehicle; calculating an installation position of a target used for adjusting a sensor mounted on the vehicle based on the center line and the floor surface; information indicating the installation position of the target is displayed by being superimposed on the image captured by the camera or on the field of view of the operator; An installation location information providing program that causes a computer to execute the above.

Images