Construction assistance method, construction assistance device, and information processing program product

Abstract

The invention provides a construction assistance method, a construction assistance device, and an information processing program product, and solves the problem that a place where an existing coating device for road marking can be used is restricted. The construction assistance method includes: acquiring a captured image (IG) representing a road surface (RS) and an object on the road surface (RS); a position determination unit that determines, on the road surface (RS), a position at which a first image for coating a road surface mark (RM) on the road surface (RS) is projected on the basis of an object on the road surface (RS) included in the captured image (IG); and projecting the first image from the optical device to a position on the road surface (RS).

Description

Technical Field

[0001] The present invention relates to a construction assisting method, a construction assisting device and a program product for information processing. Background Art

[0002] Patent Document 1 discloses a road marking coating device for painting road markings. The road marking coating device includes a positioning unit that receives positioning signal waves from satellites to determine the current position. The positioning unit receives signal waves from GPS satellites and Quasi-Zenith Satellites, allowing the road marking coating device to automatically perform positioning and apply the road markings.

[0003] Patent Document 1: Japanese Patent Application Laid-Open No. 2015-086590

[0004] However, the road marking coating device of Patent Document 1 uses satellite signals for positioning, so positioning accuracy decreases in places where satellite radio waves are difficult to reach, such as inside tunnels or in the shadows of buildings. Therefore, there are limitations on where this road marking coating device can be used. Summary of the Invention

[0005] A construction assistance method according to one embodiment of the present invention includes: obtaining a captured image representing a road surface and objects on the road surface; determining a position on the road surface for projecting a first image based on the objects on the road surface contained in the captured image, the first image being used for painting road surface markings on the road surface; and projecting the first image from an optical device to a position on the road surface.

[0006] A construction assistance device according to one embodiment of the present invention includes a processing device that performs the following processing: obtaining a captured image representing a road surface and objects on the road surface; determining a position on the road surface for projecting a first image based on the objects on the road surface contained in the captured image, the first image being used for painting road surface markings on the road surface; and causing an optical device to project the first image to a position on the road surface.

[0007] A program product for information processing in one embodiment of the present invention enables a computer to perform the following processing: obtaining a captured image representing a road surface and objects on the road surface; determining a position on the road surface for projecting a first image based on the objects on the road surface contained in the captured image, the first image being used for painting a road surface marking on the road surface; and causing an optical device to project the first image toward a position on the road surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 It is a block diagram showing the configuration of the construction support device 1 .

[0009] Figure 2 This is an external view of the construction assisting device 1 .

[0010] Figure 3 1 is a diagram showing an example of projection of an image representing a drawing pattern of a road surface marking RM on the road surface RS.

[0011] Figure 4 It is a block diagram showing a configuration example of the information processing device 10 .

[0012] Figure 5 2 is a block diagram showing a configuration example of the projector 20 .

[0013] Figure 6 1 is a flowchart showing a method of projecting a projection image PI showing a drawing pattern by the construction support device 1 .

[0014] Figure 7 1 is a flowchart showing a method of projecting a projection image PI showing a drawing pattern by the construction support device 1 .

[0015] Figure 8 This is an example of a display screen displayed on the display device 150 .

[0016] Figure 9 This is an example of a display screen displayed on the display device 150 .

[0017] Figure 10 This is an example of a display screen displayed on the display device 150 .

[0018] Figure 11 This is an example of a display screen displayed on the display device 150 .

[0019] Figure 12 This is an example of a display screen displayed on the display device 150 .

[0020] Figure 13 This is an example of a display screen displayed on the display device 150 .

[0021] Figure 14 This is an example of a display screen displayed on the display device 150 .

[0022] Figure 15 1 is a diagram showing an example of projection of an image representing a drawing pattern of a road surface marking RM on the road surface RS.

[0023] Figure 16 1 is a diagram showing an example of projection of an image representing a drawing pattern of a road surface marking RM on the road surface RS.

[0024] Figure 17 It is a block diagram showing a configuration example of an information processing device 10A.

[0025] Figure 18 This is a flowchart showing a method of projecting a projection image showing a drawing pattern by the construction support device 1A.

[0026] Figure 19 This is a flowchart showing a method of projecting a projection image showing a drawing pattern by the construction support device 1A.

[0027] Figure 20 1 is a diagram showing an example of a projection image PI showing a drawing pattern.

[0028] Figure 21 1 is a diagram showing a captured image IG when the projection image PI is projected onto the road surface RS.

[0029] Figure 22 1 is a diagram showing an example of a projection image PI indicating the outline of the first character L1 .

[0030] Figure 23 1 is a diagram showing an example of a projection image PI representing a reference pattern.

[0031] Figure 24 It is a diagram showing the captured image IG.

[0032] Description of labels

[0033] 1: Construction assistance device; 1A: Construction assistance device; 10, 10A: Information processing device; 11: Main body; 20: Projector; 21: Main body; 110: Shooting device; 120, 120A: Processing device; 121: Acquisition unit; 122: Display control unit; 123, 123A: Detection unit; 124: Acceptance unit; 125, 125A: Determination unit; 126: Projection control unit; 127: Extraction unit; 128: Comparison unit; 129: Notification unit; 140, 140A: Storage device; 150: Display device; 160: Input device; 170: Communication device; 210: Projection device; 220: Processing device; 221: Acquisition unit; 222: Projection control unit; 230: Storage device; 240: Communication device; 310: Projection device. DETAILED DESCRIPTION

[0034] The following describes embodiments of the present invention with reference to the accompanying drawings. However, the dimensions and scales of the various components in the drawings may differ from the actual dimensions and scales as appropriate. Furthermore, the embodiments described below are preferred specific examples of the present invention and therefore include various technically preferred limitations. However, the scope of the present invention is not limited to these embodiments unless otherwise specified in the following description.

[0035] 1: First Implementation

[0036] Below, refer to Figures 1 to 15 The construction support device 1 according to this embodiment will be described.

[0037] 1-1: Overall structure

[0038] Figure 1 1 is a block diagram showing the configuration of the construction support device 1 according to the present embodiment. The construction support device 1 includes an information processing device 10 and a projector 20. The information processing device 10 and the projector 20 are connected to each other so as to be able to communicate with each other.

[0039] The construction support device 1 projects an image representing a drawing pattern used to draw a road surface marking onto the road surface RS. For example, the drawing pattern includes the outline of the road surface marking RM drawn on the road surface. The drawing pattern may also include an auxiliary image AI used to draw the road surface marking RM. For example, the road surface marking RM is drawn by a user U of the construction support device 1 applying paint to the road surface using a known coating device separate from the construction support device 1.

[0040] The projector 20 projects an image representing the above-described drawing pattern onto the road surface RS. The information processing device 10 controls the projector 20. The information processing device 10 controls the projector 20, and the projector 20 projects an image representing the drawing pattern onto the road surface RS.

[0041] Figure 2 This is an example of an external view of the construction support device 1 according to the present embodiment.

[0042] exist Figure 2 In the xyz space, consider three axes: the x-axis, the y-axis, and the z-axis. The x-axis, the y-axis, and the z-axis are orthogonal to each other. In this xyz space, the z-axis direction is the vertical direction. This z-axis direction is the general term for the z1 direction and the z2 direction, which is opposite to the z1 direction. The z1 direction is the direction facing vertically downward. The z2 direction is the direction facing vertically upward. Furthermore, the x-axis direction is the same direction as the line of symmetry of the construction support device 1 in the xy plane. This x-axis direction is the general term for the x1 direction and the x2 direction, which is opposite to the x1 direction. The x1 direction is the direction facing forward of the construction support device 1. The x2 direction is the direction facing backward of the construction support device 1. More specifically, in the construction support device 1, of the first position including the information processing device 10 and the second position including the projector 20, the first position is located in the x2 direction relative to the second position. Furthermore, the y-axis direction is the general term for the y1 direction and the y2 direction. When viewing the xy plane with the x1 direction upward toward the z1 direction, the y1 direction is the right direction. When the xy plane is viewed from the x1 direction upward and the z1 direction is viewed, the y2 direction is the left direction.

[0043] exist Figure 2In the example, the construction support device 1 includes a cart PD. The cart PD has wheels W1 and W2 on its surface in the z1 direction. The wheels W1 and W2 enable the cart PD to move on the road surface RS. Furthermore, the cart PD includes a handle HD. A user U of the construction support device 1 grips the handle HD and applies force to the cart PD, thereby moving the cart PD on the road surface RS.

[0044] Furthermore, the information processing device 10 and the projector 20 are placed on the carriage PD in the z2 direction relative to the carriage PD. The information processing device 10 and the projector 20 are connected to each other via a cable CB so as to be communicable with each other.

[0045] The information processing device 10 includes a main body 11 and a display device 150. The main body 11 includes a processing device 120 described later. The display device 150 is placed on the main body 11 in the z2 direction.

[0046] The display device 150 is a device that displays images and text information. The display device 150 displays various images under the control of the processing device 120. For example, various display panels such as liquid crystal display panels and organic EL (Electro Luminescence) display panels are suitable for use as the display device 150.

[0047] The user U of the construction support device 1 can visually check various images and text information displayed on the display device 150 while gripping the handle HD.

[0048] The rotating table RT is placed on the z2-direction surface of the carriage PD. The rotating table RT is located in the x1 direction relative to the information processing device 10. The rotating table RT rotates 360 degrees around the z-axis.

[0049] The projector 20 is mounted on the z2-direction surface of the rotating stage RT. The projector 20 includes a main body 21 and a projection device 210. The main body 21 includes a processing device 220 (described later) and at least a portion of the projection device 210. As an example, the main body 21 has a cylindrical shape with the z-axis as its central axis. As an example, a portion of the projection device 210 and an opening for emitting light from the projection device 210 are provided on the side of the main body 21.

[0050] The projection device 210 is a device that projects the image generated by the information processing device 10 onto the road surface RS. Figure 2In the example shown, the projection device 210 projects the image in the x1 direction. However, the projector 20 equipped with the projection device 210 is placed on the turntable RT, and the projector 20 rotates on the turntable RT. Therefore, the projector 20 can project the image generated by the information processing device 10 in any direction among the x1 direction, x2 direction, y1 direction, and y2 direction. Also, the projector 20 can project the image generated by the information processing device 10 in any direction between the x1 direction and the y2 direction, any direction between the y2 direction and the x2 direction, any direction between the x2 direction and the y1 direction, and any direction between the y1 direction and the x1 direction.

[0051] Figure 3 FIG. is a projection example showing the construction assistance device 1 projecting an image depicting a road surface marking RM onto the road surface RS.

[0052] In Figure 3 , an XYZ space formed by three axes, the X-axis, Y-axis, and Z-axis, is envisioned. The X-axis, Y-axis, and Z-axis are mutually orthogonal. In the XYZ space, the Z-axis direction is the vertical direction. This Z-axis direction is a general term for the Z1 direction and the Z2 direction opposite to the Z1 direction. The Z1 direction is the direction toward the vertical downward. The Z2 direction is the direction toward the vertical upward. Additionally, the X-axis direction is the extension direction of the center line CL of the lane depicted on the road surface RS. This X-axis direction is a general term for the X1 direction and the X2 direction opposite to the X1 direction. The X1 direction is the direction toward the top of the road surface marking RM. The X2 direction is the direction toward the bottom of the road surface marking RM. Additionally, the Y-axis direction is the extension direction of the stop line SL depicted on the road surface RS. This Y-axis direction is a general term for the Y1 direction and the Y2 direction. When observing the road surface RS with the X1 direction as the upward direction and the Z1 direction, the Y1 direction is the right direction. When observing the road surface RS with the X1 direction as the upward direction and the Z1 direction, the Y2 direction is the left direction.

[0053] In Figure 3 In the example shown, the road surface marking RM includes three characters, the first character L1, the second character L2, and the third character L3. More specifically, in Figure 3 In the example shown, the road surface marking RM represents the three characters "止まれ". The first character L1 represents the character "止" in "止まれ". The second character L2 represents the character "ま" in "止まれ". The third character L3 represents the character "れ" in "止まれ". However, the road surface marking RM can include any number of characters. Additionally, the road surface marking RM can be any characters. Also, the road surface marking RM can include graphics other than characters. Furthermore, in Figure 3For illustration purposes, the second and third characters L2 and L3 are shown. In this embodiment, an example of sequential construction starting with the first character L1 is described. Therefore, the second and third characters L2 and L3 are not yet projected on the road surface RS at the time the first character L1 is projected. The user U can only visually recognize the image of the first character L1 of the three characters on the road surface RS.

[0054] The construction support device 1 projects the road surface marking RM onto the road surface RS one by one. Figure 3 In the example shown, the construction support device 1 projects the outline of the first character L1 in the road surface marking RM onto an area on the road surface RS that is in the Y2 direction relative to the lane centerline CL and in the X2 direction relative to the stop line SL. Details of the method for projecting the outline of the first character L1 by the construction support device 1 will be described later.

[0055] In addition, Figure 2 as well as Figure 3 In the above, the road surface RS can be an outdoor road surface or an indoor road surface. Figure 2 and Figure 3 In the example, the construction support device 1 projects the outline of the first character L1 onto the road surface RS. However, this is merely an example. The construction support device 1 may project the outline of the first character L1 onto a construction surface other than the road surface. For example, the construction support device 1 may project the outline of the first character L1 onto the construction surface of a multi-story parking garage or a sports field.

[0056] 1-2: Structure of information processing device

[0057] Figure 4 This is a block diagram showing an example configuration of an information processing device 10. While a PC (Personal Computer) is typically used as the information processing device 10, the present invention is not limited thereto and may also be a tablet terminal or a smartphone. The information processing device 10 includes an image capture device 110, a processing device 120, a storage device 140, a display device 150, an input device 160, and a communication device 170. The various components of the information processing device 10 are interconnected via a single bus or multiple buses for communicating information.

[0058] The camera 110 is a device that captures images of the road surface RS. Under the control of the processing device 120, the camera 110 captures various images. For example, cameras included in PCs, tablets, and smartphones are suitable for use as the camera 110, but the camera is not limited thereto and may also be an external camera such as a webcam.

[0059] The processing device 120 is a processor that controls the entire information processing device 10 and is composed of, for example, a single or multiple chips. The processing device 120 is composed of, for example, a central processing unit (CPU: Central Processing Unit) that includes an interface with peripheral devices, an arithmetic device, registers, and the like. In addition, part or all of the functions of the processing device 120 can also be implemented by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The processing device 120 executes various processes in parallel or sequentially.

[0060] The storage device 140 is a recording medium that can be read from and written to by the processing device 120, and stores a plurality of programs including the control program PR1 executed by the processing device 120 and the drawing pattern data DD. The storage device 140 can be composed of, for example, at least one of a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), a RAM (Random Access Memory), and the like. The storage device 140 can also be referred to as a register, a cache, a main memory, or a main storage device, etc.

[0061] The drawing pattern data DD is data related to the drawing pattern. In the present embodiment, the drawing pattern refers to the outline of the road surface marking RM. In Figure 3 the example shown, the drawing pattern is the outline of the three characters "TOMARE" as the road surface marking RM, but this is just an example. The drawing pattern includes Figure 3 the outlines of various road surface markings other than the road surface marking RM exemplified.

[0062] As described above, the display device 150 is a device that displays images and text information. The display device 150 can also be a display device separate from other components of the information processing device 10.

[0063] The input device 160 is a device that accepts operations from the user U. For example, the input device 160 is configured to include an indicating device such as a keyboard, a touchpad, a touch panel, or a mouse. Here, when the input device 160 is configured to include a touch panel, it can also serve as the display device 150.

[0064] The communication device 170 is hardware that serves as a transceiver for communicating with other devices. The communication device 170 is also known as a network device, a network controller, a network card, or a communication module. The communication device 170 may also include a connector for wired connection and an interface circuit corresponding to the connector. In addition, the communication device 170 may also include a wireless communication interface. Examples of connectors and interface circuits for wired connection include those based on wired LAN (Local Area Network), IEEE1394, and USB (Universal Serial Bus). In addition, examples of wireless communication interfaces include interfaces based on wireless LAN, Bluetooth (registered trademark), and the like.

[0065] The processing device 120 reads and executes the control program PR1 from the storage device 140, thereby functioning as an acquisition unit 121, a display control unit 122, a detection unit 123, a reception unit 124, a determination unit 125, and a projection control unit 126. Alternatively, the control program PR1 may be transmitted from another device, such as a server, that manages the information processing device 10 via a communication network.

[0066] The acquisition unit 121 acquires a captured image showing the road surface RS and objects on the road surface RS. The captured image is an image captured by the camera 110. Here, "objects on the road surface RS" include at least one of road surface markings pre-painted on the road surface RS, structures fixed to the road surface RS, and objects placed on the road surface RS, at least a portion of which is a straight line. Here, "road surface markings pre-painted on the road surface RS" is, for example, Figure 3 The lane center line CL and stop line SL are shown. Furthermore, a "structure fixed to the road surface RS" is, for example, a curb placed on the road surface RS. The curb is preferably rectangular. Furthermore, an "object placed on the road surface RS, at least a portion of which is straight," is, for example, a ruler.

[0067] exist Figure 3 In the example shown, the camera 110 captures an image of an imaging area IA on the road surface RS. The imaging area IA includes the road surface RS, a portion of the lane center line CL, and a portion of the stop line SL within the imaging area IA. The portion of the lane center line CL and the portion of the stop line SL correspond to the aforementioned "objects on the road surface RS."

[0068] The acquisition unit 121 acquires an image of the imaging area IA captured by the imaging device 110 .

[0069] The display control unit 122 causes the display device 150 to display the captured image IG acquired by the acquisition unit 121. The user U can visually recognize the captured image IG displayed on the display device 150.

[0070] The detection unit 123 detects the straight line included in the object on the road surface RS from the captured image IG. Figure 3 In the imaging area IA, the detection unit 123 detects a straight line included in the outline of the lane center line CL and a straight line included in the outline of the stop line SL.

[0071] The detection unit 123 also detects edges in the drawing pattern represented by the drawing pattern data DD. For example, the detection unit 123 divides the contour line of the road surface marking RM represented by the drawing pattern into a plurality of line segments having a maximum length within a range where each segment does not bend, and detects each line segment as an edge.

[0072] The receiving unit 124 receives an operation to select two parallel line segments representing at least a portion of the contour line of an object on the road surface RS from the captured image IG. As described above, the captured image IG is displayed on the display device 150. The user U uses the input device 160 to select two parallel line segments representing at least a portion of the contour line of the object on the road surface RS. For example, the user U selects two parallel line segments representing at least a portion of the contour line of the lane center line CL or the stop line SL displayed on the display device 150 on the touch panel included in the display device 150 as the input device 160. The receiving unit 124 receives the operation of selecting the two line segments by the user U. In addition, the two line segments are an example of the "first line segment."

[0073] When the user U selects the two line segments, the display control unit 122 may cause the display device 150 to display an auxiliary image AI representing the two line segments. As an example, the display control unit 122 preferably sets the thickness of the auxiliary image AI to a thickness different from that of the two line segments included in the captured image IG. In addition, the display control unit 122 preferably sets the color of the auxiliary image AI to a color different from that of the two line segments included in the captured image IG. Figures 8 to 14 A specific example of the captured image IG and the auxiliary image AI displayed on the display device 150 will be described.

[0074] Furthermore, the receiving unit 124 receives an input indicating the distance between two line segments. This distance is an example of a “first distance”.

[0075] When the user U inputs the distance, the display control unit 122 may cause the display device 150 to display an auxiliary image AI indicating which part of the captured image the distance corresponds to. Furthermore, the display control unit 122 causes the display device 150 to display the distance input field.

[0076] As described above, the construction support device 1 projects a projection image PI representing a depicted pattern onto the road surface RS. The projection image PI representing the depicted pattern includes multiple line segments representing at least a portion of the outline of the road surface marking RM. The receiving unit 124 receives an operation to select two parallel line segments from the projected image PI representing the depicted pattern. These two line segments are examples of "second line segments." Furthermore, the projection image PI representing the depicted pattern is an example of a "first image."

[0077] When the user U selects the two line segments included in the projected image PI representing the depicted pattern, the display control unit 122 may cause the display device 150 to display an auxiliary image AI representing the two line segments. For example, the display control unit 122 preferably sets the thickness of the auxiliary image AI to a different thickness than the thickness of the two line segments included in the projected image PI representing the depicted pattern. Furthermore, the display control unit 122 preferably sets the color of the auxiliary image AI to a different color than the color of the two line segments included in the projected image PI representing the depicted pattern.

[0078] Furthermore, the receiving unit 124 receives an input indicating the distance between the two line segments included in the projection image PI representing the drawing pattern. This distance is an example of a “second distance”.

[0079] When the user U inputs the distance, the display control unit 122 may cause the display device 150 to display an auxiliary image AI indicating to which portion of the projected image PI representing the depicted pattern the distance corresponds. Furthermore, the display control unit 122 causes the display device 150 to display an input field indicating the distance between the two line segments included in the projected image PI representing the depicted pattern.

[0080] The determination unit 125 determines the position on the road surface RS at which to project the projection image PI representing the depicted pattern used for painting the road surface marking RM on the road surface RS, based on the object on the road surface RS included in the captured image IG. More specifically, the determination unit 125 determines the position on the road surface RS at which to project the projection image PI representing the depicted pattern based on two parallel line segments selected by the user U that represent at least a portion of the outline of the object on the road surface RS, and the distance between the two line segments.

[0081] In this embodiment, the determination unit 125 further determines the position of the projection image PI representing the depicted pattern on the road surface RS based on two parallel line segments representing at least a portion of the contour line of the road surface marking RM selected by the user U and the distance between the two line segments.

[0082] In addition, regarding the specific method of determining the position of the projection image PI representing the drawing pattern projected on the road surface RS by the determination unit 125, reference will be made to the following. Figures 6 to 15 This will be described later.

[0083] The projection control unit 126 projects the projection image PI representing the drawing pattern from the projector 20 to the position on the road surface RS determined by the determination unit 125. More specifically, the projection control unit 126 transmits an image signal corresponding to the projection image PI representing the drawing pattern and a control signal for controlling the projector 20 to the projector 20 via the communication device 170.

[0084] 1-3: Structure of the projector

[0085] Figure 5 This is a block diagram showing an example of the structure of a projector 20. The projector 20 includes a projection device 210, a processing device 220, a storage device 230, and a communication device 240. The various components of the projector 20 are interconnected via a single bus or multiple buses for communicating information. Furthermore, the various components of the projector 20 may be composed of a single device or multiple devices, and some components of the projector 20 may be omitted.

[0086] The projection device 210 projects an image represented by an image signal acquired by the acquisition unit 221, described later, onto a screen, a wall, or the like. The projection device 210 projects various images under the control of the processing device 220. The projection device 210 includes, for example, a light source, a liquid crystal panel, and a projection lens. The liquid crystal panel modulates light from the light source and projects the modulated light onto a screen, a wall, or the like via the projection lens.

[0087] The processing device 220 is a processor that controls the entire projector 20 and is comprised of, for example, a single chip or multiple chips. The processing device 220 comprises, for example, a central processing unit (CPU) that includes interfaces with peripheral devices, a computing device, and registers. Alternatively, some or all of the functions of the processing device 220 may be implemented using hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The processing device 220 executes various processes in parallel or sequentially.

[0088] Storage device 230 is a recording medium readable by processing device 220 and stores a plurality of programs, including control program PR2, executed by processing device 220. Storage device 230 may be composed of, for example, at least one of ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), and RAM (Random Access Memory). Storage device 230 may also be referred to as a register, cache, main memory, or main storage device.

[0089] The communication device 240 is hardware that serves as a transceiver for communicating with other devices. The communication device 240 is also referred to as, for example, a network device, a network controller, a network card, or a communication module. The communication device 240 may also include a connector for a wired connection and an interface circuit corresponding to the connector. Furthermore, the communication device 240 may also include a wireless communication interface. Examples of connectors and interface circuits for wired connections include those based on wired LAN, IEEE1394, and USB. Furthermore, examples of wireless communication interfaces include interfaces based on wireless LAN, Bluetooth (registered trademark), and the like.

[0090] The processing device 220 reads and executes the control program PR2 from the storage device 230 to function as an acquisition unit 221 and a projection control unit 222. The control program PR2 may be transmitted from another device such as a server that manages the projector 20 via a communication network.

[0091] The acquisition unit 221 acquires an image signal corresponding to the projection image PI representing the drawing pattern and a control signal for controlling the projector 20 from the information processing device 10 .

[0092] Based on the control signal acquired by the acquisition unit 221 , the projection control unit 222 projects a projection image PI representing a drawing pattern corresponding to the image signal acquired by the acquisition unit 221 onto the road surface RS via the projection device 310 .

[0093] In this embodiment, the projection device 210 is an example of an “optical device.” The projector 20 is another example of an “optical device.”

[0094] 1-4: Projection method for depicting patterns

[0095] 1-4-1: Projection method of the first character

[0096] Figure 6 and Figure 71 is a flowchart showing a method for projecting a projection image PI showing a drawing pattern by the construction support device 1. More specifically, Figure 6 and Figure 7 1 is a flowchart showing a method of projecting a projection image PI corresponding to an image representing the first character L1 included in a drawing pattern. Figure 6 and Figure 7 In FIG, steps including operations of the user U are shown by dotted lines.

[0097] In step S1 , the user U carries the construction support device 1 to an approximate position where the road surface marking RM is drawn on the road surface RS.

[0098] In step S2, the processing device 120 functions as the projection control unit 126. The processing device 120 causes the projector 20 to project a fixed image. This fixed image is displayed in the center of the liquid crystal panel included in the projection device 210 of the projector 20 and projected onto the road surface RS. This fixed image is, for example, a square. As an example, the user U measures the length of the diagonal of the square projected onto the road surface RS. Furthermore, the user U inputs the length of this diagonal into the information processing device 10 using the input device 160.

[0099] The processing device 120 functions as an acquisition unit 121. The processing device 120 acquires the length of the diagonal line input by the user U, or more precisely, acquires information indicating the length of the diagonal line input by the user U. Furthermore, based on the length and the number of pixels corresponding to the diagonal line on the liquid crystal panel of the projection device 210, the processing device 120 calculates the correspondence between the size of a single pixel on the liquid crystal panel of the projection device 210 and the distance on the road surface RS.

[0100] In step S3, the user U uses the imaging device 110 of the information processing device 10 to capture the road surface RS. In step S3, the fixed image of step S2 may not be projected. Figure 3 In the example shown, the user U uses the camera 110 to capture an image of the capture area IA on the road surface RS. Furthermore, the processing device 120 functions as an acquisition unit 121. The processing device 120 acquires the captured image IG captured by the camera 110. Furthermore, the processing device 120 may instruct the camera 110 to capture an image if information indicating the length of a diagonal line is input in step S2.

[0101] In step S4, the processing device 120 functions as the detection unit 123. The processing device 120 detects line segments included in objects on the road surface RS from the captured image IG. Figure 3The processing device 120 detects line segments included in a portion of the stop line SL and a portion of the lane center line CL, among objects on the road surface RS within the imaging area IA. More specifically, the processing device 120 detects line segment LN1 extending in the X-axis direction, and line segments LN2 and LN3 extending in the Y-axis direction, within the outline of the stop line SL. Furthermore, the processing device 120 detects line segments LN4 and LN5 extending in the X-axis direction, and line segment LN6 extending in the Y-axis direction, within the outline of the lane center line CL. Line segments LN1 through LN6 are examples of "auxiliary images."

[0102] In step S5, processing device 120 functions as detection unit 123. Processing device 120 detects edges from projection image PI, which represents the drawing pattern projected onto road surface RS. Processing device 120 detects edges contained in projection image PI based on drawing pattern data DD. In step S5, projection image PI may or may not be projected onto road surface RS.

[0103] In step S6 , the user U uses the input device 160 to select two parallel line segments (A) from the line segments included in the object on the road surface RS as reference lines.

[0104] Figure 8 as well as Figure 9 This is an example of the display screen displayed on the display device 150 in step S6. Figure 8 Indicates the first display screen DS1. Figure 9 Indicates the second display screen DS2.

[0105] The first display screen DS1 includes a message M1, an image display window IW, a first button BT1, and a second button BT2. In the first display screen DS1, a captured image IG is displayed in the image display window IW. The captured image IG is a captured image. Figure 3 The image obtained by capturing the captured area IA in the image IG includes the stop line SL and the lane center line CL as objects on the road surface RS. Furthermore, in the first display screen DS1, a message M1 includes a statement prompting the user U to select two parallel line segments (A) from the line segments included in the objects on the road surface RS in the captured image IG. As an example, assume that the user U selects line segments LN2 and LN3 included in the outline of the stop line SL. Since the selected line segments are LN2 and LN3 and there is no problem, the user U presses or clicks the "OK" button, i.e., the first button BT1, to indicate their agreement. Furthermore, the second button BT2 is a cancel button.

[0106] When the user U presses or clicks the first button BT1, the second display screen DS2 is displayed on the display device 150. While details of the second display screen DS2 will be described later, the second display screen DS2 displays an auxiliary image AI1 for emphasizing line segment LN2 and an auxiliary image AI2 for emphasizing line segment LN3. Auxiliary image AI1 is thicker than line segment LN2 and has a different color than line segment LN2. Similarly, auxiliary image AI2 is thicker than line segment LN3 and has a different color than line segment LN3.

[0107] In step S6, when the user U selects two line segments (A), the processing device 120 functions as the receiving unit 124. The processing device 120 receives the two line segments (A) selected by the user U, or more precisely, receives information indicating the two line segments (A) selected by the user U.

[0108] In step S7, the user U inputs the distance between the two line segments (A) selected in step S6. The distance between the two line segments (A) is a dimension between the two line segments (A) predetermined as a specification of the road surface marking RM to be constructed.

[0109] Figure 9 The second display screen DS2 shown includes a message M2, an image display window IW, a first button BT1, a second button BT2, and a text box TX. The message M2 includes a message prompting the user U to enter the distance D1 between two line segments (A). Furthermore, an auxiliary image AI3 is displayed in the image display window IW, indicating that the distance to be entered is the distance D1 between line segments LN2 and LN3. The user U enters the distance D1 between the two line segments (A) in the text box TX and presses or clicks the first button BT1.

[0110] In step S7, when the user U inputs the distance D1 between the two line segments (A), the processing device 120 functions as the receiving unit 124. The processing device 120 receives the distance D1 input by the user U, or more precisely, receives information indicating the distance D1 input by the user U. The processing device 120 then calculates the correspondence between the actual distance on the road surface RS and the distance displayed on the display device 150.

[0111] In step S8 , the user U selects two parallel line segments (B) from the drawing pattern.

[0112] Figure 10 as well as Figure 11 This is an example of the display screen displayed on the display device 150 in step S8. Figure 10 Indicates the third display screen DS3. Figure 11 Indicates the 4th display screen DS4.

[0113] The third display screen DS3 includes a message M3, an image display window IW, a first button BT1, and a second button BT2. In the third display screen DS3, an image DP1 representing a drawn pattern is displayed in the image display window IW. This image DP1 includes the outline of the first character L1 included in the road surface marking RM and line segments LN7 to LN10 representing at least a portion of this outline. Line segments LN7 and LN8 are parallel to each other and extend in the height direction of the first character L1. The distance between line segments LN7 and LN8 corresponds to the width of the first character L1. Line segments LN9 and LN10 are parallel to each other and extend in the width direction of the first character L1. The distance between line segments LN9 and LN10 corresponds to the height of the first character L1. The message M3 includes a message urging the user U to select two parallel line segments (B) from the multiple line segments included in the image DP1. As an example, assume that user U selects line segments LN7 and LN8 from line segments LN7 through LN10. Since there is no problem with the selected line segments LN7 and LN8, user U presses or clicks the "OK" button, or first button BT1, to indicate their approval. Line segments LN7 through LN10 are examples of "auxiliary images."

[0114] When the user U presses or clicks the first button BT1, the fourth display screen DS4 is displayed on the display device 150. While details of the fourth display screen DS4 will be discussed later, the fourth display screen DS4 displays an auxiliary image AI4 for emphasizing line segment LN7 and an auxiliary image AI5 for emphasizing line segment LN8. Auxiliary image AI4 is thicker than line segment LN7 and has a different color than line segment LN7. Similarly, auxiliary image AI5 is thicker than line segment LN8 and has a different color than line segment LN8.

[0115] In step S8, when the user U selects two line segments (B), the processing device 120 functions as the receiving unit 124. The processing device 120 receives the two line segments (B) selected by the user U, or more precisely, receives information indicating the two line segments (B) selected by the user U.

[0116] In step S9, the user U inputs the actual distance between the two line segments (B) selected in step S8 on the road surface RS. The distance between the two line segments (B) is the dimension between the two line segments (B) predetermined as a specification for the road surface marking RM to be constructed.

[0117] Figure 11The fourth display screen DS4 shown includes a message M4, an image display window IW, a first button BT1, a second button BT2, and a text box TX. The message M4 includes a message prompting the user U to enter the actual distance on the road surface RS between two line segments (B). Furthermore, the image display window IW displays an auxiliary image AI6 indicating that the distance to be entered is the distance D2 between line segments LN7 and LN8. The user U enters the actual distance between the two line segments (B) in the text box TX and presses or clicks the first button BT1.

[0118] In step S9, when the user U inputs the actual distance between the two line segments (B), the processing device 120 functions as the receiving unit 124. The processing device 120 receives the distance D2 input by the user U, or more precisely, receives information indicating the distance D2 input by the user U.

[0119] In step S10, the processing device 120 functions as the determination unit 125. Based on the correspondence between the size of a single pixel on the liquid crystal panel of the projection device 210 and the distance on the road surface RS calculated in step S2, and the distance D1 between the two line segments (B) on the road surface RS received in step S9, the processing device 120 calculates the number of pixels between the two line segments (B) on the liquid crystal panel of the projection device 210 representing the first character L1 included in the drawn pattern. Furthermore, the processing device 120 determines the magnification of the projected image PI based on the number of pixels between the two line segments (B) on the liquid crystal panel when the projected image PI is set to a constant magnification and the calculated number of pixels.

[0120] In step S11 , the user U selects two parallel line segments (C) from each of the image representing the drawn pattern and the captured image IG.

[0121] Figure 12 as well as Figure 13 This is an example of the display screen displayed on the display device 150 in step S11. Figure 12 Indicates the 5th display screen DS5. Figure 13 Indicates the 6th display screen DS6.

[0122] The fifth display screen DS5 includes a message M5, an image display window IW, a first button BT1, and a second button BT2. In the fifth display screen DS5, the captured image IG and an image DP1 representing a drawn pattern are displayed superimposed in the image display window IW. During this superimposed display, the size of image DP1 on display device 150 is calculated based on the correspondence between the actual distance on road surface RS calculated in step S7 and the distance on display device 150, as well as the actual distance received in step S9. Furthermore, during this superimposed display, image DP1 is positioned at an angle such that line segments LN2, LN3, LN9, and LN10 are parallel to each other, and line segments LN1, LN4, LN5, LN7, and LN8 are parallel to each other. The message M5 includes a message urging the user U to select two parallel line segments (C) from each of the multiple line segments included in the captured image IG and the multiple line segments included in image DP1. As an example, assume that the user U selects line segment LN4 and line segment LN8. Since there is no problem with the selected line segments LN4 and line segment LN8, the user U presses or clicks the first button BT1, which is an "OK" button, indicating approval.

[0123] When the user U presses or clicks the first button BT1, the sixth display screen DS6 is displayed on the display device 150. While the sixth display screen DS6 will be described in detail later, it includes auxiliary image AI7 for emphasizing line segment LN4 and auxiliary image AI8 for emphasizing line segment LN8. Auxiliary image AI7 is thicker than line segment LN4 and has a different color than line segment LN4. Similarly, auxiliary image AI8 is thicker than line segment LN8 and has a different color than line segment LN8.

[0124] In step S11, when the user U selects two line segments (C), the processing device 120 functions as the receiving unit 124. The processing device 120 receives the two line segments (C) selected by the user U, or more precisely, receives information indicating the two line segments (C) selected by the user U.

[0125] In step S12, the user U inputs the actual distance between the two line segments (C) selected in step S11 on the road surface RS. The distance between the two line segments (C) is the dimension between the two line segments (C) predetermined as a specification for the road surface marking RM to be constructed.

[0126] Figure 13The sixth display screen DS6 shown includes a message M6, an image display window IW, a first button BT1, a second button BT2, and a text box TX. The message M6 includes a message prompting the user U to enter the actual distance between two line segments (C) on the road surface RS. In addition, the image display window IW displays an auxiliary image AI9 indicating that the distance to be entered is the distance D3 between line segments LN4 and LN8. The user U enters the actual distance between the two line segments (C) in the text box TX and presses or clicks the first button BT1.

[0127] In step S12, when the user U inputs the actual distance between the two line segments (C), the processing device 120 functions as the receiving unit 124. The processing device 120 receives the distance D3 input by the user U, or more precisely, receives information indicating the distance D3 input by the user U.

[0128] In step S13 , the user U selects two line segments (D) perpendicular to the two line segments (C) and parallel to each other from the drawn pattern and the captured image IG, respectively.

[0129] Figure 14 is an example of a display screen displayed on the display device 150 in step S13. Figure 12 The fifth display screen DS5 is the same as the Figure 14 The 7th display screen DS7 is shown.

[0130] Assume that the user U has selected line segments LN3 and LN9 on the fifth display screen DS5 as two line segments perpendicular to and parallel to line segments LN4 and LN8 selected in step S11. Since there is no problem with the selected line segments LN3 and LN9, the user U presses or clicks the "OK" button, or first button BT1, indicating their approval.

[0131] When the user U presses or clicks the first button BT1, the seventh display screen DS7 is displayed on the display device 150. While details of the seventh display screen DS7 will be discussed later, the seventh display screen DS7 displays auxiliary image AI10 for emphasizing line segment LN3 and auxiliary image AI11 for emphasizing line segment LN9. Auxiliary image AI10 is thicker than line segment LN3 and has a different color than line segment LN3. Similarly, auxiliary image AI11 is thicker than line segment LN9 and has a different color than line segment LN9.

[0132] In step S13, when the user U selects two line segments (D), the processing device 120 functions as the receiving unit 124. The processing device 120 receives the two line segments (D) selected by the user U, or more precisely, receives information indicating the two line segments (D) selected by the user U.

[0133] In step S14, the user U inputs the actual distance between the two line segments (D) selected in step S13 on the road surface RS. The distance between the two line segments (D) is the dimension between the two line segments (D) predetermined as a specification for the road surface marking RM to be constructed.

[0134] Figure 14 The seventh display screen DS7 shown includes a message M7, an image display window IW, a first button BT1, a second button BT2, and a text box TX. The message M7 includes a message prompting the user U to enter the actual distance between two line segments (D) on the road surface RS. In addition, the image display window IW displays an auxiliary image AI12 indicating that the distance to be entered is the distance D4 between line segments LN3 and LN9. The user U enters the actual distance between the two line segments (D) in the text box TX and presses or clicks the first button BT1.

[0135] In step S14, when the user U inputs the actual distance between the two line segments (D), the processing device 120 functions as the receiving unit 124. The processing device 120 receives the distance D4 input by the user U, or more precisely, receives information indicating the distance D4 input by the user U.

[0136] In step S15, the processing device 120 functions as the determination unit 125. The processing device 120 determines the angle and position of the image representing the first character L1 included in the depicted pattern on the road surface RS based on the actual distance between the two line segments (C) received in step S12, the actual distance between the two line segments (D) received in step S14, the inclination of the two line segments (C), and the inclination of the two line segments (D). Furthermore, the processing device 120 determines the angle and position of the projected image PI representing the first character L1 on the panel of the projection device 210 based on the angle and position of the image representing the first character L1 included in the depicted pattern on the road surface RS.

[0137] In step S16, the processing device 120 functions as the projection control unit 126. The processing device 120 projects the projection image PI from the projector 20 onto the road surface RS at the magnification determined in step S10 and the angle and position determined in step S15. As an example, the user U draws a line on the road surface RS with chalk along the outline of the first character L1 projected onto the road surface RS. Then, by painting the road surface RS with the chalk line as a reference, the road surface marking of the first character L1 is constructed on the road surface RS. In addition, when painting is performed, the projection of the projection image PI by the projector 20 may also be terminated. As another example, the user U may omit the operation of drawing the chalk line and instead paint the road surface RS with the outline of the first character L1 projected onto the road surface RS as a reference. The above-mentioned method may also be used separately according to the complexity of the characters and graphics, the positional relationship between the light from the projector 20 and the user U, the painting device, etc.

[0138] 1-4-2: Projection method after the second character

[0139] As described above, in the method for projecting the projection image PI corresponding to the first character L1, line segments LN7 through LN10 representing at least a portion of the outline of the first character L1 are used to determine the magnification of the projected image PI. Furthermore, in this projection method, line segments LN7 through LN10 representing at least a portion of the outline of the first character L1, the outline of the stop line SL, and the outline of the lane center line CL are used to determine the angle and position of the projected image PI on the liquid crystal panel included in the projection device 210.

[0140] In the method for projecting the projected image PI corresponding to the second character L2 included in the road surface marking RM, line segments LN12 to LN15 representing at least a portion of the outline of the second character L2 are typically used to determine the magnification of the projected image PI. Furthermore, in this projection method, line segments LN12 to LN15 representing at least a portion of the outline of the second character L2, line segments LN7 to LN10 representing at least a portion of the outline of the first character L1, and the outline of the lane center line CL are used to determine the angle and position of the projected image PI on the liquid crystal panel included in the projection device 210.

[0141] In other words, in the method for projecting the projection image PI corresponding to the second character L2 included in the road surface marking RM, instead of the line segments Ld7 to LN10 representing at least a portion of the outline of the first character L1 in the method for projecting the projection image PI corresponding to the first character L1, the line segments LN12 to LN15 representing at least a portion of the outline of the second character L2 are used. Furthermore, instead of the outline of the stop line SL, the line segments LN7 to LN10 representing at least a portion of the outline of the first character L1 are used. The same applies to the third and subsequent characters.

[0142] Figure 15 : is a diagram showing an example of the construction support device 1 projecting an image representing a drawing pattern of a road surface marker RM onto the road surface RS. Figure 3 In the embodiment, the construction support device 1 projects an image corresponding to the first character L1, but in Figure 15 In , the construction support device 1 projects an image corresponding to the second character L2. Figure 15 At the time of , the painting of the first character L1 on the road surface RS is completed, so the first character L1 can be used as an object on the road surface RS in the same manner as the stop line SL. Figure 3 Similarly, a third character L3 is shown in the figure for explanation, but it does not exist on the road surface RS before construction.

[0143] exist Figure 15 In the figure, line segments LN10 and LN11 represent at least a portion of the outline of the first character L1. Line segments LN10 and LN11 are parallel to each other and extend in the width direction of the first character L1. Line segment LN11 is located in the X1 direction relative to line segment LN10. The distance D5 between line segments LN10 and LN11 corresponds to the thickness of the fourth stroke of the first character L1. This thickness is the thickness of the first character L1 in the height direction.

[0144] Additionally, line segments LN12 through LN15 represent at least a portion of the outline of the second character L2. Line segments LN12 and LN13 are parallel to each other and extend in the height direction of the second character L2. Line segment LN13 is located in the Y1 direction relative to line segment LN12. The distance D6 between line segments LN12 and LN13 corresponds to the width of the second character L2. Line segments LN14 and LN15 are parallel to each other and extend in the width direction of the second character L2. Line segment LN15 is located in the X2 direction relative to line segment LN14. The distance between line segments LN14 and LN15 corresponds to the height of the second character L2.

[0145] In the projection method for the projection image PI corresponding to the second character L2, line segments LN11 and LN10 are used instead of line segments LN2 and LN3 in the projection method for the projection image PI corresponding to the first character L1. Furthermore, in the projection method for the projection image PI corresponding to the second character L2, line segments LN12 to LN15 are used instead of line segments LN7 to LN10 in the projection method for the projection image PI corresponding to the first character L1. Furthermore, in the projection method for the projection image PI corresponding to the second character L2, the distance D5 between line segments LN10 and LN11 is used instead of distance D1 in the projection method for the projection image PI corresponding to the first character L1. Furthermore, in the projection method for the projection image PI corresponding to the second character L2, the distance D6 between line segments LN12 and LN13 is used instead of distance D2 in the projection method for the projection image PI corresponding to the first character L1. Furthermore, in the projection method of the projection image PI corresponding to the second character L2, the distance D7 between the line segment LN4 and the line segment LN13 is used instead of the distance D3 used in the projection method of the projection image PI corresponding to the first character L1. Furthermore, in the projection method of the projection image PI corresponding to the second character L2, the distance D8 between the line segment LN10 and the line segment LN14 is used instead of the distance D4 used in the projection method of the projection image PI corresponding to the first character L1.

[0146] 1-5: Effects of the First Implementation

[0147] The construction support method of this embodiment provides a road marking construction support method with fewer restrictions on applicable locations compared to methods using satellite signals for positioning. Furthermore, the user U of the construction support device 1 can arbitrarily select the depicted pattern and the object on the road surface RS that determines the position of the image representing the depicted pattern on the road surface RS, allowing the optimal method to be determined at each construction site. Furthermore, the imaging area IA of the construction support device 1 can be compactly stored. Furthermore, the user U does not require advanced IT skills.

[0148] 2: Second Implementation Method

[0149] Below, refer to Figures 16 to 24 , a construction support device 1A according to this embodiment will be described. For simplicity, the following description will focus on the differences between the construction support device 1A and the construction support device 1. Components of the construction support device 1A that are identical to those of the construction support device 1 are designated by the same reference numerals, and descriptions of their functions may be omitted.

[0150] In the construction assistance device 1 of the first embodiment, the projected image PI representing the depicted pattern mainly includes an image representing the contour line of the road surface marking RM. In the construction assistance device 1A of the present embodiment, the projected image PI representing the depicted pattern includes, in addition to the image representing the contour line of the road surface marking RM, an image representing an object on the road surface RS. The object on the road surface RS includes at least one of a road surface marking pre-painted on the road surface RS and a structure fixed to the road surface RS. The road surface marking pre-painted on the road surface RS is, for example, a lane center line CL and a stop line SL. The construction assistance device 1A of the present embodiment determines the magnification, angle, and position of the projected image PI by performing pattern matching between the image representing the object on the road surface RS contained in the image representing the depicted pattern and the area of the object on the road surface RS contained in the captured image IG.

[0151] 2-1: Overall structure

[0152] The construction support device 1A of this embodiment includes an information processing device 10A instead of the information processing device 10. Figure 1 The construction assisting device 1 shown has the same overall structure, so its illustration is omitted.

[0153] Figure 16 1A is a diagram showing an example of projecting an image representing a drawing pattern of a road surface marker RM onto the road surface RS. Figure 16 As shown in FIG. 1 , in this embodiment, it is assumed that the projection range of the construction support device 1A includes objects on the road surface RS. Figure 16 In the example shown, it is assumed that the projection range includes a portion of the lane center line CL and a portion of the stop line SL as objects on the road surface RS.

[0154] 2-2: Structure of Information Processing Device

[0155] Figure 17 1 is a block diagram showing a configuration example of an information processing device 10A. The information processing device 10A differs from the information processing device 10 in that it includes a processing device 120A instead of the processing device 120 and a storage device 140A instead of the storage device 140.

[0156] The storage device 140A is different from the storage device 140 in that it stores a control program PR1A instead of the control program PR1. The storage device 140A also stores conversion data TD representing a conversion formula between camera pixel coordinates and projection pixel coordinates.

[0157] Processing device 120A reads and executes control program PR1A from storage device 140A, thereby functioning as acquisition unit 121, display control unit 122, detection unit 123A, determination unit 125A, projection control unit 126, extraction unit 127, comparison unit 128, and notification unit 129. Alternatively, control program PR1A may be transmitted from another device, such as a server that manages information processing device 10A, via a communication network.

[0158] The detection unit 123A detects the difference between the image of the object included in the captured image IG obtained by capturing the road surface RS and the projected image PI representing the pattern on the road surface RS. The projected image PI representing the pattern on the road surface RS includes an image representing the outline of the first character L1 and an image representing the object on the road surface RS. The detection unit 123A considers the image of the object included in the captured image IG obtained by capturing the road surface RS and the image representing the object on the road surface RS included in the projected image PI representing the pattern on the road surface RS to be identical. As a result, the detection unit 123A detects the image representing the outline of the first character L1 included in the projected image PI representing the pattern on the road surface RS as the difference.

[0159] The extraction unit 127 extracts the image representing the contour line of the first character L1 and the image representing the object on the road surface RS detected as a difference by the detection unit 123A from the projection image PI representing the drawn pattern.

[0160] The comparison unit 128 performs pattern matching on the image of the object on the road surface RS included in the captured image IG and the image of the object on the road surface RS included in the projected image PI representing the drawn pattern, thereby calculating a correspondence relationship between the two images.

[0161] The determination unit 125A uses the correspondence calculated by the comparison unit 128 to determine the magnification, angle and position of the projection image PI in such a way that the image of the object on the road surface RS contained in the captured image IG and the image of the object on the road surface RS contained in the projection image PI representing the depicted pattern overlap with each other.

[0162] If the correspondence calculated by the comparison unit 128 does not satisfy a predetermined condition, the notification unit 129 outputs a notification indicating that the position of the image representing the depicted pattern on the road surface RS cannot be determined. For example, the predetermined condition is that, when the comparison unit 128 performs the aforementioned pattern matching, the correlation coefficient between the image of the object on the road surface RS included in the captured image IG and the image of the object on the road surface RS included in the projected image PI representing the depicted pattern is below a threshold. This predetermined condition is an example of the "first condition."

[0163] Alternatively, the notification unit 129 may output the above notification when the projection range of the construction support device 1A does not include the object on the road surface RS. The notification may be displayed on the display device 150 , for example.

[0164] When the notification unit 129 outputs the above notification, the construction support device 1A may also execute the same projection method as the construction support device 1 according to the first embodiment.

[0165] 2-3: Projection method for depicting patterns

[0166] 2-3-1: Projection method of the first character

[0167] Figure 18 and Figure 19 1A is a flowchart showing a method for projecting a projection image PI showing a drawing pattern by the construction support device 1A. Figure 18 and Figure 19 1 is a flowchart showing a method of projecting a projection image PI corresponding to an image representing the first character L1 included in a drawing pattern. Figure 18 and Figure 19 In FIG, steps including operations of the user U are shown by dotted lines.

[0168] In step S21 , the user U carries the construction support device 1 to an approximate position where the road surface marking RM is drawn on the road surface RS.

[0169] In step S22 , the user U captures the road surface RS using the image capturing device 110 included in the information processing device 10 . The processing device 120A functions as the acquisition unit 121 . The processing device 120A acquires the captured image IG captured by the image capturing device 110 .

[0170] In step S23, the processing device 120A functions as the projection control unit 126 and the acquisition unit 121. The processing device 120A causes the projector 20 to project a projection image PI representing a drawn pattern onto a rough location on the road surface RS. In step S23, the processing device 120A causes the imaging device 110 included in the information processing device 10 to capture the road surface RS on which the projection image PI is projected. The processing device 120A acquires the captured image IG captured by the imaging device 110.

[0171] Figure 20 This figure shows an example of a projected image PI representing a drawn pattern in this embodiment. Projected image PI includes an image representing the outline of first character L1, as well as an image CL1 representing the lane center line CL and an image SL1 representing the stop line SL, representing objects on the road surface RS. The image representing the object on the road surface RS is an example of a "second image."

[0172] Figure 21 It means Figure 20 The figure shows a captured image IG when the projected image PI is projected onto the road surface RS. The captured image IG includes an image representing a depicted pattern and a captured image of an object on the road surface RS. As described above, the image representing the depicted pattern includes an image representing the outline of the first character L1, an image CL1 representing the lane center line CL, and an image SL1 representing the stop line SL. The captured image IG of the object on the road surface RS includes a captured image CL2 of the lane center line CL and a captured image SL2 of the stop line SL.

[0173] exist Figure 18 In step S24, the processing device 120A functions as the detection unit 123A. The processing device 120A detects the difference between the image of the object included in the captured image IG obtained by capturing the road surface RS and the projection image PI representing the pattern drawn on the road surface RS. Figure 21 In this example, processing device 120A considers image CL1 representing lane center line CL and captured image CL2 of lane center line CL to be identical. Furthermore, processing device 120A considers image SL1 representing stop line SL and captured image SL2 of stop line SL to be identical. As a result, processing device 120A detects the contour of first character L1 as a difference.

[0174] In step S25, processing device 120A functions as extraction unit 127. Based on the detection result in step S24, processing device 120A extracts projection image PI representing the outline of first character L1 and projection image PI representing the reference pattern from projection image PI representing the drawn pattern. Here, "projection image PI representing the reference pattern" is the image obtained by removing projection image PI representing the outline of first character L1 from projection image PI representing the drawn pattern. Figure 22 1 is a diagram showing an example of a projection image PI indicating the outline of the first character L1 . Figure 23 : is a diagram showing an example of a projection image PI representing a reference pattern. Figure 23 In the example shown, the projection image PI representing the reference pattern includes an image CL1 representing the lane center line CL and an image SL1 representing the stop line SL.

[0175] exist Figure 19 In step S26, the processing device 120A functions as the projection control unit 126. The processing device 120A causes the projector 20 to project the projection image PI representing the reference pattern onto the road surface RS.

[0176] In step S27, the processing device 120A functions as the acquisition unit 121. The processing device 120A acquires the captured image IG captured by the imaging device 110. Furthermore, the processing device 120A functions as the comparison unit 128. The processing device 120A performs pattern matching between the projection image PI representing the reference pattern and the image of the object on the road surface RS included in the captured image IG, thereby calculating the correspondence between the two images.

[0177] Figure 24 This figure shows the captured image IG in step S27. The captured image IG includes images CL1 and SL1, as well as images CL2 and SL2. Images CL1 and SL1 are images of objects included in the projected image PI. Images CL2 and SL2 are images of objects on the road surface RS included in the captured image IG.

[0178] exist Figure 19 In step S28, the processing device 120A functions as the determination unit 125A. The processing device 120A uses the correspondence relationship calculated in step S27 to determine the magnification, angle, and position of the projected image PI so that the image of the object on the road surface RS included in the captured image IG and the image of the object on the road surface RS included in the projected image PI representing the depicted pattern overlap.

[0179] In step S29, the processing device 120A functions as the projection control unit 126. The processing device 120A causes the projector 20 to project the projection image PI indicating the outline of the first character L1 onto the road surface RS based on the magnification, angle, and position determined in step S28.

[0180] 2-3-2: Projection method after the second character

[0181] The projection method for the second character L2 and subsequent characters in this embodiment may also use the same projection method as the projection method for the second character L2 and subsequent characters in the first embodiment.

[0182] Alternatively, as a projection method for the second character L2 and thereafter in this embodiment, the contour line of the first character L1 may be used as the same reference pattern as the above-mentioned reference pattern in this embodiment, and the same projection method as the projection method for the first character L1 may be used.

[0183] 2-4: Effects of the Second Implementation

[0184] According to the construction support method of this embodiment, the construction speed can be further increased compared to the construction support method of the first embodiment. In addition, compared to the construction support method of the first embodiment, the trouble of the user U in performing the operation can be further reduced.

[0185] 3: Modification

[0186] The above embodiment can be modified in various ways. Specific modifications are exemplified below. The embodiments exemplified below and the embodiments described in the above embodiments can be appropriately combined within the scope of mutual non-inconsistency. Furthermore, for elements in the modifications described below whose roles and functions are the same as those in the embodiment, the same reference numerals as those in the above description will be used, and detailed descriptions of each will be omitted as appropriate.

[0187] 3-1: Modification 1

[0188] In the first embodiment, the projection control unit 126 may cause the projector 20 to project the auxiliary image AI together with the projection image PI onto the road surface RS.

[0189] 3-2: Modification 2

[0190] In step S2 of the first embodiment, the processing device 120, acting as the projection control unit 126, causes the projector 20 to project the fixed image onto the road surface RS. However, the processing device 120 may also calculate the correspondence between the size of a single pixel on the panel of the projection device 210 and the distance on the road surface RS by projecting a projection image PI representing a drawn pattern instead of the fixed image.

[0191] 4: Summary of this disclosure

[0192] The following is a summary of the present disclosure.

[0193] (Supplementary Note 1) A construction assistance method comprising: obtaining a captured image representing a road surface and objects on the road surface; determining a position on the road surface for projecting a first image based on the objects on the road surface contained in the captured image, the first image being used for painting road surface markings on the road surface; and projecting the first image from an optical device to a position on the road surface.

[0194] Thus, a method for constructing road surface markings with fewer restrictions on usable locations than conventional methods is provided.

[0195] In addition, in general road marking construction methods, workers often use a scriber to manually draw a draft for painting road markings on the road surface. Since the time required to draw such a draft accounts for a large proportion of the overall construction time, the efficiency of the work is required. According to the construction assistance method disclosed in the present invention, the positioning work performed by the worker when manually drawing the draft using a scriber becomes simple, or the work itself of manually drawing using a scriber is no longer necessary, so the efficiency of the work can be achieved compared to the case where the entire work of drawing the draft is performed manually. In addition, the skills of a professional craftsman are required to draw the draft line using a scriber, but according to the construction assistance method disclosed in the present invention, even beginners can accurately draw the draft line.

[0196] (Note 2) The construction assistance method according to Note 1 further includes: accepting an operation of selecting two parallel first line segments representing at least a portion of the outline of the object on the road surface from the captured image; and accepting an input of a first distance between the two first line segments on the road surface, and determining a position on the road surface based on the two first line segments and the first distance.

[0197] This makes it possible to determine the position on the road surface where the first image is projected, using the size of the object on the road surface.

[0198] (Supplementary Note 3) The construction support method according to Supplementary Note 1 or 2, further comprising: causing a display device to display an auxiliary image for determining the position on the road surface.

[0199] Therefore, when executing the construction support method of the present disclosure, it is easy for the user U of the construction support method to understand the operation content. As a result, the efficiency of the work can be further improved.

[0200] (Supplement 4) The construction assistance method according to Supplement 3, wherein the auxiliary image represents at least one of the first image, the two first line segments, the first distance, and the distance between the first image on the road surface and the object on the road surface.

[0201] Therefore, when executing the construction support method of the present disclosure, it is easy for the user U of the construction support method to understand the operation content. As a result, the efficiency of the work can be further improved.

[0202] (Note 5) According to the construction assistance method described in Note 2, the first image includes multiple line segments representing at least a portion of the outline of the road surface marking, and the construction assistance method further includes: accepting an operation of selecting two parallel second line segments from the multiple line segments from the first image; and accepting an input representing a second distance between the two second line segments on the road surface, and determining the position of projecting the first image on the road surface based on the two first line segments, the first distance, the two second line segments and the second distance.

[0203] Thus, the position of the projected first image on the road surface RS can be determined more accurately based on the two second line segments and the second distance in addition to the two first line segments and the first distance.

[0204] (Note 6) A construction assistance method according to any one of Notes 1 to 5, wherein the objects on the road surface include at least one of road surface markings pre-painted on the road surface, structures fixed to the road surface, and objects placed on at least a portion of the road surface in a straight line.

[0205] In this way, the position on the road surface RS where the first image is projected can be determined by utilizing an object that exists in advance on the road surface RS.

[0206] (Supplementary Note 7) A construction assistance method according to Supplementary Note 1, wherein the object on the road surface includes at least one of a road surface marking pre-painted on the road surface and a structure fixed to the road surface, and the first image includes a second image representing the object on the road surface, and the construction assistance method includes: before determining the position on the road surface at which the first image is projected, projecting the first image from the optical device onto the road surface, and determining the position on the road surface at which the first image is projected based on the correspondence between the image of the object on the road surface contained in the captured image and the second image on the road surface.

[0207] Thus, by determining the position on the road surface RS based on the correspondence between the image of the object on the road surface included in the captured image and the second image, the position on the road surface RS where the first image is projected can be determined more simply.

[0208] (Supplementary Note 8) The construction support method according to Supplementary Note 7, wherein, when the correspondence relationship does not satisfy the first condition, a notification indicating that the position on the road surface cannot be determined is output.

[0209] Thus, when the position on the road surface RS cannot be determined because the correspondence relationship does not satisfy the first condition, the user U of the construction support method can study other construction support methods.

[0210] (Supplementary Note 9) A construction assistance device comprising a processing device, wherein the processing device performs the following processing: obtaining a captured image representing a road surface and objects on the road surface; determining a position on the road surface for projecting a first image based on the objects on the road surface contained in the captured image, the first image being used for painting road surface markings on the road surface; and projecting the first image from an optical device to a position on the road surface.

[0211] Thus, a road surface marking construction assisting device with fewer restrictions on usable locations than conventional technologies is provided.

[0212] In addition, in general road marking construction methods, workers often use a marker to manually draw a draft for painting road markings on the road surface. Since the time required to draw such a draft accounts for a large proportion of the overall construction time, the efficiency of the work is required. According to the construction assistance device disclosed in the present invention, the positioning work performed by the worker when manually drawing the draft using a marker becomes simple, or the work itself of manually drawing using a marker is no longer necessary, so the efficiency of the work can be achieved compared to the case where the entire work of drawing the draft is performed manually. In addition, the skills of a professional craftsman are required to draw the draft line using a marker, but according to the construction assistance device disclosed in the present invention, even a beginner can accurately draw the draft line.

[0213] (Supplementary Note 10) A program for information processing that causes a computer to perform the following processing: obtaining a captured image representing a road surface and objects on the road surface; determining a position on the road surface to project a first image based on the objects on the road surface contained in the captured image, the first image being used to paint a road surface marking on the road surface; and projecting the first image from an optical device to a position on the road surface.

[0214] As a result, an information processing program is provided that has fewer restrictions on usable locations than conventional programs.

[0215] In addition, in general road marking construction methods, workers often use a scriber to manually draw a draft for painting road markings on the road surface. Since the time required to draw such a draft accounts for a large proportion of the overall construction time, the efficiency of the work is required. According to the information processing program disclosed in the present invention, the positioning work performed by the worker when manually drawing the draft using a scriber becomes simple, or the work itself of manually drawing using a scriber is no longer necessary, so the efficiency of the work can be achieved compared to the case where the entire work of drawing the draft is performed manually. In addition, the skills of a professional craftsman are required to draw the draft line using a scriber, but according to the information processing program disclosed in the present invention, even beginners can accurately draw the draft line.

Claims

1. A construction assistance method, comprising: obtaining a captured image representing a road surface and objects on the road surface; determining a position on the road surface at which to project a first image, for painting a road surface marking on the road surface, based on an object on the road surface included in the captured image; and The first image is projected from an optical device onto the position on the road surface.

2. The construction assisting method according to claim 1, wherein: The construction assistance method further comprises: receiving an operation of selecting two parallel first line segments representing at least a portion of the outline of the object on the road surface from the captured image; and accepting an input indicating a first distance between the two first line segments on the road surface, The position on the road surface is determined based on the two first line segments and the first distance.

3. The construction assisting method according to claim 2, wherein: The construction assistance method further includes causing a display device to display an auxiliary image for determining the position on the road surface.

4. The construction assisting method according to claim 3, wherein: The auxiliary image indicates at least one of the first image, the two first line segments, the first distance, and a distance between the first image on the road surface and an object on the road surface.

5. The construction assisting method according to claim 2, wherein: The first image includes a plurality of line segments representing at least a portion of the outline of the road surface marking. The construction assistance method further comprises: accepting an operation of selecting two parallel second line segments from among the plurality of line segments from the first image; and accepting an input indicating a second distance between the two second line segments on the road surface, A position on the road surface where the first image is projected is determined based on the two first line segments, the first distance, the two second line segments, and the second distance.

6. The construction assisting method according to any one of claims 1 to 5, wherein: The object on the road surface includes at least one of a road surface marking pre-painted on the road surface, a structure fixed to the road surface, and an object placed on the road surface, at least a portion of which is linear.

7. The construction assisting method according to claim 1, wherein: The object on the road surface includes at least one of a road surface marking pre-painted on the road surface and a structure fixed to the road surface. The first image includes a second image representing an object on the road surface. The construction assisting method includes: before determining a position on the road surface where the first image is to be projected, projecting the first image from the optical device onto the road surface; A position on the road surface where the first image is projected is determined based on a correspondence relationship between an image of an object on the road surface included in the captured image and the second image on the road surface.

8. The construction assisting method according to claim 7, wherein: If the correspondence relationship does not satisfy the first condition, a notification indicating that the position on the road surface cannot be determined is output.

9. A construction support device comprising a processing device, wherein the processing device performs the following processing: obtaining a captured image representing a road surface and objects on the road surface; determining a position on the road surface at which to project a first image, for painting a road surface marking on the road surface, based on an object on the road surface included in the captured image; and The optical device projects the first image toward the position on the road surface.

10. An information processing program product that causes a computer to implement the following processing: obtaining a captured image representing a road surface and objects on the road surface; determining a position on the road surface at which to project a first image, for painting a road surface marking on the road surface, based on an object on the road surface included in the captured image; and The optical device projects the first image toward the position on the road surface.

Images