Construction auxiliary method, construction auxiliary apparatus, and program for information processing

Abstract

To solve the problem that a conventional coating device for road marking has restrictions on usable places.SOLUTION: A construction auxiliary method includes: acquiring a captured image IG including a road surface RS and an object on the road surface RS; determining, based upon the object on the road surface RS which is included in the captured image IG, a position on the road surface RS on which a first image used to coat the road surface RS with a road marking RM is projected; and projecting the first image at the position on the road surface RS from an optical device.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a construction assistance method, a construction assistance device, and an information processing program. [Background technology]

[0002] Patent Document 1 discloses a technology related to a road marking painting device for painting road markings. The road marking painting device is equipped with a positioning means that receives positioning signals from satellites to determine its current location. The positioning means receives signals from GPS satellites and quasi-zenith satellites, allowing the road marking painting device to automatically determine its position and paint the road markings. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-086590 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the road marking painting device disclosed in Patent Document 1 uses satellite signals for positioning, so the positioning accuracy decreases in places where radio waves from satellites are difficult to reach, such as inside tunnels or behind buildings. For this reason, there are restrictions on where the road marking painting device can be used. [Means for solving the problem]

[0005] A construction assistance method according to one embodiment of the present invention includes acquiring an image showing a road surface and an object on the road surface, determining a position on the road surface at which to project a first image used to paint a road marking on the road surface based on the object on the road surface included in the image, and projecting the first image from an optical device onto the position on the road surface.

[0006] A construction assistance device according to one embodiment of the present invention includes a processing device that acquires an image showing a road surface and an object on the road surface, determines a position on the road surface to project a first image used to paint a road marking on the road surface based on the object on the road surface included in the image, and causes an optical device to project the first image at the position on the road surface.

[0007] An information processing program according to one embodiment of the present invention causes a computer to acquire an image showing a road surface and an object on the road surface, determine a position on the road surface at which to project a first image used to paint a road marking on the road surface based on the object on the road surface included in the image, and project the first image from an optical device onto the position on the road surface. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing the configuration of a construction support device 1. [Figure 2] FIG. [Figure 3] 10A and 10B are diagrams showing examples of projection of an image showing a drawing pattern of a road marking RM onto a road surface RS. [Figure 4] FIG. 1 is a block diagram showing an example of the configuration of an information processing device 10. [Figure 5] FIG. 2 is a block diagram showing an example of the configuration of a projector 20. [Figure 6] 6 is a flowchart showing a method for projecting a projection image PI showing a drawing pattern by the construction support device 1. [Figure 7] 6 is a flowchart showing a method for projecting a projection image PI showing a drawing pattern by the construction support device 1. [Figure 8] 10 shows an example of a display screen displayed on the display device 150. [Figure 9] 10 shows an example of a display screen displayed on the display device 150. [Figure 10] 10 shows an example of a display screen displayed on the display device 150. [Figure 11] 10 shows an example of a display screen displayed on the display device 150. [Figure 12] 10 shows an example of a display screen displayed on the display device 150. [Figure 13] 10 shows an example of a display screen displayed on the display device 150. [Figure 14] 10 shows an example of a display screen displayed on the display device 150. [Figure 15] 10A and 10B are diagrams showing examples of projection of an image showing a drawing pattern of a road marking RM onto a road surface RS. [Figure 16] 10A and 10B are diagrams showing examples of projection of an image showing a drawing pattern of a road marking RM onto a road surface RS. [Figure 17] FIG. 1 is a block diagram showing an example of the configuration of an information processing device 10A. [Figure 18] 10 is a flowchart showing a method for projecting a projection image showing a drawing pattern by the construction assist device 1A. [Figure 19] 10 is a flowchart showing a method for projecting a projection image showing a drawing pattern by the construction assist device 1A. [Figure 20] FIG. 2 is a diagram showing an example of a projection image PI showing a drawing pattern. [Figure 21] 10 is a diagram showing a captured image IG when a projected image PI is projected onto a road surface RS. [Figure 22] FIG. 10 is a diagram showing an example of a projected image PI showing the contour of a first character L1. [Figure 23] FIG. 4 is a diagram showing an example of a projection image PI showing a reference pattern. [Figure 24] FIG. 2 is a diagram showing a captured image IG. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, in each drawing, the dimensions and scale of each part are appropriately different from those of the actual parts. Furthermore, since the embodiments described below are preferred specific examples of the present invention, various technically preferable limitations are applied, but the scope of the present invention is not limited to these embodiments unless otherwise specified in the following description to the effect that the present invention is limited.

[0010] 1: First embodiment Hereinafter, a construction assist device 1 according to this embodiment will be described with reference to FIGS.

[0011] 1-1: Overall structure 1 is a block diagram showing the configuration of a construction assistance device 1 according to this embodiment. The construction assistance device 1 includes an information processing device 10 and a projector 20. The information processing device 10 and the projector 20 are connected to each other so that they can communicate with each other.

[0012] The construction assistance device 1 is a device that projects, when drawing a road marking on a road surface RS, an image indicating a drawing pattern to be used for drawing the road marking on the road surface RS. The drawing pattern includes, as an example, the outline of a road marking RM to be drawn on the road surface. The drawing pattern may further include an auxiliary image AI for drawing the road marking RM. The road marking RM is drawn, for example, by a user U of the construction assistance device 1 applying paint to the road surface using a known painting device separate from the construction assistance device 1.

[0013] The projector 20 is a device that projects an image showing the drawing pattern onto the road surface RS. The information processing device 10 is a device that controls the projector 20. The information processing device 10 controls the projector 20, causing the projector 20 to project the image showing the drawing pattern onto the road surface RS.

[0014] FIG. 2 is an example of an external view of the construction auxiliary device 1 according to this embodiment. In FIG. 2, an xyz space is assumed to have three axes: an x-axis, a y-axis, and a z-axis. The x-axis, the y-axis, and the z-axis are perpendicular to one another. In the xyz space, the z-axis direction is the vertical direction. The z-axis direction is a general term for the z1 direction and the z2 direction opposite to the z1 direction. The z1 direction is a vertically downward direction. The z2 direction is a vertically upward direction. The x-axis direction is the same direction as the symmetry line of the construction assistance device 1 on the xy plane. The x-axis direction is a general term for the x1 direction and the x2 direction opposite to the x1 direction. The x1 direction is a direction facing forward of the construction assistance device 1. The x2 direction is a direction facing backward of the construction assistance device 1. More specifically, in the construction assistance device 1, of a first position where the information processing device 10 is provided and a second position where the projector 20 is provided, the first position is located in the x2 direction relative to the second position. The y-axis direction is a general term for the y1 and y2 directions. The y1 direction is the right direction when the xy plane is viewed in the z1 direction with the x1 direction at the top. The y2 direction is the left direction when the xy plane is viewed in the z1 direction with the x1 direction at the top.

[0015] In FIG. 2, the construction support device 1 is equipped with a dolly PD. The dolly PD is equipped with a wheel W1 and a wheel W2 on the surface of the dolly PD in the z1 direction. The dolly PD is capable of moving on a road surface RS due to the provision of the wheel W1 and the wheel W2. The dolly PD is also equipped with a handle HD. A user U of the construction support device 1 grips the handle HD and applies force to the dolly PD to move the dolly PD on the road surface RS.

[0016] Furthermore, the information processing device 10 and the projector 20 are placed on the dolly PD in the z2 direction with respect to the dolly PD. The information processing device 10 and the projector 20 are connected to each other via a cable CB so that they can communicate with each other.

[0017] The information processing device 10 includes a main body 11 and a display device 150. The main body 11 includes a processing device 120, which will be described later. The display device 150 is placed on the main body 11 in the z2 direction.

[0018] The display device 150 is a device that displays images and text information. The display device 150 displays various images under the control of the processing device 120. For example, various display panels such as a liquid crystal display panel and an organic EL (Electro Luminescence) display panel are suitably used as the display device 150. A user U of the construction assist device 1 can visually recognize various images and text information displayed on the display device 150 while holding the handle HD.

[0019] A rotating base RT is placed on the z2-direction surface of the dolly PD. The rotating base RT is positioned in the x1 direction relative to the information processing device 10. The rotating base RT rotates 360° around an axis in the z-axis direction.

[0020] The projector 20 is placed on the surface of the rotating table RT in the z2 direction. The projector 20 includes a main body 21 and a projection device 210. The main body 21 includes a processing device 220 (described later) and at least a part of the projection device 210. As an example, the main body 21 is cylindrical with its central axis in the z-axis direction. As an example, a part of the projection device 210 and an opening for emitting light from the projection device 210 are installed on a side surface of the main body 21.

[0021] The projection device 210 is a device that projects an image generated by the information processing device 10 onto a road surface RS. In the example shown in FIG. 2, the projection device 210 projects the image in the x1 direction. However, the projector 20 provided with the projection device 210 is mounted on a rotating table RT, and the projector 20 rotates on the rotating table RT. Therefore, the projector 20 can project the image generated by the information processing device 10 in any of the x1, x2, y1, and y2 directions. Furthermore, the projector 20 can project the image generated by the information processing device 10 in any direction between the x1 and y2 directions, any direction between the y2 and x2 directions, any direction between the x2 and y1 directions, and any direction between the y1 and x1 directions.

[0022] FIG. 3 is a diagram showing an example of an image showing a drawing pattern of a road marking RM onto a road surface RS projected by the construction assist device 1. In FIG.

[0023] In Figure 3, an XYZ space is assumed to have three axes: an X axis, a Y axis, and a Z axis. The X axis, the Y axis, and the Z axis are perpendicular to one another. In the XYZ space, the Z axis direction is the vertical direction. The Z axis direction is a general term for the Z1 direction and the Z2 direction opposite to the Z1 direction. The Z1 direction is a direction that extends vertically downward. The Z2 direction is a direction that extends vertically upward. The X axis direction is the extension direction of the roadway center line CL painted on the road surface RS. The X axis direction is a general term for the X1 direction and the X2 direction opposite to the X1 direction. The X1 direction is a direction that points toward the top of the road marking RM. The X2 direction is a direction that points toward the bottom of the road marking RM. The Y axis direction is the extension direction of the stop line SL painted on the road surface RS. The Y axis direction is a general term for the Y1 direction and the Y2 direction. The Y1 direction is the right direction when the road surface RS is viewed in the Z1 direction with the X1 direction at the top, and the Y2 direction is the left direction when the road surface RS is viewed in the Z1 direction with the X1 direction at the top.

[0024] In the example shown in FIG. 3, the road marking RM includes three characters: a first character L1, a second character L2, and a third character L3. More specifically, in the example shown in FIG. 3, the road marking RM represents the three characters "Stop." The first character L1 represents the character "to" in "Stop." The second character L2 represents the character "ma" in "Stop." The third character L3 represents the character "re" in "Stop." However, the road marking RM may include any number of characters. The road marking RM may also be any character. The road marking RM may also include graphics other than characters. Note that, for the sake of explanation, the second character L2 and the third character L3 are illustrated in FIG. 3. In this embodiment, an example will be described in which the road marks are placed in order starting with the first character L1. Therefore, these two characters have not been placed on the road surface RS at the time the first character L1 is projected, and the user U can only see the image of the first character L1 out of the three characters on the road surface RS.

[0025] The construction assist device 1 projects the road marking RM character by character onto the road surface RS. In the example shown in Fig. 3, the construction assist device 1 projects the contour of a first character L1 of the road marking RM onto the road surface RS in an area in the Y2 direction relative to the roadway center line CL and in the X2 direction relative to the stop line SL. The method of projecting the contour of the first character L1 by the construction assist device 1 will be described in detail later.

[0026] 2 and 3, the road surface RS may be an outdoor road surface or an indoor road surface. In addition, in FIGS. 2 and 3, the construction auxiliary device 1 projects the outline of the first character L1 onto the road surface RS, but this is merely an example. The construction auxiliary device 1 may also project the outline of the first character L1 onto a construction surface other than a road surface. For example, the construction auxiliary device 1 may project the outline of the first character L1 onto the construction surface of a multi-story parking lot and a sports field.

[0027] 1-2: Configuration of information processing device 4 is a block diagram showing an example of the configuration of an information processing device 10. The information processing device 10 is typically a PC (Personal Computer), but is not limited to this and may be, for example, a tablet terminal or a smartphone. The information processing device 10 includes an imaging device 110, a processing device 120, a storage device 140, a display device 150, an input device 160, and a communication device 170. The elements of the information processing device 10 are connected to each other by a single bus or multiple buses for communicating information.

[0028] The imaging device 110 is a device that captures an image of the road surface RS. The imaging device 110 captures various images under the control of the processing device 120. For example, a camera provided in a PC, a tablet terminal, or a smartphone is preferably used as the imaging device 110, but the imaging device 110 is not limited to this and may also be an external camera such as a web camera.

[0029] The processing device 120 is a processor that controls the entire information processing device 10, and is configured, for example, by one or more chips. The processing device 120 is configured, for example, by a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, a register, etc. Note that some or all of the functions of the processing device 120 may be realized by hardware such as a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA). The processing device 120 executes various processes in parallel or sequentially.

[0030] The storage device 140 is a recording medium that can be read and written by the processing device 120, and stores a plurality of programs including the control program PR1 executed by the processing device 120, and the drawing pattern data DD. The storage device 140 may be configured with at least one of, for example, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a random access memory (RAM), etc. The storage device 140 may also be called a register, a cache, a main memory, a primary storage device, etc.

[0031] The drawing pattern data DD is data relating to a drawing pattern. In this embodiment, the drawing pattern is the contour lines of a road marking RM. In the example shown in FIG. 3, the drawing pattern is the contour lines of the three characters "Stop," which is the road marking RM, but this is merely an example. The drawing pattern includes the contour lines of various road markings other than the road marking RM exemplified in FIG. 3.

[0032] As described above, the display device 150 is a device that displays images and text information. The display device 150 may be a display device separate from the other components of the information processing device 10.

[0033] The input device 160 is a device that accepts operations from the user U. For example, the input device 160 includes a keyboard, a touchpad, a touch panel, or a pointing device such as a mouse. Here, if the input device 160 includes a touch panel, it may also serve as the display device 150.

[0034] The communication device 170 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 170 is also called, for example, a network device, a network controller, a network card, or a communication module. The communication device 170 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 170 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include those compliant with wired LAN (Local Area Network), IEEE 1394, and USB (Universal Serial Bus). Examples of the wireless communication interface include those compliant with wireless LAN, Bluetooth (registered trademark), etc.

[0035] The processing device 120 reads and executes the control program PR1 from the storage device 140, thereby functioning as an acquisition unit 121, a display control unit 122, a detection unit 123, a reception unit 124, a determination unit 125, and a projection control unit 126. The control program PR1 may be transmitted from another device, such as a server that manages the information processing device 10, via a communication network.

[0036] The acquisition unit 121 acquires a captured image showing the road surface RS and an object on the road surface RS. The captured image is an image captured by the imaging device 110. Here, the "object on the road surface RS" includes at least one of a road marking painted on the road surface RS in advance, a structure fixed to the road surface RS, and an object placed on the road surface RS and having at least a portion linear. Here, the "road marking painted on the road surface RS in advance" is, for example, the roadway center line CL and the stop line SL shown in FIG. 3. Furthermore, the "structure fixed to the road surface RS" is, for example, a curbstone installed on the road surface RS. The curbstone preferably has a rectangular shape. Furthermore, the "object placed on the road surface RS and having at least a portion linear" is, for example, a ruler.

[0037] 3, the image capturing device 110 captures an image of an image capturing area IA on a road surface RS. The image capturing area IA includes the road surface RS, a portion of the roadway center line CL, and a portion of the stop line SL within the image capturing area IA. The portion of the roadway center line CL and the portion of the stop line SL correspond to the above-mentioned "object on the road surface RS." The acquisition unit 121 acquires an image of the imaging area IA captured by the imaging device 110.

[0038] The display control unit 122 causes the captured image IG acquired by the acquisition unit 121 to be displayed on the display device 150. The captured image IG displayed on the display device 150 can be viewed by the user U.

[0039] The detection unit 123 detects straight lines included in objects on the road surface RS from the captured image IG. For example, in Fig. 3, the detection unit 123 detects straight lines included in the contour of the roadway center line CL and straight lines included in the contour of the stop line SL in the captured area IA.

[0040] The detection unit 123 also detects edges in the drawing pattern represented by the drawing pattern data DD. As an example, the detection unit 123 divides the contour line of the road marking RM represented by the drawing pattern into a plurality of line segments each of which has the maximum length without bending, and detects each line segment as one edge.

[0041] The receiving unit 124 receives an operation to select, from the captured image IG, two parallel line segments that indicate at least a portion of the contour of an object on the road surface RS. As described above, the captured image IG is displayed on the display device 150. The user U uses the input device 160 to select two parallel line segments that indicate at least a portion of the contour of an object on the road surface RS. For example, the user U selects, on a touch panel as the input device 160 included in the display device 150, two parallel line segments that indicate at least a portion of the contour of the roadway center line CL or the stop line SL displayed on the display device 150. The receiving unit 124 receives an operation by the user U to select the two line segments. Note that the two line segments are an example of a "first line segment."

[0042] When the user U selects the two line segments, the display control unit 122 may cause the display device 150 to display an auxiliary image AI indicating the two line segments. As an example, the display control unit 122 preferably sets the thickness of the auxiliary image AI to a different thickness from the thickness of the two line segments included in the captured image IG. Furthermore, the display control unit 122 preferably sets the color of the auxiliary image AI to a different color from the color of the two line segments included in the captured image IG. Specific examples of the captured image IG and auxiliary image AI displayed on the display device 150 will be described later with reference to FIGS. 8 to 14.

[0043] The receiving unit 124 also receives an input indicating the distance between the two line segments. This distance is an example of a "first distance."

[0044] When the user U inputs the distance, the display control unit 122 may cause the display device 150 to display an auxiliary image AI indicating the distance to which a point in the captured image corresponds to the distance. Furthermore, the display control unit 122 causes the display device 150 to display an input field for the distance.

[0045] As described above, the construction assist device 1 projects a projection image PI showing a drawing pattern onto the road surface RS. The projection image PI showing the drawing pattern includes a plurality of line segments showing at least a portion of the contour of the road marking RM. The reception unit 124 receives an operation to select two parallel line segments from the projection image PI showing the drawing pattern. The two line segments are an example of a "second line segment." The projection image PI showing the drawing pattern is an example of a "first image."

[0046] When the user U selects the two line segments included in the projected image PI representing the drawing pattern, the display control unit 122 may cause the display device 150 to display an auxiliary image AI representing the two line segments. As an example, the display control unit 122 preferably sets the thickness of the auxiliary image AI to a different thickness from the thickness of the two line segments included in the projected image PI representing the drawing pattern. Furthermore, the display control unit 122 preferably sets the color of the auxiliary image AI to a different color from the color of the two line segments included in the projected image PI representing the drawing pattern.

[0047] The receiving unit 124 also receives an input indicating the distance between the two line segments included in the projection image PI representing the drawing pattern. This distance is an example of a "second distance."

[0048] When the user U inputs the distance, the display control unit 122 may cause the display device 150 to display an auxiliary image AI indicating the distance to which a point in the projection image PI showing the drawing pattern corresponds. Furthermore, the display control unit 122 causes the display device 150 to display an input field for the distance between the two line segments included in the projection image PI showing the drawing pattern.

[0049] The determination unit 125 determines a position on the road surface RS to project a projection image PI showing a drawing pattern used to paint a road marking RM on the road surface RS, based on an object on the road surface RS included in the captured image IG. More specifically, the determination unit 125 determines a position on the road surface RS to project the projection image PI showing the drawing pattern, based on two parallel line segments selected by the user U that show at least a part of the outline of an object on the road surface RS, and the distance between the two line segments.

[0050] In this embodiment, the determination unit 125 further determines the position on the road surface RS to project the projection image PI showing the drawing pattern, based on two parallel line segments selected by the user U that represent at least a portion of the contour line of the road marking RM and the distance between the two line segments.

[0051] A specific method for determining the position on the road surface RS onto which the projection image PI showing the drawing pattern is projected by the determination unit 125 will be described later with reference to FIGS.

[0052] The projection control unit 126 causes the projector 20 to project a projection image PI indicating the drawing pattern at the position on the road surface RS determined by the determination unit 125. More specifically, the projection control unit 126 transmits, to the projector 20, an image signal corresponding to the projection image PI indicating the drawing pattern and a control signal for controlling the projector 20 via the communication device 170.

[0053] 1-3: Projector configuration 5 is a block diagram showing an example of the configuration of the projector 20. The projector 20 includes a projection device 210, a processing device 220, a storage device 230, and a communication device 240. The elements of the projector 20 are connected to one another by one or more buses for communicating information. Furthermore, the elements of the projector 20 are configured by one or more devices, and some elements of the projector 20 may be omitted.

[0054] The projection device 210 is a device that projects an image represented by an image signal acquired by an acquisition unit 221 (described later) onto a screen, a wall, or the like. The projection device 210 projects various images under the control of the processing device 220. The projection device 210 includes, for example, a light source, a liquid crystal panel, and a projection lens, and modulates light from the light source using the liquid crystal panel and projects the modulated light onto a screen, a wall, or the like via the projection lens.

[0055] The processing device 220 is a processor that controls the entire projector 20, and is configured, for example, by one or more chips. The processing device 220 is configured, for example, by a central processing unit (CPU) that includes an interface with peripheral devices, an arithmetic unit, a register, etc. Note that some or all of the functions of the processing device 220 may be realized by hardware such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The processing device 220 executes various processes in parallel or sequentially.

[0056] The storage device 230 is a recording medium readable by the processing device 220, and stores a plurality of programs including the control program PR2 executed by the processing device 220. The storage device 230 may be configured with at least one of, for example, a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), and a random access memory (RAM). The storage device 230 may also be called a register, a cache, a main memory, a primary storage device, or the like.

[0057] The communication device 240 is hardware serving as a transmitting / receiving device for communicating with other devices. The communication device 240 is also called, for example, a network device, a network controller, a network card, or a communication module. The communication device 240 may include a connector for wired connection and an interface circuit corresponding to the connector. The communication device 240 may also include a wireless communication interface. Examples of the connector and interface circuit for wired connection include those compliant with wired LAN, IEEE1394, and USB. Examples of the wireless communication interface include those compliant with wireless LAN, Bluetooth (registered trademark), etc.

[0058] The processing device 220 reads and executes the control program PR2 from the storage device 230, thereby functioning as an acquisition unit 221 and a projection control unit 222. Note that the control program PR2 may be transmitted from another device, such as a server that manages the projector 20, via a communication network.

[0059] The acquisition unit 221 acquires, from the information processing device 10, an image signal corresponding to a projection image PI showing a drawing pattern and a control signal for controlling the projector 20.

[0060] The projection control unit 222 causes the projection device 310 to project a projection image PI showing a drawing pattern corresponding to the image signal acquired by the acquisition unit 221 onto the road surface RS based on the control signal acquired by the acquisition unit 221.

[0061] In this embodiment, the projection device 210 is an example of an "optical device." The projector 20 is another example of an "optical device."

[0062] 1-4: Drawing pattern projection method 1-4-1: First character projection method Figures 6 and 7 are flowcharts showing a method for projecting a projection image PI showing a drawing pattern by the construction assist device 1. More specifically, Figures 6 and 7 are flowcharts showing a method for projecting a projection image PI corresponding to an image showing a first character L1 included in the drawing pattern. In Figures 6 and 7, steps including operations by the user U are indicated by dotted lines.

[0063] In step S1, the user U carries the construction auxiliary device 1 to an approximate position on the road surface RS where the road marking RM is to be drawn.

[0064] In step S2, the processing device 120 functions as the projection control unit 126. The processing device 120 causes the projector 20 to project a standard image. The standard image is displayed in the center of a liquid crystal panel provided in the projection device 210 of the projector 20, and is projected onto the road surface RS. The standard image is, for example, a square. As an example, the user U measures the length of the diagonal of the square projected onto the road surface RS. The user U also inputs the length of the diagonal to the information processing device 10 using the input device 160.

[0065] The processing device 120 functions as an acquisition unit 121. The processing device 120 acquires the length of the diagonal line input by the user U, more precisely, information indicating the length of the diagonal line input by the user U. Furthermore, the processing device 120 calculates the correspondence between the size of one pixel on the liquid crystal panel of the projection device 210 and the distance on the road surface RS, based on the length and the number of pixels of the diagonal line on the liquid crystal panel of the projection device 210.

[0066] In step S3, the user U captures an image of the road surface RS with the imaging device 110 included in the information processing device 10. In step S3, the standard image of step S2 does not need to be projected. In the example shown in FIG. 3, the user U captures an image of an imaging area IA of the road surface RS with the imaging device 110. The processing device 120 also functions as an acquisition unit 121. The processing device 120 acquires the captured image IG captured by the imaging device 110. Note that the processing device 120 may instruct the imaging device 110 to capture an image when information indicating the length of the diagonal line is input in step S2.

[0067] In step S4, the processing device 120 functions as the detection unit 123. The processing device 120 detects line segments included in objects on the road surface RS from the captured image IG. For example, referring to FIG. 3, the processing device 120 detects line segments included in part of the stop line SL and line segments included in part of the roadway center line CL, among objects on the road surface RS included in the captured image IA. More specifically, the processing device 120 detects line segment LN1 extending in the X-axis direction and line segments LN2 and LN3 extending in the Y-axis direction, among the contours of the stop line SL. The processing device 120 also detects line segments LN4 and LN5 extending in the X-axis direction and line segment LN6 extending in the Y-axis direction, among the contours of the roadway center line CL. Note that line segments LN1 to LN6 are examples of "auxiliary images."

[0068] In step S5, the processing device 120 functions as the detection unit 123. The processing device 120 detects edges from the projection image PI, which indicates a drawing pattern to be projected onto the road surface RS. The processing device 120 detects edges included in the projection image PI from the drawing pattern data DD. In step S5, the projection image PI may or may not be projected onto the road surface RS.

[0069] In step S6, the user U uses the input device 160 to select, as reference lines, two line segments (A) that are parallel to each other from the line segments included in the object on the road surface RS.

[0070] 8 and 9 are examples of display screens displayed on the display device 150 in step S6. Fig. 8 shows a first display screen DS1, and Fig. 9 shows a second display screen DS2.

[0071] The first display screen DS1 includes a message M1, an image display window IW, a first button BT1, and a second button BT2. On the first display screen DS1, a captured image IG is displayed in the image display window IW. The captured image IG is an image of the captured area IA in FIG. 3 and includes a stop line SL and a roadway centerline CL as objects on the road surface RS. Furthermore, on the first display screen DS1, the message M1 includes text prompting the user U to select two parallel line segments (A) from among the line segments included in the objects on the road surface RS in the captured image IG. As an example, assume that the user U selects line segments LN2 and LN3 included in the contour of the stop line SL. Since the selected line segments LN2 and LN3 are acceptable, the user U presses or clicks the first button BT1, which is an "OK" button indicating approval. The second button BT2 is a cancel button.

[0072] When the user U presses or clicks the first button BT1, the display device 150 displays the second display screen DS2. Details of the second display screen DS2 will be described later, but on the second display screen DS2, an auxiliary image AI1 for highlighting the line segment LN2 and an auxiliary image AI2 for highlighting the line segment LN3 are displayed. The auxiliary image AI1 has a thicker line thickness than the line segment LN2 and is a different color from the line segment LN2. Similarly, the auxiliary image AI2 has a thicker line thickness than the line segment LN3 and is a different color from the line segment LN3.

[0073] In step S6, when the user U selects two line segments (A), the processing device 120 functions as the receiving unit 124. The processing device 120 receives the two line segments (A) selected by the user U, more precisely, information indicating the two line segments (A) selected by the user U.

[0074] In step S7, the user U inputs the distance between the two line segments (A) selected in step S6. The distance between the two line segments (A) is the dimension between the two line segments (A) that is planned as a specification of the road marking RM to be installed.

[0075] The second display screen DS2 shown in Fig. 9 includes a message M2, an image display window IW, a first button BT1, a second button BT2, and a text box TX. The message M2 includes text prompting the user U to input the distance D1 between the two line segments (A). In addition, an auxiliary image AI3 is displayed in the image display window IW, indicating that the distance to be input is the distance D1 between the line segments LN2 and LN3. The user U inputs the distance D1 between the two line segments (A) in the text box TX and presses or clicks the first button BT1.

[0076] In step S7, when the user U inputs the distance D1 between the two line segments (A), the processing device 120 functions as the receiving unit 124. The processing device 120 receives the distance D1 input by the user U, more precisely, information indicating the distance D1 input by the user U. Furthermore, the processing device 120 calculates the correspondence between the actual distance on the road surface RS and the distance on the display device 150.

[0077] In step S8, the user U selects two parallel line segments (B) from the drawing pattern.

[0078] 10 and 11 are examples of display screens displayed on the display device 150 in step S8. Fig. 10 shows a third display screen DS3. Fig. 11 shows a fourth display screen DS4.

[0079] The third display screen DS3 includes a message M3, an image display window IW, a first button BT1, and a second button BT2. On the third display screen DS3, an image DP1 showing a drawing pattern is displayed in the image display window IW. The image DP1 includes a contour of a first character L1 included in the road marking RM and line segments LN7 to LN10 that show at least a part of the contour. The line segments LN7 and LN8 are parallel to each other and extend in the height direction of the first character L1. The distance between the line segments LN7 and LN8 corresponds to the width of the first character L1. The line segments LN9 and LN10 are parallel to each other and extend in the width direction of the first character L1. The distance between the line segments LN9 and LN10 corresponds to the height of the first character L1. The message M3 includes wording that prompts the user U to select two parallel line segments (B) from the multiple line segments included in the image DP1. As an example, let us say that the user U selects line segments LN7 and LN8, which are included in the contour of the stop line SL, from line segments LN7 to LN10. After that, the user U is satisfied with the selected line segments LN7 and LN8, and therefore presses or clicks the first button BT1, which is an "OK" button indicating approval. Note that line segments LN7 to LN10 are an example of an "auxiliary image."

[0080] As a result of the user U pressing or clicking the first button BT1, a fourth display screen DS4 is displayed on the display device 150. The fourth display screen DS4 will be described in detail later, but on the fourth display screen DS4, an auxiliary image AI4 for highlighting the line segment LN7 and an auxiliary image AI5 for highlighting the line segment LN8 are displayed. The auxiliary image AI4 has a thicker line thickness than the line segment LN7 and is a different color from the line segment LN7. Similarly, the auxiliary image AI5 has a thicker line thickness than the line segment LN8 and is a different color from the line segment LN8.

[0081] In step S8, when the user U selects two line segments (B), the processing device 120 functions as the receiving unit 124. The processing device 120 receives the two line segments (B) selected by the user U, more precisely, information indicating the two line segments (B) selected by the user U.

[0082] In step S9, the user U inputs the actual distance on the road surface RS between the two line segments (B) selected in step S8. The distance between the two line segments (B) is the dimension between the two line segments (B) that is planned as a specification for the road marking RM to be installed.

[0083] The fourth display screen DS4 shown in Fig. 11 includes a message M4, an image display window IW, a first button BT1, a second button BT2, and a text box TX. The message M4 includes text prompting the user U to input the actual distance on the road surface RS between the two line segments (B). In addition, an auxiliary image AI6 is displayed in the image display window IW, indicating that the distance to be input is the distance D2 between the line segments LN7 and LN8. The user U inputs the actual distance between the two line segments (B) in the text box TX and presses or clicks the first button BT1.

[0084] In step S9, when the user U inputs the actual distance between the two line segments (B), the processing device 120 functions as the receiving unit 124. The processing device 120 receives the distance D2 input by the user U, more precisely, information indicating the distance D2 input by the user U.

[0085] In step S10, the processing device 120 functions as the determination unit 125. Based on the correspondence between the size of one pixel on the liquid crystal panel of the projection device 210 and the distance on the road surface RS calculated in step S2, and the distance D1 between the two line segments (B) on the road surface RS accepted in step S9, the processing device 120 calculates the number of pixels between the two line segments (B) on the liquid crystal panel of the projection image PI representing the first character L1 included in the drawing pattern. Furthermore, the processing device 120 determines the enlargement magnification of the projection image PI from the number of pixels between the two line segments (B) on the liquid crystal panel when the projection image PI is magnified to the same size and the calculated number of pixels.

[0086] In step S11, the user U selects two parallel line segments (C) from each of the image showing the drawing pattern and the captured image IG.

[0087] 12 and 13 are examples of display screens displayed on the display device 150 in step S11. Fig. 12 shows a fifth display screen DS5, and Fig. 13 shows a sixth display screen DS6.

[0088] The fifth display screen DS5 includes a message M5, an image display window IW, a first button BT1, and a second button BT2. In the fifth display screen DS5, the captured image IG and an image DP1 showing a drawing pattern are displayed superimposed in the image display window IW. When the superimposed display is performed, the size of the image DP1 on the display device 150 is calculated based on the correspondence between the actual distance on the road surface RS calculated in step S7 and the distance on the display device 150, and the actual distance received in step S9. Furthermore, when the superimposed display is performed, the image DP1 is positioned at an angle such that the line segments LN2, LN3, LN9, and LN10 are parallel to one another, and the line segments LN1, LN4, LN5, LN7, and LN8 are parallel to one another. Message M5 includes wording that prompts user U to select two parallel line segments (C) from each of the multiple line segments included in captured image IG and the multiple line segments included in image DP1. As an example, assume that user U selects line segments LN4 and LN8. After that, user U is satisfied with the selected line segments LN4 and LN8, and therefore presses or clicks first button BT1, which is an "OK" button indicating approval.

[0089] As a result of the user U pressing or clicking the first button BT1, a sixth display screen DS6 is displayed on the display device 150. The sixth display screen DS6 will be described in detail later, but on the sixth display screen DS6, an auxiliary image AI7 for highlighting the line segment LN4 and an auxiliary image AI8 for highlighting the line segment LN8 are displayed. The auxiliary image AI7 has a thicker line thickness than the line segment LN4 and is a different color from the line segment LN4. Similarly, the auxiliary image AI8 has a thicker line thickness than the line segment LN8 and is a different color from the line segment LN8.

[0090] In step S11, when the user U selects two line segments (C), the processing device 120 functions as the receiving unit 124. The processing device 120 receives the two line segments (C) selected by the user U, more precisely, information indicating the two line segments (C) selected by the user U.

[0091] In step S12, the user U inputs the actual distance on the road surface RS between the two line segments (C) selected in step S11. The distance between the two line segments (C) is the dimension between the two line segments (C) that is planned as a specification for the road marking RM to be installed.

[0092] The sixth display screen DS6 shown in FIG. 13 includes a message M6, an image display window IW, a first button BT1, a second button BT2, and a text box TX. The message M6 includes text prompting the user U to input the actual distance on the road surface RS between the two line segments (C). In addition, an auxiliary image AI9 is ​​displayed in the image display window IW, indicating that the distance to be input is the distance D3 between the line segments LN4 and LN8. The user U inputs the actual distance between the two line segments (C) in the text box TX and presses or clicks the first button BT1.

[0093] In step S12, when the user U inputs the actual distance between the two line segments (C), the processing device 120 functions as the receiving unit 124. The processing device 120 receives information indicating the distance D3 input by the user U, more precisely, the distance D3 selected by the user U.

[0094] In step S13, the user U selects two line segments (D) that are perpendicular to the two line segments (C) and parallel to each other from each of the drawing pattern and the captured image IG.

[0095] Fig. 14 shows an example of a display screen displayed on the display device 150 in step S13. In step S13, a screen identical to the fifth display screen DS5 shown in Fig. 12 and a seventh display screen DS7 shown in Fig. 14 are displayed.

[0096] Assume that the user U selects line segments LN3 and LN9 on the fifth display screen DS5 as two line segments that are perpendicular to the line segments LN4 and LN8 selected in step S11 and are parallel to each other. After that, the user U is satisfied with the selected line segments LN3 and LN9, and so presses or clicks the first button BT1, which is the "OK" button indicating approval.

[0097] When the user U presses or clicks the first button BT1, a seventh display screen DS7 is displayed on the display device 150. The seventh display screen DS7 will be described in detail later, but on the seventh display screen DS7, an auxiliary image AI10 for highlighting the line segment LN3 and an auxiliary image AI11 for highlighting the line segment LN9 are displayed. The auxiliary image AI10 has a thicker line thickness than the line segment LN3 and is a different color from the line segment LN3. Similarly, the auxiliary image AI11 has a thicker line thickness than the line segment LN9 and is a different color from the line segment LN9.

[0098] In step S13, when the user U selects two line segments (D), the processing device 120 functions as the receiving unit 124. The processing device 120 receives the two line segments (D) selected by the user U, more precisely, information indicating the two line segments (D) selected by the user U.

[0099] In step S14, the user U inputs the actual distance on the road surface RS between the two line segments (D) selected in step S13. The distance between the two line segments (D) is the dimension between the two line segments (D) that is planned as a specification of the road marking RM to be installed.

[0100] The seventh display screen DS7 shown in Fig. 14 includes a message M7, an image display window IW, a first button BT1, a second button BT2, and a text box TX. The message M7 includes text prompting the user U to input the actual distance on the road surface RS between the two line segments (D). In addition, an auxiliary image AI12 is displayed in the image display window IW, indicating that the distance to be input is the distance D4 between the line segments LN3 and LN9. The user U inputs the actual distance between the two line segments (D) in the text box TX and presses or clicks the first button BT1.

[0101] In step S14, when the user U inputs the actual distance between the two line segments (D), the processing device 120 functions as the receiving unit 124. The processing device 120 receives the distance D4 input by the user U, more precisely, information indicating the distance D4 input by the user U.

[0102] In step S15, the processing device 120 functions as the determination unit 125. The processing device 120 determines the angle and position on the road surface RS of the image showing the first character L1 included in the drawing pattern, based on the actual distance between the two line segments (C) received in step S12, the actual distance between the two line segments (D) received in step S14, the inclination of the two line segments (C), and the inclination of the two line segments (D). Furthermore, the processing device 120 determines the angle and position on the panel of the projection device 210 of the projected image PI showing the first character L1, based on the angle and position on the road surface RS of the image showing the first character L1 included in the drawing pattern.

[0103] In step S16, the processing device 120 functions as the projection control unit 126. The processing device 120 projects the projection image PI from the projector 20 onto the road surface RS at the magnification determined in step S10 and at the angle and position determined in step S15. As an example, the user U draws a line on the road surface RS with chalk along the outline of the first character L1 projected on the road surface RS. Then, the user U paints the road surface RS using the chalk line as a reference, thereby applying a road marking of the first character L1 to the road surface RS. Note that the projection of the projection image PI by the projector 20 may be completed before painting. As another example, the user U may omit the step of drawing the chalk line and instead paint the road surface RS using the outline of the first character L1 projected on the road surface RS as a reference. The above methods may be used depending on the complexity of the characters or figures, the positional relationship between the light from the projector 20 and the user U or the painting device, etc.

[0104] 1-4-2: Projection method for the second and subsequent characters As described above, in the projection method of the projection image PI corresponding to the first character L1, the line segments LN7 to LN10 showing at least a part of the outline of the first character L1 are used to determine the magnification of the projection image PI. Furthermore, in this projection method, the line segments LN7 to LN10 showing at least a part of the outline of the first character L1, the outline of the stop line SL, and the outline of the roadway center line CL are used to determine the angle and position of the projection image PI on the liquid crystal panel provided in the projection device 210.

[0105] In a projection method for a projection image PI corresponding to a second character L2 included in a road marking RM, generally, line segments LN12 to LN15 showing at least a part of the outline of the second character L2 are used to determine the magnification of the projection image PI. Furthermore, in this projection method, line segments LN12 to LN15 showing at least a part of the outline of the second character L2, line segments LN7 to LN10 showing at least a part of the outline of the first character L1, and the outline of the roadway centerline CL are used to determine the angle and position of the projection image PI on the liquid crystal panel provided in the projection device 210.

[0106] In other words, in the projection method of the projection image PI corresponding to the second character L2 included in the road marking RM, instead of the line segments LN7 to LN10 that show at least a part of the outline of the first character L1 in the projection method of the projection image PI corresponding to the first character L1, line segments LN12 to LN15 that show at least a part of the outline of the second character L2 are used, and instead of the outline of the stop line SL, line segments LN7 to LN10 that show at least a part of the outline of the first character L1 are used. The same applies to the third and subsequent characters.

[0107] FIG. 15 is a diagram showing an example of the projection of an image showing a drawing pattern of a road marking RM onto a road surface RS by the construction assist device 1. In FIG. 3, the construction assist device 1 projects an image corresponding to a first character L1, but in FIG. 15, the construction assist device 1 projects an image corresponding to a second character L2. At the time of FIG. 15, the painting of the first character L1 onto the road surface RS has been completed, so the first character L1 can be used as an object on the road surface RS, similar to the stop line SL. Also, as in FIG. 3, the third character L3 is shown for explanation purposes, but does not exist on the road surface RS because it has not yet been painted.

[0108] In FIG. 15, line segments LN10 and LN11 represent at least a portion of the contour of the first character L1. Line segments LN10 and LN11 are parallel to each other and extend in the width direction of the first character L1. Line segment LN11 is located in the X1 direction relative to line segment LN10. Distance D5 between line segments LN10 and LN11 corresponds to the thickness of the fourth stroke of the first character L1. This thickness is the thickness in the height direction of the first character L1.

[0109] Furthermore, line segments LN12 to LN15 define at least a portion of the contour of the second character L2. Line segments LN12 and LN13 are parallel to each other and extend in the height direction of the second character L2. Line segment LN13 is located in the Y1 direction relative to line segment LN12. Distance D6 between line segments LN12 and LN13 corresponds to the width of the second character L2. Line segments LN14 and LN15 are parallel to each other and extend in the height direction of the first character L1. Line segment LN15 is located in the X2 direction relative to line segment LN14. The distance between line segments LN14 and LN15 corresponds to the height of the second character L2.

[0110] In the projection method of the projection image PI corresponding to the second character L2, line segments LN11 and LN10 are used instead of line segments LN2 and LN3, respectively, in the projection method of the projection image PI corresponding to the first character L1. Also, in the projection method of the projection image PI corresponding to the second character L2, line segments LN12 to LN15 are used instead of line segments LN7 to LN10, respectively, in the projection method of the projection image PI corresponding to the first character L1. Also, in the projection method of the projection image PI corresponding to the second character L2, distance D5 between line segments LN10 and LN11 is used instead of distance D1 in the projection method of the projection image PI corresponding to the first character L1. In addition, in the projection method of the projection image PI corresponding to the second character L2, the distance D6 between the line segments LN12 and LN13 is used instead of the distance D2 in the projection method of the projection image PI corresponding to the first character L1. In addition, in the projection method of the projection image PI corresponding to the second character L2, the distance D7 between the line segments LN4 and LN13 is used instead of the distance D3 in the projection method of the projection image PI corresponding to the first character L1. In addition, in the projection method of the projection image PI corresponding to the second character L2, the distance D8 between the line segments LN10 and LN14 is used instead of the distance D4 in the projection method of the projection image PI corresponding to the first character L1.

[0111] 1-5: Effects of the First Embodiment The construction assistance method according to this embodiment provides a road marking construction assistance method that has fewer restrictions on the locations where it can be used compared to methods using satellite signals for positioning. Furthermore, the user U of the construction assistance device 1 can arbitrarily select a drawing pattern and an object on the road surface RS for determining the position on the road surface RS of an image showing the drawing pattern, allowing the optimal method to be determined for each construction site. Furthermore, the imaging area IA captured by the construction assistance device 1 can be kept compact. Furthermore, the user U does not need to have advanced IT skills.

[0112] 2: Second embodiment The construction support device 1A according to this embodiment will be described below with reference to Figures 16 to 24. For simplicity of explanation, the following mainly describes the differences between the construction support device 1A and the construction support device 1. Furthermore, among the components provided in the construction support device 1A, the same components as those provided in the construction support device 1 will be designated by the same reference numerals, and explanations of their functions may be omitted.

[0113] In the construction support device 1 according to the first embodiment, the projected image PI showing the drawing pattern mainly includes an image showing the contour lines of the road markings RM. In the construction support device 1A according to this embodiment, the projected image PI showing the drawing pattern includes an image showing an object on the road surface RS in addition to the image showing the contour lines of the road markings RM. The object on the road surface RS includes at least one of road markings pre-painted on the road surface RS and structures fixed to the road surface RS. Examples of the road markings pre-painted on the road surface RS include a roadway center line CL and a stop line SL. The construction support device 1A according to this embodiment determines the magnification, angle, and position of the projected image PI by performing pattern matching between an image showing the object on the road surface RS included in the image showing the drawing pattern and the area of ​​the object on the road surface RS included in the captured image IG.

[0114] 2-1: Overall structure The construction assistance device 1A according to this embodiment includes an information processing device 10A instead of the information processing device 10. In other respects, the construction assistance device 1A has the same overall configuration as the construction assistance device 1 shown in Fig. 1, and therefore is not shown in the figure.

[0115] Fig. 16 is a diagram showing an example of projection of an image showing a drawing pattern of a road marking RM onto a road surface RS by the construction assist device 1A. As shown in Fig. 16, in this embodiment, it is assumed that the projection range by the construction assist device 1A includes objects on the road surface RS. In the example shown in Fig. 16, it is assumed that the projection range includes a part of the roadway center line CL and a part of the stop line SL as objects on the road surface RS.

[0116] 2-2: Configuration of information processing device 17 is a block diagram showing an example of the configuration of the information processing device 10 A. The information processing device 10 A differs from the information processing device 10 in that it includes a processing device 120 A instead of the processing device 120 and a storage device 140 A instead of the storage device 140.

[0117] The storage device 140A differs from the storage device 140 in that it stores a control program PR1A instead of the control program PR1. The storage device 140A also stores conversion data TD that indicates a conversion formula between camera pixel coordinates and projected pixel coordinates.

[0118] By reading and executing the control program PR1A from the storage device 140A, the processing device 120A functions as an acquisition unit 121, a display control unit 122, a detection unit 123A, a determination unit 125A, a projection control unit 126, an extraction unit 127, a comparison unit 128, and a notification unit 129. The control program PR1A may be transmitted via a communication network from another device, such as a server that manages the information processing device 10A.

[0119] The detection unit 123A detects a difference between an image of an object included in a captured image IG obtained by capturing a road surface RS and a projection image PI showing a drawing pattern on the road surface RS. The projection image PI showing the drawing pattern on the road surface RS includes an image showing the outline of a first character L1 and an image showing an object on the road surface RS. The detection unit 123A considers the image of the object included in the captured image IG obtained by capturing a road surface RS to be the same as the image of the object on the road surface RS included in the projection image PI showing the drawing pattern on the road surface RS. As a result, the detection unit 123A detects the image showing the outline of the first character L1 included in the projection image PI showing the drawing pattern on the road surface RS as the difference.

[0120] The extraction unit 127 extracts, from the projection image PI showing the drawing pattern, an image showing the contour line of the first character L1 detected as a difference by the detection unit 123A, and an image showing an object on the road surface RS.

[0121] The matching unit 128 calculates the correspondence between the image of the object on the road surface RS included in the captured image IG and the image of the object on the road surface RS included in the projected image PI showing the drawing pattern by pattern matching.

[0122] The determination unit 125A uses the correspondence calculated by the matching unit 128 to determine the magnification, angle, and position of the projection image PI so that the image of the object on the road surface RS included in the captured image IG and the image of the object on the road surface RS included in the projection image PI showing the drawing pattern overlap each other.

[0123] If the correspondence calculated by the matching unit 128 does not satisfy a predetermined condition, the notification unit 129 outputs a notification indicating that the position of the image indicating the drawing pattern on the road surface RS cannot be determined. The predetermined condition is, for example, a condition in which, when the matching unit 128 performs the above-mentioned pattern matching, the correlation coefficient between the image of the object on the road surface RS included in the captured image IG and the image of the object on the road surface RS included in the projected image PI indicating the drawing pattern falls below a threshold. The predetermined condition is an example of a "first condition."

[0124] Alternatively, the notification unit 129 may output the above notification when the projection range of the construction assist device 1A does not include any object on the road surface RS. The notification is displayed on the display device 150, for example.

[0125] When the notification unit 129 outputs the above notification, the construction assistance device 1A may execute the same projection method as the construction assistance device 1 according to the first embodiment.

[0126] 2-3: How to project a drawing pattern 2-3-1: First character projection method Figures 18 and 19 are flowcharts showing a method for projecting a projection image PI showing a drawing pattern by the construction assist device 1A. More specifically, Figures 18 and 19 are flowcharts showing a method for projecting a projection image PI corresponding to an image showing a first character L1 included in the drawing pattern. In Figures 18 and 19, steps including operations by the user U are indicated by dotted lines.

[0127] In step S21, the user U carries the construction auxiliary device 1 to the approximate position on the road surface RS where the road marking RM is to be drawn.

[0128] In step S22, the user U captures an image of the road surface RS with the imaging device 110 provided in the information processing device 10. The processing device 120A also functions as an acquisition unit 121. The processing device 120A acquires the captured image IG captured by the imaging device 110.

[0129] In step S23, the processing device 120A functions as the projection control unit 126 and the acquisition unit 121. The processing device 120A causes the projector 20 to project a projection image PI indicating a drawing pattern onto an approximate position on the road surface RS. In step S23, the processing device 120A causes the imaging device 110 provided in the information processing device 10 to capture an image of the road surface RS onto which the projection image PI is projected. The processing device 120A acquires the captured image IG captured by the imaging device 110.

[0130] 20 is a diagram showing an example of a projection image PI showing a drawing pattern in this embodiment. The projection image PI includes an image showing the outline of a first character L1, and, as images showing objects on the road surface RS, an image CL1 showing a roadway center line CL and an image SL1 showing a stop line SL. The images showing objects on the road surface RS are an example of a "second image."

[0131] Fig. 21 is a diagram showing a captured image IG when the projected image PI shown in Fig. 20 is projected onto the road surface RS. The captured image IG includes an image showing a drawing pattern and a captured image of an object on the road surface RS. As described above, the image showing the drawing pattern includes an image showing the outline of the first character L1, an image CL1 showing the roadway center line CL, and an image SL1 showing the stop line SL. The captured image IG of the object on the road surface RS includes a captured image CL2 of the roadway center line CL and a captured image SL2 of the stop line SL.

[0132] In step S24 of FIG. 18, the processing device 120A functions as the detection unit 123A. The processing device 120A detects the difference between an image of an object included in a captured image IG of the road surface RS and a projected image PI showing a drawing pattern on the road surface RS. In FIG. 21, the processing device 120A considers an image CL1 showing the roadway centerline CL and a captured image CL2 of the roadway centerline CL to be the same. The processing device 120A also considers an image SL1 showing the stop line SL and a captured image SL2 of the stop line SL to be the same. As a result, the processing device 120A detects the contour of the first character L1 as the difference.

[0133] In step S25, the processing device 120A functions as the extraction unit 127. Based on the detection result in step S24, the processing device 120A extracts a projection image PI showing the outline of the first character L1 and a projection image PI showing the reference pattern from the projection image PI showing the drawing pattern. Here, the "projection image PI showing the reference pattern" is an image obtained by removing the projection image PI showing the outline of the first character L1 from the projection image PI showing the drawing pattern. FIG. 22 is a diagram showing an example of the projection image PI showing the outline of the first character L1. FIG. 23 is a diagram showing an example of the projection image PI showing the reference pattern. In the example shown in FIG. 23, the projection image PI showing the reference pattern includes an image CL1 showing the roadway centerline CL and an image SL1 showing the stop line SL.

[0134] 19, the processing device 120A functions as the projection control unit 126. The processing device 120A causes the projector 20 to project a projection image PI indicating a reference pattern onto the road surface RS.

[0135] In step S27, the processing device 120A functions as the acquisition unit 121. The processing device 120A acquires the captured image IG captured by the imaging device 110. The processing device 120A also functions as the matching unit 128. The processing device 120A calculates the correspondence between the projected image PI showing the reference pattern and the image of the object on the road surface RS included in the captured image IG by pattern matching the two images.

[0136] 24 is a diagram showing the captured image IG in step S27. The captured image IG includes images CL1 and SL1, and captured images CL2 and SL2. Images CL1 and SL1 are images of objects included in the captured projected image PI. Captured images CL2 and SL2 are images of objects on the road surface RS included in the captured image IG.

[0137] 19, the processing device 120A functions as a determination unit 125A. Using the correspondence calculated in step S27, the processing device 120A determines the magnification, angle, and position of the projection image PI so that the image of the object on the road surface RS included in the captured image IG and the image of the object on the road surface RS included in the projection image PI showing the drawing pattern overlap each other.

[0138] In step S29, the processing device 120A functions as the projection control unit 126. Based on the magnification, angle, and position determined in step S28, the processing device 120A causes the projector 20 to project a projection image PI showing the outline of the first character L1 onto the road surface RS.

[0139] 2-3-2: Projection method for the second and subsequent characters The projection method for the second character L2 and subsequent characters in this embodiment may be the same as the projection method for the second character L2 and subsequent characters in the first embodiment.

[0140] Alternatively, as a projection method for the second character L2 and subsequent characters in this embodiment, a projection method similar to the projection method for the first character L1 may be used, using the outline of the first character L1 as a reference pattern similar to the reference pattern described above in this embodiment.

[0141] 2-4: Effects of the Second Embodiment According to the construction assistance method of this embodiment, it is possible to further increase the speed of construction compared to the construction assistance method of embodiment 1. In addition, it is possible to further reduce the effort required for operation by the user U compared to the construction assistance method of embodiment 1.

[0142] 3: Variation The above embodiments can be modified in various ways. Specific modified embodiments are exemplified below. The embodiments exemplified below and the embodiments described above can be combined as appropriate within the scope of not mutually contradicting each other. Note that for elements in the modified embodiments exemplified below that have the same actions and functions as the embodiments, the reference numerals referenced in the above explanation will be used and detailed explanations of each element will be omitted as appropriate.

[0143] 3-1: Variation 1 In the first embodiment, the projection control unit 126 may cause the projector 20 to project an auxiliary image AI onto the road surface RS together with the projection image PI.

[0144] 3-2: Variation 2 In step S2 of the first embodiment, the processing device 120, as the projection control unit 126, causes the projector 20 to project a fixed-form image onto the road surface RS. However, the processing device 120 may calculate the correspondence between the size of one pixel on the panel provided in the projection device 210 and the distance on the road surface RS by projecting a projection image PI showing a drawing pattern instead of the fixed-form image.

[0145] 4: Summary of this disclosure A summary of this disclosure is provided below.

[0146] (Appendix 1) A construction assistance method comprising: acquiring a captured image showing a road surface and an object on the road surface; determining a position on the road surface at which to project a first image used to paint a road marking on the road surface based on the object on the road surface included in the captured image; and projecting the first image from an optical device onto the position on the road surface.

[0147] This provides a road marking construction method that has fewer restrictions on the locations where it can be used compared to conventional techniques.

[0148] Furthermore, in typical road marking construction methods, workers often manually draw a draft for painting the road marking on the road surface using a marking marker. Because the time required for drawing such a draft accounts for a large proportion of the overall construction time, there is a demand for work efficiency improvements. According to the construction assistance method disclosed herein, the positioning work performed by workers when manually drawing the draft using a marking marker is simplified, or the work of manually drawing using a marking marker is not necessary at all, thereby enabling work efficiency improvements compared to when the entire work of drawing the draft is done manually. Furthermore, while drawing a draft line using a marking marker requires the skill of a craftsman, the construction assistance method disclosed herein enables even beginners to accurately draw a draft line.

[0149] (Appendix 2) A construction assistance method as described in Appendix 1, further comprising: accepting an operation to select two parallel first line segments from the captured image, the first line segments indicating at least a portion of the contour of an object on the road surface; and accepting an input indicating a first distance between the two first line segments on the road surface; and determining a position on the road surface based on the two first line segments and the first distance.

[0150] This makes it possible to determine the position on the road surface onto which the first image is to be projected, using the dimensions of the object on the road surface, thereby further improving work efficiency.

[0151] (Supplementary Note 3) The construction assistance method according to Supplementary Note 1 or Supplementary Note 2, further comprising displaying, on a display device, an auxiliary image used to determine the position on the road surface.

[0152] This makes it easier for a user U of the construction assistance method to understand the operation details when the construction assistance method according to the present disclosure is executed, thereby enabling further improvement in work efficiency.

[0153] (Appendix 4) The construction assistance method described in Appendix 3, wherein the auxiliary image indicates at least one of the first image, the two first line segments, the first distance, and the distance between the first image on the road surface and an object on the road surface.

[0154] This makes it easier for a user U of the construction assistance method to understand the operation details when the construction assistance method according to the present disclosure is executed, thereby enabling further improvement in work efficiency.

[0155] (Appendix 5) The construction assistance method described in Appendix 2, wherein the first image includes a plurality of line segments that indicate at least a part of the contour of the road marking, and further includes accepting an operation to select two parallel second line segments from the plurality of line segments from the first image, and accepting an input that indicates a second distance between the two second line segments on the road surface, and determining a position on the road surface to project the first image based on the two first line segments, the first distance, the two second line segments, and the second distance.

[0156] As a result, the position on the road surface RS onto which the first image is to be projected can be determined more accurately based on the two second line segments and the second distance in addition to the two first line segments and the first distance.

[0157] (Appendix 6) A construction assistance method described in any one of Appendices 1 to 5, wherein the object on the road surface includes at least one of a road marking pre-painted on the road surface, a structure fixed to the road surface, and an object placed on the road surface and at least a portion of which is linear.

[0158] This makes it possible to determine the position on the road surface RS onto which the first image is to be projected, using an object that already exists on the road surface RS.

[0159] (Appendix 7) The construction assistance method described in Appendix 1, wherein the object on the road surface includes at least one of road markings pre-painted on the road surface and structures fixed to the road surface, the first image includes a second image representing the object on the road surface, and before determining the position on the road surface to project the first image, the method includes projecting the first image onto the road surface from the optical device, and determining the position on the road surface to project the first image based on the correspondence between the image of the object on the road surface included in the captured image and the second image on the road surface.

[0160] This makes it possible to more easily determine the position on the road surface RS to project the first image by determining the position on the road surface RS based on the correspondence between the image of the object on the road surface included in the captured image and the second image.

[0161] (Supplementary Note 8) The construction assistance method according to Supplementary Note 7, wherein if the correspondence does not satisfy a first condition, a notification indicating that the position on the road surface cannot be determined is output.

[0162] This allows the user U of the construction assistance method to consider other construction assistance methods when the position on the road surface RS cannot be determined because the correspondence relationship does not satisfy the first condition.

[0163] (Appendix 9) A construction assistance device comprising a processing device that performs the following operations: acquiring a captured image showing a road surface and an object on the road surface; determining a position on the road surface to project a first image used to paint a road marking on the road surface based on the object on the road surface included in the captured image; and causing an optical device to project the first image at the position on the road surface.

[0164] This provides a road marking construction assistance device that can be used in fewer locations than conventional techniques.

[0165] Furthermore, in typical road marking construction methods, workers often manually draw a draft for painting the road marking on the road surface using a marking marker. Because the time required for drawing such a draft accounts for a large proportion of the overall construction time, there is a demand for work efficiency improvements. According to the construction assistance method disclosed herein, the positioning work performed by workers when manually drawing the draft using a marking marker is simplified, or the work of manually drawing using a marking marker is not necessary at all, thereby enabling work efficiency improvements compared to when the entire work of drawing the draft is done manually. Furthermore, while drawing a draft line using a marking marker requires the skill of a craftsman, the construction assistance method disclosed herein enables even beginners to accurately draw a draft line.

[0166] (Appendix 10) An information processing program that causes a computer to acquire a captured image showing a road surface and an object on the road surface, determine a position on the road surface to project a first image used to paint a road marking on the road surface based on the object on the road surface included in the captured image, and project the first image from an optical device onto the position on the road surface.

[0167] This provides a road marking construction assistance method that has fewer restrictions on the locations where it can be used compared to conventional techniques.

[0168] Furthermore, in typical road marking construction methods, workers often manually draw a draft for painting the road marking on the road surface using a marking marker. Because the time required for drawing such a draft accounts for a large proportion of the overall construction time, there is a demand for work efficiency improvements. According to the construction assistance method disclosed herein, the positioning work performed by workers when manually drawing the draft using a marking marker is simplified, or the work of manually drawing using a marking marker is not necessary at all, thereby enabling work efficiency improvements compared to when the entire work of drawing the draft is done manually. Furthermore, while drawing a draft line using a marking marker requires the skill of a craftsman, the construction assistance method disclosed herein enables even beginners to accurately draw a draft line. [Explanation of symbols]

[0169] 1...construction support device, 1A...construction support device, 10, 10A...information processing device, 11...main body unit, 20...projector, 21...main body unit, 110...imaging device, 120, 120A...processing device, 121...acquisition unit, 122...display control unit, 123, 123A...detection unit, 124...reception unit, 125, 125A...determination unit, 126...projection control unit, 127...extraction unit, 128...collation unit, 129...notification unit, 140, 140A...storage device, 150...display device, 160...input device, 170...communication device, 210...projection device, 220...processing device, 221...acquisition unit, 222...projection control unit, 230...storage device, 240...communication device, 310...projection device

Claims

1. acquiring a captured image showing a road surface and an object on the road surface; determining a position on the road surface to project a first image used to paint a road marking on the road surface based on an object on the road surface included in the captured image; projecting the first image from an optical device onto a position on the road surface; A construction assistance method, including:

2. receiving an operation to select, from the captured image, two parallel first line segments that indicate at least a part of a contour of an object on the road surface; receiving an input indicating a first distance between the two first line segments on the road surface; Further comprising: determining a position on the road surface based on the two first line segments and the first distance; The construction assistance method according to claim 1 .

3. and causing a display device to display an auxiliary image used to determine the position on the road surface. The construction assistance method according to claim 1 .

4. the auxiliary image indicates at least one of the first image, the two first line segments, the first distance, and a distance on the road surface between the first image and an object on the road surface; The construction assistance method according to claim 3.

5. the first image includes a plurality of line segments that indicate at least a portion of the contour of the road marking; receiving an operation to select two parallel second line segments from the first image among the plurality of line segments; receiving an input indicating a second distance between the two second line segments on the road surface; Further comprising: determining a position on the road surface onto which the first image is to be projected based on the two first line segments, the first distance, the two second line segments, and the second distance; The construction assistance method according to claim 2.

6. The object on the road surface includes at least one of a road marking painted on the road surface in advance, a structure fixed to the road surface, and an object placed on the road surface and having at least a portion of a straight line. The construction assistance method according to any one of claims 1 to 5.

7. the object on the road surface includes at least one of a road marking painted on the road surface in advance and a structure fixed to the road surface; the first image includes a second image representing an object on the road surface; and projecting the first image onto the road surface from the optical device before determining the position on the road surface where the first image is to be projected; determining a position on the road surface to which the first image is to be projected based on a correspondence relationship between an image of an object on the road surface included in the captured image and the second image of the road surface; The construction assistance method according to claim 1 .

8. If the correspondence relationship does not satisfy a first condition, output a notification indicating that the position on the road surface cannot be determined. The construction assistance method according to claim 7.

9. acquiring a captured image showing a road surface and an object on the road surface; determining a position on the road surface to project a first image used to paint a road marking on the road surface based on an object on the road surface included in the captured image; causing an optical device to project the first image onto a position on the road surface; A construction assistance device comprising a processing device that executes the above.

10. acquiring a captured image showing a road surface and an object on the road surface; determining a position on the road surface to project a first image used to paint a road marking on the road surface based on an object on the road surface included in the captured image; projecting the first image from an optical device onto a position on the road surface; An information processing program that enables a computer to achieve the above.

Images