Car washing device and method for determining opening and closing of door mirror of automobile

Abstract

To accurately determine an opening / closing state of a door mirror.SOLUTION: A control unit (7) of a vehicle washing device (2) includes an image acquisition unit (71) configured to acquire an image of an automobile, a door mirror detection unit (72) configured to detect a rectangle based on a contour shape of a door mirror from the image, a first opening / closing determination unit (73) configured to determine an opening / closing state based on an aspect ratio of a bounding box, and a mechanism control unit (78) configured to control an operation of a vehicle washing machine body (4) based on a determination result.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a car wash apparatus and a method for determining whether a door mirror of an automobile is open or closed. [Background technology]

[0002] 2. Description of the Related Art A car washing apparatus capable of detecting the door mirrors of an automobile, which is a vehicle to be washed, from an image captured by a camera attached to the car washing apparatus main body is known in the prior art. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-123167 Summary of the Invention [Problem to be solved by the invention]

[0004] When using a car wash, it is necessary to keep the door mirrors in the closed position. However, there are cases where a user inadvertently washes a car with the door mirrors in the open position, resulting in damage to the door mirrors. In view of this situation, one aspect of the present invention aims to realize a car wash that can accurately detect the open / closed state of the door mirrors of an automobile being washed. [Means for solving the problem]

[0005] In order to solve the above problems, one aspect of the present invention is a car wash device comprising a car wash machine main body for washing automobiles, an imaging unit for capturing images of the automobile including the door mirrors, and a control unit, wherein the control unit comprises an image acquisition unit for acquiring the images, a door mirror detection unit for detecting a rectangle based on the contour shape of the door mirror and including at least a portion of the door mirror from the image acquired by the image acquisition unit, a first opening / closing judgment unit for judging the open / closed state of the door mirror based on the aspect ratio of the rectangle, and a mechanism control unit for controlling the operation of the car wash machine main body based on the judgment result of the first opening / closing judgment unit.

[0006] In order to solve the above problem, another aspect of the present invention is a method for determining whether a door mirror of a vehicle is open or closed, executed by an information processing device, and is characterized by including: an image acquisition step for acquiring an image of the vehicle including the door mirror; a door mirror detection step for detecting a rectangle based on the contour shape of the door mirror and including at least a portion of the door mirror from the acquired image; and an open / close determination step for determining the open / close state of the door mirror based on the aspect ratio of the rectangle.

[0007] The control unit according to each aspect of the present invention may be realized by a computer. In this case, the control program for a car wash device that realizes the control unit by making the computer operate as each part (software element) that the control unit has, and the computer-readable recording medium on which it is recorded, also fall within the scope of the present invention. [Effects of the Invention]

[0008] According to one aspect of the present invention, a car wash apparatus can be realized that can accurately detect the open / closed state of the door mirrors of an automobile, which is a vehicle to be washed. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a side view of a car wash apparatus and a front view of a car wash machine main body, for schematically illustrating the car wash apparatus according to a first embodiment. [Figure 2] 1 is a schematic diagram showing the main parts of the car wash apparatus according to the first embodiment as seen from above, with the car X parked therein. [Figure 3] 1 is a block diagram showing a configuration of a main part of a car wash apparatus according to a first embodiment. [Figure 4] 1 is a diagram for explaining training data used in machine learning for constructing a trained model that is applied to the door mirror detection unit of the car wash apparatus according to embodiment 1. It relates to an example image in which the door mirrors are closed. [Figure 5] 1 is a diagram for explaining training data used in machine learning for constructing a trained model that is applied to the door mirror detection unit of the car wash apparatus according to embodiment 1. It relates to an example image in which the door mirror is in an open state. [Figure 6] FIG. 2 is a diagram for explaining the output results of a door mirror detection unit to which a trained model is applied in the car wash apparatus according to the first embodiment. [Figure 7] 10 is a diagram for explaining the operation of the second open / close determination unit of the car wash apparatus according to the second embodiment, and relates to an example image in which the door mirror is in the open state. [Figure 8] 10 is a diagram for explaining the operation of the second open / close determination unit of the car wash apparatus according to the second embodiment, and relates to an example image in which the door mirrors are in a closed state. [Figure 9] FIG. 10 is a block diagram showing the configuration of a main part of a car wash apparatus according to a second embodiment. [Figure 10] 10 is a schematic diagram showing the main parts of a car wash apparatus according to Modification 2 as seen from above, with an automobile X parked therein. DETAILED DESCRIPTION OF THE INVENTION

[0010] [Embodiment 1] <Car wash equipment overview> Hereinafter, car wash apparatuses according to embodiments of the present invention will be described in detail. In embodiment 1, an exemplary car wash apparatus 2 that performs a car wash process to wash an automobile X will be described.

[0011] Fig. 1 is a schematic diagram showing a schematic side view 2S of a car wash machine main body 4 and a remote panel 6 provided in a car wash apparatus 2 according to embodiment 1, and a schematic front view 4F of the car wash machine main body 4. Fig. 2 is a schematic view (schematic plan view) of the main parts of the car wash apparatus 2 and an automobile X parked at a car wash position, which is a stopping position during car washing, as viewed from above.

[0012] As shown in Fig. 1, the car wash apparatus 2 includes a car wash machine main body 4 that washes the automobile X, which is a vehicle to be washed. The car wash apparatus 2 further includes a remote panel 6 that acquires the car wash conditions of the automobile X by the car wash machine main body 4. In the schematic side view 2S, the outline of the remote panel 6 is shown by a dotted line to indicate that the remote panel 6 is located further back than the automobile X toward the page.

[0013] As shown in the schematic front view 4F, the car wash machine main body 4 includes, for example, two frames 8 on the left and right, and a ceiling portion 10 connecting the upper ends of the two frames 8. The car wash machine main body 4 has a structure that allows the automobile X to pass through a space 4S surrounded by the frames 8 and the ceiling portion 10 along the approach direction DA of the automobile X, as shown in the schematic side view 2S. In this specification, the approach direction DA is defined as the direction from the front surface 4A of the car wash machine main body 4 toward the rear surface 4B.

[0014] As shown in Fig. 2, an approach guide G is formed in the car wash facility where the car wash machine main body 4 is installed. The approach guide G is formed on the ground in the car wash facility along the approach direction DA, and functions as a reference for indicating where and in which direction the automobile X to be washed by the car wash machine main body 4 should stop. In other words, the approach guide G serves as a guide for at least one of the approach direction DA of the automobile X relative to the car wash machine main body 4 or the stopping position during washing. The approach guide G includes, for example, a right approach guide GR arranged on the right side of the ideal stopping position of the automobile X, and a left approach guide GL arranged on the left side.

[0015] Furthermore, the car wash apparatus 2 may be equipped with a display unit 45 that provides various instructions and / or displays to the driver. The display unit 45 may include, for example, a first display panel 45A installed on the front surface 4A of the car wash machine main body 4 and a second display panel 45B installed behind the car wash machine main body 4 in a position visible to the driver of the automobile X being washed. Alternatively, the car wash apparatus 2 may be equipped with either the first display panel 45A or the second display panel 45B.

[0016] <Car wash machine body> The car wash machine main body 4 has wheels 12 at the bottom of each of the frames 8, and by rotating the wheels 12 using a drive unit (not shown), the car wash machine main body 4 moves relatively to the automobile X in the front-to-rear direction along rails R arranged on the ground G. The rails R are formed, for example, along the approach direction DA. Here, the car wash machine main body 4 washes the automobile X in the space 4S while moving relatively to the automobile X.

[0017] 1, of the directions of movement of the car wash machine main body 4, the direction toward the front surface 4A is referred to as the forward direction D1, and the direction toward the rear surface 4B is referred to as the backward direction D2. Hereinafter, movement of the car wash machine main body 4 in the forward direction D1 may be simply referred to as the forward movement of the car wash machine main body 4, and movement of the car wash machine main body 4 in the backward direction D2 may be simply referred to as the backward movement of the car wash machine main body 4.

[0018] The car wash machine main body 4 is equipped with a plurality of brushes that slide over and brush the automobile X as one of its cleaning parts. For example, the brushes equipped in the car wash machine main body 4 include a top brush 14, side brushes 16, and rocker brushes 18, each of which is rotated by a rotary motor (not shown). The top brush 14 slides along the top surface of the automobile X and cleans the top surface of the automobile X. The left and right side brushes 16 and the rocker brush 18 clean both sides of the automobile X.

[0019] A tank housing section 20 is disposed on the side of the car wash machine main body 4, and houses a plurality of liquid storage tanks (not shown) that store various liquid agents, including detergents and waxes. Above the tank housing section 20, a distribution piping section 22 is provided that distributes water, including city water, or liquid agents from each liquid storage tank. A plurality of nozzles, which will be described in detail below, are led out from the distribution piping section 22 via respective solenoid valves (not shown). The plurality of nozzles include a first nozzle that sprays a liquid, including city water or a cleaning liquid, onto the automobile X in order to wash the automobile X, and a second nozzle that sprays a coating agent, including a water-repellent coating agent or wax, onto the automobile X in order to form a coating film on the surface of the automobile X.

[0020] The first nozzles include a first purified water nozzle 24, a second purified water nozzle 26, a first detergent nozzle 28, and a second detergent nozzle 30. The first purified water nozzle 24 and the second purified water nozzle 26 are respectively arranged on the front surface 4A and rear surface 4B of each frame 8 of the car wash machine main body 4, and spray water including city water onto the automobile X. The first detergent nozzle 28 and the second detergent nozzle 30 are respectively arranged on the front surface 4A and rear surface 4B of each frame 8, and spray cleaning liquid including shampoo etc. onto the automobile X.

[0021] The second nozzles include a water-repellent coating nozzle 32 and a wax nozzle 34. The water-repellent coating nozzle 32 and the wax nozzle 34 are arranged on the rear surface 4B of the car wash machine main body 4. The water-repellent coating nozzle 32 sprays a liquid water-repellent coating agent onto the automobile X. The wax nozzle 34 sprays wax onto the automobile X.

[0022] The car wash machine main body 4 is also provided with a blower 36 that generates an air current to dry the automobile X. A top air blowing nozzle 38 and a side air blowing nozzle 40 are connected to the blower 36. The top air blowing nozzle 38 is provided at the upper center of the car wash machine main body 4 and blows air toward the ceiling surface of the automobile X. The side air blowing nozzles 40 are provided on both sides of the car wash machine main body 4 and blow air toward the sides of the automobile X. The car wash machine main body 4 dries the automobile X after washing by blowing air from the top air blowing nozzle 38 and the side air blowing nozzles 40.

[0023] The car wash machine main body 4 further includes a sensor 52. The sensor 52 is located, for example, closer to the front surface 4A of the car wash machine main body 4 than the side brushes 16. The sensor 52 is a sensor for acquiring information about the outer shape of the automobile X to be washed. More specifically, the sensor 52 can detect the height of the automobile X crossing a predetermined point.

[0024] For example, the sensor 52 may be configured as a multi-axis optical axis sensor in which a light emitting unit and a light receiving unit are arranged as a pair on each of the two frames 8 on the front side and space 4S side of the frames 8 of the car wash machine main body 4. Here, in the multi-axis optical axis sensor, the individual optical axis sensors are arranged so that multiple optical axes are aligned in the vertical direction and each optical axis is on a substantially horizontal plane.

[0025] Therefore, each optical axis sensor in the multi-axis optical axis sensor is configured to be able to detect the presence or absence of the automobile X at different heights. In this case, the car wash apparatus 2 can obtain information about the outer shape of the automobile X by detecting the height of the automobile X at each position where it crosses the sensor 52 while the car wash machine main body 4 moves relatively from the front side to the rear side of the automobile X.

[0026] 1 and 2, for simplicity of illustration, the devices for washing the automobile X that are provided in the car wash machine main body 4 described above may be omitted. Furthermore, the devices provided in the car wash machine main body 4 shown in Fig. 1 are merely examples, and the car wash machine main body 4 may be provided with devices for washing the automobile X and devices for assisting the washing, including conventionally known configurations, in addition to the devices described above, on the frame 8 or the ceiling portion 10.

[0027] An operation panel 42 is disposed on the front of one frame 8 of the car wash machine main body 4. The operation panel 42 has operation buttons (not shown) for setting the car wash conditions. For example, the driver who has gotten out of the car X or another technician may operate the operation buttons to set the car wash conditions, etc.

[0028] <Remote Panel> The remote panel 6 is located, for example, on the front side of the car wash machine main body 4, and is disposed roughly along the direction of movement of the car wash machine main body 4. As shown in FIG. 1 , the remote panel 6 is disposed so that its front surface faces the side of the car X before it is washed by the car wash machine main body 4, in other words, before it enters the car wash machine main body 4.

[0029] 1, the remote panel 6 includes a housing 46 and a support pole 48 that is erected on the ground G and supports the housing 46. The remote panel 6 may acquire at least some of the car wash conditions for the car X to be washed by the car wash machine main body 4 by the driver operating a touch panel or buttons (not shown) provided on the housing 46.

[0030] <Image capture unit> An imaging unit 9 is fixed to the car wash machine main body 4. The imaging unit 9 is fixed to the car wash machine main body 4 so as to capture images from above the door mirror Xd. The imaging unit 9 may include an imaging unit 9R provided on the upper right side of the car wash machine main body 4 and an imaging unit 9L provided on the upper left side. The imaging unit 9, the imaging unit 9R, and the imaging unit 9L are each an example of an imaging unit according to the present disclosure. The car wash apparatus 2 according to the first embodiment is equipped with at least one of the imaging unit 9R and the imaging unit 9L.

[0031] The imaging area of ​​the imaging unit 9R includes the right side of the automobile X parked at the parking position. Therefore, the imaging unit 9R captures an image including the door mirror Xd on the right side of the automobile X. The imaging area of ​​the imaging unit 9L includes the left side of the automobile X parked at the parking position. Therefore, the imaging unit 9L captures an image including the door mirror Xd on the left side of the automobile X.

[0032] In the first embodiment, an example is shown in which the imaging unit 9 is fixed to the car wash machine main body 4, but the arrangement of the imaging unit 9 is not limited to this example. The imaging unit 9 may be installed independently from the car wash machine main body 4 as long as it is located in a position where it can capture an image of one of the door mirrors Xd of the automobile X. For example, the imaging unit 9R and the imaging unit 9L may be attached to support pillars installed on both sides of the car wash machine main body 4, respectively.

[0033] <Control unit> Furthermore, the car wash apparatus 2 is equipped with a control unit 7 that executes required information processing and controls each unit of the car wash apparatus 2. FIG. 3 is a functional block diagram showing the configuration of the main parts of the car wash apparatus 2, focusing on the function of the control unit 7. The control unit 7 may appropriately transmit and receive information to and from the car wash machine main body 4 or the remote panel 6, and control the car wash machine main body 4. The control unit 7 may also transmit and receive information to and from devices external to the car wash apparatus 2 via the communication unit 60.

[0034] The control unit 7 controls the movement of the car wash machine main body 4 along the rail R and the operation of each of the above-mentioned parts of the car wash machine main body 4, thereby causing the car wash machine main body 4 to wash the car X. In this case, the control unit 7 may control the car wash machine main body 4 to wash the car X, reflecting the car wash conditions acquired by the operation panel 42 and the remote panel 6. Furthermore, the control unit 7 may determine the open / closed state of the door mirrors of the car X based on the image captured by the imaging unit 9, and control the car wash machine main body 4 to wash the car X, reflecting the result of the determination.

[0035] The control unit 7 may be physically located in the car wash machine main body 4 as shown in Fig. 1, or may be located outside the car wash machine main body 4. The control unit 7 is configured, for example, by a computer having a processor such as a CPU (Central Processing Unit), and various calculations and controls are realized by executing a control program stored in the memory unit 76 on the processor.

[0036] The hardware configuration of the control unit 7 is not limited to a CPU, and may be, for example, a suitable information processing device using a GPU (Graphics Processing Unit), a microprocessor, a digital signal processor, a microcontroller, an ASIC (Application Specific Integrated Circuit) such as a TPU (Tensor Processing Unit), or a combination thereof. The hardware configuration of the storage unit 76 may be, for example, a random access memory, a flash memory, a hard disk drive, or the like.

[0037] 3, the control unit 7 of the car wash apparatus 2 according to the first embodiment includes functional blocks of an image acquisition unit 71, a door mirror detection unit 72, a first open / close determination unit 73, a notification unit 77, and a mechanism control unit 78. The control unit 7 also includes the above-mentioned memory unit 76. The image acquisition unit 71 is a functional block that acquires images captured by the imaging unit 9.

[0038] The door mirror detection unit 72 is a functional block that detects a bounding box Bx surrounding a door mirror Xd from an image captured by the imaging unit 9. In the first embodiment, an example in which a trained model M constructed by machine learning is applied is shown as an example of such a functional block, but the configuration of the door mirror detection unit 72 is not limited to this example. The first open / close determination unit 73 is a functional block that determines the open / closed state of the door mirror Xd of the photographed automobile X from the bounding box Bx detected by the door mirror detection unit 72.

[0039] The mechanism control unit 78 is a functional block that controls the above-mentioned components of the car wash machine main body 4 to have the car wash machine main body 4 wash the automobile X. The notification unit 77 is a functional block that notifies drivers who use the car wash device 2 or other technicians, etc. via the display unit 45. The notification unit 77 is also a functional block that notifies external devices, such as a server, via the communication unit 60.

[0040] <Trained model M> Next, a detailed description will be given of the trained model M constructed by machine learning, which is applied to the car wash apparatus 2 of the first embodiment. The trained model M is an inference model that detects and outputs a bounding box Bx surrounding the door mirror Xd from an image including the door mirror Xd of the automobile X captured by the imaging unit 9 (at least one of the imaging unit 9R and the imaging unit 9L). A specific description will be given below using as an example an image of the right side of the automobile X captured by the imaging unit 9R.

[0041] Here, "door mirror" is a class classified by the trained model M, but in embodiment 1, the trained model M is an inference model to which only "door mirror" is applied as a class. A bounding box is a rectangular or quadrilateral box that surrounds an area in which an object is captured in a machine learning task such as object detection or image recognition, and in this application, the sides of the bounding box are either vertical or horizontal in the image.

[0042] The trained model M also outputs the bounding box Bx surrounding the door mirror Xd with a score indicating the likelihood of the inference result. In the following, such a score is calculated as a numerical value ranging from 0 to 1 according to the probability that the object indicated by being surrounded by the bounding box Bx is indeed the door mirror Xd.

[0043] In this disclosure, machine learning generally refers to a process of automatically constructing a detection algorithm based on data. In this disclosure, an algorithm for detecting an item constructed by machine learning can be applied to the trained model M. A convolutional neural network (CNN) is particularly suitable as a machine learning model for such purposes. This is because a CNN is configured to be able to accurately detect a target item regardless of its position in an image or whether the item is flipped left or right, by applying a convolution layer and a pooling layer.

[0044] 4 and 5 are diagrams for explaining the procedure for constructing a trained model M by machine learning. Reference symbol p41 in FIG. 4 is an image of the right side of the automobile X captured by the imaging unit 9R when the door mirror Xd is in a closed state. Reference symbol p51 in FIG. 5 is an image of the right side of the automobile X captured by the imaging unit 9R when the door mirror Xd is in an open state.

[0045] Also, reference numeral p42 in Fig. 4 is a diagram showing a situation in which the position of the door mirror Xd for the image represented by reference numeral p41 is taught using a bounding box Bt surrounding the door mirror Xd. In the diagram represented by reference numeral p42, the bounding box Bt is shown on the image represented by reference numeral p41, indicating that the position of the door mirror Xd for the image represented by reference numeral p41 is taught. The same applies to reference numeral p52 in Fig. 5.

[0046] 4 and 5, in the example of embodiment 1, the image of automobile X captured by the imaging unit 9 is captured from a bird's-eye view of the side of automobile X so that the longitudinal direction of the image is the longitudinal direction of the automobile X. Therefore, in the case of FIG. 4 where the door mirror Xd is in the closed state, the bounding box Bt surrounding the door mirror Xd is a vertically long rectangle. On the other hand, in the case of FIG. 5 where the door mirror Xd is in the closed state, the bounding box Bt surrounding the door mirror Xd is a horizontally long rectangle.

[0047] A large number of sets of training data are prepared, each set consisting of a learning image including the door mirror of the automobile captured by the imaging unit 9 and information on the position of a bounding box Bt surrounding the door mirror Xd in the learning image. The large number of training data sets preferably includes data for both the case where the door mirror Xd of the automobile X is in a closed state and the case where it is in an open state.

[0048] The learning images are not limited to images captured by the imaging unit 9, and separately collected images may be used as long as they are similar to the images captured by the imaging unit 9. A learned model M is constructed by using a known method to train a machine learning model such as a convolutional neural network on a large amount of the training data. The constructed learned model M is stored in the memory unit 76 of the control unit 7 in the car wash apparatus 2.

[0049] 6 shows a bounding box Bx output by the trained model M as a result of detecting the door mirror Xd when an image of the right side of the automobile X captured by the imaging unit 9R is input, the bounding box Bx being shown on the image. As described above, the trained model M outputs the bounding box Bx with a score indicating the likelihood of detecting the door mirror Xd.

[0050] <Characteristic operation of car wash equipment> The characteristic operations performed by the car wash apparatus 2 according to the first embodiment will be described in detail below. The automobile X, which is a vehicle to be washed, stops at a parking position for the car wash. The imaging unit 9 captures an image of the automobile X, including the door mirror Xd. The image acquisition unit 71 acquires the image, including the door mirror Xd of the automobile X, captured by the imaging unit 9.

[0051] It is desirable that the timing for capturing images by the image capturing unit 71 be after the automobile X has stopped at the stopping position in this way. This is because, when using the car wash device 2, there are a considerable number of drivers who close the door mirrors after stopping the automobile X at the stopping position. Therefore, it is also desirable that the timing for capturing images by the image capturing unit 71 be after the car wash operation has started, for example, after the brushes have started to rotate. Alternatively, it is also desirable that the timing be after the car wash machine main body 4 has started to move forward from the standby position.

[0052] Next, the door mirror detection unit 72 executes the learned model M stored in the memory unit 76 to detect, as described above, a bounding box Bx surrounding the door mirror Xd from the image acquired by the image acquisition unit 71. Furthermore, the first open / close determination unit 73 uses the bounding box Bx output by the door mirror detection unit 72 to determine whether the door mirror Xd is open or closed.

[0053] The open / closed state of the door mirror Xd is determined as follows: The first open / close determination unit 73 calculates the aspect ratio of the bounding box Bx. Here, for example, the aspect ratio is defined as W / H, where H is the vertical length of the bounding box Bx and W is the horizontal length of the bounding box Bx. As described above, when the door mirror Xd is closed, the bounding box Bx is vertically long and the aspect ratio W / H is small. When the door mirror Xd is open, the bounding box Bx is horizontally long and the aspect ratio W / H is large.

[0054] For example, the first open / close determination unit 73 determines that the door mirror Xd is closed if the aspect ratio W / H is equal to or less than 1, and determines that the door mirror Xd is open if the aspect ratio W / H is greater than 1. Thus, according to the first embodiment, it was possible to correctly determine the open / closed state of the door mirror Xd.

[0055] It should be noted that the judgment threshold value 1 here is an example and should be set appropriately depending on the arrangement and orientation of the imaging unit 9 relative to the automobile X. Furthermore, if the score output by the door mirror detection unit 72 based on the trained model M is lower than a predetermined value, the first open / close judgment unit 73 may determine that it is impossible to judge the open / closed state, since it is unlikely that the door mirror Xd has been correctly detected.

[0056] Furthermore, when the control unit 7 determines the open / closed state of the door mirror Xd for both the image of the right side of the automobile X captured by the imaging unit 9R and the image of the left side of the automobile X captured by the imaging unit 9L, the final open / closed state determination may be made as follows: If both open / closed state determinations match, that determination result is used as the final determination. If one of the determinations is impossible, the other open / closed state determination result is used. If the two determination results are opposite, the final determination is made to be on the safe side that the door mirror Xd is in the open state.

[0057] The control unit 7 performs a car wash according to the determined open / closed state of the door mirror Xd by controlling each part of the car wash machine main body 4 using the mechanism control unit 78. For example, when it is determined that the door mirror Xd is in the open state, the side brushes 16 wash the side of the automobile X at a distance less than when the door mirror Xd is in the closed state.

[0058] Furthermore, when it is determined that the door mirror Xd is in the open state, the notification unit 77 may immediately notify the driver of this fact via the display unit 45. Alternatively, the notification unit 77 may transmit the result of the determination of the open / closed state and an image to the server via the communication unit 60. By storing the image and the information on the determination result in the server, it becomes possible to detect, for example, when an erroneous determination of the open / closed state of the door mirror Xd occurs in a particular vehicle model.

[0059] In addition, the car wash device 2 may be configured not to determine the open / closed state of the above-mentioned door mirror Xd if it is declared by operating the remote panel 6 prior to the car wash that the vehicle to be washed does not have opening / closing door mirrors.

[0060] [Embodiment 2] Other embodiments of the present invention will be described below. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the above embodiment, and the description thereof will not be repeated.

[0061] 9 is a functional block diagram showing the main configuration of the car wash apparatus 2 according to the second embodiment, focusing on the functions of the control unit 7. The second embodiment differs from the first embodiment in that the control unit 7 of the car wash apparatus 2 further includes a second open / close determination unit 74 and an integrated determination unit 75. The second open / close determination unit 74 is a functional block that determines the open / closed state of the door mirror Xd using a method different from that of the first open / close determination unit 73. The integrated determination unit 75 is a functional block that makes a final determination of the open / closed state of the door mirror Xd by comprehensively determining the open / closed state based on the open / closed state determination results by the first open / close determination unit 73 and the open / closed state determination results by the second open / close determination unit 74.

[0062] 7 and 8 are diagrams for explaining the procedure for determining the open / closed state in the second open / close determination unit 74. Fig. 7 shows an example in which the door mirror Xd is in the open state. Fig. 8 shows an example in which the door mirror Xd is in the closed state. The second open / close determination unit 74 determines whether the door mirror Xd is in the open / closed state as follows.

[0063] Reference symbol p71 in Fig. 7 indicates a first region, which is a part of the image including the region of the bounding box Bx detected by the door mirror detection unit 72. In the example of Fig. 7, the first region is assumed to coincide with the region of the bounding box Bx in the image. However, the first region does not necessarily have to coincide completely with the region of the bounding box Bx.

[0064] For example, the first region may be a region that contains the region of the bounding box Bx but is slightly larger than the region of the bounding box Bx. Conversely, the first region may be a region that is contained in the region of the bounding box Bx but is slightly smaller than the region of the bounding box Bx.

[0065] The second open / close determination unit 74 acquires the image captured by the imaging unit 9 from the image acquisition unit 71, and cuts out the first region from the image based on the detection result of the door mirror detection unit 72. Next, the second open / close determination unit 74 performs appropriate preprocessing on the image of the first region indicated by reference symbol p71. Reference symbol p72 in FIG. 7 is the image of the first region after such preprocessing.

[0066] As shown in the image p72, such pre-processing is preferably a process that performs so-called edge extraction. It is also preferable that the pre-processing be appropriately combined with gray-scaling and negative-positive inversion. The purpose of such pre-processing is to obtain a clear Fourier transform image for detecting the open / closed state of the door mirror Xd in subsequent processing, and any appropriate processing may be performed for this purpose.

[0067] Next, the second open / close determination unit 74 performs a two-dimensional fast Fourier transform (2D FFT) on the image p72 to obtain a Fourier transform image as shown in p73. The two-dimensional fast Fourier transform is an example of a two-dimensional Fourier transform. In the Fourier transform image p73, the horizontal axis represents the frequency kx (wavenumber) in the horizontal direction of the image, and the vertical axis represents the frequency ky (wavenumber) in the vertical direction of the image.

[0068] In the Fourier transform, a straight line portion such as the edge of the door mirror Xd in the image indicated by reference symbol p72 is expressed as a superposition of waves of various frequency components (wave number components) perpendicular to the extension direction of the straight line. Therefore, in the Fourier transform image indicated by reference symbol p73, a strong signal intensity is obtained on a line passing through the origin in the extension direction of the straight line. In the example of Figure 7, this corresponds to the signal on a line rotated approximately 20° from the ky axis in the direction of the kx axis.

[0069] Next, the second open / close determination unit 74 calculates the average intensity distribution in the angular direction in the Fourier transform image, as shown by reference symbol p74 in Fig. 7. The angle on the horizontal axis of the graph shown by reference symbol p74 in Fig. 7 is defined as the rotation angle from the positive direction of the ky axis, which is 0°, toward the positive direction of the kx axis. In other words, the angle on the horizontal axis is the angle obtained when the positive direction of the ky axis is 0° and the image is rotated clockwise from there.

[0070] The intensity on the vertical axis of the graph indicated by reference numeral p74 in Figure 7 is the average value of the Fourier component intensity on the radius vector of the angle in the range of the graph. Here, the radius vector is a half line starting from the origin. Reference numeral s1 on the graph indicated by reference numeral p74 is the angular average of the intensity on the vertical axis of the graph over the entire range from 0° to 180°.

[0071] Furthermore, the symbol s2 in the graph indicated by symbol p74 represents the angular average of the intensity on the vertical axis of the graph within the angle range of 5° to 35°. Note that the angle range of 5° to 35° corresponds to the angle range in which the Fourier component intensity is large when the door mirror Xd is open. This angle range corresponds to the angle in which the extension direction of the straight line is 95° to 125° in the real space image before the Fourier transform.

[0072] In Figure 8, which shows an example in which the door mirror Xd is in the closed state, reference symbols p81 to p84 are images or graphs corresponding to reference symbols p71 to p73 in the example in Figure 7. In the graph on p84 in Figure 8, no significant Fourier component intensity is obtained in the angle range from 5° to 35°. In this way, the second open / close determination unit 74 determines whether the door mirror Xd is in the open state or the closed state depending on whether or not a significant Fourier component intensity is obtained in a specific angular range in the average intensity distribution in the angular direction in the Fourier transform image.

[0073] More specifically, by comparing the angle average S1 of the intensity over the entire angle range (0 to 180°) with the angle average S2 of the intensity over the specific angle range, it can be determined that the door mirror Xd is in the open state if, for example, the magnitude of S2 / S1 is greater than a predetermined value, and that the door mirror Xd is in the closed state if it is less than the predetermined value. Note that the angle range for determination here is an example and should be set appropriately depending on the arrangement and orientation of the imaging unit 9 relative to the automobile X. In particular, such angle ranges should be in a left-right reversed relationship between an image captured of the right side of the automobile X and an image captured of the left side of the automobile X.

[0074] The integrated determination unit 75 comprehensively determines the open / closed state of the door mirror Xd of the automobile X by comprehensively determining the open / closed state determination result by the first open / closed state determination unit 73 and the open / closed state determination result by the second open / closed state determination unit. Such a comprehensive determination by the integrated determination unit 75 may be performed as appropriate. For example, as a first example, the comprehensive determination may be performed as follows.

[0075] In the first embodiment, an example has been shown in which the open / closed state of the door mirror Xd is determined using a threshold value of the aspect ratio W / H of 1. In this case, if the calculated aspect ratio W / H is sufficiently smaller than the threshold value of 1, it can be said that the door mirror Xd is definitely in the closed state. Also, if the calculated aspect ratio W / H is sufficiently larger than the threshold value of 1, it can be said that the door mirror Xd is definitely in the open state. On the other hand, if the calculated aspect ratio W / H is close to the threshold value of 1, it is considered that there is a high possibility of an erroneous determination.

[0076] Therefore, when the aspect ratio W / H calculated by the first open / close determination unit 73 is equal to or less than a first threshold value (e.g., 0.8) that is smaller than threshold value 1, or is equal to or greater than a second threshold value (e.g., 1.2) that is larger than threshold value 1, the overall determination result follows the determination result of the first open / close determination unit 73. On the other hand, when the aspect ratio W / H calculated by the first open / close determination unit 73 is larger than the first threshold value and smaller than the second threshold value, the overall determination result follows the determination result of the second open / close determination unit 74.

[0077] Alternatively, as a second example, a comprehensive determination may be made as follows: If a clear peak appears in the angle range of 5° to 35° in the average intensity distribution in the angular direction in the Fourier transform image calculated by the second open / close determination unit 74, that is, if the value S2 / S1 is sufficiently large, it can be said that the state is definitely open. Therefore, in such cases, the determination result of the second open / close determination unit 74 is followed, and in other cases, the determination result of the first open / close determination unit 73 is followed.

[0078] [Variation 1] The car wash apparatus 2 according to Modification 1 is a modification of the car wash apparatus 2 according to Embodiment 1 or the car wash apparatus 2 according to Embodiment 2. Modification 1 differs from the above-described embodiments in that the door mirror detection unit 72 is configured to detect the door mirror Xd from the image captured by the imaging unit 9 using a known pattern matching technique.

[0079] The door mirror detection unit 72 in Modification 1 detects the door mirror Xd by pattern matching using a pre-registered pattern. The pattern registered to detect the door mirror Xd is an image of the door mirror Xd or an image that extracts features of the door mirror Xd, and may be what is also called a template image in pattern matching technology. In order to accommodate various door mirrors Xd, the door mirror detection unit 72 has a variety of registered patterns.

[0080] Furthermore, the door mirror detection unit 72 in Modification 1 generates a bounding box Bx of the detected door mirror Xd. Note that if the pattern matching algorithm in the door mirror detection unit 72 is configured to extract the bounding box of the door mirror Xd, there is no need to generate the bounding box Bx again.

[0081] The operation of each functional block of the control unit 7 of the car wash apparatus 2 according to the first modification, other than the door mirror detection unit 72, is the same as in the above-described embodiments. In this way, the car wash apparatus 2 according to the first modification can detect the open / closed state of the door mirror Xd in the same manner as in the above-described embodiments.

[0082] [Variation 2] The car wash apparatus 2 according to Modification 2 is a modification of the car wash apparatus 2 according to Embodiment 1 or the car wash apparatus 2 according to Embodiment 2. In Modification 2, the mounting position and imaging direction of the imaging unit 9 on the car wash machine main body 4 are different from those shown in FIG.

[0083] 10 is a schematic diagram showing the main parts of the car wash apparatus 2 according to Modification 2 as seen from above, and shows the installation position and shooting direction of the imaging unit 9 in Modification 2. In FIG. 2, the imaging unit 9 captures images at an angle that captures the sides of the automobile X. However, in Modification 2, the imaging unit 9 captures images of the automobile X at an angle that captures images outward from the left and right sides of the automobile in the horizontal direction.

[0084] Therefore, the imaging unit 9 of the second modification captures an image of the automobile X, including the door mirror Xd, from above at an angle that makes it difficult for the side of the automobile X parked at the parking position to be captured. According to the second modification, since the side of the automobile X is unlikely to be captured in the image of the automobile X captured by the imaging unit 9, there is a possibility that the detection accuracy of the door mirror Xd by the door mirror detection unit 72 (trained model M) may be further improved.

[0085] <Additional Notes> In the description of the embodiment, the term "bounding box" refers to the smallest rectangle (rectangle) that encloses the outline of the door mirror Xd in the image, as is commonly used in the field of image processing.

[0086] However, in the spirit of the present invention, any rectangle in an image whose aspect ratio allows determination of whether the door mirror is open or closed is not limited to a "bounding box." A rectangle that includes at least a portion of the door mirror and is based on the outline shape of the door mirror may be used instead of the "bounding box" in the description of the invention.

[0087] An example of such a rectangle is a bounding box surrounding the door mirror Xd, in which at least one of the boundaries of the upper side, lower side, vehicle body side, or opposite side of the door mirror Xd has been moved toward the outside or inside of the door mirror Xd according to a predetermined rule.

[0088] For example, if a rectangle that always follows a certain rule, such as moving a specific boundary outward or inward by a specific number of pixels or a predetermined multiple of the bounding box, is used, it is easy to understand that the open / closed state of the door mirror can also be determined based on the aspect ratio of such a rectangle. Note that when a trained model M constructed by machine learning is applied to the door mirror detection unit 72, such a rectangle can be applied to the training data instead of the bounding box Bt.

[0089] 〔summary〕 Aspect 1 of the present disclosure is a car wash device comprising a car wash machine main body for washing automobiles, an imaging unit for capturing images of the automobile including the door mirrors, and a control unit, wherein the control unit comprises an image acquisition unit for acquiring the images, a door mirror detection unit for detecting a rectangle including at least a portion of the door mirror and based on the contour shape of the door mirror from the image acquired by the image acquisition unit, a first opening / closing determination unit for determining the open / closed state of the door mirror based on the aspect ratio of the rectangle, and a mechanism control unit for controlling the operation of the car wash machine main body based on the determination result of the first opening / closing determination unit.

[0090] Aspect 2 of the present disclosure is a car wash device according to aspect 1, wherein the control unit further has a second opening / closing judgment unit that judges the opening / closing state of the door mirror based on the average intensity distribution in the angular direction in a Fourier transform image obtained by performing a two-dimensional Fourier transform on a first area, which is a portion of the image and includes the rectangular area detected by the door mirror detection unit, and the mechanism control unit may be configured to control the operation of the car wash machine main body based on the judgment results of the first opening / closing judgment unit and the second opening / closing judgment unit.

[0091] Aspect 3 of the present disclosure is a car wash device according to aspect 1 or 2, wherein the rectangle is a bounding box surrounding the door mirror, and the door mirror detection unit detects the bounding box surrounding the door mirror using a trained model constructed by machine learning, and the trained model may be constructed by machine learning using training data consisting of a set of training images of a car that include door mirrors and information on the position of the bounding box surrounding the door mirror in the training images.

[0092] A fourth aspect of the present disclosure is that, in the car wash device of the first or second aspect, the door mirror detection unit may be configured to detect the rectangle by detecting the door mirror through pattern matching using a pattern indicating a pre-registered door mirror.

[0093] Aspect 5 of the present disclosure is a method for determining whether a door mirror of a vehicle is open or closed, executed by an information processing device, and includes an image acquisition step for acquiring an image of the vehicle including the door mirror, a door mirror detection step for detecting a rectangle including at least a portion of the door mirror and based on the contour shape of the door mirror from the acquired image, and an open / close determination step for determining whether the door mirror is open or closed based on the aspect ratio of the rectangle.

[0094] A sixth aspect of the present disclosure is the car wash apparatus of any one of the first to fourth aspects, wherein the imaging unit may be fixed to the car wash machine body so as to capture the image from above the door mirror.

[0095] A seventh aspect of the present disclosure is a control program for causing a computer to function as a control unit of a car wash device of any one of aspects 1 to 4 or a car wash device of aspect 6, and is a control program for causing a computer to function as the image acquisition unit, the door mirror detection unit, the first opening / closing determination unit, and the mechanism control unit.

[0096] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0097] 2 Car wash equipment 4 Car wash machine body 6 Remote Panel 7 Control Unit 71 Image acquisition unit 72 Door mirror detector 73 First open / close determination unit 74 Second open / close determination unit 75 Integrated Judgment Department 76 Memory section 77 Information Department 78 Mechanism control unit 9, 9R, 9L imaging unit 45 Display section 60 Communications Department X car Xd door mirror Bt Bounding Box (training data) Bx bounding box

Claims

1. A car wash machine body for washing cars, an imaging unit that captures an image of the automobile including the door mirrors; A car wash device comprising: The control unit an image acquisition unit that acquires the image; a door mirror detection unit that detects a rectangle that includes at least a portion of the door mirror and is based on a contour shape of the door mirror from the image acquired by the image acquisition unit; a first open / close determination unit that determines whether the door mirror is open or closed based on an aspect ratio of the rectangle; a mechanism control unit that controls the operation of the car wash machine main body based on the determination result of the first opening / closing determination unit.

2. The control unit a second open / close determination unit that determines whether the door mirror is open or closed based on an average intensity distribution in an angular direction in a Fourier transform image obtained by two-dimensional Fourier transforming a first region that is a part of the image and includes the rectangular region detected by the door mirror detection unit, The car wash apparatus according to claim 1 , wherein the mechanism control unit controls the operation of the car wash machine main body based on a determination result of the first open / close determination unit and a determination result of the second open / close determination unit.

3. the rectangle is a bounding box surrounding the door mirror, The door mirror detection unit A bounding box surrounding the door mirror is detected using a trained model constructed by machine learning; 3. The car wash device according to claim 1, wherein the trained model is constructed by machine learning using training data consisting of a set of training images of a car that include door mirrors and information on the position of a bounding box surrounding the door mirrors in the training images.

4. The door mirror detection unit 3. The car wash apparatus according to claim 1, wherein the rectangle is detected by detecting the door mirror through pattern matching using a pattern representing a door mirror that has been registered in advance.

5. A method for determining whether a door mirror of an automobile is open or closed, executed by an information processing device, comprising: an image capturing step of capturing an image of the vehicle including the door mirror; a door mirror detection step of detecting a rectangle including at least a portion of the door mirror based on a contour shape of the door mirror from the acquired image; and an open / close determination step of determining whether the door mirror is open or closed based on the aspect ratio of the rectangle.

Images