Lane determination device, lane determination method and program

Abstract

To suppress deterioration of detection accuracy of a detection object vehicle in the case vehicles in a plurality of lanes are reflected within a camera view angle.SOLUTION: A lane determination device includes a processor, and a memory for storing a camera image imaged by a camera arranged so as to be able to image a plurality of vehicles respectively traveling in a plurality of adjacent lanes. The processor acquires information of a mask area for specifying a part to be excluded from a prescribed processing object in parts constituting the vehicles included in the camera image in cooperation with the memory, sets a specific detection frame to the vehicle reflected in the camera image and outputs parts of the vehicles having determination points inside / outside the specific detection frame which are not included in the mask area as the prescribed processing object.SELECTED DRAWING: Figure 13

Description

[Technical Field]

[0001] The present disclosure relates to a lane determination device, a lane determination method, and a program. [Background technology]

[0002] Patent Document 1 discloses a license plate recognition device that detects multiple quadrangles of license plate area candidates from an input image and performs character recognition on character areas included in the license plate area candidates. The license plate recognition device selects a license plate area candidate to output from the multiple detected license plate area candidates based on the character recognition results and quadrangle information for each license plate area candidate, and outputs information about the selected license plate area candidate. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-217347 Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, an input image to be processed in the license plate detection process shows multiple vehicles, each with its own license plate. However, it does not take into account the case where multiple vehicles are captured in the camera's field of view, which is a public or private road with multiple lanes. In other words, when detecting some kind of identification information (e.g., license plate or vehicle-specific ID) of a target vehicle, if vehicles are captured in multiple lanes, the detection results (i.e., vehicle identification numbers) of lanes different from the lane originally intended to be detected may be mixed together, or the lane may overlap with parts of other vehicles, making it impossible to detect, resulting in a problem of reduced detection accuracy for the identification numbers of target vehicles in the lane intended to be detected.

[0005] The present disclosure has been devised in consideration of the above-described conventional situation, and aims to suppress a decrease in the detection accuracy of a target vehicle when vehicles in multiple lanes are captured within the camera's field of view. [Means for solving the problem]

[0006] The present disclosure provides a lane determination device comprising a processor and a memory that stores camera images captured by a camera positioned so as to be able to capture images of multiple vehicles traveling in each of multiple adjacent lanes, wherein the processor cooperates with the memory to obtain information on a mask area for identifying parts of the vehicle included in the camera image that should be excluded from the target of specified processing, sets a specific detection frame for the vehicle shown in the camera image, and outputs parts of the vehicle that have inside and outside determination points of the specific detection frame that are not included in the mask area as the target of the specified processing.

[0007] The present disclosure also provides a lane determination method executed by a lane determination device, the lane determination method comprising: storing camera images captured by a camera positioned so as to capture images of multiple vehicles traveling in each of multiple adjacent lanes; acquiring information about a mask area for identifying parts of the vehicle included in the camera image that are to be excluded from the target of predetermined processing; setting a specific detection frame for the vehicle shown in the camera image; and outputting parts of the vehicle that have inside and outside determination points of the specific detection frame that are not included in the mask area as the target of the predetermined processing.

[0008] The present disclosure also provides a program for causing a lane determination device, which is a computer, to perform the following operations: storing camera images captured by a camera positioned so as to capture images of multiple vehicles traveling in each of multiple adjacent lanes; acquiring information about a mask area for identifying parts of the vehicle included in the camera image that are to be excluded from the target of specified processing; setting a specific detection frame for the vehicle shown in the camera image; and outputting parts of the vehicle that have inside and outside determination points of the specific detection frame that are not included in the mask area as the target of the specified processing.

[0009] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]

[0010] According to the present disclosure, it is possible to suppress a decrease in the detection accuracy of a target vehicle when vehicles in multiple lanes are captured within the camera's field of view. [Brief explanation of the drawings]

[0011] [Figure 1] A diagram showing an example of the exterior of a yard, a large facility where vehicles enter and exit. [Figure 2] (a) A diagram showing an example of the problem with ID detection processing when there are vehicles in multiple lanes traveling in the same direction, and (b) a diagram showing an example of the problem with ID detection processing when there are vehicles in multiple lanes traveling in opposite directions. [Figure 3] A diagram showing an example of the problem with ID detection processing when vehicles are present in multiple lanes. [Figure 4] FIG. 1 is a block diagram showing an example of the hardware configuration of a lane determination system according to an embodiment of the present invention. [Figure 5] FIG. 1 is a diagram showing an example of an outline of the operation of a lane determination device according to an embodiment of the present invention; [Figure 6] A diagram showing an example of the mask area setting screen [Figure 7] (a) A diagram showing an example of modifying a mask area in which a self-intersecting rectangle is specified. (b) A diagram showing an example of modifying a mask area in which a concave rectangle is specified. [Figure 8] (a) A diagram showing an example in which the center point of a specific detection frame is the inside / outside determination point, (b) A diagram showing an example in which the upper left point of a specific detection frame is the inside / outside determination point, (c) A diagram showing an example in which the upper right point of a specific detection frame is the inside / outside determination point, (d) A diagram showing an example in which the lower left point of a specific detection frame is the inside / outside determination point, and (e) A diagram showing an example in which the lower right point of a specific detection frame is the inside / outside determination point. [Figure 9] (a) A diagram showing an example of setting the inside and outside judgment points for the tractor part when a vehicle is traveling on the left front side of the camera's field of view. (b) A diagram showing an example of setting the inside and outside judgment points for the trailer part when a vehicle is traveling on the left front side of the camera's field of view. [Figure 10] (a) A diagram showing an example of setting the inside / outside determination points for the tractor part when a vehicle is traveling on the right front side of the camera's field of view. (b) A diagram showing an example of setting the inside / outside determination points for the trailer part when a vehicle is traveling on the right front side of the camera's field of view. [Figure 11] A diagram showing an example of setting inside and outside decision points when vehicles pass each other in multiple lanes. [Figure 12] (a) An example of setting the mask area of ​​the trailer part for each vehicle. (b) An example of setting the mask area of ​​the tractor part for each vehicle. [Figure 13] 1 is a flowchart showing an example of an operation procedure of the lane determination device according to the present embodiment in chronological order. DETAILED DESCRIPTION OF THE INVENTION

[0012] 1. Background to this disclosure First, the process leading to the configuration of the lane determination device according to the present disclosure will be described with reference to Figs. 1 to 3. Fig. 1 is a diagram showing an example of the exterior of a yard YRD, a large facility where vehicles enter and exit. Fig. 2(a) is a diagram showing an example of a problem with ID detection processing when vehicles in multiple lanes traveling in the same direction exist. Fig. 2(b) is a diagram showing an example of a problem with ID detection processing when vehicles in multiple lanes traveling in opposite directions exist. Fig. 3 is a diagram showing an example of a problem with ID detection processing when vehicles in multiple lanes exist.

[0013] The following explanation assumes that when a road, such as a public road or private road, has a center line and multiple lanes, each vehicle traveling on each lane (i.e., traffic lane) can enter and exit the yard YRD. The vehicle TR0 (see FIG. 1 ) is, for example, a transportation vehicle, and is configured with a trailer TRA0 capable of carrying one or more containers for carrying cargo to be transported, and a tractor TRC0 capable of towing the trailer TRA0. In other words, the vehicle is configured with a tractor and a trailer. In the following, the trailer TRA0 will be described as a combination of a container that stores cargo but does not have a mechanism for movement, such as wheels, and a dolly on which the container is loaded. However, the trailer TRA0 may also be configured as a vehicle in which the cargo storage section and the mechanism for movement are integrated. When a tractor tows a combination of a container and a dolly, only the dolly may be referred to as the trailer. However, for the sake of convenience, in this specification, the entire object towed by the tractor, i.e., both the combination of the cart and the container, or the vehicle equipped with an area for storing cargo and a mechanism for moving it, will be referred to as a trailer.

[0014] Here, we will briefly explain the yard YRD.

[0015] As shown in Figure 1, the yard YRD is a large facility where one or more vehicles enter and exit loaded with cargo to be transported, and has space for parking one or more trailers. For example, the yard YRD may include a vehicle entrance GIN to the yard YRD, a vehicle exit GOUT from the yard YRD, a dock DCK for collecting or shipping cargo, one or more trailer parking spaces PK1, and a parking space PK2 where vehicles temporarily stop to collect or ship cargo.

[0016] In actual operation, the following requirements exist: Specifically, in order to accurately transfer cargo from dock DCK to trailers and track trailers within yard YRD, it is necessary to accurately manage trailers entering and exiting yard YRD using camera images taken near the entrances and exits of yard YRD (i.e., vehicle entrance GIN and vehicle exit GOUT).

[0017] To meet this requirement, the identification information used to identify a trailer or tractor may be, for example, the trailer or tractor ID or the ID of a container loaded in the trailer, and may be information consisting of only letters, only numbers, or a combination of letters and numbers. In other words, the trailer or tractor identification information can be said to be information indicating how many trailers or tractors have entered or exited the yard YRD. In actual operation, it is expected that after recognizing the trailer or tractor area within the camera image, the trailer or tractor identification information will be searched for within that area, or the number of trailers or tractors will be counted. In the following description, the identification information will sometimes be referred to as "ID."

[0018] With this assumption in mind, the issues that arise when reading the identification information of a trailer or tractor will be explained with reference to Figures 2(a), 2(b), and 3. An example will be shown in which multiple vehicles TRz1, TRz2 attempting to enter the yard YRD are traveling in multiple lanes (i.e., lanes LN1, LN2). The same components will be assigned the same reference numerals, and explanations will be simplified or omitted, with only differences being explained.

[0019] In FIG. 2(a), vehicle TRz1 is traveling in lane LN1 so as to approach camera 30, and vehicle TRz2 is traveling in lane LN2 so as to similarly approach camera 30. Vehicle TRz1 is equipped with a trailer TRza1 and a tractor TRzc1. Vehicle TRz2 is equipped with a trailer TRza2 and a tractor TRzc2. When vehicles TRz1 and TRz2 traveling in the same direction are present within the viewing angle AG1 (i.e., the angle of view) of camera 30, the read result of the identification information of the vehicle in the lane that is originally intended to be detected (e.g., the ID of the trailer TRza1 or tractor TRzc1 of vehicle TRz1 in lane LN1) will be mixed with the read result of the identification information of the vehicle in a lane different from that lane (e.g., the ID of the trailer TRza2 or tractor TRzc2 of vehicle TRz2 in lane LN2). This creates a problem in that it is unclear which identification information to use.

[0020] In FIG. 2(b), vehicle TRz1 travels in lane LN1 approaching camera 30, and vehicle TRz travels in lane LN2 moving away from camera 30. Even in this case, if vehicles TRz1 and TRz2 traveling in the same direction are present within the viewing angle AG1 (i.e., the angle of view) of camera 30 as in FIG. 2(a), the read result of the identification information of the vehicle in the lane that is originally intended to be detected (e.g., the ID of the trailer TRza1 or tractor TRzc1 of vehicle TRz1 in lane LN1) will be mixed with the read result of the identification information of the vehicle in a lane different from that lane (e.g., the ID of the trailer TRza2 or tractor TRzc2 of vehicle TRz2 in lane LN2). This creates a problem in that it is unclear which identification information to use.

[0021] Here, the problem with the scene in FIG. 2(a) will be described more specifically with reference to FIG.

[0022] Camera image PIC1 shows vehicles TR1 and TR2 traveling in the same direction in multiple lanes, each with a center line WHL. Camera image PIC2 is an image captured by a camera a certain time after camera image PIC1. In camera image PIC2, ​​vehicles TR1 and TR2 coexist within the field of view (angle of view). Trailer identification information ID1 is depicted on the front of the trailer within a specific detection frame WK1 for detecting the trailer of vehicle TR1. Similarly, trailer identification information ID2 is depicted on the front of the trailer within a specific detection frame WK2 for detecting the trailer of vehicle TR2. In other words, when vehicles TR1 and TR2 travel in multiple adjacent lanes as shown in Figure 3, the trailer identification information ID1 and ID2 of both vehicles TR1 and TR2 are detected in camera image PIC2, ​​posing a problem of not knowing which identification information to use. When vehicles TR1 and TR2 are traveling in adjacent lanes, there are many cases in which it is not necessary to read the identification information of either of the vehicles TR1 and TR2. For example, if the purpose is to read the identification information of vehicles entering and leaving the yard YRD and it is known that the lanes on which vehicles entering and leaving the yard YRD travel are limited to the front lanes, it is not necessary to read the identification information of vehicles traveling in the back lanes. For this reason, there is a need to appropriately manage the presence of target vehicles.

[0023] Therefore, in the following embodiments, examples of a lane determination device, lane determination method, and program that suppress a decrease in the detection accuracy of identification information of a vehicle to be detected when each vehicle in multiple lanes is captured within the camera's field of view will be described.

[0024] Hereinafter, with appropriate reference to the drawings, embodiments specifically disclosing a lane determination device, a lane determination method, and a program according to the present disclosure will be described in detail. However, more detailed description than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter of the claims.

[0025] 2. Configuration of lane determination device First, with reference to FIG. 4, an example configuration of a lane determination system 100 according to the present disclosure will be described. FIG. 4 is a block diagram illustrating an example hardware configuration of the lane determination system 100 according to the present embodiment. The lane determination system 100 includes at least a lane determination device 10, a display device 20, and a camera 30. The lane determination device 10 is connected to another external device (not shown) via a network NW1 so as to enable data communication with the other external device. The network NW1 may be a wired network or a wireless network. The wired network may be, for example, an optical communication network, a wired wide area network (WAN), or a wired local area network (LAN), and is not limited to those exemplified here. The wireless network may be, for example, a wide area network (WAN), a local area network (LAN), mobile communications such as Long Term Evolution (LTE), 4G, or 5G, power line communications, short-range wireless communications (e.g., Bluetooth (registered trademark) communications), or communications for mobile phones, and is not limited to those exemplified here.

[0026] The lane determination device 10 includes at least a processor 11, a communication I / F 14, and a memory 15. The lane determination device 10 is, for example, a computer such as a personal computer (PC). Note that, in this embodiment, an example in which the lane determination device 10 is configured as a single computer will be described, but the functions of the lane determination device 10 may be realized by a plurality of computers working together, or may be realized as a cloud computer.

[0027] The processor 11 is configured using at least one of, for example, a central processing unit (CPU), a digital signal processor (DSP), a graphical processing unit (GPU), and a field programmable gate array (FPGA). The processor 11 functions as a controller that manages the overall operation of the lane determination device 10. The processor 11 performs control processing for overseeing the operation of each part of the lane determination device 10, data input / output processing between each part of the lane determination device 10, data calculation processing, and data storage processing. The processor 11 operates according to various programs stored in the memory 15. The processor 11 uses the memory 15 during operation and temporarily stores data generated or acquired by the processor 11 in the memory 15. The processor 11 realizes the respective functions of the vehicle detection unit 12 and the lane determination unit 13 by using the programs and data stored in the memory 15.

[0028] The vehicle detection unit 12 references the tractor detection engine 16 stored in the memory 15, inputs the camera image captured by the camera 30, and detects the vehicle and tractor portions in the camera image. The vehicle detection unit 12 references the trailer detection engine 17 stored in the memory 15, inputs the camera image captured by the camera 30, and detects the vehicle and trailer portions in the camera image. The detection results of the vehicle detection unit 12 are referenced by the lane determination unit 13. The tractor detection engine 16 and the trailer detection engine 17 output the range of the tractor or trailer portion that was successfully detected in the camera image, and the specific detection frame is set based on this output. The specific detection frame may be a range that matches the range output by each engine, or may be another range set based on the range output by each engine. In the latter case, for example, the specific detection frame may be a range output by each engine plus a predetermined margin. The tractor detection engine 16 and the trailer detection engine 17 are configured, for example, using machine learning, and detect the range of the tractor or trailer that corresponds to the learned portion. Therefore, if the learned range does not match the range desired to be used as the specific detection frame, adjustment may be required when setting the specific detection frame. As will be described later, in this embodiment, the entire tractor portion and the front or rear of the trailer portion are used as the specific detection frame. Therefore, if the detection ranges of the tractor detection engine 16 and the trailer detection engine 17 match these ranges, the detection range of each engine can be used as the specific detection frame. Furthermore, if the detection ranges of the tractor detection engine 16 and the trailer detection engine 17 do not match these ranges, the specific detection frame can be set based on the detection range of each engine. For example, if the trailer detection engine 17 is configured to detect the entire trailer, only a portion of the detection range of the trailer detection engine 17 corresponding to the front or rear is set as the specific detection frame of the trailer.

[0029] In addition, in this embodiment, the trailer detection engine 17 is configured to detect a range that does not include tires so that it can also detect trailers that have been separated from their tires. Because this detection result does not include tire information, it is difficult to set a range that includes tires as a specific detection frame. Therefore, in the following description, it is assumed that tires are not included in the specific detection frame for a trailer.

[0030] The lane determination unit 13 refers to various setting values ​​(described later) stored in the memory 15 and detects and extracts the ID of the target vehicle (i.e., the vehicle whose ID is to be read) traveling in the lane in the camera image based on the detection results of the vehicle detection unit 12. The various setting values ​​specifically include coordinate information of each vertex of the mask area in the camera image and the type of each inside / outside determination point for the vehicle's tractor and trailer for each lane. If the mask area in the camera image does not contain an inside / outside determination point corresponding to the specific detection frame indicated by the detection results of the vehicle detection unit 12 (i.e., a detection frame indicating the tractor or trailer portion of the target vehicle traveling in the lane in the camera image), the lane determination unit 13 reads the ID of the vehicle within the specific detection frame. The lane determination unit 13 stores the ID read result in the memory 15, displays (outputs) it on the display device 20, or transmits it to another external device via the network NW1. The processing content of the lane determination unit 13 and the various setting values ​​described above will be described in detail below.

[0031] The communication I / F 14 is an interface circuit for the lane determination device 10 to communicate with each of the display device 20 and the camera 30 via a wired or wireless connection. Here, I / F stands for interface. The communication I / F 14 may communicate with each of the display device 20 and the camera 30 via a network. The communication method used by the communication I / F 14 is, for example, mobile communication such as WAN, LAN, LTE, 4G, or 5G, power line communication, short-range wireless communication (e.g., Bluetooth (registered trademark) communication), or communication for mobile phones.

[0032] The memory 15 is configured using, for example, random access memory (RAM) and read-only memory (ROM) and temporarily stores programs required for operation of the lane judgment device 10, as well as data generated during operation. The RAM is, for example, a work memory used during operation of the lane judgment device 10. The ROM, for example, stores and holds programs for controlling the lane judgment device 10 in advance. The memory 15 also stores programs and data for the tractor detection engine 16 and the trailer detection engine 17. The tractor detection engine 16 and the trailer detection engine 17 are each referenced and executed by the processor 11 as appropriate. The memory 15 also stores information regarding the imaging direction of the camera 30. The memory 15 also stores coordinate information for each vertex of a mask area in a camera image captured by the camera 30, and the types of inside / outside determination points for the tractor and trailer of a vehicle for each lane. Details of the mask area and the inside / outside determination points will be described later.

[0033] The tractor detection engine 16 is configured with an artificial intelligence (AI) model that has been trained in advance by machine learning so that it can detect and output the vehicle portion and the tractor portion within the vehicle portion in the camera image. The tractor detection engine 16 inputs the camera image captured by the camera 30, detects and outputs the vehicle portion and the tractor portion within the vehicle portion in the camera image.

[0034] The trailer detection engine 17 is configured with an artificial intelligence (AI) model that has been trained in advance by machine learning so that it can detect and output the vehicle portion and the trailer portion within that vehicle portion in the camera image. The trailer detection engine 17 inputs the camera image captured by the camera 30, and detects and outputs the vehicle portion and the trailer portion within that vehicle portion in the camera image.

[0035] The display device 20 is configured using, for example, a Liquid Crystal Display (LCD) or an organic EL display. The display device 20 displays, for example, the processing results of the lane determination device 10 (for example, the results of reading the ID of the tractor or trailer of the vehicle to be read), or a setting screen for a mask area (see FIG. 6) described below. Note that, although the display device 20 is provided separately from the lane determination device 10 in FIG. 4, it may be disposed within the lane determination device 10.

[0036] The cameras 30 are devices arranged at various locations in the yard YRD and capture images of subjects within the viewing angle (angle of view) of the cameras 30. The cameras 30 are installed, for example, near the entrance GIN of the yard YRD, near the exit GOUT of the yard YRD, or near the vehicle parking area of ​​the dock DCK. The cameras 30 capture images of vehicles traveling in each of multiple lanes on a road near the entrance GIN of the yard YRD. Similarly, the cameras 30 capture images of vehicles traveling in each of multiple lanes on a road near the exit GOUT of the yard YRD or near the vehicle parking area of ​​the dock DCK. The cameras 30 are configured to include at least a lens (not shown) and an image sensor (not shown) as optical elements. The lens receives light reflected by an object within the viewing angle (angle of view) of the area captured by the camera 30 and forms an optical image of the object on the light-receiving surface of the image sensor. The image sensor is a solid-state imaging element such as a Charged Coupled Device (CCD) or a Complementary Metal Oxide Semiconductor (CMOS). The image sensor converts an optical image formed on a light receiving surface via a lens into an electrical signal at predetermined intervals (e.g., 1 / 30 seconds). For example, if the predetermined interval is 1 / 30 seconds, the frame rate of the camera 30 is 30 fps. The camera 30 also generates a camera image, which is image data, by performing predetermined signal processing on the electrical signal at the predetermined intervals. The camera 30 outputs the camera image data to the lane determination device 10. Note that, although the example in FIG. 4 shows one camera 30, the lane determination system 100 may be configured to include multiple cameras 30 arranged at different locations.

[0037] 3. Mask area settings Next, the mask area will be described with reference to Fig. 5 to Fig. 7. Fig. 5 is a diagram showing an example of an outline of the operation of the lane determination device according to this embodiment. Fig. 6 is a diagram showing an example of a setting screen for the mask area. Fig. 7(a) is a diagram showing an example of correction of a mask area in which a self-intersecting quadrangle is specified. Fig. 7(b) is a diagram showing an example of correction of a mask area in which a concave quadrangle is specified.

[0038] The mask area is an area in the camera image captured by the camera 30 for excluding vehicles traveling in one of the lanes (see FIG. 2) from targets for ID reading when the vehicles travel in each of the lanes. The mask area may be set manually (see FIG. 6) by a user on a setting screen WD1 (see FIG. 6) that displays the camera image, or may be automatically set after being corrected by the processor 11 of the lane determination device 10 based on the positions of multiple vertices specified by a user.

[0039] As shown in FIG. 5, the camera image PIC3 shows vehicles traveling in each of a plurality of adjacent lanes, each separated by a center line WHL. For example, if it is necessary to exclude the ID of a vehicle in the left lane of the camera image PIC3 from being read, a mask area MSK1 is set in the left region of the camera image PIC3. This allows the lane determination device 10 to determine that, as the ID reading target, it is sufficient to read the identification information (see identification information ID2 in FIG. 5) within the specific detection frame WK2 of the detected vehicle in the right lane, rather than the vehicle in the left lane. In other words, the lane determination device 10 can determine that it is not necessary to read the identification information of the vehicle in the left lane. However, if even a portion of the tractor or trailer of a detected vehicle in the right lane overlaps the mask area MSK1, the tractor or trailer is excluded from the vehicle identification information reading target, and the identification information of the vehicle in the right lane cannot be detected.

[0040] Therefore, the lane determination device 10 according to this embodiment narrows down the overlap with the mask area MSK1 to the entire tractor portion of the vehicle, or to only the front or rear of the trailer portion. Specifically, the processor 11 reads the identification information ID2 within the specific detection frame WK3 because the inside / outside determination point KPT1 (see below) of the specific detection frame WK3 corresponding to the front of the trailer portion detected with reference to the trailer detection engine 17 is outside the mask area in the camera image. The processor 11 does not read the identification information within the specific detection frame WK3 because the inside / outside determination point KPT2 (see below) of the specific detection frame WK4 corresponding to the front of the trailer portion detected with reference to the trailer detection engine 17 is inside the mask area in the camera image. The reason for using the entire tractor portion or the front or rear of the trailer portion as the target for determining overlap with the mask area MSK1 will be described in detail below.

[0041] As shown in FIG. 6, when setting a mask area, the processor 11 of the lane determination device 10 displays a setting screen WD1 on the display device 20, which displays a camera image captured by the camera 30. The setting screen WD1 in FIG. 6 shows an example in which a camera image PIC4 and a mask area MSK2 set for the camera image PIC4 are displayed. That is, an example is shown in which, of multiple lanes captured within the viewing angle (angle of view) of the camera 30, the IDs of vehicles in the lane on the right side as seen from the camera 30 are excluded from the target for reading. The mask area MSK2 has a polygonal shape, for example, a quadrilateral or greater, but may also be triangular. When the processor 11 of the lane determination device 10 detects four-point position designation by a user operation, it sets a polygonal mask area with the positions as vertices, and stores position information (coordinate information in the camera image) of each vertex of the set mask area in the memory 15. Note that multiple types of mask areas may be set. However, if the shape of this set mask area corresponds to a specific shape shown in FIG. 7 (e.g., a shape that is not a convex rectangle), the processor 11 modifies and sets the mask area to that specific shape (see FIG. 7). A shape that is not a convex rectangle is, for example, a self-intersecting rectangle (see FIG. 7(a)) or a concave rectangle (see FIG. 7(b)), but is not limited to these. The reason why self-intersecting rectangles and concave rectangles are treated as specific shapes is as follows. As shown in FIG. 7(a), a self-intersecting rectangle has a configuration in which its width increases on either side of a region where its width becomes zero. However, because a mask area is set as an area including a lane through which a vehicle can pass, and whose width cannot become zero, it is unlikely that such a shape will occur. Therefore, if the shape of the mask area is a self-intersecting rectangle, it is treated as if there was an error in the order in which the vertices were specified. Furthermore, as shown in FIG. 7(b), a concave rectangle has a configuration in which its width increases on either side of a region where its width becomes narrower. However, if the lane is linear, it is unlikely that the mask area including the lane will have such a shape. Therefore, if the shape of the mask area is a concave rectangle, the vertices will be treated as incorrect. However, if the lane is curved, specifying the vertices along the lane may result in the mask area becoming a concave rectangle.Therefore, a concave quadrangle may not be treated as a special shape, or the user may be asked whether or not to treat it as a special shape.

[0042] Although a rectangle has been used as an example above, other shapes, such as polygons with pentagons or more sides, may also be treated as special shapes as long as they contain recesses. For the same reason as above, if a recess is included, there is a high possibility that the vertices of the mask area have been specified incorrectly.

[0043] 7(a) in the order of [1], [2], [3], and [4], the processor 11 of the lane determination device 10 modifies and sets the mask area of ​​the self-intersecting quadrilateral FG1 obtained by connecting the lines in that order to a quadrilateral mask area MSK3 having vertices at the positions of [1], [2], [3], and [4]. By making the area of ​​the mask area larger than the shape obtained by the initial connection, the lane determination device 10 can more easily exclude vehicles traveling in each of the multiple lanes from being read targets.

[0044] 7(b) , the processor 11 of the lane determination device 10 modifies and sets the mask area of ​​the concave rectangle FG2 obtained by connecting the lines in that order to a triangular mask area MSK4 with vertices at the positions of [1], [3], and [4]. By making the area of ​​the mask area larger than the shape obtained by the initial connection, the lane determination device 10 can more easily exclude vehicles traveling in each of the multiple lanes from being read.

[0045] 4. Setting the inside / outside decision point Next, the inside / outside determination points of the specific detection frame for the tractor portion or trailer portion will be described with reference to FIGS. 8(a) to 8(e), 9(a) to 9(b), and 10(a) to 10(b). Figure 8(a) is a diagram showing an example in which the center points of specific detection frames WK5 and WK6 are inside / outside judgment points KPT3 and KPT4, Figure 8(b) is a diagram showing an example in which the upper left points of specific detection frames WK5 and WK6 are inside / outside judgment points KPT5 and KPT6, Figure 8(c) is a diagram showing an example in which the upper right points of specific detection frames WK5 and WK6 are inside / outside judgment points KPT7 and KPT8, Figure 8(d) is a diagram showing an example in which the lower left points of specific detection frames WK5 and WK6 are inside / outside judgment points KPT9 and KPT10, and Figure 8(e) is a diagram showing an example in which the lower right points of specific detection frames WK5 and WK6 are inside / outside judgment points KPT11 and KPT12. Fig. 9(a) is a diagram showing an example of setting the inside / outside determination points for the tractor portion when a vehicle is traveling on the left front side of the field of view of camera 30, and Fig. 9(b) is a diagram showing an example of setting the inside / outside determination points for the trailer portion when a vehicle is traveling on the left front side of the field of view of camera 30. Fig. 10(a) is a diagram showing an example of setting the inside / outside determination points for the tractor portion when a vehicle is traveling on the right front side of the field of view of camera 30, and Fig. 10(b) is a diagram showing an example of setting the inside / outside determination points for the trailer portion when a vehicle is traveling on the right front side of the field of view of camera 30.

[0046] The inside / outside determination point is a position within a specific detection frame that indicates the tractor portion or trailer portion of a vehicle in a camera image detected by the processor 11 and is used to determine whether the specific detection frame can be used to read an ID (in other words, whether the specific detection frame is inside or outside a mask area). In the example of FIGS. 8(a) to 8(e), the inside / outside determination point is manually designated and set by a user operation, and position information of the inside / outside determination point (i.e., coordinate information in the camera image) is stored in memory 15. In the example of FIGS. 9(a) to 9(b), the inside / outside determination point is set by the processor 11 of the lane determination device 10 according to a predetermined rule (see below).

[0047] In the example of Figure 8(a), mask areas MSK3 and MSK4 are set in camera image PIC5, but either mask area is set to be used when reading an ID using actual camera image PIC5 as input. The center point within specific detection frame WK5 of the vehicle's trailer portion on the mask area MSK3 side is set as inside / outside judgment point KPT3. Also, the center point within specific detection frame WK6 of the vehicle's trailer portion on the mask area MSK4 side is set as inside / outside judgment point KPT4. In other words, when the center point of specific detection frame WK5 or specific detection frame WK6 is located outside the corresponding mask area, processor 11 of lane determination device 10 reads the ID within that specific detection frame WK5 or specific detection frame WK6.

[0048] In the example of Figure 8(b), mask areas MSK3 and MSK4 are set in camera image PIC5, and either mask area is set to be used when reading an ID using actual camera image PIC5 as input. The upper left point within specific detection frame WK5 of the trailer portion of the vehicle on the mask area MSK3 side is set as inside / outside determination point KPT5. Furthermore, the upper left point within specific detection frame WK6 of the trailer portion of the vehicle on the mask area MSK4 side is set as inside / outside determination point KPT6. In other words, when the upper left point of specific detection frame WK5 or specific detection frame WK6 is located outside the corresponding mask area, processor 11 of lane determination device 10 reads the ID within that specific detection frame WK5 or specific detection frame WK6.

[0049] In the example of Figure 8(c), mask areas MSK3 and MSK4 are set in camera image PIC5, and either mask area is set to be used when reading an ID using actual camera image PIC5 as input. The upper right point within specific detection frame WK5 of the trailer portion of the vehicle on the mask area MSK3 side is set as inside / outside determination point KPT7. Furthermore, the upper right point within specific detection frame WK6 of the trailer portion of the vehicle on the mask area MSK4 side is set as inside / outside determination point KPT8. In other words, when the upper right point of specific detection frame WK5 or specific detection frame WK6 is located outside the corresponding mask area, processor 11 of lane determination device 10 reads the ID within that specific detection frame WK5 or specific detection frame WK6.

[0050] In the example of Figure 8(d), mask areas MSK3 and MSK4 are set in camera image PIC5, and either mask area is set to be used when reading an ID using actual camera image PIC5 as input. The lower left point within specific detection frame WK5 of the trailer portion of the vehicle on the mask area MSK3 side is set as inside / outside judgment point KPT9. Furthermore, the lower left point within specific detection frame WK6 of the trailer portion of the vehicle on the mask area MSK4 side is set as inside / outside judgment point KPT10. In other words, when the lower left point of specific detection frame WK5 or specific detection frame WK6 is located outside the corresponding mask area, processor 11 of lane determination device 10 reads the ID within that specific detection frame WK5 or specific detection frame WK6.

[0051] In the example of Figure 8(e), mask areas MSK3 and MSK4 are set in camera image PIC5, and either mask area is set to be used when reading an ID using actual camera image PIC5 as input. The lower right point within specific detection frame WK5 of the vehicle's trailer portion on the mask area MSK3 side is set as inside / outside judgment point KPT11. Furthermore, the lower right point within specific detection frame WK6 of the vehicle's trailer portion on the mask area MSK4 side is set as inside / outside judgment point KPT12. In other words, when the lower right point of specific detection frame WK5 or specific detection frame WK6 is located outside the corresponding mask area, processor 11 of lane determination device 10 reads the ID within that specific detection frame WK5 or specific detection frame WK6.

[0052] As shown in Figures 9(a) to 9(b) and Figures 10(a) to 10(b), for camera images PIC6, PIC7, PIC8, and PIC9 showing vehicles traveling in each of multiple adjacent lanes with a center line WHL laid down, the processor 11 of the lane determination device 10 automatically sets inside and outside determination points within a specific detection frame in accordance with both of the predetermined rules (1) and (2).

[0053] Here, the predetermined rule will be explained. For ease of explanation, the specific detection frame has a rectangular shape. (1) Vertical rule: Set points on the bottom edge of the specific detection frame, but do not set points on the top edge of the specific detection frame. This is because the bottom edge of the specific detection frame is closest to the road, and using points on the bottom edge is likely to accurately determine the positional relationship between the points and the lane. (2) Horizontal rule: A point on the left side or the right side of the specific detection frame is set based on whether the ID reading target is a tractor or trailer vehicle and information about the imaging direction of the camera 30.

[0054] For example, for a tractor, a point on either the left or right side corresponding to a position farthest from the camera 30 is set as the inside / outside determination point. Whether the left or right side corresponds to a position farthest from the camera 30 can be determined based on the imaging direction of the camera 30. For example, when the camera 30 is imaging the vehicle from the left as shown in FIG. 9(a), the left side corresponds to a position farther from the camera 30 than the right side. Also, when the camera 30 is imaging the vehicle from the right as shown in FIG. 10(a), the right side corresponds to a position farther from the camera 30 than the left side. In this embodiment, since the specific detection frame for the tractor includes the entire tractor, a side corresponding to a position close to the camera 30 is likely to include the depth of the trailer. In particular, when selecting a point on the bottom side according to the vertical direction rule, a point corresponding to a position close to the camera 30 is likely to be outside the lane in which the trailer is actually traveling. For example, for the tractor TRC1 in FIG. 9(a), the right side of the specific detection frame WK7 is the side farthest from the camera 30. The lower part of this right side is outside the lane that trailer TRC1 is actually traveling in. Also, for tractor TRC2 in Fig. 10(a), the left side of specific detection frame WK12 is the side farthest from camera 30. Although this left side is included in the lane that the trailer is actually traveling in, it can be seen that if the position of tractor TRC2 is slightly shifted, it will deviate from the lane that TRC2 is traveling in.

[0055] Furthermore, for trailers, points on the left or right side that correspond to positions closest to camera 30 are set as inside / outside determination points. Here, sides that correspond to positions close to camera 30 can be determined based on the imaging direction of camera 30, in the same way as determining sides that correspond to positions far from camera 30. Because the specific detection frame for trailers does not include tires, it is easy to make an error in determining whether an inside / outside determination point is included in the mask area due to errors or camera distortion. Therefore, for trailers, the effects of errors and distortion are reduced by using points that correspond to positions as close as possible to the lane and camera as inside / outside determination points.

[0056] 9(a), a mask area MSK1 is set in the camera image PIC6 and an inside / outside determination point KPT14 is automatically set in order to read the ID of the tractor TRC2 of vehicle TR2 without reading the ID of the tractor TRC1 of vehicle TR1. That is, when the ID of vehicle TR2 is to be read, the processor 11 automatically sets the lower left point, which is on the lower side and on the upper left side of the specific detection frame WK8 of the tractor TRC2 of vehicle TR2 detected by referring to the tractor detection engine 16, as the inside / outside determination point KPT14. This is because, while satisfying the above-mentioned rule (1), in rule (2), the camera 30 images the vehicles TR1 and TR2 from the right side (in other words, the vehicles TR1 and TR2 are traveling toward the left front side of the field of view of the camera 30 so as to approach the camera 30), and a tractor is the detection target. Even if the mask area is set to cover vehicle TR2, the processor 11 similarly references the tractor detection engine 16 and automatically sets the lower left point, which is located on the lower side and on the left side of the specific detection frame WK7 of the tractor TRC1 of the detected vehicle TR1, as the inside / outside determination point KPT13.

[0057] 9(b), a mask area MSK1 is set in the camera image PIC7 and an inside / outside determination point KPT16 is automatically set in order to read the ID of the trailer TRA2 of vehicle TR2 without reading the ID of the trailer TRA1 of vehicle TR1. That is, when the ID of vehicle TR2 is to be read, the processor 11 automatically sets the inside / outside determination point KPT16 to a lower right point that is on the lower side and on the right side of the specific detection frame WK10 in front of the trailer TRA2 of vehicle TR2 detected with reference to the trailer detection engine 17. This is because, while satisfying the above-mentioned rule (1), in rule (2), the camera 30 images the vehicles TR1 and TR2 from the right side (in other words, the vehicles TR1 and TR2 are traveling toward the left front side of the field of view of the camera 30 so as to approach the camera 30), and a trailer is the detection target. Even if the mask area is set to cover vehicle TR2, the processor 11 similarly references the trailer detection engine 17 and automatically sets the lower left point, which is located on the lower side and on the right side of the specific detection frame WK9 of the trailer TRA1 of vehicle TR1 detected, as the inside / outside determination point KPT15.

[0058] 10(a), in order to read the ID of the tractor TRC1 of vehicle TR1 without reading the ID of the tractor TRC2 of vehicle TR2, a mask area MSK2 is set in the camera image PIC8, and the automatic setting of the inside / outside determination point KPT17 is also shown. In other words, when the ID of vehicle TR1 is to be read, the processor 11 automatically sets the lower right point, which is on the lower side and on the right side of the specific detection frame WK11 of the tractor TRC1 of vehicle TR1 detected with reference to the tractor detection engine 16, as the inside / outside determination point KPT17. This is because, while satisfying the above-mentioned rule (1), in rule (2), the camera 30 images the vehicles TR1 and TR2 from the left side (in other words, the vehicles TR1 and TR2 are traveling toward the front right side of the field of view of the camera 30 so as to approach the camera 30), and a tractor is the detection target. Even if the mask area is set to cover vehicle TR1, the processor 11 similarly references the tractor detection engine 16 and automatically sets the lower right point, which is located on the lower side and on the right side of the specific detection frame WK12 of the tractor TRC2 of vehicle TR2 detected, as the inside / outside determination point KPT18.

[0059] 10(b), in order to read the ID of the trailer TRA1 of vehicle TR1 without reading the ID of the trailer TRA2 of vehicle TR2, a mask area MSK2 is set in the camera image PIC9, and the automatic setting of the inside / outside determination point KPT19 is also shown. In other words, when the ID of vehicle TR1 is to be read, the processor 11 automatically sets, as the inside / outside determination point KPT19, the lower left point that is on the lower side and on the left side of the specific detection frame WK13 in front of the trailer TRA1 of vehicle TR1 detected with reference to the trailer detection engine 17. This is because, while satisfying the above-mentioned rule (1), in rule (2), the camera 30 images the vehicles TR1 and TR2 from the left side (in other words, the vehicles TR1 and TR2 are traveling toward the left front side of the field of view of the camera 30 so as to approach the camera 30), and the trailer is the detection target. Even if the mask area is set to cover vehicle TR1, the processor 11 similarly references the trailer detection engine 17 and automatically sets the lower left point, which is located on the lower side and on the left side of the specific detection frame WK14 of the trailer TRA2 of vehicle TR2 detected, as the inside / outside determination point KPT20.

[0060] 11 is a diagram showing an example of setting the inside / outside determination points when vehicles TR1 and TR2 pass each other in multiple lanes. Specifically, camera 30 captures images of vehicles TR1 and TR2 from the right side, with vehicle TR1 traveling on the far right side of the field of view of camera 30 and vehicle TR2 traveling on the near left side of the field of view of camera 30.

[0061] 11, in order to read the ID of the trailer TRA3 of vehicle TR1 but not the ID of the tractor TRC4 of vehicle TR2 or the ID of the trailer TRA4, a mask area MSK3 is set in the camera image PIC10, and the inside / outside determination point KPT22 or the inside / outside determination point KPT23 is automatically set. That is, when the ID of the tractor TRC4 of vehicle TR2 is to be read, the processor 11 automatically sets the lower left point, which is on the lower side and on the upper left side of the specific detection frame WK16 of the tractor TRC4 of vehicle TR2 detected with reference to the tractor detection engine 16, as the inside / outside determination point KPT22. This is because, while satisfying the above-mentioned rule (1), in rule (2), the camera 30 images the vehicles TR1 and TR2 from the right side (in other words, the vehicle TR2 is traveling toward the left front side of the field of view of the camera 30 so as to approach the camera 30), and a tractor is the detection target.

[0062] Alternatively, when the ID of trailer TRA4 of vehicle TR2 is to be read, processor 11 automatically sets the lower right point, which is on the lower side and on the upper right side of specific detection frame WK17 of trailer TRA4 of vehicle TR2 detected with reference to trailer detection engine 17, as the inside / outside determination point KPT23. This is because, while satisfying rule (1) above, rule (2) states that camera 30 images vehicles TR1 and TR2 from the right side (in other words, vehicle TR2 is traveling toward camera 30 toward the front left side of the field of view of camera 30), and the trailer is the detection target. Note that even if the mask area is set to cover vehicle TR2, processor 11 similarly automatically sets the lower right point, which is on the lower side and on the upper right side of specific detection frame WK15 of the rear of trailer TRA1 of vehicle TR1 detected with reference to trailer detection engine 17, as the inside / outside determination point KPT21.

[0063] 12(a) is a diagram showing an example of setting an individual mask area for the trailer portion for each vehicle, and FIG. 12(b) is a diagram showing an example of setting an individual mask area for the tractor portion for each vehicle. The processor 11 of the lane determination device 10 can also set an individual mask area for each vehicle traveling in each of a plurality of adjacent lanes.

[0064] 12(a), the processor 11 sets a mask area MSK4a corresponding to the trailer TRA5 of the vehicle in the left lane in the camera image PIC12, and in conjunction with this setting, automatically sets the inside / outside judgment point KPT23 in accordance with the predetermined rule described above. Similarly, the processor 11 sets a mask area MSK4b corresponding to the trailer TRA6 of the vehicle in the right lane in the camera image PIC12, and in conjunction with this setting, automatically sets the inside / outside judgment point KPT24 in accordance with the predetermined rule described above.

[0065] 12(b), processor 11 sets a mask area MSK5a corresponding to tractor TRC5 of a vehicle in the left lane in camera image PIC12, and in conjunction with this setting, automatically sets inside / outside judgment point KPT25 in accordance with the predetermined rule described above. Similarly, processor 11 sets a mask area MSK5b corresponding to tractor TRC6 of a vehicle in the right lane in camera image PIC12, and in conjunction with this setting, automatically sets inside / outside judgment point KPT26 in accordance with the predetermined rule described above.

[0066] 5. Lane Judgment Device Operation Procedure Next, the operation procedure of the lane determination device 10 according to this embodiment will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an example of the operation procedure of the lane determination device 10 according to this embodiment in chronological order. The series of processes shown in Fig. 13 is mainly executed by the processor 11 of the lane determination device 10. Furthermore, as a premise for the explanation of Fig. 13, information indicating a mask area to be set in a camera image captured by the camera 30 and information indicating whether the target for ID reading is a tractor or trailer of a vehicle are each stored in the memory 15.

[0067] 13, the processor 11 reads and acquires the coordinate information of each vertex of the mask area and the type of each inside / outside determination point of the tractor or trailer (for example, the bottom left point, the bottom right point of the specific detection frame, etc.) from the memory 15 (St1). The camera 30 captures images of each vehicle traveling in each of a plurality of adjacent lanes, and sends the camera image data obtained by capturing the images to the lane determination device 10. The processor 11 acquires the camera image data captured by the camera 30 (St2).

[0068] Processor 11 detects the tractor or trailer portion of each vehicle in the camera image based on the camera image acquired in step St2, and extracts position information of the tractor or trailer portion in the camera image (St3). The processing of processor 11 proceeds to either step St4a or step St4b depending on whether the ID reading target is the tractor or trailer of the vehicle.

[0069] (When the ID to be read is a tractor vehicle) Processor 11 determines whether the inside / outside determination point of the specific detection frame indicating the tractor portion extracted in step St3 is within the mask area acquired in step St1 (St4a). If processor 11 determines that the inside / outside determination point of the specific detection frame indicating the tractor portion is not within the mask area (St4a, NO), it detects and reads the ID within the tractor portion (St5a). Note that the ID can be read using a known technique using, for example, well-known image processing, and the details of the process will not be mentioned here. Processor 11 outputs the ID detected in step St5a (St6a). On the other hand, if processor 11 determines that the inside / outside determination point of the specific detection frame indicating the tractor portion is outside the mask area (St4a, YES), it decides not to use the ID within the tractor portion (St7a).

[0070] (When the ID to be read is the trailer of a vehicle) Processor 11 determines whether the inside / outside determination point of the specific detection frame indicating the trailer portion extracted in step St3 is within the mask area acquired in step St1 (St4b). If processor 11 determines that the inside / outside determination point of the specific detection frame indicating the trailer portion is not within the mask area (St4b, NO), processor 11 detects and reads the ID within the trailer portion (St5b). Processor 11 outputs the ID detected in step St5b (St6b). On the other hand, if processor 11 determines that the inside / outside determination point of the specific detection frame indicating the trailer portion is outside the mask area (St4b, YES), processor 11 decides not to adopt the ID within the trailer portion (St7b).

[0071] After the processes of steps St6a, St7a, St6b, and St7b, processor 11 repeatedly executes the processes of step St2 and subsequent steps every time a camera image is acquired in step St2.

[0072] In the above-described embodiment, the trailer identification information (ID) may be a trailer ID that identifies the trailer itself, or a container ID that identifies a container loaded in the trailer. The container ID is an ID that identifies a container and is determined by a standard. The trailer ID is an ID set by the trailer manager, and currently, no unified standard has been established.

[0073] If the purpose is to detect the trailer ID, it is possible to detect only the trailer without detecting the tractor itself. Note that when counting trailers that enter or exit the yard YRD, the tractor detection results may also be used. For example, if a new tractor is detected while a preceding trailer is being detected, it may be treated as if the trailer of the next vehicle was detected.

[0074] The trailer detection engine and the tractor detection engine may be merely examples. That is, by referring to the trailer detection engine 17, a bounding box including not only the front or back of the trailer but also the entire trailer, including its sides (in other words, its depth), may be detected. However, because trailers are generally long, there is a high possibility that the bounding box will fall outside the lane. As a result, when a bounding box is used as the specific detection frame, the inside / outside determination point may not accurately reflect a lane other than the lane in which the trailer is located. Note that when a bounding box including the entire trailer is used as the specific detection frame, the possibility of an incorrect inside / outside determination can be reduced by using a point corresponding to a position far from the camera as the inside / outside determination point. However, in the case of a trailer, a position far from the camera may be hidden by the tractor, and the edge corresponding to the position far from the camera may itself be erroneously detected. For this reason, in this example, only the surface closest to the camera 30 (front or back) is targeted for detection. Furthermore, while the trailer detection engine 17 has targeted the portion of the trailer that does not include the tires as its detection target, it may also target the portion that includes the tires. By detecting a bounding box within the range that includes the tires and using this bounding box as the specific detection frame for the trailer, the bottom edge of the specific detection frame corresponds to the position of the tires that are in contact with the lane. Therefore, setting the inside / outside determination point to the bottom edge of the specific detection frame allows for a more accurate evaluation of the relationship between the trailer and the lane. In other words, if the specific detection frame is configured to only cover the range corresponding to the front or rear of the trailer, the point corresponding to the bottom edge of the specific detection frame can be used as the inside / outside determination point regardless of whether the specific detection frame includes the tires.

[0075] Also, only a part of the tractor excluding the tires may be targeted for detection by referring to the tractor detection engine 16. However, since the entire area of ​​the tractor is likely to be included in the viewing angle (in other words, the imaging range) of the camera 30, in this embodiment, the accuracy of tractor detection is improved by using the tractor detection engine 16 that detects the entire tractor.

[0076] Furthermore, in the above-described embodiment, a configuration has been described in which a mask area, which is a non-detection area, is set, but a configuration may also be adopted in which a non-mask area (detection area) is set, which is an area for making the opposite judgment (i.e., an area that is to be detected if it contains an inside / outside judgment point).

[0077] Furthermore, in the above-described embodiment, the area of ​​the detected tractor or trailer is used to detect the ID written on the tractor or trailer, but it may also be used for other purposes. For example, it may be used to count the number of tractors or trailers, or as a clue to determine which of multiple vehicles constitutes a single vehicle. In other words, the mask area only needs to be used to determine whether or not to select the detected tractor or trailer as a target for subsequent processing, and the specific content of the subsequent processing is not important.

[0078] Furthermore, in the above-described embodiment, an example in which the number of lanes is two has been shown, but the number of lanes may be three or more.

[0079] In the above-described embodiment, the inside / outside determination points in Figures 8(a) to 8(e) are manually designated and set by the user, and the inside / outside determination points in Figures 9(a) to 9(b) are set according to predetermined rules. However, the inside / outside determination points in Figures 8(a) to 8(e) may be set according to predetermined rules, or the inside / outside determination points in Figures 9(a) to 9(b) may be manually designated and set by the user. Furthermore, points different from those shown in Figures 8(a) to 8(e) or 9(a) to 9(b) may be set as inside / outside determination points manually or according to predetermined rules.

[0080] Furthermore, the vertical and horizontal rules used in the above-described embodiment to set the inside / outside determination points in FIGS. 9(a) and 9(b) are merely examples. Depending on the camera 30's angle of view, the lane width, the presence or absence of obstructions, and other factors, changing the rules may result in more appropriate inside / outside determination points being set, or deleting one of the rules may not affect the accuracy of the inside / outside determination. For example, if the lane is sufficiently wide or the camera 30 captures the vehicle from a more frontal angle, the points on the upper edge of the specific detection frame are less likely to be located outside the lane in which the vehicle is actually traveling. Therefore, in such cases, the vertical rules may be deleted to eliminate restrictions on the vertical positions of the inside / outside determination points.

[0081] Summary of the Disclosure The above description of the embodiments discloses technical concepts corresponding to the following items.

[0082] (Item 1) A processor (11); a memory (15) for storing camera images captured by a camera (30) arranged so as to be able to capture images of a plurality of vehicles traveling in each of a plurality of adjacent lanes; The processor, in cooperation with the memory, Acquire information about a mask area for identifying a portion of the vehicle included in the camera image that is to be excluded from a predetermined processing target; A specific detection frame having a rectangular shape of the vehicle (for example, a tractor portion or a trailer portion) is set in the camera image; a part of the vehicle having an inside / outside determination point of the specific detection frame that is not included in the mask area is output as a target for the predetermined processing; Lane determination device. As a result, the lane determination device can suppress a decrease in detection accuracy of a vehicle to be detected (e.g., a vehicle that is the subject of a specified processing) when each vehicle traveling in each of multiple adjacent lanes is captured within the camera's field of view.

[0083] (Item 2) the specific detection frame is a front portion or a rear portion of a trailer towed by a tractor of the vehicle; Item 1. The lane determination device according to item 1. As a result, the lane determination device can prevent the inside / outside determination point based on the specific detection frame set with a focus on the trailer from being set outside the lane the vehicle is traveling in, for example, when the trailer is long, thereby preventing a decrease in the detection accuracy of the target vehicle.

[0084] (Item 3) the predetermined process is reading an ID written in the trailer, The lane determination device further reads and outputs the ID from the specific detection frame. Item 2. The lane determination device. This allows the lane determination device to read and output the ID, which is identification information printed on the trailer.

[0085] (Item 4) the processor sets the horizontal position of the inside / outside determination point of the specific detection frame to a point on a left side or a right side of the specific detection frame based on the imaging direction of the camera of the plurality of lanes. Item 2 or 3. The lane determination device according to item 2 or 3. As a result, the lane determination device can appropriately select the trailer or tractor of the vehicle to be subjected to the specified processing and output it without erroneous determination, even if there are vehicles traveling in each of multiple adjacent lanes captured in the camera image.

[0086] (Item 5) the processor sets the horizontal position of one point on the left side or the right side of the specific detection frame that corresponds to a position closest to the camera as the horizontal position of the inside / outside determination point. Item 4. The lane determination device according to item 4. As a result, the lane determination device can set the horizontal position of the specific detection frame that is closer to the camera to the horizontal position of the inside / outside determination point, thereby making it possible to appropriately select the trailer or tractor of the vehicle that is the subject of the specified processing.

[0087] (Item 6) the processor sets the vertical position of the inside / outside determination point of the specific detection frame to a point on the bottom side of the specific detection frame. Item 2. The lane determination device. As a result, the lane determination device can set the vertical position of the specific detection frame that is closer to the camera to the vertical position of the inside / outside determination point, thereby making it possible to appropriately select the trailer or tractor of the vehicle that is the subject of the specified processing.

[0088] (Item 7) the specific detection frame is a part of a tractor that pulls a trailer of the vehicle, Item 1. The lane determination device according to item 1. As a result, the lane determination device can set the inside / outside determination points based on the position of the tractor. Here, since a tractor is shorter than a trailer, the specific detection frame set with the tractor in mind is less likely to overlap with other lanes. Therefore, this configuration can prevent the inside / outside determination points from being set outside the lane in which the vehicle is traveling, and can prevent a decrease in the detection accuracy of the target vehicle.

[0089] (Item 8) The processor detects the specific detection frame using an AI model (e.g., a tractor detection engine 16 or a trailer detection engine 17) that is stored in the memory and that is capable of detecting the specific detection frame. The lane determination device according to any one of items 1 to 7. As a result, the lane determination device can properly detect the trailer or tractor portion of a vehicle even when there are vehicles traveling in each of multiple adjacent lanes captured in the camera image.

[0090] (Item 9) the processor sets the mask area based on detection of a user operation for specifying each vertex of the mask area having a polygonal shape; The lane determination device according to any one of items 1 to 8. As a result, the lane determination device allows the user to easily set the mask area by specifying each vertex through an operation.

[0091] (Item 10) when the mask area has a shape including a recess due to the designation order of the vertices, the processor modifies and sets the shape of the mask area. 10. The lane determination device according to any one of items 1 to 9. As a result, even if an inappropriate mask area is designated by a user operation, the lane determination device can correct the mask area to an appropriate one.

[0092] (Item 11) the processor sets the inside / outside determination point to one of the center point, the upper left point, the lower left point, the upper right point, and the lower right point of the specific detection frame based on detection of a user operation. The lane determination device according to any one of items 1 to 10. As a result, the lane determination device allows the inside / outside determination points to be easily selected and set by a user operation.

[0093] (Item 12) A lane determination method executed by a lane determination device, comprising: storing camera images captured by cameras arranged so as to be able to capture images of a plurality of vehicles traveling in each of a plurality of adjacent lanes; acquiring information on a mask area for identifying a portion of the vehicle included in the camera image that is to be excluded from a predetermined processing target; Setting a specific detection frame for a vehicle captured in the camera image; and outputting a portion of the vehicle having an inside / outside determination point of the specific detection frame that is not included in the mask area as a target for the predetermined processing. Lane determination method. As a result, the lane determination method can suppress a decrease in detection accuracy of a vehicle to be detected (e.g., a vehicle that is the subject of a specified processing) when each vehicle traveling in each of multiple adjacent lanes is captured within the camera's field of view.

[0094] (Item 13) The lane determination device is a computer. storing camera images captured by cameras arranged so as to be able to capture images of a plurality of vehicles traveling in each of a plurality of adjacent lanes; acquiring information on a mask area for identifying a portion of the vehicle included in the camera image that is to be excluded from a predetermined processing target; Setting a specific detection frame for a vehicle captured in the camera image; outputting a portion of the vehicle having an inside / outside determination point of the specific detection frame that is not included in the mask area as a target of the predetermined processing; program. As a result, according to the program, the lane determination device can suppress a decrease in the detection accuracy of vehicles to be detected (e.g., vehicles that are the subject of specified processing) when each vehicle traveling in each of multiple adjacent lanes is captured within the camera's field of view.

[0095] Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also fall within the technical scope of the present disclosure. Furthermore, the components in the above-described embodiments may be combined in any manner without departing from the spirit of the invention. [Industrial Applicability]

[0096] The technology disclosed herein is useful as a lane determination device, lane determination method, and program that suppresses a decrease in detection accuracy of a target vehicle when vehicles in multiple lanes are captured within the camera's field of view. [Explanation of symbols]

[0097] 10 Lane Judgment Device 11 processors 12 Vehicle detection unit 13 Lane Judgment Unit 14 Communication I / F 15 memory 16 Tractor detection engine 17 Trailer Detector Engine 20 Display Devices 30 Camera 100 Lane Judgment System NW1 Network

Claims

1. a processor; a memory that stores camera images captured by a camera that is arranged so as to be able to capture images of a plurality of vehicles traveling in each of a plurality of adjacent lanes; The processor, in cooperation with the memory, Acquire information about a mask area for identifying a portion of the vehicle included in the camera image that is to be excluded from a predetermined processing target; A specific detection frame is set for a vehicle captured in the camera image; a part of the vehicle having an inside / outside determination point of the specific detection frame that is not included in the mask area is output as a target for the predetermined processing; Lane determination device.

2. the specific detection frame is a front portion or a rear portion of a trailer towed by a tractor of the vehicle; The lane determination device according to claim 1 .

3. the predetermined process is reading an ID written on the trailer, The lane determination device further reads and outputs the ID from the specific detection frame. The lane determination device according to claim 2 .

4. the processor sets the horizontal position of the inside / outside determination point of the specific detection frame to a point on a left side or a right side of the specific detection frame based on the imaging direction of the camera of the plurality of lanes. The lane determination device according to claim 2 .

5. the processor sets the horizontal position of one point on the left side or the right side of the specific detection frame that corresponds to a position closest to the camera as the horizontal position of the inside / outside determination point. The lane determination device according to claim 4 .

6. the processor sets the vertical position of the inside / outside determination point of the specific detection frame to a point on the bottom side of the specific detection frame. The lane determination device according to claim 2 .

7. the specific detection frame is a part of a tractor that pulls a trailer of the vehicle, The lane determination device according to claim 1 .

8. the processor detects the specific detection window using an AI model capable of detecting the specific detection window, which is stored in the memory; The lane determination device according to claim 1 .

9. the processor sets the mask area based on detection of a user operation for specifying each vertex of the mask area having a polygonal shape; The lane determination device according to claim 1 .

10. when the mask area has a shape including a recess due to the designation order of the vertices, the processor modifies and sets the shape of the mask area. The lane determination device according to claim 9.

11. the processor sets the inside / outside determination point to one of the center point, the upper left point, the lower left point, the upper right point, and the lower right point of the specific detection frame based on detection of a user operation. The lane determination device according to claim 1 .

12. A lane determination method executed by a lane determination device, comprising: storing camera images captured by cameras arranged so as to be able to capture images of a plurality of vehicles traveling in each of a plurality of adjacent lanes; acquiring information about a mask area for identifying a portion of the vehicle included in the camera image that is to be excluded from a processing target of a predetermined process; Setting a specific detection frame for a vehicle captured in the camera image; and outputting a portion of the vehicle having an inside / outside determination point of the specific detection frame that is not included in the mask area as a target for the predetermined processing. Lane determination method.

13. The lane determination device is a computer. storing camera images captured by cameras arranged so as to be able to capture images of a plurality of vehicles traveling in each of a plurality of adjacent lanes; acquiring information about a mask area for identifying a portion of the vehicle included in the camera image that is to be excluded from a predetermined processing target; Setting a specific detection frame for a vehicle captured in the camera image; outputting a portion of the vehicle having an inside / outside determination point of the specific detection frame that is not included in the mask area as a target of the predetermined processing; program.

Images