Lane determination device, lane determination method, and program

Abstract

This lane determination device comprises a processor and a memory for storing a camera image captured by a camera which is disposed so as to be able to image a plurality of vehicles that travel in a plurality of respective adjacent lanes. The processor, in collaboration with the memory, acquires mask area information for identifying, from among portions which constitute vehicles that are included in the camera image, a portion to be excluded from the target of prescribed processing, sets a specific detection frame for the vehicles that are shown in the camera image, and outputs, as the target of the prescribed processing, a portion of the vehicle that is not included in the mask area and that has an inside / outside determination point of the specific detection frame.

Description

Lane determination device, lane determination method and program

[0001] The present disclosure relates to a lane determination device, a lane determination method, and a program.

[0002] Patent Literature 1 discloses a license plate recognition device that detects multiple quadrangles of license plate area candidates from an input image and performs character recognition on character areas included in the license plate area candidates. The license plate recognition device selects a license plate area candidate to output from the multiple detected license plate area candidates based on the character recognition results and quadrangle information for each license plate area candidate, and outputs information about the selected license plate area candidate.

[0003] Japanese Patent Application Laid-Open No. 2008-217347

[0004] In Patent Document 1, an input image to be processed in the license plate detection process shows multiple vehicles, each with its own license plate. However, the camera's angle of view, which is the subject of processing on a public or private road with multiple lanes, does not anticipate the case in which multiple vehicles are shown together. In other words, when detecting some kind of identification information (e.g., license plates or vehicle-specific IDs) of a target vehicle, if vehicles are shown in multiple lanes, the detection results (i.e., vehicle identification numbers) of lanes other than the lane originally intended to be detected may be mixed together, or the lane may overlap with parts of other vehicles, making it impossible to detect, resulting in a problem of reduced detection accuracy for the identification numbers of target vehicles in the lane intended to be detected.

[0005] The present disclosure has been devised in consideration of the above-described conventional situation, and aims to suppress a decrease in the detection accuracy of a target vehicle when vehicles in multiple lanes are captured within the camera's field of view.

[0006] The present disclosure provides a lane determination device comprising a processor and a memory that stores camera images captured by a camera positioned so as to be able to capture images of multiple vehicles traveling in each of multiple adjacent lanes, wherein the processor cooperates with the memory to obtain information on a mask area for identifying parts of the vehicle included in the camera image that should be excluded from the target of specified processing, sets a specific detection frame for the vehicle shown in the camera image, and outputs parts of the vehicle that have inside and outside determination points of the specific detection frame that are not included in the mask area as the target of the specified processing.

[0007] The present disclosure also provides a lane determination method executed by a lane determination device, the lane determination method comprising: storing camera images captured by a camera positioned so as to capture images of multiple vehicles traveling in each of multiple adjacent lanes; acquiring information about a mask area for identifying parts of the vehicle included in the camera image that are to be excluded from the target of predetermined processing; setting a specific detection frame for the vehicle shown in the camera image; and outputting parts of the vehicle that have inside and outside determination points of the specific detection frame that are not included in the mask area as the target of the predetermined processing.

[0008] The present disclosure also provides a program for causing a lane determination device, which is a computer, to perform the following operations: storing camera images captured by a camera positioned so as to capture images of multiple vehicles traveling in each of multiple adjacent lanes; acquiring information about a mask area for identifying parts of the vehicle included in the camera image that are to be excluded from the target of specified processing; setting a specific detection frame for the vehicle shown in the camera image; and outputting parts of the vehicle that have inside and outside determination points of the specific detection frame that are not included in the mask area as the target of the specified processing.

[0009] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0010] According to the present disclosure, it is possible to suppress a decrease in the detection accuracy of a target vehicle when vehicles in multiple lanes are captured within the camera's field of view.

[0011] FIG. 1 shows an example of the appearance of a yard, which is a large facility where vehicles enter and exit. FIG. 2 shows an example of a problem with ID detection processing when there are vehicles in multiple lanes traveling in the same direction. FIG. 3 shows an example of a problem with ID detection processing when there are vehicles in multiple lanes traveling in opposite directions. FIG. 4 shows an example of a problem with ID detection processing when there are vehicles in multiple lanes. FIG. 5 shows an example of a block diagram of a hardware configuration of a lane determination system according to this embodiment. FIG. 6 shows an example of an outline of the operation of a lane determination device according to this embodiment. FIG. 7 shows an example of a mask area setting screen. FIG. 8 shows an example of a correction to a mask area where a self-intersecting rectangle is specified. FIG. 9 shows an example of a correction to a mask area where a concave rectangle is specified. FIG. 10 shows an example where the center point of a specific detection frame becomes the inside / outside determination point. FIG. 11 shows an example where the upper left point of a specific detection frame becomes the inside / outside determination point. FIG. 12 shows an example where the upper right point of a specific detection frame becomes the inside / outside determination point. FIG. 1 shows an example where the bottom left point of the exit frame is the inside / outside judgment point. FIG. 2 shows an example where the bottom right point of the specific detection frame is the inside / outside judgment point. FIG. 3 shows an example where the bottom right point of the specific detection frame is the inside / outside judgment point. FIG. 4 shows an example where the inside / outside judgment point of the tractor part is set when a vehicle is traveling on the left front side of the camera's field of view. FIG. 5 shows an example where the inside / outside judgment point of the trailer part is set when a vehicle is traveling on the left front side of the camera's field of view. FIG. 6 shows an example where the inside / outside judgment point of the tractor part is set when a vehicle is traveling on the right front side of the camera's field of view. FIG. 7 shows an example where the inside / outside judgment point of the trailer part is set when a vehicle is traveling on the right front side of the camera's field of view. FIG. 8 shows an example where the inside / outside judgment point of the trailer part is set when a vehicle is traveling on the right front side of the camera's field of view.

[0012] 1. Background to the Present Disclosure First, the background to the configuration of the lane determination device according to the present disclosure will be described with reference to Figures 1 to 3. Figure 1 is a diagram showing an example of the exterior of a yard YRD, a large facility where vehicles enter and exit. Figure 2A is a diagram showing an example of a problem with ID detection processing when vehicles in multiple lanes traveling in the same direction exist. Figure 2B is a diagram showing an example of a problem with ID detection processing when vehicles in multiple lanes traveling in opposite directions exist. Figure 3 is a diagram showing an example of a problem with ID detection processing when vehicles in multiple lanes exist.

[0013] The following explanation assumes that when a road, such as a public road or private road, has a center line and multiple lanes, each vehicle traveling on each lane (i.e., traffic lane) can enter and exit the yard YRD. The vehicle TR0 (see FIG. 1 ) is, for example, a transportation vehicle, and is configured with a trailer TRA0 having a towed vehicle function capable of carrying one or more containers for carrying cargo to be transported, and a tractor TRC0 having a towing vehicle function for towing the trailer TRA0. In other words, the vehicle is configured with a tractor and a trailer. In the following, the trailer TRA0 will be described as a combination of a container that stores cargo but does not have a mechanism for movement, such as wheels, and a dolly on which the container is loaded. However, the trailer TRA0 may also be configured as a vehicle in which the cargo storage section and the mechanism for movement are integrated. When a tractor tows a combination of a container and a dolly, only the dolly may be referred to as the trailer. However, for the sake of convenience, in this specification, the entire object towed by the tractor, i.e., both the combination of the cart and the container, or the vehicle equipped with an area for storing cargo and a mechanism for moving it, will be referred to as a trailer.

[0014] Here, the yard YRD will be briefly explained.

[0015] As shown in Figure 1, the yard YRD is a large facility where one or more vehicles enter and exit loaded with cargo to be transported, and has space for parking one or more trailers. For example, the yard YRD includes a vehicle entrance GIN to the yard YRD, a vehicle exit GOUT from the yard YRD, a dock DCK for collecting or shipping cargo, one or more trailer parking spaces PK1, and a parking space PK2 where vehicles temporarily stop for collecting or shipping cargo.

[0016] In actual operation, the following requirements exist: Specifically, in order to accurately transfer cargo from the dock DCK to the trailer, track trailers within the yard YRD, and so on, it is necessary to accurately manage trailers entering and exiting the yard YRD based on camera images taken near the entrances and exits of the yard YRD (i.e., the vehicle entrance GIN and vehicle exit GOUT).

[0017] To meet this requirement, the identification information used to identify a trailer or tractor may be, for example, the ID of the trailer or tractor or the ID of a container loaded in the trailer, and may be information consisting of only letters, only numbers, or a combination of letters and numbers. In other words, the identification information of a trailer or tractor can be said to be information indicating how many trailers or tractors have entered or exited the yard YRD. In addition, in actual operation, it is expected that after recognizing the area of ​​a trailer or tractor in a camera image, the identification information of the trailer or tractor will be searched for within that area, or the number of trailers or tractors will be counted. In the following description, the identification information may be referred to as ID.

[0018] With this assumption in mind, the issues involved in reading the identification information of a trailer or tractor will be described with reference to Figures 2A, 2B, and 3. An example will be shown in which multiple vehicles TRz1, TRz2 attempting to enter the yard YRD are present and traveling in multiple lanes (i.e., lanes LN1, LN2). The same components will be assigned the same reference numerals, and explanations will be simplified or omitted, with differences being described.

[0019] In FIG. 2A , vehicle TRz1 is traveling in lane LN1 approaching camera 30, and vehicle TRz2 is traveling in lane LN2 approaching camera 30. Vehicle TRz1 includes a trailer TRza1 and a tractor TRzc1. Vehicle TRz2 includes a trailer TRza2 and a tractor TRzc2. When vehicles TRz1 and TRz2 traveling in the same direction are present within the viewing angle AG1 (i.e., the angle of view) of camera 30, the read result of the identification information of the vehicle in the lane that is intended to be detected (e.g., the ID of the trailer TRza1 or tractor TRzc1 of vehicle TRz1 in lane LN1) will be mixed with the read result of the identification information of the vehicle in a lane different from the lane that is intended to be detected (e.g., the ID of the trailer TRza2 or tractor TRzc2 of vehicle TRz2 in lane LN2). This creates a problem in that it is unclear which identification information to use.

[0020] 2B , vehicle TRz1 travels in lane LN1 approaching camera 30, and vehicle TRz travels in lane LN2 moving away from camera 30. Even in this case, if vehicles TRz1 and TRz2 traveling in the same direction are present within the viewing angle AG1 (i.e., the angle of view) of camera 30, as in FIG. 2A , the read result of the identification information of the vehicle in the lane that is originally intended to be detected (e.g., the ID of the trailer TRza1 or tractor TRzc1 of vehicle TRz1 in lane LN1) will be mixed with the read result of the identification information of the vehicle in a lane different from that lane (e.g., the ID of the trailer TRza2 or tractor TRzc2 of vehicle TRz2 in lane LN2). This creates a problem in that it is unclear which identification information to use.

[0021] Now, with reference to FIG. 3, the problem will be described more specifically for the scene in FIG. 2A.

[0022] Camera image PIC1 shows vehicles TR1 and TR2 traveling in the same direction in multiple lanes, each marked with a center line WHL. Camera image PIC2 is an image captured by a camera a certain time after camera image PIC1. In camera image PIC2, ​​vehicles TR1 and TR2 coexist within the same field of view (angle of view). Trailer identification information ID1 is depicted on the front of the trailer within a specific detection frame WK1 for detecting the trailer of vehicle TR1. Similarly, trailer identification information ID2 is depicted on the front of the trailer within a specific detection frame WK2 for detecting the trailer of vehicle TR2. In other words, when vehicles TR1 and TR2 travel in multiple adjacent lanes as shown in FIG. 3 , the trailer identification information ID1 and ID2 of both vehicles TR1 and TR2 are detected in camera image PIC2, ​​creating a problem in that it is unclear which identification information to use. When vehicles TR1 and TR2 are traveling in adjacent lanes, there are many cases in which it is not necessary to read the identification information of either of the vehicles TR1 and TR2. For example, if the purpose is to read the identification information of vehicles entering and leaving the yard YRD and it is known that the lanes on which vehicles entering and leaving the yard YRD travel are limited to the front lanes, it is not necessary to read the identification information of vehicles traveling in the back lanes. For this reason, there is a reality in which it is necessary to appropriately manage the presence of target vehicles.

[0023] Therefore, in the following embodiments, examples of a lane determination device, lane determination method, and program that suppress a decrease in the detection accuracy of identification information of a vehicle to be detected when each vehicle in multiple lanes is captured within the camera's field of view are described.

[0024] Hereinafter, with appropriate reference to the drawings, embodiments specifically disclosing a lane determination device, a lane determination method, and a program according to the present disclosure will be described in detail. However, more detailed description than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter of the claims.

[0025] 2. Configuration of the Lane Determination Device First, with reference to FIG. 4, an example configuration of a lane determination system 100 according to the present disclosure will be described. FIG. 4 is a block diagram showing an example hardware configuration of the lane determination system 100 according to the present embodiment. The lane determination system 100 includes at least a lane determination device 10, a display device 20, and a camera 30. The lane determination device 10 is connected to another external device (not shown) via a network NW1 so as to enable data communication therebetween. The network NW1 may be a wired network or a wireless network. The wired network may be, for example, an optical communication network, a wired wide area network (WAN), or a wired local area network (LAN), and is not limited to those exemplified here. The wireless network may be, for example, a Wide Area Network (WAN), a Local Area Network (LAN), a Long Term Evolution (LTE), mobile communications such as 4G and 5G, power line communications, short-range wireless communications (e.g., Bluetooth (registered trademark) communications), or communications for mobile phones, and need not be limited to those exemplified here.

[0026] The lane judgment device 10 includes at least a processor 11, a communication I / F 14, and a memory 15. The lane judgment device 10 is, for example, a computer such as a personal computer (PC). Note that, in the present embodiment, an example in which the lane judgment device 10 is configured as a single computer will be described, but the functions of the lane judgment device 10 may be realized by a plurality of computers working together, or may be realized as a cloud computer.

[0027] The processor 11 is configured using at least one of, for example, a central processing unit (CPU), a digital signal processor (DSP), a graphic processing unit (GPU), and a field programmable gate array (FPGA). The processor 11 functions as a controller that manages the overall operation of the lane judgment device 10. The processor 11 performs control processing for overseeing the operation of each part of the lane judgment device 10, data input / output processing between each part of the lane judgment device 10, data arithmetic processing, and data storage processing. The processor 11 operates according to various programs stored in the memory 15. The processor 11 uses the memory 15 during operation, and temporarily stores data generated or acquired by the processor 11 in the memory 15. The processor 11 realizes the functions of the vehicle detection unit 12 and the lane determination unit 13 by using the programs and data stored in the memory 15 .

[0028] The vehicle detection unit 12 references the tractor detection engine 16 stored in the memory 15, inputs the camera image captured by the camera 30, and detects the vehicle and tractor portions in the camera image. The vehicle detection unit 12 references the trailer detection engine 17 stored in the memory 15, inputs the camera image captured by the camera 30, and detects the vehicle and trailer portions in the camera image. The detection results of the vehicle detection unit 12 are referenced by the lane determination unit 13. The tractor detection engine 16 and the trailer detection engine 17 output the range of the tractor or trailer portion that was successfully detected in the camera image, and the specific detection frame is set based on this output. The specific detection frame may be a range that matches the range output by each engine, or may be another range set based on the range output by each engine. In the latter case, for example, the specific detection frame may be a range that includes a predetermined margin added to the range output by each engine. The tractor detection engine 16 and the trailer detection engine 17 are configured, for example, using machine learning, and detect the range of the tractor or trailer that corresponds to the learned portion. Therefore, if the learned range does not match the range desired to be used as the specific detection frame, adjustment may be necessary when setting the specific detection frame. As will be described later, in this embodiment, the entire tractor portion and the front or rear of the trailer portion are used as the specific detection frame. Therefore, if the detection ranges of the tractor detection engine 16 and the trailer detection engine 17 match these ranges, the detection ranges of each engine can be used as the specific detection frame. Furthermore, if the detection ranges of the tractor detection engine 16 and the trailer detection engine 17 do not match these ranges, the specific detection frame can be set based on the detection ranges of each engine. For example, if the trailer detection engine 17 is configured to detect the entire trailer, only a portion of the detection range of the trailer detection engine 17 corresponding to the front or rear is set as the specific detection frame for the trailer.

[0029] In addition, in this embodiment, the trailer detection engine 17 is configured to detect a range that does not include tires so that it can also detect trailers that have been separated from their tires. Because this detection result does not include tire information, it is difficult to set a range that includes tires as a specific detection frame. Therefore, in the following description, it is assumed that tires are not included in the specific detection frame for a trailer.

[0030] The lane determination unit 13 references various setting values ​​(described later) stored in the memory 15 and detects and extracts the ID of the target vehicle (i.e., the vehicle whose ID is to be read) traveling in the lane in the camera image based on the detection results of the vehicle detection unit 12. The various setting values ​​specifically include coordinate information for each vertex of the mask area in the camera image and the type of each inside / outside determination point for the vehicle's tractor and trailer for each lane. If the mask area in the camera image does not contain an inside / outside determination point corresponding to the specific detection frame indicated by the detection results of the vehicle detection unit 12 (i.e., a detection frame indicating the tractor or trailer portion of the target vehicle traveling in the lane in the camera image), the lane determination unit 13 reads the ID of the vehicle within the specific detection frame. The lane determination unit 13 stores the ID read result in the memory 15, displays (outputs) it on the display device 20, or transmits it to another external device via the network NW1. The processing content of the lane determination unit 13 and the various setting values ​​described above will be described in detail below.

[0031] The communication I / F 14 is an interface circuit that allows the lane determination device 10 to communicate with each of the display device 20 and the camera 30 via a wired or wireless connection. Here, I / F stands for interface. The communication I / F 14 may communicate with each of the display device 20 and the camera 30 via a network. The communication method used by the communication I / F 14 is, for example, mobile communication such as WAN, LAN, LTE, 4G, or 5G, power line communication, short-range wireless communication (e.g., Bluetooth (registered trademark) communication), or communication for mobile phones.

[0032] The memory 15 is configured using, for example, random access memory (RAM) and read-only memory (ROM) and temporarily stores programs necessary for the operation of the lane determination device 10, as well as data generated during operation. The RAM is, for example, a work memory used during operation of the lane determination device 10. The ROM pre-stores and holds, for example, programs for controlling the lane determination device 10. The memory 15 also stores programs and data for the tractor detection engine 16 and the trailer detection engine 17. The tractor detection engine 16 and the trailer detection engine 17 are each referenced and executed by the processor 11 as appropriate. The memory 15 also stores information regarding the imaging direction of the camera 30. The memory 15 also stores coordinate information for each vertex of a mask area in the camera image captured by the camera 30, and the types of inside and outside determination points for the tractor and trailer of the vehicle for each lane. Details of the mask area and the inside and outside determination points will be described later.

[0033] The tractor detection engine 16 is configured with an artificial intelligence (AI) model that has been machine-learned in advance so that it can detect and output a vehicle portion and a tractor portion within that vehicle portion in a camera image. The tractor detection engine 16 inputs a camera image captured by the camera 30, and detects and outputs a vehicle portion and a tractor portion within that vehicle portion in the camera image.

[0034] The trailer detection engine 17 is configured with an artificial intelligence (AI) model that has been machine-learned in advance so that it can detect and output a vehicle portion and a trailer portion within that vehicle portion in a camera image. The trailer detection engine 17 inputs a camera image captured by the camera 30, and detects and outputs a vehicle portion and a trailer portion within that vehicle portion in the camera image.

[0035] The display device 20 is configured using, for example, a liquid crystal display (LCD) or an organic EL display. The display device 20 displays, for example, the processing results of the lane determination device 10 (for example, the results of reading the ID of a tractor or trailer of a vehicle to be read), or a mask area setting screen (see FIG. 6 ) described below. Note that, although the display device 20 is provided separately from the lane determination device 10 in FIG. 4 , it may also be disposed within the lane determination device 10.

[0036] The cameras 30 are devices arranged at various locations in the yard YRD and capture images of subjects within the field of view (angle of view) of the cameras 30. The cameras 30 are installed, for example, near the entrance GIN of the yard YRD, near the exit GOUT of the yard YRD, or near the vehicle parking area of ​​the dock DCK. The cameras 30 capture images of vehicles traveling in each of multiple lanes on a road near the entrance GIN of the yard YRD. Similarly, the cameras 30 capture images of vehicles traveling in each of multiple lanes on a road near the exit GOUT of the yard YRD or near the vehicle parking area of ​​the dock DCK. The cameras 30 are configured to include at least a lens (not shown) and an image sensor (not shown) as optical elements. The lens receives light reflected by an object within the field of view (angle of view) of the area captured by the camera 30 and forms an optical image of the object on the light-receiving surface of the image sensor. The image sensor is a solid-state imaging element such as a charged coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The image sensor converts an optical image formed on a light receiving surface via a lens into an electrical signal at predetermined intervals (e.g., 1 / 30 seconds). For example, if the predetermined interval is 1 / 30 seconds, the frame rate of the camera 30 is 30 fps. The camera 30 also generates a camera image, which is image data, by performing predetermined signal processing on the electrical signal at the predetermined intervals. The camera 30 outputs the camera image data to the lane determination device 10. Note that while the example in FIG. 4 shows one camera 30, the lane determination system 100 may be configured to include multiple cameras 30 arranged at different locations.

[0037] 3. Setting the Mask Area Next, the mask area will be described with reference to Figs. 5 to 7. Fig. 5 is a diagram showing an example of an outline of the operation of the lane determination device according to this embodiment. Fig. 6 is a diagram showing an example of a screen for setting the mask area. Fig. 7A is a diagram showing an example of correction of a mask area in which a self-intersecting quadrangle is specified. Fig. 7B is a diagram showing an example of correction of a mask area in which a concave quadrangle is specified.

[0038] The mask area is an area in the camera image captured by the camera 30 for excluding vehicles traveling in one of the lanes (see FIG. 2) from targets for ID reading when the vehicles travel in each of the lanes. The mask area may be set manually (see FIG. 6) by a user on the setting screen WD1 (see FIG. 6) that displays the camera image, or may be automatically set after being corrected by the processor 11 of the lane determination device 10 based on the positions of multiple vertices specified by a user.

[0039] As shown in FIG. 5 , the camera image PIC3 shows vehicles traveling in each of a plurality of adjacent lanes, each separated by a center line WHL. For example, if it is necessary to exclude the ID of a vehicle in the left lane of the camera image PIC3 from being read, a mask area MSK1 is set in the left region of the camera image PIC3. This allows the lane determination device 10 to determine that, as the ID reading target, it is sufficient to read the identification information (see identification information ID2 in FIG. 5 ) within the specific detection frame WK2 of the detected vehicle in the right lane, rather than the vehicle in the left lane. In other words, the lane determination device 10 can determine that it is not necessary to read the identification information of the vehicle in the left lane. However, if even a portion of the tractor or trailer of a detected vehicle in the right lane overlaps the mask area MSK1, the tractor or trailer is excluded from the vehicle identification information reading target, and the identification information of the vehicle in the right lane cannot be detected.

[0040] Therefore, the lane determination device 10 according to this embodiment narrows down the overlap with the mask area MSK1 to the entire tractor portion of the vehicle, or to only the front or rear of the trailer portion. Specifically, the processor 11 reads the identification information ID2 within the specific detection frame WK3 because the inside / outside determination point KPT1 (described later) of the specific detection frame WK3 corresponding to the front of the trailer portion detected with reference to the trailer detection engine 17 is outside the mask area in the camera image. The processor 11 does not read the identification information within the specific detection frame WK3 because the inside / outside determination point KPT2 (described later) of the specific detection frame WK4 corresponding to the front of the trailer portion detected with reference to the trailer detection engine 17 is inside the mask area in the camera image. The reason for using the entire tractor portion or the front or rear of the trailer portion as the target for determining overlap with the mask area MSK1 will be described in detail below.

[0041] As shown in FIG. 6 , when setting a mask area, the processor 11 of the lane determination device 10 displays a setting screen WD1 on the display device 20, which displays a camera image captured by the camera 30. The setting screen WD1 in FIG. 6 shows an example in which a camera image PIC4 and a mask area MSK2 set for the camera image PIC4 are displayed. That is, an example is shown in which, of the multiple lanes captured within the viewing angle (angle of view) of the camera 30, the IDs of vehicles in the lane on the right side as viewed from the camera 30 are excluded from the target for reading. The mask area MSK2 has a polygonal shape, e.g., a square or larger polygon, but may also be triangular. When the processor 11 of the lane determination device 10 detects four positions specified by a user operation, it sets a polygonal mask area with the positions as vertices, and stores position information (coordinate information in the camera image) of each vertex of the set mask area in the memory 15. Note that multiple types of mask areas may be set. However, if the shape of this set mask area corresponds to a specific shape shown in FIG. 7 (e.g., a shape other than a convex rectangle), the processor 11 modifies and sets the mask area to that specific shape (see FIG. 7). Shapes that are not convex rectangles include, but are not limited to, a self-intersecting rectangle (see FIG. 7A) or a concave rectangle (see FIG. 7B). The reason why self-intersecting rectangles and concave rectangles are treated as specific shapes is as follows. As shown in FIG. 7A, a self-intersecting rectangle has a structure in which its width increases on either side of a region where its width becomes zero. However, because a mask area is set as an area including a lane through which a vehicle can pass, and whose width cannot become zero, such a shape is unlikely to occur. Therefore, if the shape of the mask area is a self-intersecting rectangle, it is treated as if there was an error in the order in which the vertices were specified. Furthermore, as shown in FIG. 7B, a concave rectangle has a structure in which its width increases on either side of a region where its width becomes narrower. However, if the lane is linear, it is unlikely that the mask area including the lane will have such a shape. Therefore, if the shape of the mask area is a concave rectangle, the vertices will be treated as incorrect. However, if the lane is curved, specifying the vertices along the lane may result in the mask area becoming a concave rectangle.Therefore, a concave quadrangle may not be treated as a special shape, or the user may be asked whether or not to treat it as a special shape.

[0042] Although a rectangle has been used as an example above, other shapes, such as polygons with pentagons or more sides, may also be treated as special shapes as long as they contain recesses. For the same reason as above, if a recess is included, there is a high possibility that the vertices of the mask area have been specified incorrectly.

[0043] 7A, the processor 11 of the lane determination device 10 modifies and sets the mask area of ​​the self-intersecting quadrilateral FG1 obtained by connecting the lines in that order to a quadrilateral mask area MSK3 having vertices at the positions of [1], [2], [3], and [4]. By making the area of ​​the mask area larger than the shape obtained by the initial connection, the lane determination device 10 can more easily exclude vehicles traveling in each of the multiple lanes from being read.

[0044] 7B , the processor 11 of the lane determination device 10 modifies and sets the mask area of ​​the concave rectangle FG2 obtained by connecting the lines in that order to a triangular mask area MSK4 with vertices at the positions of [1], [3], and [4]. By making the area of ​​the mask area larger than the shape obtained by the initial connection, the lane determination device 10 can more easily exclude vehicles traveling in each of the multiple lanes from being read.

[0045] 4. Setting of Interior / Exterior Judgment Points Next, the interior / exterior judgment points of the specific detection frame for the tractor portion or trailer portion will be described with reference to Figures 8A to 8E, 9A to 9B, and 10A to 10B. Figure 8A shows an example in which the center points of the specific detection frames WK5 and WK6 are the interior / exterior judgment points KPT3 and KPT4, Figure 8B shows an example in which the upper left points of the specific detection frames WK5 and WK6 are the interior / exterior judgment points KPT5 and KPT6, Figure 8C shows an example in which the upper right points of the specific detection frames WK5 and WK6 are the interior / exterior judgment points KPT7 and KPT8, Figure 8D shows an example in which the lower left points of the specific detection frames WK5 and WK6 are the interior / exterior judgment points KPT9 and KPT10, and Figure 8E shows an example in which the lower right points of the specific detection frames WK5 and WK6 are the interior / exterior judgment points KPT11 and KPT12. Fig. 9A is a diagram showing an example of setting the inside / outside determination points for the tractor portion when a vehicle is traveling on the left front side of the field of view of camera 30, and Fig. 9B is a diagram showing an example of setting the inside / outside determination points for the trailer portion when a vehicle is traveling on the left front side of the field of view of camera 30. Fig. 10A is a diagram showing an example of setting the inside / outside determination points for the tractor portion when a vehicle is traveling on the right front side of the field of view of camera 30, and Fig. 10B is a diagram showing an example of setting the inside / outside determination points for the trailer portion when a vehicle is traveling on the right front side of the field of view of camera 30.

[0046] The inside / outside determination point is a position within a specific detection frame that indicates the tractor portion or trailer portion of a vehicle in a camera image detected by the processor 11 and is used to determine whether the specific detection frame can be used to read an ID (in other words, whether the specific detection frame is inside or outside a mask area). In the examples of FIGS. 8A to 8E, the inside / outside determination point is manually designated and set by a user operation, and the position information of the inside / outside determination point (i.e., coordinate information within the camera image) is stored in the memory 15. In the examples of FIGS. 9A to 9B, the inside / outside determination point is set by the processor 11 of the lane determination device 10 according to predetermined rules (see below).

[0047] In the example of Figure 8A, mask areas MSK3 and MSK4 are set in camera image PIC5, but either mask area is set to be used when reading an ID using actual camera image PIC5 as input. The center point within specific detection frame WK5 of the vehicle's trailer portion on the mask area MSK3 side is set as inside / outside judgment point KPT3. Furthermore, the center point within specific detection frame WK6 of the vehicle's trailer portion on the mask area MSK4 side is set as inside / outside judgment point KPT4. In other words, when the center point of specific detection frame WK5 or specific detection frame WK6 is located outside the corresponding mask area, processor 11 of lane determination device 10 reads the ID within that specific detection frame WK5 or specific detection frame WK6.

[0048] In the example of Figure 8B, mask areas MSK3 and MSK4 are set in camera image PIC5, and either mask area is set to be used when reading an ID using actual camera image PIC5 as input. The upper left point within specific detection frame WK5 of the trailer portion of the vehicle on the mask area MSK3 side is set as inside / outside determination point KPT5. Furthermore, the upper left point within specific detection frame WK6 of the trailer portion of the vehicle on the mask area MSK4 side is set as inside / outside determination point KPT6. In other words, when the upper left point of specific detection frame WK5 or specific detection frame WK6 is located outside the corresponding mask area, processor 11 of lane determination device 10 reads the ID within that specific detection frame WK5 or specific detection frame WK6.

[0049] In the example of Figure 8C, mask areas MSK3 and MSK4 are set in camera image PIC5, and either mask area is set to be used when reading an ID using actual camera image PIC5 as input. The upper right point within specific detection frame WK5 of the trailer portion of the vehicle on the mask area MSK3 side is set as inside / outside determination point KPT7. Furthermore, the upper right point within specific detection frame WK6 of the trailer portion of the vehicle on the mask area MSK4 side is set as inside / outside determination point KPT8. In other words, when the upper right point of specific detection frame WK5 or specific detection frame WK6 is located outside the corresponding mask area, processor 11 of lane determination device 10 reads the ID within that specific detection frame WK5 or specific detection frame WK6.

[0050] In the example of Figure 8D, mask areas MSK3 and MSK4 are set in camera image PIC5, and either mask area is set to be used when reading an ID using actual camera image PIC5 as input. The lower left point within specific detection frame WK5 of the trailer portion of the vehicle on the mask area MSK3 side is set as inside / outside judgment point KPT9. Furthermore, the lower left point within specific detection frame WK6 of the trailer portion of the vehicle on the mask area MSK4 side is set as inside / outside judgment point KPT10. In other words, when the lower left point of specific detection frame WK5 or specific detection frame WK6 is located outside the corresponding mask area, the processor 11 of the lane determination device 10 reads the ID within that specific detection frame WK5 or specific detection frame WK6.

[0051] In the example of Figure 8E, mask areas MSK3 and MSK4 are set in camera image PIC5, and either mask area is set to be used when reading an ID using actual camera image PIC5 as input. The lower right point within specific detection frame WK5 of the trailer portion of the vehicle on the mask area MSK3 side is set as inside / outside judgment point KPT11. Furthermore, the lower right point within specific detection frame WK6 of the trailer portion of the vehicle on the mask area MSK4 side is set as inside / outside judgment point KPT12. In other words, when the lower right point of specific detection frame WK5 or specific detection frame WK6 is located outside the corresponding mask area, the processor 11 of the lane determination device 10 reads the ID within that specific detection frame WK5 or specific detection frame WK6.

[0052] As shown in Figures 9A to 9B and Figures 10A to 10B, for camera images PIC6, PIC7, PIC8, and PIC9 showing vehicles traveling in each of multiple adjacent lanes with a center line WHL laid out, the processor 11 of the lane determination device 10 automatically sets inside and outside determination points within a specific detection frame in accordance with both of the predetermined rules (1) and (2).

[0053] Here, the predetermined rule will be explained. For ease of explanation, the specific detection frame has a rectangular shape.

[0054] (1) Vertical rule: Set a point on the bottom edge of the specific detection frame, but do not set a point on the top edge of the specific detection frame. This is because the bottom edge of the specific detection frame is closest to the road, and so using a point on the bottom edge is likely to accurately determine the positional relationship between the point and the lane.

[0055] (2) Horizontal rule: A point on the left side or the right side of the specific detection frame is set based on whether the ID reading target is a tractor or trailer vehicle and information regarding the imaging direction of the camera 30.

[0056] For example, for a tractor, a point on either the left or right side corresponding to a position farthest from the camera 30 is set as the inside / outside determination point. Whether the left or right side corresponds to a position farther from the camera 30 can be determined based on the imaging direction of the camera 30. For example, when the camera 30 is imaging the vehicle from the left as shown in FIG. 9A , the left side corresponds to a position farther from the camera 30 than the right side. Also, when the camera 30 is imaging the vehicle from the right as shown in FIG. 10A , the right side corresponds to a position farther from the camera 30 than the left side. In this embodiment, since the specific detection frame for the tractor includes the entire tractor, a side corresponding to a position close to the camera 30 is likely to include the depth of the trailer. In particular, when selecting a point on the bottom side according to the vertical direction rule, a point corresponding to a position close to the camera 30 is likely to be outside the lane in which the trailer is actually traveling. For example, for the tractor TRC1 in FIG. 9A , the right side of the specific detection frame WK7 is the side farthest from the camera 30. The lower part of this right side is outside the lane that the trailer TRC1 is actually traveling in. Also, for the tractor TRC2 in Fig. 10A, the left side of the specific detection frame WK12 is the side farthest from the camera 30. Although this left side is included in the lane that the trailer is actually traveling in, it can be seen that if the position of the tractor TRC2 is slightly shifted, the TRC2 will deviate from the lane that it is traveling in.

[0057] Furthermore, for the trailer, a point on the left side or a point on the right side that corresponds to a position closest to the camera 30 is set as the inside / outside determination point. Here, the side that corresponds to a position close to the camera 30 can be determined based on the imaging direction of the camera 30, in the same way as determining the side that corresponds to a position far from the camera 30. Because the specific detection frame for the trailer does not include the tires, it is easy to make an error in determining whether the inside / outside determination point is included in the mask area due to errors or camera distortion. Therefore, for the trailer, the effects of errors and distortion are reduced by using a point that corresponds to a position as close as possible to the lane and the camera as the inside / outside determination point.

[0058] 9A , in order to read the ID of the tractor TRC2 of vehicle TR2 without reading the ID of the tractor TRC1 of vehicle TR1, a mask area MSK1 is set in the camera image PIC6, and the inside / outside determination point KPT14 is automatically set. In other words, when the ID of vehicle TR2 is to be read, the processor 11 automatically sets the inside / outside determination point KPT14 to a lower left point that is located on the lower side and on the upper left side of the specific detection frame WK8 of the tractor TRC2 of vehicle TR2 detected by referring to the tractor detection engine 16. This is because, while satisfying the above-described rule (1), in rule (2), the camera 30 captures the images of vehicles TR1 and TR2 from the right side (in other words, the vehicles TR1 and TR2 are traveling toward the left front side of the field of view of the camera 30 so as to approach the camera 30), and a tractor is the detection target. Even if the mask area is set to cover vehicle TR2, the processor 11 similarly references the tractor detection engine 16 and automatically sets the lower left point, which is on the lower side and on the left side of the specific detection frame WK7 of the tractor TRC1 of vehicle TR1 detected, as the inside / outside determination point KPT13.

[0059] 9B , in order to read the ID of the trailer TRA2 of vehicle TR2 without reading the ID of the trailer TRA1 of vehicle TR1, a mask area MSK1 is set in the camera image PIC7, and the inside / outside determination point KPT16 is automatically set. In other words, when the ID of vehicle TR2 is to be read, the processor 11 automatically sets the inside / outside determination point KPT16 to a lower right point that is located on the lower side and on the right side of the specific detection frame WK10 on the front side of the trailer TRA2 of vehicle TR2, detected by referring to the trailer detection engine 17. This is because, while satisfying the above-described rule (1), rule (2) stipulates that the camera 30 captures images of vehicles TR1 and TR2 from the right side (in other words, the vehicles TR1 and TR2 are traveling toward the left front side of the field of view of the camera 30 so as to approach the camera 30), and the trailer is the detection target. Even if the mask area is set to cover vehicle TR2, the processor 11 similarly references the trailer detection engine 17 and automatically sets the lower left point, which is located on the lower side and on the right side of the specific detection frame WK9 of the trailer TRA1 of vehicle TR1 detected, as the inside / outside judgment point KPT15.

[0060] 10A , in order to read the ID of the tractor TRC1 of vehicle TR1 without reading the ID of the tractor TRC2 of vehicle TR2, a mask area MSK2 is set in the camera image PIC8, and the inside / outside determination point KPT17 is automatically set. That is, when the ID of vehicle TR1 is to be read, the processor 11 automatically sets the inside / outside determination point KPT17 to a lower right point that is located on the lower side and on the right side of the specific detection frame WK11 of the tractor TRC1 of vehicle TR1 detected by referring to the tractor detection engine 16. This is because, while satisfying the above-described rule (1), in rule (2), the camera 30 captures images of vehicles TR1 and TR2 from the left side (in other words, the vehicles TR1 and TR2 are traveling toward the front right side of the field of view of the camera 30 so as to approach the camera 30), and a tractor is the detection target. Even if the mask area is set to cover vehicle TR1, the processor 11 similarly references the tractor detection engine 16 and automatically sets the lower right point, which is on the lower side and on the right side of the specific detection frame WK12 of the tractor TRC2 of vehicle TR2 detected, as the inside / outside determination point KPT18.

[0061] 10B , in order to read the ID of the trailer TRA1 of vehicle TR1 without reading the ID of the trailer TRA2 of vehicle TR2, a mask area MSK2 is set in the camera image PIC9, and the inside / outside determination point KPT19 is automatically set. In other words, when the ID of vehicle TR1 is to be read, the processor 11 automatically sets the inside / outside determination point KPT19 to a lower left point that is located on the lower side and on the left side of the specific detection frame WK13 on the front side of the trailer TRA1 of vehicle TR1, detected by referring to the trailer detection engine 17. This is because, while satisfying the above-described rule (1), rule (2) indicates that the camera 30 captures images of vehicles TR1 and TR2 from the left side (in other words, the vehicles TR1 and TR2 are traveling toward the left front side of the field of view of the camera 30 so as to approach the camera 30), and the trailer is the detection target. Even if the mask area is set to cover vehicle TR1, the processor 11 similarly references the trailer detection engine 17 and automatically sets the lower left point, which is located on the lower side and on the left side of the specific detection frame WK14 of the trailer TRA2 of vehicle TR2 detected, as the inside / outside judgment point KPT20.

[0062] 11 is a diagram showing an example of setting the inside / outside determination points when vehicles TR1 and TR2 pass each other in a multi-lane traffic. Specifically, camera 30 captures images of vehicles TR1 and TR2 from the right side, with vehicle TR1 traveling on the far right side of the field of view of camera 30 and vehicle TR2 traveling on the near left side of the field of view of camera 30.

[0063] 11 shows that a mask area MSK3 is set in the camera image PIC10 and an inside / outside determination point KPT22 or an inside / outside determination point KPT23 is automatically set in order to read the ID of the tractor TRC4 of vehicle TR2 or the trailer TRA4 without reading the ID of the trailer TRA3 of vehicle TR1. That is, when the ID of the tractor TRC4 of vehicle TR2 is to be read, the processor 11 automatically sets the lower left point, which is located on the lower side and on the left side of the specific detection frame WK16 of the tractor TRC4 of vehicle TR2 detected with reference to the tractor detection engine 16, as the inside / outside determination point KPT22. This is because, while satisfying the above-described rule (1), in rule (2), the camera 30 captures images of vehicles TR1 and TR2 from the right side (in other words, vehicle TR2 is traveling toward the left front side of the field of view of camera 30 so as to approach camera 30), and a tractor is the detection target.

[0064] Alternatively, when the ID of trailer TRA4 of vehicle TR2 is to be read, processor 11 automatically sets the lower right point, which is located on the lower edge and on the right edge of specific detection frame WK17 of trailer TRA4 of vehicle TR2 detected with reference to trailer detection engine 17, as the inside / outside determination point KPT23. This is because, while satisfying rule (1) above, rule (2) states that camera 30 images vehicles TR1 and TR2 from the right side (in other words, vehicle TR2 is traveling so as to approach camera 30 toward the front left side of the field of view of camera 30), and the trailer is the detection target. Note that even if the mask area is set to cover vehicle TR2, processor 11 similarly automatically sets the lower right point, which is located on the lower edge and on the right edge of specific detection frame WK15 of the rear of trailer TRA1 of vehicle TR1 detected with reference to trailer detection engine 17, as the inside / outside determination point KPT21.

[0065] 12A is a diagram showing an example of setting an individual mask area for the trailer portion for each vehicle, and FIG. 12B is a diagram showing an example of setting an individual mask area for the tractor portion for each vehicle. The processor 11 of the lane determination device 10 can also set an individual mask area for each vehicle traveling in each of a plurality of adjacent lanes.

[0066] 12A, the processor 11 sets a mask area MSK4a corresponding to the trailer TRA5 of the vehicle in the left lane in the camera image PIC12, and in conjunction with this setting, automatically sets an inside / outside judgment point KPT23 in accordance with the predetermined rule described above. Similarly, the processor 11 sets a mask area MSK4b corresponding to the trailer TRA6 of the vehicle in the right lane in the camera image PIC12, and in conjunction with this setting, automatically sets an inside / outside judgment point KPT24 in accordance with the predetermined rule described above.

[0067] 12B, the processor 11 sets a mask area MSK5a corresponding to the tractor TRC5 of the vehicle in the left lane in the camera image PIC12, and in conjunction with this setting, automatically sets the inside / outside judgment point KPT25 in accordance with the predetermined rule described above. Similarly, the processor 11 sets a mask area MSK5b corresponding to the tractor TRC6 of the vehicle in the right lane in the camera image PIC12, and in conjunction with this setting, automatically sets the inside / outside judgment point KPT26 in accordance with the predetermined rule described above.

[0068] 5. Operation Procedure of Lane Determination Device Next, the operation procedure of the lane determination device 10 according to this embodiment will be described with reference to Fig. 13. Fig. 13 is a flowchart showing an example of the operation procedure of the lane determination device 10 according to this embodiment in chronological order. The series of processes shown in Fig. 13 is mainly executed by the processor 11 of the lane determination device 10. Furthermore, as a premise for the explanation of Fig. 13, information indicating a mask area to be set in a camera image captured by the camera 30 and information indicating whether the target vehicle whose ID is to be read is a tractor or a trailer are each stored in the memory 15.

[0069] 13 , the processor 11 reads and acquires the coordinate information of each vertex of the mask area and the type of each inside / outside determination point of the tractor or trailer (for example, the bottom left point, the bottom right point of the specific detection frame, etc.) from the memory 15 (Step 1). The camera 30 captures images of each vehicle traveling in each of a plurality of adjacent lanes and sends the camera image data obtained by the capture to the lane determination device 10. The processor 11 acquires the camera image data captured by the camera 30 (Step 2).

[0070] The processor 11 detects the tractor or trailer portion of each vehicle in the camera image based on the camera image acquired in step St2, and extracts the position information of the tractor or trailer portion in the camera image (St3). The processing of the processor 11 proceeds to either step St4a or step St4b depending on whether the ID reading target is the tractor or trailer of the vehicle.

[0071] (When the ID reading target is a tractor vehicle) Processor 11 determines whether the inside / outside determination point of the specific detection frame indicating the tractor portion extracted in step St3 is within the mask area acquired in step St1 (St4a). If processor 11 determines that the inside / outside determination point of the specific detection frame indicating the tractor portion is not within the mask area (St4a, NO), processor 11 detects and reads the ID within the tractor portion (St5a). Note that the ID reading method may be, for example, a well-known technique using image processing, and the details of the process will not be discussed here. Processor 11 outputs the ID detected in step St5a (St6a). On the other hand, if processor 11 determines that the inside / outside determination point of the specific detection frame indicating the tractor portion is outside the mask area (St4a, YES), processor 11 decides not to use the ID within the tractor portion (St7a).

[0072] (When the ID reading target is a vehicle trailer) Processor 11 determines whether the inside / outside determination point of the specific detection frame indicating the trailer portion extracted in step St3 is within the mask area acquired in step St1 (St4b). If processor 11 determines that the inside / outside determination point of the specific detection frame indicating the trailer portion is not within the mask area (St4b, NO), processor 11 detects and reads the ID within the trailer portion (St5b). Processor 11 outputs the ID detected in step St5b (St6b). On the other hand, if processor 11 determines that the inside / outside determination point of the specific detection frame indicating the trailer portion is outside the mask area (St4b, YES), processor 11 decides not to adopt the ID within the trailer portion (St7b).

[0073] After the processes of steps St6a, St7a, St6b, and St7b, the processor 11 repeatedly executes the processes of step St2 and subsequent steps every time it acquires a camera image obtained in step St2.

[0074] In the above-described embodiment, the trailer identification information (ID) may be a trailer ID that identifies the trailer itself, or a container ID that identifies a container loaded in the trailer. The container ID is an ID that identifies a container and is determined by a standard. The trailer ID is an ID set by the trailer manager, and currently, no unified standard has been established.

[0075] If the purpose is to detect the trailer ID, it is possible to detect only the trailer without detecting the tractor itself. Note that when counting trailers entering or leaving the yard YRD, the tractor detection results may also be used. For example, if a new tractor is detected while a preceding trailer is being detected, it may be treated as if the trailer of the next vehicle was detected.

[0076] The trailer detection engine and the tractor detection engine may be merely examples. In other words, by referencing the trailer detection engine 17, a bounding box including not only the front or back of the trailer but also the entire trailer, including its sides (i.e., its depth), may be detected. However, because trailers are generally long, there is a high possibility that the bounding box will fall outside the lane. As a result, when a bounding box is used as the specific detection frame, the inside / outside determination point may not accurately reflect a lane other than the lane in which the trailer is located. When a bounding box including the entire trailer is used as the specific detection frame, the possibility of an incorrect inside / outside determination can be reduced by using a point corresponding to a position far from the camera as the inside / outside determination point. However, in the case of a trailer, a position far from the camera may be hidden by the tractor, and the edge corresponding to the position far from the camera may itself be erroneously detected. For this reason, in this example, only the surface closest to the camera 30 (front or back) is targeted for detection.

[0077] Furthermore, although the trailer detection engine 17 has targeted the portion of the trailer that does not include the tires as its detection target, it may also target the portion that includes the tires. By detecting a bounding box within the range that includes the tires and using this bounding box as the specific detection frame for the trailer, the bottom edge of the specific detection frame corresponds to the position of the tires that are in contact with the lane. Therefore, setting the inside / outside determination point to the bottom edge of the specific detection frame allows for a more accurate evaluation of the relationship between the trailer and the lane. In other words, if the specific detection frame is configured to only cover the range corresponding to the front or rear of the trailer, then the point corresponding to the bottom edge of the specific detection frame can be used as the inside / outside determination point, regardless of whether the specific detection frame includes the tires.

[0078] Also, only a portion of the tractor excluding the tires may be targeted for detection by referring to the tractor detection engine 16. However, since the entire area of ​​the tractor is likely to be included in the viewing angle (in other words, the imaging range) of the camera 30, in this embodiment, the accuracy of tractor detection is improved by using the tractor detection engine 16 that detects the entire tractor.

[0079] Furthermore, in the above-described embodiment, a configuration has been described in which a mask area, which is a non-detection area, is set, but a configuration may also be adopted in which a non-mask area (detection area) is set, which is an area for making the opposite judgment (i.e., an area that is to be detected if it contains an inside / outside judgment point).

[0080] Furthermore, in the above-described embodiment, the area of ​​the detected tractor or trailer is used to detect the ID written on the tractor or trailer, but it may also be used for other purposes. For example, it may be used to count the number of tractors or trailers, or as a clue to determine which of multiple vehicles constitutes a single vehicle. In other words, the mask area only needs to be used to determine whether or not to select the detected tractor or trailer as a target for subsequent processing, and the specific content of the subsequent processing is not important.

[0081] Furthermore, in the above-described embodiment, an example in which the number of lanes is two has been shown, but the number of lanes may be three or more.

[0082] In the above-described embodiment, the inside / outside determination points in Figures 8A to 8E are manually designated and set by the user, and the inside / outside determination points in Figures 9A to 9B are set according to predetermined rules. However, the inside / outside determination points in Figures 8A to 8E may be set according to predetermined rules, or the inside / outside determination points in Figures 9A to 9B may be manually designated and set by the user. Furthermore, points different from the points shown in Figures 8A to 8E or 9A to 9B may be set as inside / outside determination points manually or according to predetermined rules.

[0083] Furthermore, the vertical and horizontal rules used in the above-described embodiment to set the inside / outside determination points in Figures 9A and 9B are merely examples. Depending on the camera 30's angle of view, the lane width, the presence or absence of obstructions, and other factors, changing the rules may result in more appropriate inside / outside determination points being set, or deleting any of the rules may not affect the accuracy of the inside / outside determination. For example, if the lane is sufficiently wide or the camera 30 captures the vehicle from a more frontal angle, the points on the top edge of the specific detection frame are less likely to be located outside the lane in which the vehicle is actually traveling. Therefore, in such cases, the vertical rules may be deleted so that no restrictions are placed on the vertical positions of the inside / outside determination points.

[0084] (Summary of the present disclosure) The above description of the embodiments discloses technical ideas corresponding to the following items.

[0085] (Item 1) A lane determination device comprising: a processor (11); and a memory (15) that stores camera images captured by a camera (30) positioned so as to be able to capture images of a plurality of vehicles traveling in each of a plurality of adjacent lanes, wherein the processor, in cooperation with the memory, acquires information on a mask area for identifying portions of the vehicle included in the camera image that are to be excluded from the target of predetermined processing, sets a specific detection frame (e.g., a tractor portion or a trailer portion) having a rectangular shape of the vehicle shown in the camera image, and outputs portions of the vehicle that have inside and outside determination points of the specific detection frame that are not included in the mask area as the target of the predetermined processing.

[0086] As a result, the lane determination device can suppress a decrease in detection accuracy of vehicles to be detected (e.g., vehicles that are the subject of specified processing) when each vehicle traveling in each of multiple adjacent lanes is captured within the camera's field of view.

[0087] (Item 2) The lane determination device according to Item 1, wherein the specific detection frame is a front portion or a rear portion of a trailer towed by a tractor of the vehicle.

[0088] As a result, the lane determination device can prevent the inside / outside determination point based on the specific detection frame set with a focus on the trailer from being set outside the lane the vehicle is traveling in, for example, when the trailer is long, thereby preventing a decrease in the detection accuracy of the target vehicle.

[0089] (Item 3) The lane determination device according to Item 2, wherein the predetermined process is reading an ID written on the trailer, and the lane determination device further reads and outputs the ID from the specific detection frame.

[0090] This allows the lane determination device to read and output the ID, which is identification information printed on the trailer.

[0091] (Item 4) The lane determination device according to Item 2 or 3, wherein the processor sets the horizontal position of the inside / outside determination point of the specific detection frame to a point on a left side or a right side of the specific detection frame based on the imaging direction of the camera of the plurality of lanes.

[0092] As a result, the lane determination device can appropriately select the trailer or tractor of the vehicle to be subjected to the specified processing and output it without erroneous determination, even if there are vehicles traveling in each of multiple adjacent lanes captured in the camera image.

[0093] (Item 5) The lane determination device according to Item 4, wherein the processor sets the horizontal position of one of the points on the left side or the right side of the specific detection frame that corresponds to a position closer to the camera as the horizontal position of the inside / outside determination point.

[0094] As a result, the lane determination device can set the horizontal position of the specific detection frame that is closer to the camera to the horizontal position of the inside / outside determination point, thereby making it possible to appropriately select the trailer or tractor of the vehicle that is the subject of the specified processing.

[0095] (Item 6) The lane determination device according to Item 2, wherein the processor sets the vertical position of the inside / outside determination point of the specific detection frame to a point on a bottom side of the specific detection frame.

[0096] As a result, the lane determination device can set the vertical position of the specific detection frame that is closer to the camera to the vertical position of the inside / outside determination point, thereby making it possible to appropriately select the trailer or tractor of the vehicle that is the subject of the specified processing.

[0097] (Item 7) The lane determination device according to Item 1, wherein the specific detection frame is a part of a tractor that tows a trailer of the vehicle.

[0098] As a result, the lane determination device can set the inside / outside determination points based on the position of the tractor. Here, since a tractor is shorter than a trailer, the specific detection frame set with the tractor in mind is less likely to overlap with other lanes. Therefore, this configuration can prevent the inside / outside determination points from being set outside the lane in which the vehicle is traveling, and can prevent a decrease in the detection accuracy of the target vehicle.

[0099] (Item 8) The lane determination device according to any one of Items 1 to 7, wherein the processor detects the specific detection frame using an AI model (e.g., tractor detection engine 16 or trailer detection engine 17) that is stored in the memory and that can detect the specific detection frame.

[0100] As a result, the lane determination device can properly detect the trailer or tractor portion of a vehicle even when there are vehicles traveling in each of multiple adjacent lanes captured in the camera image.

[0101] (Item 9) The lane determination device according to any one of items 1 to 8, wherein the processor sets the mask area based on detection of a user operation for specifying each vertex of the mask area having a polygonal shape.

[0102] As a result, the lane determination device allows the user to easily set the mask area by specifying each vertex through an operation.

[0103] (Item 10) The lane determination device according to any one of Items 1 to 9, wherein the processor modifies and sets the shape of the mask area when the mask area has a shape including a recess due to the designation order of the vertices.

[0104] As a result, even if an inappropriate mask area is designated by a user operation, the lane determination device can correct the mask area to an appropriate one.

[0105] (Item 11) The lane determination device according to any one of items 1 to 10, wherein the processor sets the inside / outside determination point to one of a center point, an upper left point, a lower left point, an upper right point, and a lower right point of the specific detection frame based on detection of a user operation.

[0106] As a result, the lane determination device allows the inside / outside determination points to be easily selected and set by a user operation.

[0107] (Item 12) A lane determination method executed by a lane determination device, comprising: storing camera images captured by a camera positioned so as to capture images of multiple vehicles traveling in each of multiple adjacent lanes; acquiring information on a mask area for identifying portions of the vehicle included in the camera image that are to be excluded from a target for predetermined processing; setting a specific detection frame for the vehicle shown in the camera image; and outputting portions of the vehicle that have inside and outside determination points of the specific detection frame that are not included in the mask area as a target for the predetermined processing.

[0108] As a result, the lane determination method can suppress a decrease in detection accuracy of a vehicle to be detected (e.g., a vehicle that is the subject of a specified processing) when each vehicle traveling in each of multiple adjacent lanes is captured within the camera's field of view.

[0109] (Item 13) A program for causing a lane determination device that is a computer to perform the following: storing camera images captured by a camera positioned so as to be able to capture images of multiple vehicles traveling in each of multiple adjacent lanes; acquiring information on a mask area for identifying parts of the vehicle included in the camera image that are to be excluded from the target of predetermined processing; setting a specific detection frame for the vehicle shown in the camera image; and outputting parts of the vehicle that have inside and outside determination points of the specific detection frame that are not included in the mask area as the target of the predetermined processing.

[0110] As a result, according to the program, the lane determination device can suppress a decrease in the detection accuracy of the vehicle to be detected (e.g., a vehicle that is the subject of a specified processing) when each vehicle traveling in each of multiple adjacent lanes is captured within the camera's field of view.

[0111] Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also fall within the technical scope of the present disclosure. Furthermore, the components in the above-described embodiments may be combined in any manner without departing from the spirit of the invention.

[0112] The disclosures of the specification, drawings and abstracts contained in the U.S. provisional application No. 63 / 575010 filed on April 5, 2024, and the Japanese patent application No. 2024-191021 filed on October 30, 2024, are incorporated herein by reference in their entirety.

[0113] The technology disclosed herein is useful as a lane determination device, lane determination method, and program that suppresses a decrease in detection accuracy of a target vehicle when vehicles in multiple lanes are captured within the camera's field of view.

[0114] REFERENCE SIGNS LIST 10 Lane determination device 11 Processor 12 Vehicle detection unit 13 Lane determination unit 14 Communication I / F 15 Memory 16 Tractor detection engine 17 Trailer detection engine 20 Display device 30 Camera 100 Lane determination system NW1 Network

Claims

1. A lane determination device comprising: a processor; and a memory for storing camera images captured by a camera positioned so as to be able to capture images of a plurality of vehicles traveling in each of a plurality of adjacent lanes, wherein the processor, in cooperation with the memory, obtains information on a mask area for identifying portions of the vehicle included in the camera image that are to be excluded from the target of predetermined processing; sets a specific detection frame for the vehicle shown in the camera image; and outputs portions of the vehicle that have inside / outside determination points of the specific detection frame that are not included in the mask area as the target of the predetermined processing.

2. The lane determination device according to claim 1, wherein the specific detection frame is the front or rear portion of a trailer towed by a tractor of the vehicle.

3. The lane determination device according to claim 2, wherein the predetermined process is reading an ID written on the trailer, and the lane determination device further reads and outputs the ID from the specific detection frame.

4. The lane determination device according to claim 2, wherein the processor sets the horizontal position of the inside / outside determination point of the specific detection frame to a point on the left or right side of the specific detection frame based on the imaging direction of the camera of the multiple lanes.

5. The lane determination device according to claim 4, wherein the processor sets the horizontal position of one of the points on the left or right side of the specific detection frame that corresponds to a position closest to the camera as the horizontal position of the inside / outside determination point.

6. The lane determination device according to claim 2, wherein the processor sets the vertical position of the inside / outside determination point of the specific detection frame to a point on the bottom side of the specific detection frame.

7. The lane determination device according to claim 1, wherein the specific detection frame is a part of a tractor that tows a trailer of the vehicle.

8. The lane determination device according to claim 1, wherein the processor detects the specific detection frame using an AI model capable of detecting the specific detection frame stored in the memory.

9. The lane determination device according to claim 1, wherein the processor sets the mask area based on detection of a user operation for specifying each vertex of the mask area having a polygonal shape.

10. The lane determination device according to claim 9, wherein the processor modifies and sets the shape of the mask area when the mask area has a shape that includes a recess due to the order in which the vertices are specified.

11. The lane determination device according to claim 1, wherein the processor sets the inside / outside determination point to one of the center point, upper left point, lower left point, upper right point, or lower right point of the specific detection frame based on detection of a user operation.

12. A lane determination method executed by a lane determination device, comprising: storing camera images captured by a camera positioned so as to capture images of multiple vehicles traveling in each of multiple adjacent lanes; acquiring information on a mask area for identifying parts of the vehicle included in the camera image that are to be excluded from processing by a specified process; setting a specific detection frame for the vehicle shown in the camera image; and outputting parts of the vehicle that have inside and outside determination points of the specific detection frame that are not included in the mask area as targets for the specified process.

13. A program for causing a lane determination device, which is a computer, to perform the following operations: storing camera images captured by a camera positioned so as to capture images of multiple vehicles traveling in each of multiple adjacent lanes; acquiring information on a mask area for identifying parts of the vehicle included in the camera image that are to be excluded from the target of specified processing; setting a specific detection frame for the vehicle shown in the camera image; and outputting parts of the vehicle that have inside and outside determination points of the specific detection frame that are not included in the mask area as the target of the specified processing.

Images