Intersection edge terminal, method, and computer program

Abstract

To provide an intersection edge terminal that reduces the computational load of recognition processing when recognizing objects around intersections.SOLUTION: An intersection edge terminal includes: a communication unit that receives images captured at an intersection; and a processing unit that detects the presence of a high-speed moving vehicle in the images and performs recognition processing of the high-speed moving vehicle using information associated with the high-speed moving vehicle acquired at other intersections.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present disclosure relates to an intersection edge terminal, a method, and a computer program.

Background Art

[0002] Patent Document 1 discloses a technique in which an object detection device arranged at an intersection generates movement information and transmits it to an object detection device arranged near an adjacent intersection.

[0003] In recent years, smart utilization methods using IT and AI have spread to all industries, and transportation, which is a social infrastructure, is no exception. Especially regarding the utilization of AI, the progress since the Transformer technology has been remarkable, and large-scale and highly accurate AI engines such as GPT have emerged, and their utilization in a wide range of fields is expected in the future.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the above-described applications for social infrastructure, reliable real-time processing is a very important factor. In cloud-based AI utilization from a remote location such as GPT, there are various problems regarding processing time including delays due to communication networks and response requirements.

[0006] Applications of smart intersections utilizing images from cameras deployed at intersection edge terminals include intersection state recognition and monitoring, efficient and safe signal control and information transmission to intersection users using recognition information, and identification and response to issues related to intersection safety, efficiency, and convenience. In these applications, the images collected by the cameras must be highly detailed and accurate, reliably recognizing vehicles, pedestrians, pets, and backgrounds, and the results must be utilized for various purposes. To meet this demand, methods such as increasing the number of cameras, increasing the resolution of camera pixels, increasing the number of processing frames per unit time, and increasing the parameters of the AI ​​processing model can be considered, but all of these significantly increase the computational load on the intersection edge terminals. Furthermore, for vehicles and other high-speed moving objects at intersections, the aforementioned massive amount of information processing must be performed within an extremely limited time, requiring even more difficult and large-scale processing in terms of computational load per unit of time. To handle this massive computational load challenge using only edge terminals installed at intersections, it is necessary to equip them with a large amount of hardware resources such as high-performance CPUs, GPUs, and memory, which leads to the challenge of large-scale equipment and high power consumption.

[0007] One example of the purpose of this disclosure is to provide an intersection edge terminal, method, and computer program that can solve the above-mentioned problems. [Means for solving the problem]

[0008] An intersection edge terminal according to one aspect of the present disclosure includes a communication unit that receives an image captured at its own intersection, and a processing unit that detects whether the image includes a high-speed moving object and performs a recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at another intersection.

[0009] A method relating to one aspect of the present disclosure includes: a communication unit of an intersection edge terminal receiving an image captured at its own intersection; a processing unit of the intersection edge terminal detecting that the image includes a high-speed moving object; and the processing unit performing a high-speed moving object recognition process using information associated with a high-speed moving object acquired at another intersection.

[0010] A computer program according to one aspect of the present disclosure causes an intersection edge terminal to execute communication means for receiving an image captured at the intersection, and processing means for detecting whether the image includes a high-speed moving object and for performing a recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at another intersection. [Effects of the Invention]

[0011] According to the above embodiment, the computational complexity of object recognition processing at intersection edge terminals can be significantly reduced, and efficient object recognition processing can be achieved. [Brief explanation of the drawing]

[0012] [Figure 1] This is a schematic diagram showing the external configuration of the system for recognizing the state of an intersection related to this disclosure. [Figure 2] This figure shows the functional configuration of an intersection edge terminal for recognizing the state of an intersection related to this disclosure. [Figure 3] This is a flowchart of the method for recognizing the state of an intersection related to this disclosure. [Figure 4] This figure shows an example of detailed information related to this disclosure. [Figure 5] This is a flowchart of the abstraction classification process related to this disclosure. [Figure 6] This diagram shows the functional configuration of the system for recognizing the state of an intersection related to this disclosure. [Figure 7] This is a flowchart of the method for recognizing the state of an intersection related to this disclosure. [Figure 8] This figure shows an example of detailed information related to this disclosure. [Figure 9]It is a diagram showing an example of the arrangement of the associated camera according to the present disclosure. [Figure 10] It is a diagram showing another example of the configuration of the intersection edge terminal according to the present disclosure. [Figure 11] It is a diagram showing another example of a method for recognizing the state of an intersection according to the present disclosure. [Figure 12] It is a diagram showing an example of the configuration of a computer according to the present disclosure.

Mode for Carrying Out the Invention

[0013] Hereinafter, each embodiment will be described with reference to the drawings. In all the drawings, the same or corresponding components are denoted by the same reference numerals, and common descriptions are omitted.

[0014] (Overall Configuration) FIG. 1 is a schematic diagram showing the external configuration of a system 1 for recognizing the state of an intersection according to the present disclosure. In FIG. 1, an intersection having a traffic signal 11 is shown.

[0015] The system 1 includes at least one camera 2 and an intersection edge terminal 3. The camera 2 acquires an image of the intersection. The camera 2 is arranged at a position where it can monitor fixed bodies such as roads, sidewalks, and guardrails at the intersection, and moving bodies passing through the intersection, such as automobiles 13, motorcycles or bicycles 14, pedestrians 15, and pets. The camera 2 may be installed on the traffic signal 11 or the pillar 12 of the intersection. The intersection edge terminal 3 may be installed on the traffic signal 11 or the pillar 12 of the intersection. The intersection edge terminal 3 acquires image information using the camera 2 attached to the traffic signal 11 and / or the pillar 12 as an input source regarding the state of the intersection.

[0016] The image may include a plurality of objects including the automobile 13, the motorcycle or bicycle 14, and the pedestrian 15. The plurality of objects may include any objects such as houses and animals such as pets. The intersection edge terminal 3 can communicate with other intersection edge terminals installed at neighboring intersections 5. The intersection edge terminal 3 can communicate with a neighboring road camera 6 installed on a neighboring road. The above-described communication is performed by wireless communication or wired communication via the network 4.

[0017] <First Embodiment> Hereinafter, an embodiment according to the present disclosure will be described with reference to FIGS. 2 to 4.

[0018] (Functional Configuration) FIG. 2 is a diagram showing a functional configuration of the intersection edge terminal 3 for recognizing the state of an intersection according to the present disclosure. The intersection edge terminal 3 includes a processing unit 31 and a communication unit 32. The processing unit 31 may be a processor such as a CPU or a GPU, for example. The communication unit 32 can communicate with any device other than the intersection edge terminal 3 provided in the communication unit 32. For example, the communication unit 32 can communicate with the camera 2 in the system 1, other intersection edge terminals at the neighboring intersection 5, and the like.

[0019] (Processing Flow) FIG. 3 is a flowchart showing the operation of the intersection edge terminal 3 according to an embodiment of the present disclosure. First, the communication unit 32 acquires information about a high-speed moving object acquired by another intersection edge terminal at a neighboring intersection 5 (step S301). For example, the information may be detailed information indicated by the object ID1 and the object ID2 in FIG. For example, the detailed information may include the vehicle type of the vehicle, the passengers, and their expressions. The camera 2 acquires an image at the self-intersection (step S302). The camera 2 transmits the acquired image to the communication unit 32.

[0020] Next, the processing unit 31 determines whether or not the acquired image contains a high-speed moving object (step S303). For example, the processing unit 31 may perform AI object detection processing to extract objects from the image and determine whether or not a high-speed moving object is included by recognizing whether or not the extracted object is a high-speed moving object such as a background, pedestrian, or vehicle. The above-described AI object detection processing may be performed based on a trained neural network acquired by deep learning.

[0021] If it is determined that the image does not contain a high-speed moving object (step S303: no), the processing unit 31 returns to executing step S302. If it is determined that the image contains a high-speed moving object (step S303: yes), it is determined whether the information about the high-speed moving object acquired by the other intersection edge terminals includes information corresponding to the high-speed moving object acquired by the local intersection edge terminal 3 (step S304). For example, the intersection edge terminal 3 may detect the type of vehicle, body color, license plate, etc., of the high-speed moving object and perform the determination in step S304 by comparing it with the information from the other intersection edge terminals. If it is determined that the information from the other intersection edge terminals does not include information corresponding to the high-speed moving object acquired by the local intersection edge terminal 3 (step S304: no), the processing unit 31 returns to executing step S302.

[0022] If it is determined that the corresponding information is included in the information from the other intersection edge terminals mentioned above (step S304: yes), the information associated with the high-speed mobile object is obtained from the information from the other intersection edge terminals (step S305). The processing unit 31 uses the information associated with the high-speed mobile object to perform recognition processing of the high-speed mobile object (step S306).

[0023] The processing unit 31 acquires detailed information about the high-speed moving object through the recognition process in step S305 (step S307). For example, the detailed information acquired by the intersection edge terminal 3 may be additional information to the detailed information acquired by other intersection edge terminals acquired in step S301. For example, the additional information may include a characteristic image of the vehicle, the type of vehicle, the occupants, and their facial expressions.

[0024] Furthermore, the additional information may be used to fill in the missing portion of the license plate indicated by "??" in object ID 2 of Figure 4. The missing information to be filled in is not limited to license plate information. This makes it possible to have complete processing results of detailed and unique object information and to utilize that information in the process of passing through multiple intersections, even when computation processing and image collection cannot be completed at a single intersection. In addition, this function is expected to have the effect of enabling data completion in the event of sudden information collection or processing delays at individual intersections, thereby improving the redundancy and reliability of the system.

[0025] The processing unit 31 performs synthesis processing based on the detailed information obtained in step S307 (step S308). The processing unit 31 may perform synthesis processing according to various applications. For example, the processing unit 31 may perform synthesis processing based only on the necessary information from the detailed information, according to applications such as traffic accident analysis or traffic signal control. Furthermore, the synthesized information may be used for various applications such as signal display switching timing control, providing information to intersection users before they enter, monitoring accidents and violations, accident analysis, traffic problem identification, traffic problem countermeasures, image verification, and search. It may also be stored for these applications.

[0026] (Effect, Action) As described above, the intersection edge terminal 3 includes a communication unit 32 that receives images captured at its own intersection, and a processing unit 31 that detects whether the image contains a high-speed moving object and performs a recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at other intersections.

[0027] In other words, before a high-speed moving object entering the intersection enters it, the results already processed at other nearby intersections 5 become available at the intersection edge terminal 3. This significantly reduces the computational load required for object recognition processing at the intersection edge terminal 3 and enables efficient object recognition processing. As a result, there is no need to increase the size of the device or consume large amounts of power, and it is possible to perform high-load calculations such as increasing the number of cameras, increasing the resolution of camera pixels, increasing the number of processing frames per unit time, and increasing the parameters of the AI ​​processing model.

[0028] <First Modification of the First Embodiment> A modified example of one embodiment of the present disclosure will be described below with reference to Figure 5. Figure 5 is a flowchart of the abstraction classification process related to this disclosure.

[0029] First, camera 2 acquires an image containing an object (step S501). Camera 2 transmits the acquired image to the communication unit 32.

[0030] The processing unit 31 detects objects in the image received via the communication unit 32 (step S502). For example, the processing unit 31 performs AI object detection processing to extract objects from the image. The processing unit 31 performs abstraction classification processing to abstract the extracted objects and classify them into background, pedestrians, vehicles and other high-speed moving objects, etc.

[0031] Next, the intersection edge terminal 3 determines whether or not a request for detailed recognition processing has been received in the communication unit 32 (step S503). For example, a sound sensor linked to the processing unit 31 detects the occurrence of an emergency, such as a traffic accident, based on the detected volume. At this time, the sound sensor transmits a request for detailed recognition processing to the communication unit 32. The above emergency may also be detected by various sensors such as light, vibration, distance, laser, and ultrasound.

[0032] If a request for detailed recognition processing is received (step S503: Yes), the intersection edge terminal 3 executes detailed recognition processing (steps S302 to S307). Furthermore, the processing unit 31 may perform detailed recognition processing (steps S302 to S307) based on the information about the object acquired in step S502. That is, if a high-speed moving object is detected in step S502, the processing unit 31 may perform the detailed recognition processing in steps S304 to S307. Alternatively, detailed recognition processing may be performed by a simplified process that overlays detailed recognition processing onto the information obtained in the object extraction process in the abstraction classification process.

[0033] If no request for detailed recognition processing is received (step S503: No), the processing unit 31 continues the abstraction classification processing (steps S504 to S506).

[0034] The processing unit 31 recognizes the relative relationship between the objects (step S504). For example, the processing unit 31 measures the distance between multiple pedestrians on the road (background), or the distance between multiple cars.

[0035] The processing unit 31 performs traffic condition recognition processing (step S505). For example, the processing unit 31 detects that there is a high concentration of pedestrians at the intersection where the intersection edge terminal 3 is located, based on the small distances between multiple pedestrians measured in step S504. The processing unit 31 also detects that there is a high probability of traffic congestion occurring, or that traffic congestion has already occurred, at the intersection where the intersection edge terminal 3 is located, based on the small distances between multiple vehicles.

[0036] The processing unit 31 executes a process to make effective use of the acquired traffic condition information (step S506). For example, if the processing unit 31 detects that pedestrians are densely packed at an intersection where the intersection edge terminal 3 is located, it may extend the duration of the green light for pedestrian signals to control the signals and alleviate the congestion. Alternatively, if the processing unit 31 detects that there is a high probability of traffic congestion occurring, or that traffic congestion is already occurring, it may extend the duration of the green light for vehicle signals to control the signals and avoid traffic congestion.

[0037] (Effect, Action) As described above, the processing unit 31 of the intersection edge terminal 3 further detects multiple objects, performs an abstraction classification process to abstract and classify the multiple objects, and, in response to the detection of a predetermined event at the intersection that requires detailed recognition processing, performs detailed recognition processing on the multiple objects detected in the abstraction classification process.

[0038] This system separates two distinct functions: a computationally intensive abstraction classification process and a detailed recognition process. It also offers two processing configurations: one where each function operates independently, and another where they are combined for result processing. This allows for optimization of the computational load according to the application's requirements. In other words, it enables the reduction of the load on the intersection edge terminal 3 and optimization of the computational load depending on traffic conditions and the application.

[0039] <Second modified example of the first embodiment> Information from other intersection edge terminals at neighboring intersection 5 may include the recording time and recorded vehicle speed at the other intersection edge terminals. The processing unit 31 may predict at least one of the order and time of vehicle entry based on the distance from its own intersection to neighboring intersection 5. The processing unit 31 may prepare and execute recognition processing based on at least one of the predicted order and time of entry of vehicles, etc. For example, the processing unit 31 may prepare relevant information in the memory of the intersection edge terminal 3 so that it can read and update information on the vehicle that is predicted to enter the intersection first.

[0040] As described above, the processing unit 31 predicts at least one of the order and time in which high-speed moving objects will enter its own intersection based on information associated with high-speed moving objects acquired at other intersections, and performs recognition processing at its own intersection based on at least one of the predicted order and time.

[0041] This makes it possible to predict the order in which a vehicle will enter an intersection in its direction of travel, as well as the approximate time of entry, and by having a method to wait for the vehicle to enter before it actually does, the linking process can be simplified and made more reliable, and the results of the image unique detail recognition process can be prepared in a timely manner.

[0042] <Third modified example of the first embodiment> After obtaining detailed information in step S307, the processing unit 31 may transmit the detailed information to other intersection edge terminals at nearby intersections. Here, the intersection edge terminal 3 may obtain the direction of travel of the high-speed moving object and transmit the detailed information only to other intersection edge terminals at nearby intersections corresponding to the direction of travel.

[0043] In other words, the intersection edge terminal 3 has the function of sending data only in the direction in which individual vehicles are leaving the intersection. With this function, the data will not be sent to unnecessary nearby intersections 5, or intersection edge terminals at nearby intersections that deem the data unnecessary may discard the data without receiving it. This significantly reduces the amount of data transmitted through the communication network between neighboring intersections, and also reduces the hardware resources required for transmission and reception processing at the intersection edge terminals 3 between neighboring intersections.

[0044] <Second Embodiment> Hereinafter, one embodiment relating to this disclosure will be described with reference to Figures 6 to 9.

[0045] (Functional Configuration) Figure 6 is a diagram showing the functional configuration of System 1 according to one embodiment of the present disclosure. Since the functional configuration common to Figure 2 is the same as that of Figure 2, the differences will be explained below.

[0046] The intersection edge terminal 3 includes a memory 33 and a cooling unit 34. As a result, the cooling unit 34 can prevent thermal failures even in harsh environments such as outdoor ambient temperatures.

[0047] The memory 33 may be any storage medium, such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or NVMe (Non-Volatile Memory Express). The cooling unit 34 may be any cooling device.

[0048] The intersection edge terminal 3 communicates with at least one of the neighboring intersections 5 and neighboring road cameras 6 via the communication unit 32 and the network 4. This communication is performed by wireless or wired communication. The nearby intersection 5 may refer to an adjacent intersection or an intersection that is not adjacent but is located nearby. The nearby road camera 6 is a camera placed on a traffic light 11 or pole 12 on a road that is not an intersection.

[0049] System 1 further includes at least one of the following: a linked camera 7, a linked wireless terminal 8, a linked in-vehicle terminal 9, and a linked satellite terminal 10.

[0050] The linked camera 7 is positioned at a different location from camera 2, which is located at the intersection corresponding to the intersection edge terminal 3. The linked camera 7 is linked to camera 2. As shown in Figure 9, the linked camera 7 can capture images of the subject simultaneously or nearly simultaneously from a different angle than the linked camera 2. For example, one method for realizing the linking processing function is to use image recognition of vehicle license plates. If camera 2 in an intersection is simply installed from above on a traffic light 11 or an intersection support pole 12, when multiple vehicles enter in a line, the vehicle in front will be in a blind spot, and camera 2 will only be able to confirm the vehicle license plate after the vehicle in front just before entering the intersection has passed, resulting in a problem where the time available for processing after linking is very short. As a solution to this problem, the linking camera 7 for vehicle linking can be configured to be positioned and oriented so that the vehicle in front of the target vehicle passes by, and the vehicle license plate is not in a blind spot due to the vehicle in front. The attached camera 7 may be positioned below the road surface before the intersection, with its camera surface angled from upward to away from the intersection. Alternatively, it may be positioned on the side of the road before the intersection, with its camera surface angled from facing the vehicle to away from the intersection.

[0051] As described above, the image is acquired by camera 2 at the intersection, and the communication unit 32 further acquires a linked image from linked camera 7 which is linked to camera 2, and linked camera 7 is located at a different position from camera 2. The processing unit 31 performs recognition processing based on the image and the linked image. This increases the probability of acquiring vehicle information, even when the vehicle in front has just passed the intersection, resulting in a very short processing time after linking.

[0052] The linked wireless terminal 8 is a short-range wireless terminal mounted on the vehicle. For example, the linked wireless terminal 8 is a vehicle-mounted short-range wireless terminal with strong directivity and high-speed processing capabilities, similar to those used in ETC systems on toll roads. System 1 and the vehicle are each equipped with wireless equipment for short-range communication. The intersection edge terminal 3 recognizes each vehicle's unique ID for linking, links it with the image-processed vehicle information, and sends this linked set of information to other nearby intersections 5. At the next nearby intersection 5 where the vehicle moves to, the vehicle can be identified using the unique ID for linking, and the sent vehicle information can be utilized.

[0053] The linked in-vehicle terminal 9 is a drive recorder or vehicle-mounted camera of a car 13 entering the intersection, and is linked to the intersection edge terminal 3. For example, the linked in-vehicle terminal 9 receives information from in-vehicle cameras such as a drive recorder and autonomous driving cameras, and before entering an intersection, it wirelessly transmits information such as the license plates of the vehicles in front of and behind the vehicle, and whether the vehicles have been swapped, to the intersection edge terminal 3 from the vehicle. This simplifies and improves the accuracy of the linking process, and enables the timely preparation of the results of the image unique detail recognition process.

[0054] The linked satellite terminal 10 is a satellite terminal linked to the intersection edge terminal 3. The linked satellite terminal 10 transmits satellite images to the intersection edge terminal 3. The intersection edge terminal 3 can trace the movement status of individual vehicles from the satellite images and collect image unique detail recognition processing information collected at other nearby intersections 5 before each vehicle enters each intersection, and perform the linking process. This simplifies and improves the accuracy of the linking process, and enables the timely preparation of the results of the image unique detail recognition process.

[0055] System 1 may further include other sensors, such as sound, light, vibration, distance, laser, and ultrasonic sensors, to monitor the conditions of the intersection.

[0056] (Processing flow) Figure 7 is a flowchart showing the operation of an intersection edge terminal 3 according to one embodiment of the present disclosure. First, camera 2 acquires an image containing an object that was captured at the intersection (step S701). Camera 2 transmits the acquired image to the communication unit 32.

[0057] Next, the processing unit 31 detects objects contained in the received image (step S702). For example, the processing unit 31 performs AI object detection processing to extract objects from the image, recognize whether the extracted objects are background, vehicles, or pedestrians, and uniquely identify each of them. In this embodiment, it is assumed that the processing unit 31 has recognized four objects. The processing unit 31 assigns a unique identifier (object ID 1 to object ID 4) to each of the four recognized objects. The above-described AI object detection processing may be performed based on a trained neural network obtained by deep learning.

[0058] Next, the processing unit 31 calculates the amount of time variation for each detected object (step S703). For example, the processing unit 31 calculates the difference in the positions of each object included in the first image and the second image, which is taken a certain time after the first image was taken, and calculates the amount of time variation. The amount of time variation may be calculated by averaging the amount of time variation in a series of three or more images. For objects recognized as background in step S702, the calculation of the amount of time variation may be omitted in order to reduce the computational load.

[0059] The processing unit 31 classifies each object into a predetermined category based on the amount of time variation (step S704). For example, the predetermined categories are four: fixed background, temporarily stationary object, slow-moving object, and fast-moving object. The number of predetermined categories may be two or more. The predetermined categories may also be classified into categories with different content than the four mentioned above.

[0060] For example, when comparing a second image taken after a predetermined period (e.g., 5 seconds) has elapsed since the first image was taken, if the time variation value of the first object corresponding to object ID 1 during that predetermined period is 0m, the first object is classified into a first category indicating a fixed background. If the second object corresponding to object ID2 has a speed of 0 m / s during a predetermined period, and then has a speed greater than 0 m / s and less than 1.3 m / s during a certain period after the predetermined period has elapsed, the second object is classified into a second category indicating a temporarily stationary object. When the third object corresponding to object ID3 travels at a speed greater than 1.3 m / s and less than 5.0 m / s during a predetermined period, the third object is classified into a third category indicating a slow-moving object. If the fourth object corresponding to object ID 4 travels at a speed greater than 5.0 m / s during a predetermined period, the fourth object is classified into the fourth category, indicating a high-speed moving object.

[0061] The processing unit 31 recognizes each of the multiple objects according to a predetermined recognition method corresponding to the classified category (steps S705-1 to S705-4). For example, the processing unit 31 recognizes a first object classified into a first category according to a method for recognizing a fixed background. For example, the processing unit 31 recognizes the first object at low resolution and over a long period of time. Furthermore, the processing unit 31 recognizes the second object classified into the second category according to a method for recognizing a temporarily stationary object. For example, the method for recognizing a temporarily stationary object may specify recognition with a higher resolution and shorter period than the method for recognizing the fixed background described above. Furthermore, the processing unit 31 recognizes the third object classified into the third category according to a method for recognizing slow-moving objects. For example, the method for recognizing slow-moving objects may specify recognition with a higher resolution and shorter period than the method for recognizing temporarily stationary objects described above. Unlike the method for recognizing a fixed background and the method for recognizing temporarily stationary objects, the method for recognizing slow-moving objects may acquire color information of the third object. Furthermore, the processing unit 31 recognizes the fourth object, which has been classified into the fourth category, according to a method for recognizing high-speed moving objects. For example, the method for recognizing high-speed moving objects may specify recognition at a higher resolution and shorter period than the method for recognizing low-speed moving objects described above. In the method for recognizing high-speed moving objects, unlike the method for recognizing a fixed background and the method for recognizing a temporarily stationary object, the color information of the fourth object may be acquired. Also, in the method for recognizing high-speed moving objects, unlike the method for recognizing a fixed background and the method for recognizing a low-speed moving object, information on the vehicle's license plate or images of the vehicle from multiple directions may be acquired. The differences in settings for each recognition method should not be limited to the resolution and period described above; arbitrary settings may be used. Furthermore, the detailed settings for different recognition methods do not necessarily have to differ. For example, the resolution settings for a method recognizing a fixed background may be the same as those for a method recognizing a temporary, stationary object. Each recognition method may be performed in parallel.

[0062] The processing unit 31 obtains detailed information by performing recognition processing corresponding to each recognition method in steps S705-1 to S705-4 (step S706). For example, the information may be detailed as shown in Figure 8, but it is not limited to this form of information and may also include other information such as the type of vehicle, the occupants and their facial expressions.

[0063] The processing unit 31 performs synthesis processing based on the detailed information obtained in step S706 (step S707). The processing unit 31 may perform synthesis processing according to various applications. For example, if the application is limited to pedestrian information at an intersection, the synthesis processing may be limited to detailed information of objects in the second category. If the application is for traffic accident analysis, the synthesis processing may be performed based on detailed information of the third and fourth categories. If the application is for traffic signal control, the synthesis processing may be limited to detailed information of objects in the second to fourth categories. Furthermore, the synthesized information may be used for various applications such as signal display switching timing control, providing information to intersection users before they enter, monitoring accidents and violations, accident analysis, traffic problem identification, traffic problem countermeasures, image verification, and search. The information may also be stored for these applications.

[0064] (Effect, Action) As described above, the communication unit 32 receives multiple images captured at the intersection, the processing unit 31 detects objects included in the multiple images, calculates the amount of time variation of the objects based on the multiple images, classifies the objects into predetermined categories based on the amount of time variation, and performs object recognition processing according to a predetermined recognition method corresponding to the category.

[0065] In other words, the intersection edge terminal 3 has a function to classify the image data collected from the camera into categories such as high-speed moving objects like cars 13 and motorcycles, low-speed moving objects like bicycles 14, pedestrians 15 and pets, temporary stationary objects (temporarily stationary objects) in the intersection such as garbage and fallen objects, and fixed background information such as road surfaces, signs, utility poles, signal poles, and guardrails that are always present, based on the amount of time variation of each object per unit of time. Furthermore, different methods are used for object recognition processing cycles and recognition methods for each classified category, and recognition processing is performed at different cycles that match the changes in the state and position of each object.

[0066] This allows for recognition processing using the optimal recognition method for each object, thereby reducing the computational load on recognition processing at intersection edge terminals and enabling efficient recognition of objects in images.

[0067] <Modified form of the second embodiment> The intersection edge terminal at nearby intersection 5 or the terminal at nearby road camera 6 may communicate with a vehicle-mounted terminal (linked wireless terminal 8) and receive a vehicle identifier (for example, a vehicle-specific identifier such as a serial number) from the vehicle-mounted terminal. At this time, the intersection edge terminal at nearby intersection 5 or the terminal at nearby road camera 6 links the received vehicle identifier with the detailed information acquired at the terminal. The intersection edge terminal at nearby intersection 5 or the terminal at nearby road camera 6 transmits the linked detailed information to the intersection edge terminal 3 at nearby intersection. The intersection edge terminal 3 receives the linked detailed information and uses the linked detailed information to perform detailed recognition processing.

[0068] The above description explains the linking process by which the intersection edge terminal at nearby intersection 5 or the terminal at nearby road camera 6 communicates with the vehicle-mounted terminal. However, the intersection edge terminal 3 shown in Figure 1, which has a similar configuration to the intersection edge terminal at nearby intersection 5, may also perform the same process. In other words, the communication unit 32 of the intersection edge terminal 3 receives a vehicle identifier from the vehicle-mounted terminal, the processing unit 31 associates the vehicle identifier with information generated by the recognition process, and transmits the associated information to other intersection edge terminals.

[0069] This ensures the unique identification of each vehicle and improves the reliability of the information obtained.

[0070] <Third Embodiment> Figure 10 shows another example of the configuration of an intersection edge terminal according to this disclosure. In the configuration shown in Figure 10, the intersection edge terminal 100 comprises a processing unit 101 and a communication unit 102.

[0071] In this configuration, the communication unit 102 receives an image captured at its own intersection. The processing unit 101 detects that the image contains a high-speed moving object and performs recognition processing of the high-speed moving object using information associated with the high-speed moving object acquired at other intersections.

[0072] According to the intersection edge terminal 100, before a high-speed moving object entering its own intersection enters the intersection, the results already processed at other nearby intersections 5 become available to the intersection edge terminal 3. This significantly reduces the computational load for object recognition processing at the intersection edge terminal 3 and enables efficient object recognition processing. As a result, there is no need to increase the size of the device or consume large amounts of power, and it is possible to perform high-load calculations such as increasing the number of cameras, increasing the resolution of camera pixels, increasing the number of processing frames per unit time, and increasing the parameters of the AI ​​processing model.

[0073] <Fourth Embodiment> Figure 11 shows another example of a method for recognizing the state of an intersection according to this disclosure. The method shown in Figure 11 includes the communication unit 102 of the intersection edge terminal receiving an image captured at its own intersection (step S1101). The method includes the processing unit 101 of the intersection edge terminal detecting that the image contains a high-speed moving object (step S1102). The method includes the processing unit 101 performing a high-speed moving object recognition process using information associated with a high-speed moving object acquired at another intersection (step S1103).

[0074] According to the method shown in Figure 11, before a high-speed moving object entering the intersection enters the intersection, the results already processed at other nearby intersections 5 become available at the intersection edge terminal 3. This significantly reduces the computational load for object recognition processing at the intersection edge terminal 3 and enables efficient object recognition processing. As a result, there is no need to increase the size of the device or consume large amounts of power, and it is possible to achieve high-load calculations such as increasing the number of cameras, increasing the resolution of camera pixels, increasing the number of processing frames per unit time, and increasing the parameters of the AI ​​processing model.

[0075] Figure 12 shows an example of the computer configuration related to this disclosure. As shown in Figure 12, the computer 1200 comprises a CPU 1201, a main memory 1202, an auxiliary memory 1203, an interface 1204, and a non-volatile recording medium 1205.

[0076] One or more of the above-mentioned intersection edge terminals 3 and 100, or parts thereof, may be implemented in the computer 1200. In that case, the operation of each processing unit described above is stored in the auxiliary storage device 1203 in the form of a program. The CPU 1201 reads the program from the auxiliary storage device 1203, expands it in the main memory device 1202, and executes the above processing according to the program. The CPU 1201 also allocates memory areas in the main memory device 1202 corresponding to each of the above-mentioned memory units according to the program. Communication between each device and other devices is performed by the interface 1204 having a communication function and performing communication according to the control of the CPU 1201. The interface 1204 also has a port for the non-volatile recording medium 1205, and reads information from the non-volatile recording medium 1205 and writes information to the non-volatile recording medium 1205.

[0077] When the intersection edge terminal 3 is implemented in the computer 1200, the operation of the processing unit 31 and each of its parts is stored in auxiliary storage device 1203 in the form of a program. The CPU 1201 reads the program from auxiliary storage device 1203, expands it into main memory device 1202, and executes the above processing according to the program.

[0078] Furthermore, the CPU 1201 allocates memory in the main memory 1202 for processing by the intersection edge terminal 3 according to the program. Communication with other devices by the communication unit 32 is performed by the interface 1204 having a communication function and operating under the control of the CPU 1201. Interaction between the intersection edge terminal 3 and the user is performed by the interface 1204 having input and output devices, presenting information to the user via the output device and accepting user operations via the input device under the control of the CPU 1201.

[0079] When the intersection edge terminal 100 is implemented in the computer 1200, the operation of the processing unit 101 is stored in auxiliary storage device 1203 in the form of a program. The CPU 1201 reads the program from auxiliary storage device 1203, expands it into main memory 1202, and executes the above processing according to the program.

[0080] Furthermore, the CPU 1201 allocates memory in the main memory 1202 for processing by the intersection edge terminal 100 according to the program. Communication between the intersection edge terminal 100 and other devices is performed by the interface 1204 having a communication function and operating under the control of the CPU 1201. Interaction between the intersection edge terminal 100 and the user is performed by the interface 1204 having input and output devices, presenting information to the user via the output device and accepting user operations via the input device under the control of the CPU 1201.

[0081] One or more of the above-mentioned programs may be recorded on the non-volatile recording medium 1205. In this case, the interface 1204 may read the program from the non-volatile recording medium 1205. The CPU 1201 may then either directly execute the program read by the interface 1204, or it may temporarily save it in the main memory 1202 or auxiliary memory 1203 before executing it.

[0082] Alternatively, a program for executing all or part of the processing performed by the intersection edge terminal 3 and the intersection edge terminal 100 may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be loaded into a computer system and executed to perform the processing of each part. The term "computer system" here includes hardware such as the OS (Operating System) and peripheral devices. Furthermore, "computer-readable recording media" refers to portable media such as flexible disks, magneto-optical disks, ROMs (Read Only Memory), CD-ROMs (Compact Disc Read Only Memory), and storage devices such as hard disks built into computer systems. The above-mentioned program may be intended to implement only a part of the functions described above, and may also be able to implement the above-mentioned functions in combination with programs already recorded in the computer system.

[0083] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure are possible, as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0084] Some or all of the above embodiments may also be described as follows, but are not limited to the following:

[0085] (Note 1) A communication unit that receives images captured at the intersection, The image is detected to contain a high-speed moving object. The recognition process for the high-speed moving object is performed using information associated with the high-speed moving object acquired at other intersections. Processing section and An intersection edge terminal equipped with this feature.

[0086] (Note 2) The aforementioned processing unit further, The image is detected to contain an object. The aforementioned objects are abstracted and classified, In response to the detection of a predetermined event at the aforementioned intersection, the recognition process is performed on the classified object. The intersection edge terminal described in Appendix 1.

[0087] (Note 3) The aforementioned communications unit further, Multiple images captured at the aforementioned intersection are received. The aforementioned processing unit further, The object contained in the aforementioned multiple images is detected, Based on the aforementioned multiple images, the amount of time variation of the object is calculated, Based on the aforementioned time variation, the objects are classified into a predetermined category. The recognition process of the object is performed according to a predetermined recognition method corresponding to the aforementioned category. Intersection edge terminal as described in Appendix 1 or 2.

[0088] (Note 4) Performing the recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at other intersections means that If the information associated with the high-speed moving object includes missing portions, the recognition process includes supplementing the missing portions. An intersection edge terminal as described in any one of the notes 1 to 3.

[0089] (Note 5) The aforementioned processing unit, Based on the information associated with the high-speed moving object obtained at other intersections, at least one of the order and time in which the high-speed moving object enters the current intersection is predicted. Based on at least one of the predicted order and time, the recognition process at the intersection is performed. An intersection edge terminal as described in any one of the appendices 1 to 4.

[0090] (Note 6) The aforementioned processing unit, The direction of travel of the high-speed moving object is obtained, The aforementioned communications unit is The information generated by the recognition process of the high-speed moving object is transmitted only to other intersection edge terminals at nearby intersections corresponding to the direction of travel. An intersection edge terminal as described in any one of the notes 1 through 5.

[0091] (Note 7) The aforementioned image was acquired by a camera at the aforementioned intersection. The communication unit further acquires linked images from a linked camera linked to the camera, and the linked camera is positioned differently from the camera. The processing unit performs the recognition process based on the image and the associated image. An intersection edge terminal as described in any one of the notes 1 through 6.

[0092] (Note 8) The aforementioned communication unit receives a vehicle identifier from the vehicle-mounted terminal. The processing unit associates the vehicle identifier with the information generated by the recognition process and transmits the associated information to other intersection edge terminals. An intersection edge terminal as described in any one of the notes 1 through 7.

[0093] (Note 9) The communication unit of the intersection edge terminal receives images captured at its own intersection, The processing unit of the aforementioned intersection edge terminal detects that the image contains a high-speed moving object, The processing unit performs recognition processing of the high-speed moving object using information associated with the high-speed moving object acquired at other intersections. Methods that include...

[0094] (Note 10) The aforementioned processing unit further, The image is detected to contain an object. The aforementioned objects are abstracted and classified, In response to the detection of a predetermined event at the aforementioned intersection, the recognition process is performed on the classified object. The method described in Appendix 9.

[0095] (Note 11) The aforementioned communications unit further, Multiple images captured at the aforementioned intersection are received. The aforementioned processing unit further, The object contained in the aforementioned multiple images is detected, Based on the aforementioned multiple images, the amount of time variation of the object is calculated, Based on the aforementioned time variation, the objects are classified into a predetermined category. The recognition process of the object is performed according to a predetermined recognition method corresponding to the aforementioned category. The method described in Appendix 9 or 10.

[0096] (Note 12) Performing the recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at other intersections means that If the information associated with the high-speed moving object includes missing portions, the recognition process includes supplementing the missing portions. The method described in any one of the appendices 9 to 11.

[0097] (Note 13) The aforementioned processing unit, Based on the information associated with the high-speed moving object obtained at other intersections, at least one of the order and time in which the high-speed moving object enters the current intersection is predicted. Based on at least one of the predicted order and time, the recognition process at the intersection is performed. The method described in any one of the appendices 9 to 12.

[0098] (Note 14) The aforementioned processing unit, The direction of travel of the high-speed moving object is obtained, The aforementioned communications unit is The information generated by the recognition process of the high-speed moving object is transmitted only to other intersection edge terminals at nearby intersections corresponding to the direction of travel. The method described in any one of the appendices 9 to 13.

[0099] (Note 15) The aforementioned image was acquired by a camera at the aforementioned intersection. The communication unit further acquires linked images from a linked camera linked to the camera, and the linked camera is positioned differently from the camera. The processing unit performs the recognition process based on the image and the associated image. The method described in any one of the appendices 9 to 14.

[0100] (Note 16) The aforementioned communication unit receives a vehicle identifier from the vehicle-mounted terminal. The processing unit associates the vehicle identifier with the information generated by the recognition process and transmits the associated information to other intersection edge terminals. The method described in any one of the appendices 9 to 15.

[0101] (Note 17) At the intersection edge terminal, A communication means for receiving images captured at the intersection, The image is detected to contain a high-speed moving object. The recognition process for the high-speed moving object is performed using information associated with the high-speed moving object acquired at other intersections. Processing means and A computer program that executes an action. (Note 18) The processing means further, The image is detected to contain an object. The aforementioned objects are abstracted and classified, In response to the detection of a predetermined event at the aforementioned intersection, the recognition process is performed on the classified object. The computer program described in Appendix 17.

[0102] (Note 19) The aforementioned communication means further, Multiple images captured at the aforementioned intersection are received. The processing means further, The object contained in the aforementioned multiple images is detected, Based on the aforementioned multiple images, the amount of time variation of the object is calculated, Based on the aforementioned time variation, the objects are classified into a predetermined category. The recognition process of the object is performed according to a predetermined recognition method corresponding to the aforementioned category. The computer programs described in Appendix 17 or 18.

[0103] (Note 20) Performing the recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at other intersections means that If the information associated with the high-speed moving object includes missing portions, the recognition process includes supplementing the missing portions. A computer program described in any one of the appendices 17 to 19.

[0104] (Note 21) The processing means is Based on the information associated with the high-speed moving object obtained at other intersections, at least one of the order and time in which the high-speed moving object enters the current intersection is predicted. Based on at least one of the predicted order and time, the recognition process at the intersection is performed. A computer program described in any one of the appendices 17 to 20.

[0105] (Note 22) The processing means is The direction of travel of the high-speed moving object is obtained, The aforementioned communication means is The information generated by the recognition process of the high-speed moving object is transmitted only to other intersection edge terminals at nearby intersections corresponding to the direction of travel. A computer program described in any one of the appendices 17 to 21.

[0106] (Note 23) The aforementioned image was acquired by a camera at the aforementioned intersection. The communication means further acquires linked images from a linked camera linked to the camera, and the linked camera is positioned differently from the camera. The processing means performs the recognition process based on the image and the associated image. A computer program described in any one of the appendices 17 to 22.

[0107] (Note 24) The aforementioned communication means receives a vehicle identifier from a vehicle-mounted terminal. The processing means associates the vehicle identifier with the information generated by the recognition process and transmits the associated information to other intersection edge terminals. A computer program described in any one of the appendices 17 to 23.

[0108] (Note 25) It is a system, A camera that acquires images at the intersection, Intersection edge terminals listed in any one of the appendices 1 to 8 and A system equipped with these features. [Explanation of Symbols]

[0109] 1 System 2 cameras 3. Intersection edge terminals 4 Network 5. Neighborhood intersections 6. Nearby road cameras 7. Corded camera 8. Corded wireless terminals 9. Linked in-vehicle terminals 10 Linked Satellite Terminals 11 Traffic lights 12 pillars 13. Automobiles 14 Bicycle 15 Pedestrians 31 Processing Unit 32 Communications Department 33 memory 34 Cooling section 100 Intersection Edge Terminals 101 Processing Unit 102 Communications Department 1200 Computers 1202 Main storage 1203 Auxiliary storage device 1204 Interface 1205 Non-volatile recording media

Claims

1. A communication unit that receives information regarding a first image taken at its own intersection and a second image taken at another intersection, Information relating to the second image, which includes the vehicle type, body color, information contained in the license plate, and passengers obtained from the second image taken at the other intersection, A processing unit that uses feature information based on the second image associated with the vehicle received from the other intersection and the first image to perform recognition processing for a vehicle that has newly passed through the other intersection and the current intersection, and performs recognition processing for additional information indicating the vehicle's features, including at least one of the vehicle type, body color, information contained in the license plate, and passengers, as indicated by the feature information, and transmits the information obtained from the first image through this recognition processing to an intersection identified based on the direction of travel of the vehicle, in addition to the information associated with the vehicle based on the second image acquired at the other intersection. An intersection edge terminal equipped with this feature.

2. The aforementioned communications unit further, Multiple images of the first image taken at the aforementioned intersection are received. The aforementioned processing unit further, The object including the vehicle included in the plurality of first images is detected, Based on the plurality of first images, the amount of time variation of the object is calculated, Based on the aforementioned time variation, the objects are classified into a predetermined category. Recognition processing of information indicating the characteristics of the object is performed according to a predetermined recognition method corresponding to the aforementioned category. The intersection edge terminal according to claim 1.

3. If the recognition of the characters on the license plate in the feature information associated with the vehicle acquired at the aforementioned other intersection includes a portion that is poorly recognized, the recognition process includes supplementing the missing portion by synthesizing the feature information calculated from the first image captured at the aforementioned intersection. The intersection edge terminal according to claim 1.

4. The aforementioned processing unit, Based on the characteristic information associated with the vehicle acquired at another intersection, which includes at least one of the recording time and vehicle speed, and the distance between the current intersection and the other intersection, the system predicts at least one of the order and time at which the vehicle enters the current intersection. Based on at least one of the predicted order and time, the recognition process is performed, which includes identifying feature information about the vehicle included in the first image captured at the intersection, and generating information by combining the feature information associated with the vehicle acquired at other intersections with the feature information calculated from the first image captured at the intersection. The intersection edge terminal according to claim 1.

5. The aforementioned processing unit, The direction of travel of the aforementioned vehicle is obtained, The aforementioned communications unit is The characteristic information generated by the vehicle recognition process is transmitted only to other intersection edge terminals at nearby intersections corresponding to the direction of travel. The intersection edge terminal according to claim 1.

6. The aforementioned first image is an image acquired by a plurality of different cameras positioned at different locations in the intersection. The processing unit links the characteristic information of the vehicle obtained as a result of the recognition process using the first images acquired by the multiple different cameras. The intersection edge terminal according to claim 1.

7. The aforementioned communication unit receives a vehicle identifier from the vehicle-mounted terminal. The processing unit associates the vehicle identifier with the feature information generated by the recognition process, and transmits the associated feature information to another intersection edge terminal. The intersection edge terminal according to claim 1.

8. The communication unit of the intersection edge terminal receives information regarding the first image captured at its own intersection and the second image captured at another intersection, The processing unit of the intersection edge terminal acquires information relating to the second image, which includes the vehicle type, body color, information contained in the license plate, and passengers, based on the second image taken at the other intersection. Using the feature information based on the second image associated with the vehicle received from the other intersection and the first image, a recognition process is performed for a vehicle that has newly passed through the other intersection and the current intersection, and the recognition process is performed for additional information indicating the vehicle's features, including the vehicle type, the color of the vehicle body, the information contained in the license plate, and at least one of the passengers, as indicated by the feature information, and the information obtained from the first image through this recognition process is added to the information associated with the vehicle based on the second image acquired at the other intersection and transmitted to an intersection identified based on the direction of travel of the vehicle. Methods that include...

9. At the intersection edge terminal, A communication means for receiving information regarding a first image captured at the intersection and a second image captured at another intersection, Information relating to the second image, which includes the vehicle type, body color, information contained in the license plate, and passengers obtained from the second image taken at the other intersection, Processing means that uses feature information based on the second image associated with the vehicle received from the other intersection and the first image to perform recognition processing of a vehicle that has newly passed through the other intersection and the current intersection, and performs recognition processing of additional vehicle features indicating information including at least one of the vehicle type, body color, information contained in the license plate, and passengers indicated by the feature information, and transmits the information obtained from the first image by the recognition processing to an intersection identified based on the direction of travel of the vehicle, in addition to the information associated with the vehicle based on the second image acquired at the other intersection, A computer program that executes an action.

Images