Intersection edge device, method, and computer program

Abstract

To provide an intersection edge terminal that reduces the computational load of recognition processing when recognizing objects around intersections.SOLUTION: An intersection edge terminal includes: a communication unit that receives images captured at an intersection; and a processing unit that detects the presence of a high-speed moving vehicle in the images and performs recognition processing of the high-speed moving vehicle using information associated with the high-speed moving vehicle acquired at other intersections.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to intersection edge terminals, methods and computer programs. [Background technology]

[0002] Patent Document 1 discloses a technique in which an object detection device placed at an intersection generates movement information and transmits it to an object detection device placed near an adjacent intersection.

[0003] In recent years, the use of IT and AI has led to smart applications that are spreading to all industries, including transportation, which is a part of social infrastructure. In particular, the use of AI has made tremendous progress since the advent of Transformer technology, and large-scale, highly accurate AI engines such as GPT have emerged, which are expected to be used in a wide range of fields in the future. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2022-037998 Summary of the Invention [Problem to be solved by the invention]

[0005] In the aforementioned transportation infrastructure applications, reliable real-time processing is a very important factor, and when using cloud-based AI such as GPT from remote locations, there are various issues regarding processing time and response requirements, including delays caused by communication networks.

[0006] Applications of smart intersections utilizing images from cameras installed at intersection edge terminals include intersection status recognition and monitoring, efficient and safe traffic signal control using recognition information, information transmission to intersection users, and issue identification and resolution related to intersection safety, efficiency, and convenience. For these applications, images collected by cameras must be processed with higher resolution and precision, reliably, and able to recognize vehicles, pedestrians, pets, and backgrounds, and the results must be utilized for various purposes. To meet this demand, methods such as increasing the number of cameras, increasing the camera resolution, increasing the number of frames processed per unit time, and increasing the parameters of AI processing models have been considered. However, all of these methods significantly increase the computational load on intersection edge terminals. Furthermore, for vehicles (high-speed moving objects) at intersections, the aforementioned massive amount of information processing must be performed within the extremely limited time they are present, necessitating even greater difficulty and larger-scale processing in terms of the amount of computational processing per hour. To address this massive computational load per hour using only edge terminals installed at intersections, a large number of hardware resources, such as high-performance CPUs, GPUs, and memory, would be required, leading to issues such as larger equipment and greater power consumption.

[0007] An example of an objective of the present disclosure is to provide an intersection edge terminal, a method, and a computer program that can solve the above-mentioned problems. [Means for solving the problem]

[0008] An intersection edge terminal according to one aspect of the present disclosure includes a communication unit that receives an image captured at the intersection, and a processing unit that detects that the image contains a high-speed moving object and performs a recognition process for the high-speed moving object using information associated with the high-speed moving object obtained at another intersection.

[0009] A method according to one aspect of the present disclosure includes a communication unit of an intersection edge terminal receiving an image captured at the intersection, a processing unit of the intersection edge terminal detecting that the image contains a high-speed moving object, and the processing unit performing a recognition process for the high-speed moving object using information associated with high-speed moving objects obtained at other intersections.

[0010] A computer program according to one aspect of the present disclosure causes an intersection edge terminal to execute a communication means for receiving an image captured at the intersection, and a processing means for detecting that the image contains a high-speed moving object and for performing a recognition process for the high-speed moving object using information associated with the high-speed moving object obtained at another intersection. [Effects of the Invention]

[0011] According to the above aspect, it is possible to significantly reduce the amount of calculation required for object recognition processing at intersection edge terminals, and to achieve efficient object recognition processing. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a schematic diagram showing the external configuration of a system for recognizing the state of an intersection according to the present disclosure; [Figure 2] FIG. 2 is a diagram illustrating the functional configuration of an intersection edge terminal for recognizing the state of an intersection according to the present disclosure. [Figure 3] FIG. 1 is a flow diagram of a method for recognizing intersection conditions according to the present disclosure. [Figure 4] FIG. 10 is a diagram illustrating an example of detailed information according to the present disclosure. [Figure 5] FIG. 1 is a flow diagram of an abstraction classification process according to the present disclosure. [Figure 6] 1 is a diagram illustrating a functional configuration of a system for recognizing the state of an intersection according to the present disclosure. [Figure 7] FIG. 1 is a flow diagram of a method for recognizing intersection conditions according to the present disclosure. [Figure 8] FIG. 10 is a diagram illustrating an example of detailed information according to the present disclosure. [Figure 9]FIG. 1 is a diagram illustrating an example of the placement of tied cameras according to the present disclosure. [Figure 10] FIG. 10 is a diagram illustrating another example of the configuration of an intersection edge terminal according to the present disclosure. [Figure 11] FIG. 10 illustrates another example of a method for recognizing an intersection state according to the present disclosure. [Figure 12] FIG. 1 illustrates an example of a computer configuration according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0013] Each embodiment will be described below with reference to the drawings. In all drawings, the same or corresponding components are designated by the same reference numerals, and common descriptions will be omitted.

[0014] (Overall composition) 1 is a schematic diagram showing the external configuration of a system 1 for recognizing the state of an intersection according to the present disclosure. In FIG. 1, an intersection having a traffic light 11 is shown.

[0015] The system 1 includes at least one camera 2 and an intersection edge terminal 3. The camera 2 acquires images of the intersection. The camera 2 is arranged in a position where it can monitor fixed objects such as the road, crosswalk, and guardrail at the intersection, as well as moving objects passing through the intersection such as automobiles 13, motorcycles or bicycles 14, pedestrians 15, and pets. The camera 2 may be installed on a traffic light 11 or a support pole 12 at the intersection. The intersection edge terminal 3 may be installed on a traffic light 11 or a support pole 12 at the intersection. The intersection edge terminal 3 acquires image information about the state of the intersection from the camera 2 attached to the traffic light 11 and / or the support pole 12 as an input source.

[0016] The image may include multiple objects including a car 13, a motorbike or bicycle 14, and a pedestrian 15. The multiple objects may also include any object such as a house or an animal such as a pet. The intersection edge terminal 3 can communicate with other intersection edge terminals installed at nearby intersections 5. The intersection edge terminal 3 can communicate with nearby road cameras 6 installed on nearby roads. The above-mentioned communication is performed via a network 4 by wireless communication or wired communication.

[0017] First Embodiment Hereinafter, an embodiment according to the present disclosure will be described with reference to FIGS. 2 to 4. FIG.

[0018] (Functional configuration) FIG. 2 is a diagram showing the functional configuration of the intersection edge terminal 3 for recognizing the state of an intersection according to the present disclosure. The intersection edge terminal 3 includes a processing unit 31 and a communication unit 32. The processing unit 31 may be a processor such as a CPU or a GPU. The communication unit 32 can communicate with any device other than the intersection edge terminal 3 provided in the communication unit 32. For example, the communication unit 32 can communicate with the camera 2 in the system 1, other intersection edge terminals at nearby intersections 5, etc.

[0019] (Processing flow) FIG. 3 is a flow diagram illustrating the operation of the intersection edge terminal 3 according to one embodiment of the present disclosure. First, the communication unit 32 acquires information about a high-speed moving object acquired by another intersection edge terminal at another nearby intersection 5 (step S301). For example, the information may be detailed information indicated by object ID1 and object ID2 in Fig. 4. For example, the detailed information may include the vehicle model, the occupants, their facial expressions, etc. The camera 2 captures an image of the current intersection (step S302). The camera 2 transmits the captured image to the communication unit 32.

[0020] Next, the processing unit 31 determines whether or not a high-speed moving object is included in the acquired image (step S303). For example, the processing unit 31 may perform an AI object detection process to extract an object from the image and determine whether or not the extracted object is included in the image by recognizing whether the extracted object corresponds to the background, a pedestrian, a high-speed moving object such as a vehicle, etc. The above-described AI object detection process may be performed based on a trained neural network acquired by deep learning.

[0021] If it is determined that the image does not include a high-speed moving object (step S303: NO), the processing unit 31 returns to the execution of step S302. If it is determined that the image includes a high-speed moving object (step S303: YES), the processing unit 31 determines whether or not the information about the high-speed moving object acquired by the other intersection edge terminals includes information corresponding to the high-speed moving object acquired by the current intersection edge terminal 3 (step S304). For example, the intersection edge terminal 3 may execute the determination of step S304 by detecting the type of vehicle, body color, license plate, etc. of the high-speed moving object and comparing it with information from other intersection edge terminals. If it is determined that the information from the other intersection edge terminals does not include information corresponding to the high-speed moving object acquired by the current intersection edge terminal 3 (step S304: NO), the processing unit 31 returns to the execution of step S302.

[0022] If it is determined that the information from the other intersection edge terminals includes corresponding information (step S304: YES), the processing unit 31 acquires information associated with the high-speed moving object from the information from the other intersection edge terminals (step S305).The processing unit 31 executes a high-speed moving object recognition process using the information associated with the high-speed moving object (step S306).

[0023] The processing unit 31 acquires detailed information about the high-speed moving object through the recognition process of step S305 (step S307). For example, the detailed information acquired by the current intersection edge terminal 3 may be information added to the detailed information acquired by another intersection edge terminal in step S301. For example, the added information may be a characteristic image of the vehicle, the vehicle model, or the occupants and their facial expressions.

[0024] Furthermore, the added information may be used to complement missing parts of license plates, as indicated by the "??" in Object ID2 in Figure 4. The missing information to be complemented is not limited to license plate information. This makes it possible to retain and utilize complete processing results of detailed and unique object information as the vehicle passes through multiple intersections, addressing the issue of computational processing and image collection not being completed at a single intersection. This function is also expected to enable data complementation in the event of unexpected information collection or processing stagnation at individual intersections, improving system redundancy and reliability.

[0025] The processing unit 31 performs a synthesis process based on the detailed information acquired in step S307 (step S308). The processing unit 31 may execute the synthesis process in accordance with various applications. For example, the processing unit 31 may execute the synthesis process based only on necessary information from the detailed information in accordance with applications such as traffic accident analysis and traffic light control. Furthermore, the synthesized information may be used for various applications such as traffic light display switching timing control, providing information to intersection users before entering, accident and violation monitoring, accident analysis, traffic problem extraction and response, image confirmation, and search. Furthermore, the synthesized information may be stored for these applications.

[0026] (Action, effect) As described above, the intersection edge terminal 3 includes a communication unit 32 that receives an image captured at the intersection, and a processing unit 31 that detects that the image contains a high-speed moving object and performs a recognition process for the high-speed moving object using information associated with the high-speed moving object obtained at another intersection.

[0027] That is, before a high-speed moving object entering the current intersection enters the current intersection, the results of processing already performed at other nearby intersections 5 become available to the intersection edge terminal 3, which significantly reduces the amount of calculation required for object recognition processing at the intersection edge terminal 3 and enables efficient object recognition processing. This eliminates the need for larger devices or large amounts of power consumption, and makes it possible to realize high-load calculations such as an increase in the number of cameras, higher resolution camera pixels, an increase in the number of processed frames per unit time, and an increase in the number of parameters for the AI ​​processing model.

[0028] <First Modification of First Embodiment> A modified example of an embodiment according to the present disclosure will be described below with reference to FIG. FIG. 5 is a flow diagram of an abstraction classification process according to the present disclosure.

[0029] First, the camera 2 captures an image including an object (step S501). The camera 2 transmits the captured image to the communication unit 32.

[0030] The processing unit 31 detects objects in the image received via the communication unit 32 (step S502). For example, the processing unit 31 performs AI object detection processing to extract objects from the image. The processing unit 31 then performs abstraction classification processing to abstract the extracted objects and classify them into background, pedestrians, fast-moving objects such as vehicles, and the like.

[0031] Next, the intersection edge terminal 3 determines whether or not a request for detailed recognition processing has been received at the communication unit 32 (step S503). For example, a sound sensor linked to the processing unit 31 detects the occurrence of an emergency such as a traffic accident based on the detected volume, etc. At this time, the sound sensor transmits a request for detailed recognition processing to the communication unit 32. The emergency may be detected by various sensors such as light, vibration, distance, laser, and ultrasonic sensors.

[0032] If a request for detailed recognition processing has been received (step S503: YES), the intersection edge terminal 3 executes detailed recognition processing (steps S302 to S307). Furthermore, the processing unit 31 may execute detailed recognition processing (steps S302 to S307) based on the information about the object acquired in step S502. That is, when a high-speed moving object is detected in step S502, the processing unit 31 may execute detailed recognition processing from steps S304 to S307. Further, the detailed recognition process may be performed by a simple process of superimposing the detailed recognition process on the information acquired in the object extraction process in the abstract classification process.

[0033] If a request for detailed recognition processing has not been received (step S503: NO), the processing unit 31 continues the abstraction classification processing (steps S504 to S506).

[0034] The processing unit 31 recognizes the relative relationship between the objects (step S504). For example, the processing unit 31 measures the distance between multiple pedestrians on the road (background) or the distance between multiple automobiles.

[0035] The processing unit 31 executes a traffic condition recognition process (step S505). For example, in response to the short distance between the multiple pedestrians measured in step S504, the processing unit 31 detects that pedestrians are concentrated at the intersection where the intersection edge terminal 3 is located. In addition, in response to the short distance between the multiple vehicles, the processing unit 31 detects that there is a high possibility of traffic congestion occurring at the intersection where the intersection edge terminal 3 is located, or that traffic congestion has already occurred.

[0036] The processing unit 31 executes a process for effectively utilizing the acquired traffic condition information (step S506). For example, when detecting that pedestrians are crowded at an intersection where the intersection edge terminal 3 is located, the processing unit 31 may control the traffic light by lengthening the time that the green light of the pedestrian traffic light is on so as to eliminate the crowding. Also, when detecting that there is a high possibility of traffic congestion occurring or that traffic congestion has already occurred, the processing unit 31 may control the traffic light by lengthening the time that the green light of the automobile traffic light is on so as to avoid the traffic congestion.

[0037] (Action, effect) As described above, the processing unit 31 of the intersection edge terminal 3 further detects multiple objects, performs abstraction classification processing to abstract and classify the multiple objects, and, in response to the detection of a predetermined event at the intersection that requires detailed recognition processing, performs detailed recognition processing on the multiple objects detected in the abstraction classification processing.

[0038] This allows the system to have two separate functions, abstract classification processing, which requires a light amount of calculation, and detailed recognition processing, and to have two processing configurations where each function operates independently and where they are combined to process the results, thereby optimizing the amount of calculation processing according to the conditions of the application.In other words, the load on the intersection edge terminal 3 can be reduced according to the traffic state and the application, making it possible to optimize the amount of calculation processing.

[0039] <Second Modification of First Embodiment> The information from the other intersection edge terminals of the neighboring intersection 5 may include the recording time at the other intersection edge terminals and the recorded speed of the vehicle, etc. The processing unit 31 may predict at least one of the order and time of entry of the vehicle, etc., based on the distance from the current intersection to the neighboring intersection 5, etc. The processing unit 31 may prepare and execute the recognition process based on at least one of the predicted order and time of entry of vehicles, etc. For example, the processing unit 31 may prepare related information in the memory of the intersection edge terminal 3 so as to be able to read and update information on the vehicle predicted to enter the intersection first.

[0040] As described above, the processing unit 31 predicts at least one of the order and time at which a high-speed moving object will enter the current intersection based on information associated with the high-speed moving object obtained at another intersection, and performs recognition processing at the current intersection based on at least one of the predicted order and time.

[0041] This makes it possible to predict the order in which a vehicle will enter the intersection relative to other vehicles and the approximate time of entry when the vehicle enters the intersection in the direction of travel. Having a method of waiting for a vehicle to enter before the vehicle actually does so simplifies and ensures the linking process, and also enables the timely preparation of image unique detailed recognition process results.

[0042] <Third Modification of First Embodiment> After the detailed information is acquired in step S307, the processing unit 31 may transmit the detailed information to other intersection edge terminals at nearby intersections. Here, the intersection edge terminal 3 may obtain the traveling direction of the high-speed moving object and transmit the detailed information only to other intersection edge terminals at nearby intersections corresponding to the traveling direction.

[0043] That is, the intersection edge terminal 3 has a function to send data only in the direction in which each vehicle etc. is leaving the intersection. By having this function, the data will not be sent to unnecessary neighboring intersections 5, or the intersection edge terminal of a neighboring intersection that is determined to be unnecessary may discard the data without receiving it. This makes it possible to significantly reduce the amount of data traffic in the communication network between neighboring intersections, and also to reduce the hardware resources required for processing transmission and reception by the intersection edge terminals 3 between neighboring intersections.

[0044] Second Embodiment An embodiment of the present disclosure will be described below with reference to FIGS.

[0045] (Functional configuration) Fig. 6 is a diagram showing the functional configuration of system 1 according to an embodiment of the present disclosure. The functional configuration common to Fig. 2 is the same as the functional configuration of Fig. 2, so differences will be described below.

[0046] The intersection edge terminal 3 includes a memory 33 and a cooling unit 34. As a result, the cooling unit 34 can prevent breakdowns due to heat even in harsh environments such as outdoor environmental temperature conditions.

[0047] The memory 33 may be any storage medium, such as a hard disk drive (HDD), a solid state drive (SSD), or a non-volatile memory express (NVMe). The cooling unit 34 may be any cooling device.

[0048] The intersection edge terminal 3 communicates with at least one of the nearby intersections 5 and the nearby road cameras 6 via the communication unit 32 and the network 4. The communication is performed by wireless communication or wired communication. The nearby intersection 5 may indicate an adjacent intersection or a non-adjacent but nearby intersection. The nearby road camera 6 is a camera installed on a traffic light 11 or a pole 12 of a road that is not an intersection.

[0049] The system 1 further includes at least one of a tied camera 7 , a tied wireless terminal 8 , a tied in-vehicle terminal 9 , and a tied satellite terminal 10 .

[0050] The tied camera 7 is placed at a different position from the camera 2 at the intersection where the camera 2 corresponding to the intersection edge terminal 3 is placed. The tied camera 7 is tied to the camera 2. As shown in FIG. 9 , the tied camera 7 can capture an image of a subject from a different angle from the tied camera 2 at the same time or almost the same time. For example, one method for implementing the linking processing function is to use image recognition of vehicle license plates. If a camera 2 within an intersection is simply installed from above on a traffic light 11 or a support pole 12 within the intersection, when multiple vehicles enter the intersection in a line, the vehicle in front will be in the blind spot, and the camera 2 will only be able to confirm the vehicle license plate after the vehicle in front has passed just before entering the intersection, resulting in a very short processing time after linking. As a means of solving this problem, the linking camera 7 for vehicle linking can be positioned and oriented so that the vehicle traveling just ahead of the target vehicle passes by, and the vehicle license plate is not in the blind spot of the vehicle in front. The tied camera 7 may be installed below the road surface just before the intersection, with the camera's imaging surface facing upward and diagonally away from the intersection, or it may be installed on the side of the road just before the intersection, with the camera's imaging surface facing diagonally away from the intersection and away from the vehicle.

[0051] As described above, the image is acquired by the camera 2 at the current intersection, and the communication unit 32 further acquires a linked image from the linked camera 7 linked to the camera 2, the linked camera 7 being located at a different position from the camera 2. The processing unit 31 executes recognition processing based on the image and the linked image. This increases the probability of acquiring vehicle information even when the vehicle ahead has already left the intersection and the processing time after linking is extremely short.

[0052] The tied wireless terminal 8 is a short-range wireless terminal mounted on a vehicle. For example, the linked wireless terminal 8 is a short-distance wireless terminal mounted on a vehicle that has strong directivity over short distances and is capable of high-speed processing, as used in ETC systems on toll roads, etc. The system 1 and the vehicle are each equipped with wireless equipment for short-distance wireless communication, and the intersection edge terminal 3 recognizes each vehicle's unique ID for linking, links it with image-processed vehicle information, and sends this linked set of information to other nearby intersections 5. Next, at the nearby intersection 5 to which the vehicle has moved, the vehicle can be identified from the unique ID for linking and the sent vehicle information can be used.

[0053] The tied in-vehicle terminal 9 is a drive recorder or a vehicle-mounted camera of an automobile 13 entering an intersection, and is tied to the intersection edge terminal 3. For example, the linked vehicle-mounted terminal 9 receives information from a drive recorder or a vehicle-mounted camera such as an autonomous driving camera, and wirelessly transmits information such as the license plate numbers of the vehicles in front and behind the vehicle and whether the vehicles have been switched to the intersection edge terminal 3 from the vehicle before entering the intersection. This simplifies and ensures the linking process, and enables the timely preparation of image unique detailed recognition process results.

[0054] The linked satellite terminal 10 is a satellite terminal linked to the intersection edge terminal 3. The linked satellite terminal 10 transmits satellite images to the intersection edge terminal 3. The intersection edge terminal 3 is able to trace the movement state of each vehicle from the satellite images, collect image unique detailed recognition processing information collected at other nearby intersections 5 before each vehicle enters each intersection, and perform linking processing. This simplifies and ensures the linking process, and enables the timely preparation of image unique detailed recognition process results.

[0055] The system 1 may further include other sensors, such as sound, light, vibration, distance, laser, ultrasonic, and other sensors, to monitor the state of the intersection.

[0056] (Processing flow) FIG. 7 is a flow diagram illustrating the operation of the intersection edge terminal 3 according to one embodiment of the present disclosure. First, the camera 2 acquires an image including an object captured at the current intersection (step S701). The camera 2 transmits the acquired image to the communication unit 32.

[0057] Next, the processing unit 31 detects objects included in the received image (step S702). For example, the processing unit 31 performs AI object detection processing to extract objects from the image, recognize whether the extracted objects correspond to the background, a vehicle, or a pedestrian, and uniquely identify each of them. In this embodiment, it is assumed that the processing unit 31 recognizes four objects. The processing unit 31 assigns unique identifiers (object ID1 to object ID4) to each of the four recognized objects. The above-mentioned AI object detection processing may be performed based on a trained neural network acquired by deep learning.

[0058] Next, the processing unit 31 calculates the amount of time variation for each of the detected objects (step S703). For example, the processing unit 31 calculates the difference in the position of each object contained in a first image when comparing the first image with a second image captured a certain time after the first image was captured, and calculates the amount of time variation. The amount of time variation may be calculated by averaging the amounts of time variation in three or more series of images. For objects recognized as background in step S702, the calculation process of the amount of time variation may be omitted in order to reduce the amount of calculation.

[0059] The processing unit 31 classifies each object into a predetermined category based on the amount of time variation (step S704). For example, the predetermined categories are four: fixed background, temporary stationary object, slow-moving object, and fast-moving object. The number of predetermined categories may be two or more. The predetermined categories may be classified into categories with different contents other than the above four.

[0060] For example, when comparing a second image captured after a predetermined period (e.g., 5 seconds) has elapsed since the first image was captured, if the value of the time variation of the first object corresponding to object ID 1 during the predetermined period is 0 m, the first object is classified into a first category indicating a fixed background. When the second object corresponding to object ID2 has a speed of 0 m / s during a predetermined period, and during a certain period after the predetermined period has elapsed, the speed is greater than 0 m / s and less than 1.3 m / s, the second object is classified into a second category indicating a temporarily stationary object. When the third object corresponding to object ID3 moves at a speed greater than 1.3 m / s and less than 5.0 m / s during the predetermined period, the third object is classified into a third category indicating a slow-moving object. If the fourth object corresponding to object ID4 moves at a speed greater than 5.0 m / s during a predetermined period, the fourth object is classified into a fourth category indicating a high-speed moving object.

[0061] The processing unit 31 recognizes each of the plurality of objects according to a predetermined recognition method corresponding to the classified category (steps S705-1 to S705-4). For example, the processing unit 31 recognizes the first object classified into the first category according to a method for recognizing a fixed background. For example, the processing unit 31 recognizes the first object at a low resolution and over a long period of time. Furthermore, the processing unit 31 recognizes the second object classified into the second category according to a method for recognizing a temporary stationary object. For example, the method for recognizing a temporary stationary object may specify recognition at a higher resolution and shorter period than the method for recognizing a fixed background described above. Furthermore, the processing unit 31 recognizes the third object classified into the third category according to a method for recognizing a slow-moving object. For example, the method for recognizing a slow-moving object may specify recognition at a higher resolution and shorter period than the method for recognizing a temporarily stationary object described above. Unlike the method for recognizing a fixed background and the method for recognizing a temporarily stationary object, the method for recognizing a slow-moving object may acquire color information of the third object. Furthermore, the processing unit 31 recognizes the fourth object classified into the fourth category according to a method for recognizing a high-speed moving object. For example, the method for recognizing a high-speed moving object may specify recognition at a higher resolution and a shorter period than the method for recognizing a low-speed moving object described above. Unlike the method for recognizing a fixed background and the method for recognizing a temporarily stationary object, the method for recognizing a high-speed moving object may acquire color information of the fourth object. Furthermore, unlike the method for recognizing a fixed background and the method for recognizing a low-speed moving object, the method for recognizing a high-speed moving object may acquire information about the license plate of the vehicle or images of the vehicle from multiple directions. The differences in the settings for each recognition method should not be limited to the resolution and period described above, and any settings may be used. Furthermore, the detailed settings for the recognition methods do not necessarily need to be different between different recognition methods. For example, the resolution setting for the method for recognizing a fixed background and the resolution setting for the method for recognizing a temporarily stationary object may be the same. Each recognition method may be performed in parallel.

[0062] The processing unit 31 acquires detailed information by performing recognition processing corresponding to each recognition method in steps S705-1 to S705-4 (step S706). For example, the detailed information may be as shown in FIG. 8, but is not limited to this form of information, and may also include other information such as the vehicle model, the occupants, and their facial expressions.

[0063] The processing unit 31 performs a synthesis process based on the detailed information acquired in step S706 (step S707). The processing unit 31 may execute the synthesis process in accordance with various applications. For example, if it is desired to use only information about pedestrians at an intersection, the synthesis process may be performed only on detailed information about objects in the second category. If it is desired to use the information for traffic accident analysis, the synthesis process may be performed based on detailed information about objects in the third and fourth categories. If it is desired to use the information for traffic light control, the synthesis process may be performed only on detailed information about objects in the second to fourth categories. The synthesized information may be used for various applications, such as controlling the timing of switching traffic light displays, providing information to intersection users before entering, monitoring accidents and violations, accident analysis, traffic issue extraction and response, image confirmation, and search. The information may also be stored for these applications.

[0064] (Action, effect) As described above, the communication unit 32 receives multiple images captured at the current intersection, and the processing unit 31 detects objects contained in the multiple images, calculates the amount of change over time of the objects based on the multiple images, classifies the objects into predetermined categories based on the amount of change over time, and performs object recognition processing in accordance with a predetermined recognition method corresponding to the category.

[0065] That is, the intersection edge terminal 3 has the function of classifying objects from the image data collected from the camera according to the amount of time fluctuation of each object per unit of time, such as high-speed moving objects such as cars 13 and motorbikes, low-speed moving objects such as bicycles 14, pedestrians 15 and pets, temporary fixed objects (temporarily stationary objects) such as garbage and fallen objects within the intersection, and fixed background information such as the road surface, signs, telegraph poles, traffic light poles and guardrails that are always present. Furthermore, the object recognition processing cycle and recognition method differ for each classified category, and the recognition processing is performed at different cycles according to the changes in the state and position of each object.

[0066] This allows recognition processing to be performed using the optimum recognition method for each object, thereby reducing the amount of calculation required for recognition processing at intersection edge terminals and achieving efficient recognition of objects in images.

[0067] <Modification of the second embodiment> The intersection edge terminal of the nearby intersection 5 or the terminal of the nearby road camera 6 may communicate with the vehicle-mounted terminal (linked wireless terminal 8) and receive a vehicle identifier (for example, a vehicle-specific identifier such as a serial number) from the vehicle-mounted terminal. At this time, the intersection edge terminal of the nearby intersection 5 or the terminal of the nearby road camera 6 links the received vehicle identifier to the detailed information acquired by the terminal. The intersection edge terminal of the nearby intersection 5 or the terminal of the nearby road camera 6 transmits the linked detailed information to the intersection edge terminal 3 of the nearby intersection. The intersection edge terminal 3 receives the linked detailed information and performs detailed recognition processing using the linked detailed information.

[0068] In the above explanation, the linking process is performed by the intersection edge terminal of the nearby intersection 5 or the terminal of the nearby road camera 6 communicating with the vehicle-mounted terminal. However, the intersection edge terminal 3 shown in FIG. 1, which has the same configuration as the intersection edge terminal of the nearby intersection 5, may also perform the same process. That is, the communication unit 32 of the intersection edge terminal 3 receives a vehicle identifier from the vehicle-mounted terminal, and the processing unit 31 links the vehicle identifier to the information generated by the recognition process and transmits the linked information to other intersection edge terminals.

[0069] This can ensure unique identification of the vehicle and improve the reliability of the information obtained.

[0070] Third Embodiment 10 is a diagram showing another example of the configuration of an intersection edge terminal according to the present disclosure. In the configuration shown in FIG.

[0071] With this configuration, the communication unit 102 receives an image captured at the current intersection. The processing unit 101 detects that the image contains a high-speed moving object, and performs a recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at another intersection.

[0072] According to the intersection edge terminal 100, the results of processing already performed at other nearby intersections 5 before a high-speed moving object entering the intersection can be used by the intersection edge terminal 3, so the amount of calculation required for object recognition processing by the intersection edge terminal 3 can be significantly reduced and efficient object recognition processing can be achieved. This eliminates the need for larger devices or large amounts of power consumption, and makes it possible to achieve high-load calculations such as an increase in the number of cameras, higher resolution camera pixels, an increase in the number of processed frames per unit time, and an increase in the parameters of AI processing models.

[0073] <Fourth embodiment> FIG. 11 is a diagram illustrating another example of a method for recognizing the state of an intersection according to the present disclosure. 11 includes a step S1101 in which the communication unit 102 of the intersection edge terminal receives an image captured at the intersection. The method includes a step S1102 in which the processing unit 101 of the intersection edge terminal detects that the image contains a high-speed moving object. The method also includes a step S1103 in which the processing unit 101 executes a recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at another intersection.

[0074] According to the method shown in Fig. 11, the results of processing already performed at other nearby intersections 5 before a high-speed moving object entering the current intersection are available to the intersection edge terminal 3, so the amount of calculation required for object recognition processing at the intersection edge terminal 3 can be significantly reduced and efficient object recognition processing can be achieved. This eliminates the need for larger devices or large amounts of power consumption, and makes it possible to achieve high-load calculations such as an increase in the number of cameras, higher resolution camera pixels, an increase in the number of processed frames per unit time, and an increase in the number of parameters for the AI ​​processing model.

[0075] FIG. 12 is a diagram illustrating an example of the configuration of a computer according to the present disclosure. In the configuration shown in FIG. 12, a computer 1200 includes a CPU 1201 , a main memory device 1202 , an auxiliary memory device 1203 , an interface 1204 , and a non-volatile recording medium 1205 .

[0076] One or more of the intersection edge terminal 3 and the intersection edge terminal 100, or a part thereof, may be implemented in a computer 1200. In this case, the operation of each of the above-mentioned processing units is stored in the auxiliary storage device 1203 in the form of a program. The CPU 1201 reads the program from the auxiliary storage device 1203, loads it into the main storage device 1202, and executes the above-mentioned processing in accordance with the program. The CPU 1201 also allocates storage areas in the main storage device 1202 corresponding to each of the above-mentioned storage units in accordance with the program. Communication between each device and other devices is executed by the interface 1204, which has a communication function and performs communication under the control of the CPU 1201. The interface 1204 also has a port for the nonvolatile storage medium 1205, and reads information from the nonvolatile storage medium 1205 and writes information to the nonvolatile storage medium 1205.

[0077] When the intersection edge terminal 3 is implemented in the computer 1200, the operations of the processing unit 31 and each unit thereof are stored in the form of a program in the auxiliary storage device 1203. The CPU 1201 reads the program from the auxiliary storage device 1203, loads it into the main storage device 1202, and executes the above-mentioned processing in accordance with the program.

[0078] Furthermore, the CPU 1201 allocates a storage area in the main memory device 1202 for the intersection edge terminal 3 to perform processing in accordance with the program. Communication with other devices via the communication unit 32 is performed by the interface 1204, which has a communication function and operates under the control of the CPU 1201. Interaction between the intersection edge terminal 3 and a user is performed by the interface 1204, which has an input device and an output device, presenting information to the user via the output device under the control of the CPU 1201 and accepting user operations via the input device.

[0079] When the intersection edge terminal 100 is implemented in a computer 1200, the operation of the processing unit 101 is stored in the form of a program in the auxiliary storage device 1203. The CPU 1201 reads the program from the auxiliary storage device 1203, loads it into the main storage device 1202, and executes the above-mentioned processing in accordance with the program.

[0080] Furthermore, the CPU 1201 allocates a storage area in the main memory device 1202 for the intersection edge terminal 100 to perform processing in accordance with a program. Communication between the intersection edge terminal 100 and other devices is carried out by the interface 1204, which has a communication function and operates under the control of the CPU 1201. Interaction between the intersection edge terminal 100 and a user is carried out by the interface 1204, which has an input device and an output device, presenting information to the user via the output device under the control of the CPU 1201 and accepting user operations via the input device.

[0081] One or more of the above-described programs may be recorded on the nonvolatile recording medium 1205. In this case, the interface 1204 may read the program from the nonvolatile recording medium 1205. The CPU 1201 may then directly execute the program read by the interface 1204, or may temporarily store the program in the main storage device 1202 or the auxiliary storage device 1203 and then execute it.

[0082] It is also possible to record a program for executing all or part of the processing performed by the intersection edge terminal 3 and the intersection edge terminal 100 on a computer-readable recording medium, and have a computer system read and execute the program recorded on the recording medium to perform the processing of each part. Note that the term "computer system" here includes the OS (Operating System) and hardware such as peripheral devices. Furthermore, "computer-readable recording media" refers to portable media such as flexible disks, optical magnetic disks, ROMs (Read Only Memory), and CD-ROMs (Compact Disc Read Only Memory), as well as storage devices such as hard disks built into computer systems. The program may be one that realizes part of the aforementioned functions, or may be one that can realize the aforementioned functions in combination with a program already stored in the computer system.

[0083] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0084] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0085] (Appendix 1) a communication unit that receives an image captured at the current intersection; Detecting that the image contains a fast moving object; Execute a recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at another intersection. Processing section and An intersection edge terminal comprising:

[0086] (Appendix 2) The processing unit further comprises: Detecting that the image contains an object; Abstracting and classifying the object; performing the recognition processing on the classified object in response to detection of a predetermined event at the current intersection; 1. An intersection edge terminal as described in Appendix 1.

[0087] (Appendix 3) The communication unit further comprises: receiving a plurality of images captured at the intersection; The processing unit further comprises: Detecting an object contained in the plurality of images; calculating a time variation of the object based on the plurality of images; classifying the object into a predetermined category based on the amount of time variation; performing a recognition process for the object in accordance with a predetermined recognition method corresponding to the category; 3. An intersection edge terminal as defined in claim 1 or 2.

[0088] (Appendix 4) performing a recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at another intersection, and when the information associated with the high-speed moving object includes a missing portion, complementing the missing portion by the recognition processing. 4. The intersection edge terminal of any one of claims 1 to 3.

[0089] (Appendix 5) The processing unit predicting at least one of the order and time at which the high-speed moving object will enter the intersection based on the information associated with the high-speed moving object acquired at another intersection; performing the recognition process at the current intersection based on at least one of the predicted order and time; 5. The intersection edge terminal of any one of claims 1 to 4.

[0090] (Appendix 6) The processing unit acquiring a direction of travel of the high-speed moving object; The communication unit transmitting information generated by the recognition processing of the high-speed moving object to only other intersection edge terminals of the neighboring intersection corresponding to the traveling direction; 6. The intersection edge terminal of any one of Supplementary Notes 1 to 5.

[0091] (Appendix 7) the image is acquired by a camera at the current intersection, the communication unit further acquires a linked image from a linked camera that is linked to the camera, the linked camera being located at a different position from the camera; the processing unit executes the recognition process based on the image and the associated image. 7. The intersection edge terminal of any one of claims 1 to 6.

[0092] (Appendix 8) the communication unit receives a vehicle identifier from a vehicle-mounted terminal; the processing unit associates the vehicle identifier with the information generated by the recognition process, and transmits the associated information to another intersection edge terminal. 8. The intersection edge terminal of any one of Supplementary Notes 1 to 7.

[0093] (Appendix 9) a communication unit of the intersection edge terminal receiving an image captured at the intersection; a processing unit of the intersection edge terminal detecting that the image contains a high-speed moving object; the processing unit executes a recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at another intersection; A method comprising:

[0094] (Appendix 10) The processing unit further comprises: Detecting that the image contains an object; Abstracting and classifying the object; performing the recognition processing on the classified object in response to detection of a predetermined event at the current intersection; The method described in Appendix 9.

[0095] (Appendix 11) The communication unit further comprises: receiving a plurality of images captured at the intersection; The processing unit further comprises: Detecting an object contained in the plurality of images; calculating a time variation of the object based on the plurality of images; classifying the object into a predetermined category based on the amount of time variation; performing a recognition process for the object in accordance with a predetermined recognition method corresponding to the category; 11. The method of claim 9 or 10.

[0096] (Appendix 12) performing a recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at another intersection, and when the information associated with the high-speed moving object includes a missing portion, complementing the missing portion by the recognition processing. 12. The method of any one of appendices 9 to 11.

[0097] (Appendix 13) The processing unit predicting at least one of the order and time at which the high-speed moving object will enter the intersection based on the information associated with the high-speed moving object acquired at another intersection; performing the recognition process at the current intersection based on at least one of the predicted order and time; 13. The method of any one of appendices 9 to 12.

[0098] (Appendix 14) The processing unit acquiring a direction of travel of the high-speed moving object; The communication unit transmitting information generated by the recognition processing of the high-speed moving object to only other intersection edge terminals of the neighboring intersection corresponding to the traveling direction; 14. The method of any one of appendices 9 to 13.

[0099] (Appendix 15) the image is acquired by a camera at the current intersection, the communication unit further acquires a linked image from a linked camera that is linked to the camera, the linked camera being located at a different position from the camera; the processing unit executes the recognition process based on the image and the associated image. 15. The method of any one of appendices 9 to 14.

[0100] (Appendix 16) the communication unit receives a vehicle identifier from a vehicle-mounted terminal; the processing unit associates the vehicle identifier with the information generated by the recognition process, and transmits the associated information to another intersection edge terminal. 16. The method of any one of appendices 9 to 15.

[0101] (Appendix 17) At the intersection edge terminal, a communication means for receiving an image captured at the current intersection; Detecting that the image contains a fast moving object; Execute a recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at another intersection. Processing means and A computer program that executes the following: (Appendix 18) The processing means further comprises: Detecting that the image contains an object; Abstracting and classifying the object; performing the recognition processing on the classified object in response to detection of a predetermined event at the current intersection; 18. The computer program of claim 17.

[0102] (Appendix 19) The communication means further comprises: receiving a plurality of images captured at the intersection; The processing means further comprises: Detecting an object contained in the plurality of images; calculating a time variation of the object based on the plurality of images; classifying the object into a predetermined category based on the amount of time variation; performing a recognition process for the object in accordance with a predetermined recognition method corresponding to the category; 19. The computer program of claim 17 or 18.

[0103] (Appendix 20) performing a recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at another intersection, and when the information associated with the high-speed moving object includes a missing portion, complementing the missing portion by the recognition processing. 20. The computer program of any one of appendices 17 to 19.

[0104] (Appendix 21) The processing means predicting at least one of the order and time at which the high-speed moving object will enter the intersection based on the information associated with the high-speed moving object acquired at another intersection; performing the recognition process at the current intersection based on at least one of the predicted order and time; 21. The computer program of any one of appendices 17 to 20.

[0105] (Appendix 22) The processing means acquiring a direction of travel of the high-speed moving object; The communication means is transmitting information generated by the recognition processing of the high-speed moving object to only other intersection edge terminals of the neighboring intersection corresponding to the traveling direction; 22. The computer program of any one of appendices 17 to 21.

[0106] (Appendix 23) the image is acquired by a camera at the current intersection, the communication means further acquires a linked image from a linked camera linked to the camera, the linked camera being located at a different position from the camera; The processing means executes the recognition process based on the image and the associated image. 23. The computer program of any one of appendices 17 to 22.

[0107] (Appendix 24) the communication means receives a vehicle identifier from a vehicle-mounted terminal; The processing means associates the vehicle identifier with the information generated by the recognition process, and transmits the associated information to another intersection edge terminal. 24. The computer program of any one of appendices 17 to 23.

[0108] (Appendix 25) 1. A system comprising: a camera for capturing images at an intersection; An intersection edge terminal according to any one of Supplementary Notes 1 to 8; A system comprising: [Explanation of symbols]

[0109] 1 System 2 Cameras 3. Intersection edge terminal 4 Network 5 Nearby intersections 6. Neighborhood road cameras 7. Tied Camera 8. Linked wireless terminal 9. Linked in-vehicle terminal 10. Linked satellite terminal 11 Traffic lights 12 pillars 13. Automobiles 14 Bicycle 15 Pedestrians 31 Processing section 32 Communications Department 33 Memory 34 Cooling section 100 Intersection Edge Terminal 101 Processing section 102 Communications Department 1200 Computer 1202 Main storage 1203 Auxiliary storage device 1204 Interface 1205 Non-volatile recording media

Claims

1. a communication unit that receives an image captured at the current intersection; Detecting that the image contains a fast moving object; Execute a recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at another intersection. Processing section and An intersection edge terminal comprising:

2. The processing unit further comprises: Detecting that the image contains an object; Abstracting and classifying the object; performing the recognition processing on the classified object in response to detection of a predetermined event at the current intersection; The intersection edge terminal of claim 1 .

3. The communication unit further comprises: receiving a plurality of images captured at the intersection; The processing unit further comprises: Detecting an object contained in the plurality of images; calculating a time variation of the object based on the plurality of images; classifying the object into a predetermined category based on the amount of time variation; performing a recognition process for the object in accordance with a predetermined recognition method corresponding to the category; The intersection edge terminal of claim 1 .

4. performing a recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at another intersection, and when the information associated with the high-speed moving object includes a missing portion, complementing the missing portion by the recognition processing. The intersection edge terminal of claim 1 .

5. The processing unit predicting at least one of the order and time at which the high-speed moving object will enter the intersection based on the information associated with the high-speed moving object acquired at another intersection; performing the recognition process at the current intersection based on at least one of the predicted order and time; The intersection edge terminal of claim 1 .

6. The processing unit acquiring a direction of travel of the high-speed moving object; The communication unit transmitting information generated by the recognition processing of the high-speed moving object to only other intersection edge terminals of the neighboring intersection corresponding to the traveling direction; The intersection edge terminal of claim 1 .

7. the image is acquired by a camera at the current intersection, the communication unit further acquires a linked image from a linked camera that is linked to the camera, the linked camera being located at a different position from the camera; the processing unit executes the recognition process based on the image and the associated image. The intersection edge terminal of claim 1 .

8. the communication unit receives a vehicle identifier from a vehicle-mounted terminal; the processing unit associates the vehicle identifier with the information generated by the recognition process, and transmits the associated information to another intersection edge terminal. The intersection edge terminal of claim 1 .

9. a communication unit of the intersection edge terminal receiving an image captured at the intersection; a processing unit of the intersection edge terminal detecting that the image contains a high-speed moving object; the processing unit executes a recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at another intersection; A method comprising:

10. At the intersection edge terminal, a communication means for receiving an image captured at the current intersection; Detecting that the image contains a fast moving object; Execute a recognition process for the high-speed moving object using information associated with the high-speed moving object acquired at another intersection. Processing means and A computer program that executes the following:

Images