Assistance control device, assistance control method, and program

Abstract

To reduce the processing load on an assistance control device in preventive safety technology.SOLUTION: An assistance control device includes a first acquisition unit that acquires position information on a first moving object detected from an image captured by an imaging device, a second acquisition unit that acquires position information on second moving objects measured at each of the second moving objects, a recognition processing unit that performs a recognition process to compare the position information on the first moving object with the position information on the second moving objects and recognize a combination of moving objects whose position information has a degree of coincidence higher than a predetermined value as one moving object, and an assistance control unit that performs control for traffic assistance for the moving objects based on the position information on the moving objects obtained by applying the recognition process to the position information on the first moving object and the position information on the second moving objects. The recognition processing unit limits the position information to be subject to the recognition process of the first moving object and the second moving objects to position information within a predetermined range set based on the position of the imaging device.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to an assistance control device, an assistance control method, and a program. [Background technology]

[0002] In recent years, efforts to provide access to sustainable transportation systems that take into consideration vulnerable people among transportation participants have been gaining momentum. To achieve this, we are focusing on research and development to further improve the safety and convenience of transportation through research and development of preventive safety technology. Patent documents 1-6 describe technologies related to vehicle operation support. [Prior art document] [Patent documents] Patent Document 1: JP 2019-175201 A Patent Document 2: JP 2021-92840 A Patent Document 3: JP 2019-78554 A Patent Document 4: International Publication No. 2020-165981 Patent Document 5: International Publication No. 2019-188429 Patent Document 6: JP 2021-167770 A Summary of the Invention [Problem to be solved by the invention]

[0003] However, in preventive safety technology, an increase in the processing load required to perform support is an issue. In order to solve the above issue, the present application aims to reduce the processing load in the support control device, thereby contributing to the development of a sustainable transportation system. [Means for solving the problem]

[0004] In a first aspect of the present invention, an assistance control device is provided. The assistance control device includes a first acquisition unit that acquires position information of a first moving object detected from an image captured by an imaging device. The assistance control device includes a second acquisition unit that acquires position information of the second moving object measured for each of the second moving objects. The assistance control device includes a recognition processing unit that performs a recognition process in which the position information of the first moving object and the position information of the second moving object are compared for each combination of the first moving object and the second moving object, and a combination of moving objects having a degree of coincidence of position information higher than a predetermined value is recognized as one moving object. The assistance control device includes an assistance control unit that performs control for traffic assistance for the moving object based on the position information of the moving object obtained by applying the recognition process to the position information of the first moving object and the position information of the second moving object. The recognition processing unit limits the position information to be subjected to the recognition process among the position information of the first moving object and the position information of the second moving object to position information within a predetermined range set based on the position of the imaging device.

[0005] In the above assistance control device, the predetermined range may be a predetermined fixed range that includes an installation position of the imaging device.

[0006] Any of the above-mentioned assistance control devices may further include a range setting unit that selects at least four position information having the outermost coordinate components for each of two coordinate axes defined within the predetermined range from the position information of the first moving object acquired by the first acquisition unit, and sets a range defined by lines passing through each of the selected position information as the predetermined range.

[0007] In any of the above-mentioned assistance control devices, the recognition processing unit may compare the position indicated by the position information of the first moving object with the position indicated by the position information of the second moving object for each combination of the first moving object and the second moving object, and recognize a combination of moving objects whose positional similarity is higher than a predetermined value as a single moving object.

[0008] In any of the above assistance control devices, the recognition processing unit may recognize, as one moving object, a combination of moving objects in which the degree of coincidence of the positions is higher than a predetermined value for a predetermined period of time.

[0009] In any of the above-mentioned assistance control devices, the recognition processing unit may further compare a moving direction identified from the history of position information of the first moving object with a moving direction identified from the history of position information of the second moving object for each combination of the first moving object and the second moving object, and recognize a combination of moving objects whose positional matching degree is higher than a predetermined value and whose moving direction matching degree is higher than a predetermined value as one moving object.

[0010] In any of the above-mentioned assistance control devices, the recognition processing unit may recognize a combination of moving objects whose positional consistency is higher than a predetermined value for a predetermined period of time and whose moving direction consistency is higher than a predetermined value for a predetermined period of time as one moving object.

[0011] In any of the above-mentioned assistance control devices, the recognition processing unit may determine over time, for each combination of the first moving object and the second moving object, whether the degree of positional coincidence is higher than a predetermined value or not, and each time it is determined that the degree of positional coincidence is higher than the predetermined value, it may increase an evaluation value by a predetermined value, and each time it is determined that the degree of positional coincidence is equal to or lower than the predetermined value, it may decrease the evaluation value by a predetermined value, and the recognition processing unit may perform a recognition process to recognize a combination of moving objects for which the evaluation value has reached the predetermined value as a single moving object.

[0012] In any of the above-mentioned assistance control devices, the recognition processing unit may perform a recognition process to determine over time, for each combination of the first moving object and the second moving object, whether the degree of coincidence of the positions is higher than a predetermined value and the degree of coincidence of the moving directions is higher than a predetermined value, and increase an evaluation value by a predetermined value each time it is determined that the degree of coincidence of the positions is higher than a predetermined value and the degree of coincidence of the moving directions is higher than a predetermined value, and decrease the evaluation value by a predetermined value each time it is determined that the degree of coincidence of the positions is lower than a predetermined value or the degree of coincidence of the moving directions is lower than a predetermined value, and to recognize a combination of moving objects for which the evaluation value has reached the predetermined value as a single moving object.

[0013] In any of the above-mentioned assistance control devices, the assistance control unit may perform control to send an alert to at least one of the moving objects that may approach each other within a predetermined time period when it is determined that there are moving objects that may approach each other within a predetermined time period based on the position information of the moving objects obtained by applying the recognition processing to the position information of the first moving object and the position information of the second moving object.

[0014] In any of the above-mentioned assistance control devices, the assistance control unit may predict a position of a moving object that has moved out of an imaging range captured by the imaging device based on a history of position information of the moving objects obtained by applying the recognition processing to the position information of the first moving object and the position information of the second moving object, and may determine whether there are moving objects approaching each other within a predetermined time based on the predicted position of the moving object and a position indicated by the position information of another moving object.

[0015] In a second aspect of the present invention, there is provided an assistance control method. The assistance control method includes a step of acquiring position information of a first moving object detected from an image captured by an imaging device. The assistance control method includes a step of acquiring position information of the second moving object measured for each of the second moving objects. The assistance control method includes a step of performing a recognition process in which the position information of the first moving object and the position information of the second moving object are compared for each combination of the first moving object and the second moving object, and a combination of moving objects having a degree of coincidence of position information higher than a predetermined value is recognized as one moving object. The assistance control method includes a step of performing control for traffic assistance for the moving object based on the position information of the moving object obtained by applying the recognition process to the position information of the first moving object and the position information of the second moving object. In the step of performing the recognition process, the position information to be the target of the recognition process out of the position information of the first moving object and the position information of the second moving object is limited to position information within a predetermined range set based on the position of the imaging device.

[0016] In a third aspect of the present invention, there is provided a program that causes a computer to function as any one of the above-mentioned assistance control devices.

[0017] The above summary of the invention does not list all of the features of the present invention. Also, subcombinations of these features may also be inventions. [Brief description of the drawings]

[0018] [Figure 1] 1 illustrates a schematic diagram of a scenario in which the support system 10 is used. [Diagram 2] 2 shows the functional configuration of the support device 60. [Diagram 3] 1 shows the first reception information received by the support device 60 from the imaging device 70 in table form. [Figure 4] 4 shows the second received information received by the support device 60 from the user terminal 82 in table format. [Diagram 5]11 is a diagram for explaining the recognition process in the recognition processing unit 240. FIG. [Figure 6] 13 is another diagram for explaining the recognition processing in the recognition processing unit 240. FIG. [Figure 7] 11A and 11B show schematic diagrams of other examples of the range that is the target of the recognition process. [Figure 8] 13 is a diagram for explaining a support method when a user 80f moves out of the imaging range of an imaging device 70. FIG. [Figure 9] 1 shows an example of a flowchart relating to an assistance control method executed by the assistance device 60. [Figure 10] An example of a computer 2000 is shown. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0019] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0020] 1 is a schematic diagram showing a usage scenario of the support system 10. The support system 10 includes a vehicle 20a, a vehicle 20b, a vehicle 20c, and a vehicle 20d, an imaging device 70, a support device 60, and a user terminal 82a, a user terminal 82b, a user terminal 82e, and a user terminal 82g.

[0021] In this embodiment, vehicle 20a, vehicle 20b, vehicle 20c, and vehicle 20d may be collectively referred to as "vehicle 20." User terminal 82a, user terminal 82b, user terminal 82e, and user terminal 82g may be collectively referred to as "user terminal 82." Users 80e, user 80f, and user 80g may be collectively referred to as "users 80."

[0022] In this embodiment, the vehicle 20 is a vehicle that travels on a road 90. The vehicle 20 is an example of a moving body.

[0023] The user terminal 82a and the user terminal 82b are terminals carried by passengers of the vehicle 20a and the vehicle 20b, respectively. The user terminal 82e is a terminal carried by the user 80e. The user terminal 82g is a terminal carried by the user 80g. The user terminal 82f does not carry a user terminal. In this embodiment, the users 80e, 80f, and 80g are pedestrians.

[0024] The user terminal 82 is, for example, a mobile terminal such as a smartphone. The user terminal 82 is an example of a moving body. The user terminal 82 periodically transmits current position information of the user terminal 82 detected by a position sensor including a GNSS receiver to the support device 60. For example, the user terminal 82 transmits latitude and longitude information indicating the current position of the user terminal 82 to the support device 60.

[0025] The imaging device 70 is an imaging device installed on a traffic infrastructure. The imaging device 70 acquires the positions of the vehicle 20 and the user 80 that are present within the imaging range of the imaging device 70 by analyzing the captured image, and transmits the acquired positions of the vehicle 20 and the user 80 to the assistance device 60.

[0026] The vehicle 20 and the user 80 are traffic participants. The vehicle 20 and the user 80 are examples of moving objects that can be supported by the support device 60.

[0027] The support device 60 receives, by mobile communication, the position information transmitted from the user terminal 82. The support device 60 may receive the information transmitted from the user terminal 82 and the imaging device 70 through any combination of mobile communication and a communication line such as the Internet or a dedicated line.

[0028] 1, the vehicle 20a, the vehicle 20c, the vehicle 20d, and the user 80f are located within the imaging range of the imaging device 70. The imaging device 70 analyzes the images acquired by imaging to identify the positions of the vehicle 20a, the vehicle 20c, the vehicle 20d, and the user 80f, and transmits position information indicating the identified positions to the assistance device 60.

[0029] The assistance device 60 selects a moving object to be supported from among the vehicles 20 and the users 80 based on the position information of each of the user terminals 82 received from the user terminal 82 and the position information of the vehicles 20a, 20c, 20d, and the users 80f received from the imaging device 70, and provides assistance to the selected moving objects.

[0030] As an example, the support device 60 determines that the vehicle 20a and the user 80e will approach each other within a predetermined time based on the history of the position information of the vehicle 20a acquired by the imaging device 70 and the history of the position information of the user 80e detected by the user terminal 82e. In this case, the support device 60 provides support to the user 80e by transmitting support information instructing the user terminal 82e to output an alarm.

[0031] Here, the assistance device 60 not only receives position information of the user terminal 82a from the user terminal 82a carried by a passenger of the vehicle 20a, but also receives position information of the vehicle 20a acquired by analyzing an image by the imaging device 70. The user terminal 82a moves together with the vehicle 20a. Therefore, it is not preferable to perform assistance processing assuming that moving objects exist at the position detected by the user terminal 82a and the position of the vehicle 20a acquired by the imaging device 70, and it is preferable to recognize the user terminal 82a and the vehicle 20a as a single moving object.

[0032] Therefore, the assistance device 60 performs a recognition process to compare the position information of the user terminal 82a detected by the user terminal 82a with the position information of the vehicle 20a acquired by the imaging device 70, and recognizes the user terminal 82a and the vehicle 20a as one moving object according to the comparison result. The assistance device 60 provides assistance to the traffic participants based on the position information aggregated by applying the recognition process.

[0033] When performing the recognition process, the support device 60 does not compare all combinations of the position information acquired by the imaging device 70 and the position information detected by the user terminal 82, but limits the position information to be compared to position information within a range 100. In one embodiment, the range 100 is set to a predetermined range including the position where the imaging device 70 is installed. In one embodiment, the range 100 may be a range in which the distance from the position where the imaging device 70 is installed is equal to or less than a predetermined set value. The set value may be, for example, 300 m.

[0034] According to the assistance system 10, it is possible to limit the position information to be subjected to the recognition process, thereby reducing the processing load on the assistance device 60.

[0035] 2 shows a functional configuration of the assistance device 60. The assistance device 60 includes an assistance control device 200, a communication device 290, and a storage device 280.

[0036] The communication device 290 is responsible for communication between the user terminal 82 and the imaging device 70 and the assistance device 60 under the control of the assistance control device 200. The assistance control device 200 is realized by including a circuit such as an arithmetic processing device including a processor. The assistance control device 200 may be realized by a microcomputer including a CPU, ROM, RAM, I / O, buses, etc. The storage device 280 is realized by including a non-volatile storage medium. The assistance control device 200 performs processing using information stored in the storage device 280. Map information including information indicating the position of roads 90 is stored in the storage device 280.

[0037] The assistance control device 200 includes an acquisition unit 202, an assistance control unit 260, a recognition processing unit 240, and a range setting unit 230. The acquisition unit 202 includes a first acquisition unit 210 and a second acquisition unit 220. A form in which the assistance device 60 does not have some of the functions of the functional configuration shown in FIG. 2 may be adopted.

[0038] The first acquisition unit 210 acquires position information of a first moving object detected from an image captured by the imaging device 70. The first acquisition unit 210 acquires the position information of the first moving object from the position information transmitted from the imaging device 70.

[0039] The second acquisition unit 220 acquires the position information of the second moving object measured for each of the second moving objects. In this embodiment, the second moving object is the user terminal 82. The second acquisition unit 220 acquires the position information of the user terminal 82 from the position information transmitted from the user terminal 82.

[0040] In this embodiment, the first acquisition unit 210 acquires position information of a first moving object detected from an image captured by the imaging device 70, and the second acquisition unit 220 acquires position information of the user terminal 82 measured in each of the user terminals 82. However, in a form in which the vehicle 20 is equipped with an imaging device that captures an image of the surroundings of the vehicle 20, the first acquisition unit 210 may acquire position information of a first moving object detected from an image captured by the imaging device equipped in the vehicle 20. In a case in which the vehicle 20 is equipped with a position sensor including a GNSS receiver, the position information second acquisition unit 220 may acquire position information of the vehicle 20 measured by the position sensor equipped in the vehicle 20.

[0041] The recognition processing unit 240 performs a recognition process in which the position information of the first moving object and the position information of the second moving object are compared for each combination of the first moving object and the second moving object, and a combination of moving objects having a degree of coincidence of the position information higher than a predetermined value is recognized as one moving object. At this time, the recognition processing unit 240 limits the position information to be subjected to the recognition process among the position information of the first moving object and the position information of the second moving object to position information within a predetermined range set based on the position of the imaging device 70. The assistance control unit 260 performs control for traffic assistance for the moving object based on the position information of the moving object obtained by applying the recognition process to the position information of the first moving object and the position information of the second moving object.

[0042] The predetermined range may be a predetermined fixed range 100 including the installation position of the imaging device 70. The range setting unit 230 may select at least four pieces of position information having the outermost coordinate components for each of two coordinate axes defined within the predetermined range from among the position information of the first moving object acquired by the first acquisition unit 210, and set a range defined by lines passing through each of the selected pieces of position information as the predetermined range.

[0043] As an example, the recognition processing unit 240 compares the position indicated by the position information of the first moving object with the position indicated by the position information of the second moving object for each combination of the first moving object and the second moving object, and recognizes a combination of moving objects whose positional coincidence is higher than a predetermined value as one moving object. The recognition processing unit 240 may recognize a combination of moving objects whose positional coincidence is higher than a predetermined value for a predetermined time as one moving object.

[0044] The recognition processing unit 240 may further compare the moving direction identified from the history of the position information of the first moving object with the moving direction identified from the history of the position information of the second moving object for each combination of the first moving object and the second moving object, and recognize a combination of moving objects whose position coincidence is higher than a predetermined value and whose moving direction coincidence is higher than a predetermined value for a predetermined time as one moving object. The recognition processing unit 240 may recognize a combination of moving objects whose position coincidence is higher than a predetermined value for a predetermined time and whose moving direction coincidence is higher than a predetermined value for a predetermined time as one moving object.

[0045] The recognition processing unit 240 may perform a first evaluation value calculation process for each combination of a first moving object and a second moving object. The first evaluation value calculation process includes a process of (i) determining whether or not the degree of coincidence of positions is higher than a predetermined value over time, and (ii) increasing the evaluation value by a predetermined value each time it is determined that the degree of coincidence of positions is higher than the predetermined value, and decreasing the evaluation value by a predetermined value each time it is determined that the degree of coincidence of positions is equal to or lower than the predetermined value. The recognition processing unit 240 performs a recognition process to recognize a combination of moving objects whose evaluation value has reached a predetermined value as one moving object.

[0046] The recognition processing unit 240 may perform a second evaluation value calculation process for each combination of the first moving object and the second moving object. The second evaluation value calculation process includes a process of determining over time whether the degree of coincidence of positions is higher than a predetermined value and the degree of coincidence of moving directions is higher than a predetermined value, and (ii) increasing the evaluation value by a predetermined value each time it is determined that the degree of coincidence of positions is higher than a predetermined value and the degree of coincidence of moving directions is higher than a predetermined value, and decreasing the evaluation value by a predetermined value each time it is determined that the degree of coincidence of positions is equal to or lower than a predetermined value or the degree of coincidence of moving directions is equal to or lower than a predetermined value. The recognition processing unit 240 performs a recognition process of recognizing a combination of moving objects whose evaluation value has reached a predetermined value as one moving object.

[0047] When it is determined that there are moving objects approaching each other within a predetermined time based on the position information of the moving objects obtained by applying recognition processing to the position information of the first moving object and the position information of the second moving object, the assistance control unit 260 performs control to send an alert to at least one of the moving objects that may approach each other.

[0048] The support control unit 260 may predict the position of a moving object that has moved from within the imaging range of the imaging device 70 to outside the imaging range based on the history of the position information of the moving object obtained by applying the recognition process to the position information of the first moving object and the position information of the second moving object, and may determine whether there are moving objects that approach each other within a predetermined time based on the predicted position of the moving object and the positions indicated by the position information of other moving objects. This allows support to be continued for a certain period of time even for a moving object that has moved outside the imaging range of the imaging device 70.

[0049] 3 shows in table format the first reception information received by the support device 60 from the imaging device 70. The first reception information includes, as data items, an ID, a time, a position, a type, an acquisition means, and an acquisition source position.

[0050] The ID indicates the identification information of the moving object. The imaging device 70 assigns the identification information to the moving object acquired by analyzing the image. The position information transmitted from the imaging device 70 includes the identification information assigned by the imaging device 70 to the moving object.

[0051] The time indicates the time when the position is acquired by the imaging device. The position indicates the position of the moving object. Specifically, the position indicates the position of the moving object acquired by the imaging device 70 by analyzing the image. The position may be, for example, latitude and longitude information.

[0052] The type indicates the type of the moving object. The imaging device 70 identifies the type of the moving object captured by analyzing the image. The position information transmitted from the imaging device 70 includes the type of the moving object identified by the imaging device 70.

[0053] The acquisition means is information indicating the means of acquiring the location information. "Infrastructure camera" in Fig. 3 indicates that the means of acquiring the location information is the imaging device 70 installed in the road infrastructure.

[0054] The acquisition source location indicates location information of the location where the image capturing device 70, which is the acquisition unit, is installed. The acquisition source location may be latitude and longitude information.

[0055] 3 are identification information of the vehicles 20a, 20c, and 20d, respectively, given by the imaging device 70. The ID "104" is identification information of the user 80f given by the imaging device 70.

[0056] 4 shows, in table format, the second received information received by the support device 60 from the user terminal 82. The second received information includes, as data items, an ID, a time, a location, and an acquisition means.

[0057] The ID indicates the identification information of the moving object. The position information transmitted from the user terminal 82 includes the identification information that is permanently assigned to the user terminal 82.

[0058] The time indicates the time when the position information was acquired by the user terminal 82. The position indicates the position of a moving object. Specifically, the position indicates the position of the vehicle 20 detected by the user terminal 82. The position may be, for example, latitude and longitude information.

[0059] The acquisition means is a means for acquiring location information. "Smartphone" in Fig. 4 indicates that the acquisition means for location information is a smartphone.

[0060] The IDs "301", "302", "303" and "304" in FIG. 4 are identification information for the user terminal 82a, the user terminal 82b, the user terminal 82e and the user terminal 82g, respectively.

[0061] As described in relation to FIG. 1, the recognition processing unit 240 targets only moving objects whose position information is detected and exists within the range 100 for the recognition process. That is, in the example of FIG. 3 and FIG. 4, the recognition processing unit 240 targets only combinations of four positions corresponding to ID "101", ID "102", ID "103", and ID "104" in FIG. 3, and three positions corresponding to ID "301", ID "302", and ID "303" in FIG. 4. That is, the position of ID "304" in FIG. 4 is excluded from the recognition process. This can reduce the amount of calculation required for the recognition process.

[0062] Fig. 5 is a diagram for explaining the recognition processing in the recognition processing unit 240. Fig. 5 shows a case where it is determined that the first moving object and the second moving object are to be recognized as one moving object.

[0063] FIG. 5 shows a case where the comparison between the position of the first moving object and the position of the second moving object is performed five times at a predetermined time interval. In FIG. 5, positions 511, 512, 513, 514, and 515 indicate the positions of the first moving object. Positions 521, 522, 523, 524, and 525 indicate the positions of the second moving object. In FIG. 5, the moving directions of the first moving object and the second moving object, which are identified from the history of multiple pieces of position information, are indicated by arrows. Here, the moving direction indicates whether the moving object is facing in the forward direction or the reverse direction, with the moving direction of each moving object being positive.

[0064] The initial value of the evaluation value for recognizing the first moving object and the second moving object as one moving object is 0. At a first time, the position of the first moving object is position 511, and the position of the second moving object is position 521. The recognition processing unit 240 determines whether the position 521 of the second moving object is within a predetermined range A1 centered on position 511, and whether the moving directions are the same. Position 521 is within range A1, and the moving directions are also the same. Therefore, the evaluation value is incremented by 1. This makes the evaluation value 1.

[0065] Similar processing is performed at times 2 to 5. That is, at time 2, position 522 of the second moving object is within a predetermined range A2 centered on position 512 of the first moving object, and the moving direction is also the same, so recognition processing unit 240 adds 1 to the evaluation value. As a result, the evaluation value at time 2 becomes 2.

[0066] At the third time point, the position 523 of the second moving object is not present within the predetermined range A3 centered on the position 513 of the first moving object. Therefore, the recognition processing unit 240 decreases the evaluation value by 1. As a result, the evaluation value becomes 1 at the third time point.

[0067] At the fourth time, the position 524 of the second moving object is within a predetermined range A4 centered on the position 514 of the first moving object, and the moving directions are also the same, so the recognition processing unit 240 adds 1 to the evaluation value. As a result, the evaluation value at the fourth time becomes 2. At the fifth time, the position 525 of the second moving object is within a predetermined range A5 centered on the position 515 of the first moving object, and the moving directions are also the same, so the recognition processing unit 240 adds 1 to the evaluation value. As a result, the evaluation value at the fifth time becomes 3.

[0068] When the evaluation value becomes equal to or greater than the first threshold value 3, the recognition processing unit 240 determines to recognize the first moving object and the second moving object as one moving object.

[0069] Fig. 6 is another diagram for explaining the recognition processing in the recognition processing unit 240. Fig. 6 shows a case where it is determined that the first moving object and the second moving object are not to be recognized as one moving object.

[0070] As in Fig. 5, Fig. 6 shows a case where the comparison between the position of the first moving object and the position of the second moving object is performed five times at predetermined time intervals. At the first time, the position of the first moving object is position 511, and the position of the second moving object is not within range A1. Therefore, the recognition processing unit 240 reduces the evaluation value by 1. As a result, the evaluation value at the first time becomes -1.

[0071] Similarly, at the second time instant, since the position 622 of the second moving object is not present within the range A2 centered on the position 512 of the first moving object, the recognition processing unit 240 reduces the evaluation value by 1. As a result, the evaluation value at the second time instant becomes −2.

[0072] At the third time, the position 623 of the second moving object is within the range A3 centered on the position 513 of the first moving object, and the moving direction is also the same. Therefore, the recognition processing unit 240 adds 1 to the evaluation value. As a result, the evaluation value at the third time becomes −1.

[0073] At the fourth time, the second moving object's position 624 is not present within range A4 centered on the first moving object's position 514, so the recognition processing unit 240 reduces the evaluation value by 1. As a result, the evaluation value at the fourth time becomes -2. At the fifth time, the second moving object's position 625 is not present within range A5 centered on the first moving object's position 515, so the recognition processing unit 240 reduces the evaluation value by 1. As a result, the evaluation value at the fifth time becomes -3.

[0074] When the evaluation value becomes equal to or less than the second threshold value −3, the recognition processing unit 240 determines to recognize the first moving object and the second moving object as one moving object.

[0075] Fig. 7 shows another example of the range of the recognition process. In Fig. 7, a circle indicates the position of a first moving object detected by the imaging device 70, and a cross indicates the position of a second moving object detected by the user terminal 82 in the xy coordinate system.

[0076] Range setting unit 230 identifies P1 having the smallest x coordinate value, P2 having the largest x coordinate value, P3 having the smallest y coordinate value, and P4 having the largest y coordinate value among the positions of the first moving object detected by imaging device 70. Range setting unit 230 sets range 700 defined by line L1 passing through P1 and parallel to the y axis, line L2 passing through P2 and parallel to the y axis, line L3 passing through P3 and parallel to the x axis, and line L4 passing through P4 and parallel to the x axis as a range to be subjected to recognition processing.

[0077] Whether the coordinates of the second moving object are included in the range 700 can be determined by determining whether the x coordinate of the position of the second moving object is between the x coordinate of P1 and the x coordinate of P2, and whether the y coordinate of the position of the second moving object is between the y coordinate of P3 and the y coordinate of P4. Since the determination can be made based on the magnitude relationship of the coordinate values ​​as described above, the amount of calculation involved in the process for extracting the coordinates of the second moving object included in the range 700 can be reduced. Furthermore, the second moving object whose position is detected near the first moving object whose position the imaging device 70 has actually been able to acquire can be targeted for the recognition process. This is expected to further reduce the amount of calculation involved in the recognition process.

[0078] FIG. 8 is a diagram for explaining a method of supporting the user 80f when the user 80f moves out of the imaging range of the imaging device 70. In FIG.

[0079] The support control unit 260 holds the position P and the moving speed v of the user 80f immediately before the user 80f goes out of the imaging range. The support control unit 260 estimates the position of the user 80f based on the position P and the moving speed v for a predetermined time from the time when the user 80f goes out of the imaging range. For example, the support control unit 260 can estimate the position P' T seconds later based on the position P and the moving speed v.

[0080] The support control unit 260 estimates the position Q of the vehicle 20b after T seconds from the history of the position information of the user terminal 82b. When the support control unit 260 determines that the distance between the positions P' and Q is smaller than a predetermined value, it determines to provide support to the user 80f and / or the vehicle 20b. For example, the support control unit 260 provides support by instructing the user terminal 82b to output an alarm. In this case, the support control unit 260 may weaken the strength of the alarm. For example, when the user 80f is located within the imaging range of the imaging device 70, an alarm that "a pedestrian is nearby" is output, whereas after the user 80f goes out of the imaging range of the imaging device 70, an alarm that "a pedestrian may be nearby" is output. This allows support to be provided while taking into consideration the possibility that the accuracy of the support may be reduced.

[0081] FIG. 9 shows an example of a flowchart relating to the assistance control method executed by the assistance device 60.

[0082] In S900, the range setting unit 230 sets a range for extracting position information to be a target of the recognition process. The range setting unit 230 may set a range based on the position of the imaging device 70, such as the range 100 described in relation to FIG. 1. The installation position of the imaging device 70 is included in the position information transmitted from the imaging device 70. The range setting unit 230 may set a range based on the position detected by the imaging device 70, such as the range 700 described in relation to FIG.

[0083] In S902, the recognition processing unit 240 extracts position information of a second moving object to be subjected to the recognition processing within the range set in S900.

[0084] In S904, the recognition processing unit 240 executes a loop process of repeating the processes from S906 to S910 for the combination of the position of the first moving object and the position of the second moving object extracted in S902.

[0085] In S906, the recognition processing unit 240 calculates an evaluation value. For example, the recognition processing unit 240 calculates the evaluation value by the method described in relation to Fig. 5 and Fig. 6. Although the case where the calculation of the evaluation value is completed in five calculation steps has been described in Fig. 5 and Fig. 6, an upper limit may be set for the number of calculation steps for calculating the evaluation value. The recognition processing unit 240 may end the calculation of the evaluation value when the number of calculation steps for the evaluation value reaches the upper limit.

[0086] In S908, it is determined whether the evaluation value calculated in S906 is equal to or greater than the first threshold. If it is determined that the evaluation value is equal to or greater than the first threshold, the process of S910 is executed, and if it is determined that the evaluation value is less than the first threshold, the process proceeds to the next step of the loop process. In S910, the recognition processing unit 240 determines that the first moving object and the second moving object are to be recognized as one moving object.

[0087] In S912, the assistance control unit 260 makes an assistance decision as to whether or not to execute assistance. For example, the assistance control unit 260 predicts the future position of each of the multiple moving objects after the recognition processing by the recognition processing unit 240 is applied, and when it is predicted that the moving object will approach another moving object within a predetermined time, it determines to execute assistance for the moving object and / or the other moving object. The assistance control unit 260 may determine whether or not the moving object will approach another moving object based on whether or not the distance between the moving objects predicted after a predetermined time is shorter than a predetermined value.

[0088] When the first moving object and the second moving object are recognized as a single moving object and an assistance judgment is made, if both the position detected by the imaging device 70 and the position detected by the user terminal 82 are detected, either one of the positions may be used for the assistance judgment, and the average value of the position detected by the imaging device 70 and the position detected by the user terminal 82 may be used for the assistance judgment.

[0089] In S914, the assistance control unit 260 executes assistance for the moving object in response to the assistance determination in S912. For example, the assistance control unit 260 controls the communication device 290 to transmit warning information instructing to output a warning to at least one of the moving object predicted to approach another moving object in the determination in S912 and the other moving object. For example, in the example of FIG. 1, when the assistance control unit 260 determines the user terminal 82e as the assistance target, the assistance control unit 260 transmits warning information instructing the user terminal 82e to output warning information through voice and / or screen display to the user terminal 82e. Upon receiving the warning information, the user terminal 82e notifies the user 80e of the presence of a vehicle approaching the user 80e through an HMI (Human Machine Interface) function of the user terminal 82e.

[0090] According to the support system 10 described above, the location information to be subjected to the recognition process for determining whether or not to recognize an object as a single moving object can be limited to a specific range. This can reduce the processing load of the recognition process. This can prevent a large delay in the transmission of warning information.

[0091] 10 shows an example of a computer 2000 in which multiple embodiments of the present invention may be embodied in whole or in part. A program installed in the computer 2000 may cause the computer 2000 to function as an apparatus such as the support device 60 according to an embodiment or each part of the apparatus, to execute operations associated with the apparatus or each part of the apparatus, and / or to execute a process or steps of the process according to an embodiment. Such a program may be executed by the CPU 2012 to cause the computer 2000 to execute specific operations associated with some or all of the blocks of the process steps and block diagrams described herein.

[0092] The computer 2000 according to this embodiment includes a CPU 2012 and a RAM 2014, which are connected to each other by a host controller 2010. The computer 2000 also includes a ROM 2026, a flash memory 2024, a communication interface 2022, and an input / output chip 2040. The ROM 2026, the flash memory 2024, the communication interface 2022, and the input / output chip 2040 are connected to the host controller 2010 via the input / output controller 2020.

[0093] The CPU 2012 operates according to the programs stored in the ROM 2026 and RAM 2014, thereby controlling each unit.

[0094] The communication interface 2022 communicates with other electronic devices via a network. The flash memory 2024 stores programs and data used by the CPU 2012 in the computer 2000. The ROM 2026 stores a boot program and the like executed by the computer 2000 upon activation, and / or a program that depends on the hardware of the computer 2000. The input / output chip 2040 may also connect various input / output units, such as a keyboard, mouse, and monitor, to the input / output controller 2020 via input / output ports, such as a serial port, a parallel port, a keyboard port, a mouse port, a monitor port, a USB port, an HDMI port, and the like.

[0095] The programs are provided via a computer-readable storage medium such as a CD-ROM, a DVD-ROM, or a memory card, or a network. The RAM 2014, the ROM 2026, or the flash memory 2024 are examples of computer-readable storage media. The programs are installed in the flash memory 2024, the RAM 2014, or the ROM 2026, and executed by the CPU 2012. Information processing described in these programs is read by the computer 2000, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by implementing an operation or processing of information according to the use of the computer 2000.

[0096] For example, when communication is performed between the computer 2000 and an external device, the CPU 2012 may execute a communication program loaded in the RAM 2014 and instruct the communication interface 2022 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 2012, the communication interface 2022 reads transmission data stored in a transmission buffer processing area provided in a recording medium such as the RAM 2014 and the flash memory 2024, transmits the read transmission data to a network, and writes reception data received from the network to a reception buffer processing area or the like provided on the recording medium.

[0097] Furthermore, the CPU 2012 may cause all or a necessary portion of a file or database stored on a recording medium such as the flash memory 2024 to be read into the RAM 2014, and perform various types of processing on the data on the RAM 2014. The CPU 2012 then writes back the processed data to the recording medium.

[0098] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and subjected to information processing. The CPU 2012 may execute various types of processing, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., described herein and specified by the instruction sequence of the program, on the data read from the RAM 2014, and write back the results to the RAM 2014. The CPU 2012 may also search for information in a file, database, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 2012 may search for an entry that matches a condition in which an attribute value of the first attribute is specified from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0099] The above-described programs or software modules may be stored in a computer-readable storage medium on the computer 2000 or in the vicinity of the computer 2000. A recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as the computer-readable storage medium. The programs stored in the computer-readable storage medium may be provided to the computer 2000 via a network.

[0100] A program installed in the computer 2000 and causing the computer 2000 to function as the support device 60 may act on the CPU 2012, etc., to cause the computer 2000 to function as each part of the support device 60 (e.g., the support control device 200, etc.). When the information processing described in these programs is read into the computer 2000, it functions as each part of the support device 60, which is a specific means in which the software and the various hardware resources described above work together. These specific means then realize calculation or processing of information according to the intended use of the computer 2000 in this embodiment, thereby constructing a specific support device 60 according to the intended use.

[0101] Various embodiments have been described with reference to block diagrams or the like. In the block diagrams, each block may represent (1) a stage of a process where an operation is performed or (2) a portion of an apparatus responsible for performing an operation. Particular stages and portions may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuitry and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuitry including logical AND, logical OR, logical XOR, logical NAND, logical NOR, and other logical operations, memory elements such as flip-flops, registers, field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like.

[0102] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon constitutes at least a portion of a product that includes instructions that can be executed to provide a means for performing the operations specified in a process or block diagram. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray (RTM) disks, memory sticks, integrated circuit cards, etc.

[0103] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0104] The computer readable instructions may be provided to a processor or programmable circuitry of a general purpose computer, special purpose computer, or other programmable data processing apparatus, either locally or over a wide area network (WAN) such as a local area network (LAN), the Internet, etc., to cause the computer readable instructions to execute means for performing operations specified in the process steps or block diagrams described. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0105] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0106] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically indicated as "before," "prior to," etc., and may be realized in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, it does not mean that it is essential to perform the process in this order. [Explanation of symbols]

[0107] 10 Support System 20 Vehicles 60 Support equipment 70 Imaging Device 80 users 82 User terminal 90 road 100 range 200 Assistance control device 202 Acquisition Department 210 First acquisition part 220 Second Acquisition Department 230 Range setting section 240 Recognition processing section 260 Support control section 280 Storage device 290 Communication Equipment 700 range 2000 Computer 2010 Host Controller 2012 CPU 2014 RAM 2020 Input / Output Controller 2022 Communication Interface 2024 Flash Memory 2026 ROM 2040 Input / Output Chip

Claims

1. a first acquisition unit that acquires position information of a first moving object detected from an image captured by the imaging device; a second acquisition unit that acquires position information of each of the second moving objects measured at the second moving objects; a recognition processing unit that performs a recognition process of comparing position information of the first moving object and position information of the second moving object for each combination of the first moving object and the second moving object, and recognizing a combination of moving objects whose degree of coincidence of position information is higher than a predetermined value as one moving object; an assistance control unit that performs control for traffic assistance for the moving object based on position information of the moving object obtained by applying the recognition processing to the position information of the first moving object and the position information of the second moving object; Equipped with The recognition processing unit limits position information to be subjected to the recognition processing out of the position information of the first moving object and the position information of the second moving object to position information within a predetermined range set based on the position of the imaging device. Assistive control device.

2. The predetermined range is a predetermined fixed range that includes an installation position of the imaging device. The assistance control device according to claim 1 .

3. a range setting unit that selects at least four pieces of position information having outermost coordinate components for each of two coordinate axes defined within the predetermined range from among the position information of the first moving object acquired by the first acquisition unit, and sets a range defined by lines passing through each of the selected pieces of position information as the predetermined range. The assistance control device according to claim 1 , further comprising:

4. The recognition processing unit compares a position indicated by the position information of the first moving object with a position indicated by the position information of the second moving object for each combination of the first moving object and the second moving object, and recognizes a combination of moving objects having a positional coincidence higher than a predetermined value as one moving object. The assistance control device according to any one of claims 1 to 3.

5. The recognition processing unit recognizes a combination of moving objects whose position coincidence is higher than a predetermined value for a predetermined time as one moving object. The assistance control device according to claim 4.

6. The recognition processing unit further compares a moving direction identified from a history of position information of the first moving object with a moving direction identified from a history of position information of the second moving object for each combination of the first moving object and the second moving object, and recognizes a combination of moving objects having a degree of coincidence of positions higher than a predetermined value and a degree of coincidence of moving directions higher than a predetermined value as one moving object. The assistance control device according to claim 4.

7. The recognition processing unit recognizes, as one moving object, a combination of moving objects having a degree of coincidence of positions higher than a predetermined value for a predetermined time period and a degree of coincidence of moving directions higher than a predetermined value for a predetermined time period. The assistance control device according to claim 6.

8. The recognition processing unit, for each combination of the first moving object and the second moving object, determining whether the degree of coincidence of the positions is higher than a predetermined value over time; Each time the degree of coincidence of the positions is determined to be higher than a predetermined value, the evaluation value is increased by the predetermined value, and each time the degree of coincidence of the positions is determined to be equal to or lower than the predetermined value, the evaluation value is decreased by the predetermined value. Processing is performed, The recognition processing unit performs a recognition process for recognizing a combination of moving objects whose evaluation value has reached a predetermined value as one moving object. The assistance control device according to claim 4.

9. The recognition processing unit, for each combination of the first moving object and the second moving object, determining whether the degree of coincidence of the positions is higher than a predetermined value and whether the degree of coincidence of the movement directions is higher than a predetermined value over time; Each time it is determined that the degree of coincidence of the positions is higher than a predetermined value and that the degree of coincidence of the movement direction is higher than a predetermined value, the evaluation value is increased by a predetermined value, and each time it is determined that the degree of coincidence of the positions is equal to or lower than a predetermined value or that the degree of coincidence of the movement direction is equal to or lower than a predetermined value, the evaluation value is decreased by a predetermined value. Processing is performed, A recognition process is performed to recognize a combination of moving objects whose evaluation value has reached a predetermined value as one moving object. The assistance control device according to claim 6.

10. The assistance control unit performs control to transmit an alarm to at least one of the moving objects that may approach each other when it is determined that there are moving objects that approach each other within a predetermined time based on the position information of the moving objects obtained by applying the recognition processing to the position information of the first moving object and the position information of the second moving object. The assistance control device according to any one of claims 1 to 3.

11. The assistance control unit performs, based on a history of position information of the moving objects obtained by applying the recognition processing to the position information of the first moving object and the position information of the second moving object, predicting a position of a moving object that has moved out of an imaging range captured by the imaging device; Based on the predicted position of the moving object and the positions indicated by the position information of other moving objects, it is determined whether there are moving objects approaching each other within a predetermined time period. The assistance control device according to claim 10.

12. acquiring position information of a first moving object detected from an image captured by an imaging device; obtaining position information of the second moving objects measured at each of the second moving objects; performing a recognition process for comparing position information of the first moving object and position information of the second moving object for each combination of the first moving object and the second moving object, and recognizing a combination of moving objects whose degree of coincidence of position information is higher than a predetermined value as one moving object; performing control for traffic assistance for the moving object based on position information of the moving object obtained by applying the recognition processing to the position information of the first moving object and the position information of the second moving object; Equipped with In the step of performing the recognition process, the position information of the first moving object and the position information of the second moving object to be subjected to the recognition process is limited to position information within a predetermined range set based on the position of the imaging device. Assistive control methods.

13. A program for causing a computer to function as the assistance control device according to any one of claims 1 to 3.

Images