Object recognition system, object recognition method, and program

Abstract

[Problem] To inhibit degradation of recognition performance of an object recognition system due to environmental changes including the weather and the time period such as day and night. [Solution] An object recognition system comprises: an object recognition means that uses a recognition dictionary to execute object recognition on a moving body captured by a camera; a first acquisition means that acquires a first index value indicating the reliability of an object recognition result for the moving body; a second acquisition means that acquires a second index value indicating an imaging environment of the camera; and a control means that determines, on the basis of the first index value, whether the recognition dictionary used in the object recognition needs to be changed. When it is determined that the recognition dictionary is to be changed, the control means of the object recognition system selects, on the basis of the second index value, a recognition dictionary to be used in the object recognition from among a plurality of recognition dictionaries.

Description

Technical Field

[0001] The present invention relates to an object recognition system, an object recognition method, and a recording medium.

Background Art

[0002] Patent Document 1 discloses a video processing device that can recognize a video with high reliability even when the recognition environment that affects the recognition accuracy of the video captured by the imaging device changes. According to this document, this video processing device includes a recognition environment acquisition unit that acquires recognition environment factors at the time of imaging that affect the recognition accuracy of the video captured by the imaging device, and a recognition environment factor that stores recognition environment conditions that are the correspondence between the recognition accuracy of the video and the recognition environment factors. A recognition accuracy calculation unit that calculates the recognition accuracy based on the recognition environment factors acquired by the recognition environment acquisition unit, and a recognition reliability calculation unit that calculates the recognition reliability from the calculated recognition accuracy. Further, this document describes that when the result of the transmission of this video processing device and another video processing device for the same recognition target is different, this video processing device determines that there is an abnormality in its own recognition result, and presents an improvement plan for the function of outputting the recognition result, such as changing the algorithm (paragraphs 0057, etc.).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] A common problem with object recognition systems installed outdoors is that the recognition accuracy changes depending on the weather, day and night, or weather. In this regard, although Patent Document 1 describes that when it is determined that there is an abnormality in the recognition result, an improvement plan for the function of outputting the recognition result, such as changing the algorithm, is presented, it is difficult to automatically improve the function.

[0005] The present invention aims to provide an object recognition system, an object recognition method, and a recording medium that can suppress the deterioration of recognition performance due to environmental changes such as time of day or weather. [Means for solving the problem]

[0006] From a first perspective, an object recognition system is provided, comprising: object recognition means that performs object recognition on a moving object captured by a camera using a recognition dictionary; first acquisition means that acquires a first index value indicating the reliability of the object recognition result of the moving object; second acquisition means that acquires a second index value representing the shooting environment of the camera; and control means that determines whether or not it is necessary to change the recognition dictionary used for object recognition based on the first index value. If the control means of this object recognition system determines that it is necessary to change the recognition dictionary, it selects a recognition dictionary to be used for object recognition from among a plurality of recognition dictionaries based on the second index value.

[0007] From a second perspective, an object recognition method is provided, which involves performing object recognition on a moving object captured by a camera using a recognition dictionary, obtaining a first index value indicating the reliability of the object recognition result of the moving object, determining whether or not to change the recognition dictionary used for object recognition based on the first index value, obtaining a second index value representing the camera's shooting environment if it is determined that the recognition dictionary should be changed, and selecting a recognition dictionary to be used for object recognition from among a plurality of recognition dictionaries based on the second index value.

[0008] From a third perspective, a recording medium is provided which contains a program that causes a computer to execute the following: a process of performing object recognition on a moving object captured by a camera using a recognition dictionary; a process of obtaining a first index value indicating the reliability of the object recognition result of the moving object; a process of determining whether or not it is necessary to change the recognition dictionary used for object recognition based on the first index value; and, if it is determined that the recognition dictionary should be changed, a process of obtaining a second index value representing the shooting environment of the camera, and selecting a recognition dictionary to be used for object recognition from among a plurality of recognition dictionaries based on the second index value. [Effects of the Invention]

[0009] According to the present invention, an object recognition system, an object recognition method, and a recording medium are provided that can suppress the deterioration of recognition performance due to environmental changes such as time of day or weather. [Brief explanation of the drawing]

[0010] [Figure 1] This is a diagram showing the configuration of one embodiment of the present invention. [Figure 2] This is a flowchart illustrating the operation of one embodiment of the system. [Figure 3] This is a diagram illustrating the operation of one embodiment of the present invention. [Figure 4] This is a block diagram showing the configuration of an object recognition system according to the first embodiment of the present invention. [Figure 5] This figure shows an example of a recognition dictionary set held in the recognition dictionary storage means of the object recognition system of the first embodiment of the present invention. [Figure 6] This is a flowchart illustrating the operation of the object recognition system according to the first embodiment of the present invention. [Figure 7] This is a diagram illustrating the operation of the object recognition system according to the first embodiment of the present invention. [Figure 8] This is another diagram illustrating the operation of the object recognition system according to the first embodiment of the present invention. [Figure 9] This is another diagram illustrating the operation of the object recognition system according to the first embodiment of the present invention. [Figure 10] This is another diagram illustrating the operation of the object recognition system according to the first embodiment of the present invention. [Figure 11] This is a block diagram showing the configuration of an object recognition system according to a second embodiment of the present invention. [Figure 12] This is a diagram illustrating the operation of an object recognition system according to a second embodiment of the present invention. [Figure 13] This is a block diagram showing the configuration of an object recognition system according to a third embodiment of the present invention. [Figure 14]It is a flowchart showing the operation of the object recognition system according to the third embodiment of the present invention. [Figure 15] It is a block diagram showing the configuration of the object recognition system according to the fourth embodiment of the present invention. [Figure 16] It is a flowchart showing the operation of the object recognition system according to the fourth embodiment of the present invention. [Figure 17] It is a diagram for explaining the operation of the object recognition system according to the fourth embodiment of the present invention. [Figure 18] It is a diagram showing the configuration of a computer that can function as the object recognition system of the present invention.

Embodiments for Carrying Out the Invention

[0011] First, an overview of an embodiment of the present invention will be described with reference to the drawings. Note that the reference numerals in the drawings appended to this overview are for convenience of each element as an example to assist understanding, and are not intended to limit the present invention to the illustrated embodiments. Also, the connection lines between the blocks in the drawings and the like referred to in the following description include both bidirectional and unidirectional ones. The one-way arrow schematically shows the flow of the main signal (data) and does not exclude bidirectionality. The program is executed via a computer device, and the computer device includes, for example, a processor, a storage device, an input device, a communication interface, and a display device as necessary. Further, this computer device is configured to be communicable with devices inside or outside the device (including computers) via a communication interface, regardless of wired or wireless. Also, ports or interfaces are provided at the input / output connection points of each block in the figure, but the illustration is omitted.

[0012] In one embodiment of the present invention, as shown in FIG. 1, it can be realized by an object recognition system 10 including an object recognition means 11, a first acquisition means 12, a second acquisition means 13, and a control means 14.

[0013] The object recognition means 11 performs object recognition on the moving body shown in the camera 20 using the recognition dictionaries 15-1 to 15-2. The recognition dictionaries 15-1 to 15-2 are a set of data necessary for recognition applied to the discriminator used by the object recognition means 11 for object recognition, and are switched by the control means 14. The recognition dictionaries 15-1 to 15-2 are created in multiple types according to the shooting environment of the camera, such as daytime, nighttime, sunny day, rainy day, etc. Such recognition dictionaries can be created by preparing images obtained under different shooting environments as teacher data and using methods such as machine learning and deep learning. The discriminator receives an input value and outputs a recognition result for it, and may also be called a learning model or an AI (Artificial Intelligence) model.

[0014] The first acquisition means 12 acquires a first index value indicating the reliability of the result of the object recognition of the moving body. As this first index value, mAP (mean Average Precision), IoU (Intersection over Union), etc. obtained in the process of object recognition of the moving body can be used. Of course, as the first index value, a value indicating the reliability of the result of object recognition of other moving bodies may be calculated.

[0015] The second acquisition means 13 acquires a second index value representing the shooting environment of the camera 20. For example, when the recognition dictionary is created by dividing day and night, the second acquisition means 13 can obtain the second index value by acquiring time information. Also, when the recognition dictionary is created by dividing the weather, the second acquisition means 13 may acquire weather information from an external network, a sensor, etc. For example, the second acquisition means 13 can also acquire the second index value by estimating day and night division or weather from the image taken by the camera.

[0016] The control means 14 determines whether it is necessary to change the recognition dictionary used for object recognition based on the first index value. If the control means 14 determines that it is necessary to change the recognition dictionary, it selects a recognition dictionary to be used for object recognition from among a plurality of recognition dictionaries based on the second index value and instructs the object recognition means 11 to switch the recognition dictionary.

[0017] Figure 2 shows an object recognition method used in the object recognition system 10 according to this embodiment. The object recognition system 10 configured as described above first performs object recognition on a moving object captured by the camera using a recognition dictionary, as shown in Figure 2 (step S001). Next, the object recognition system 10 obtains a first index value indicating the reliability of the object recognition result of the moving object (step S002). Next, the object recognition system 10 determines whether or not it is necessary to change the recognition dictionary used for object recognition based on the first index value (step S003).

[0018] If the determination in step S003 determines that the recognition dictionary should be changed (Yes), the object recognition system 10 obtains a second index value representing the camera's shooting environment (Step S004), and based on this second index value, selects and switches to a recognition dictionary from among multiple recognition dictionaries to be used for object recognition (Step S005). If the determination in step S003 determines that the recognition dictionary should not be changed (No), the object recognition system 10 omits obtaining the second index value and changing the recognition dictionary.

[0019] Figure 3 is a diagram illustrating the operation of one embodiment of the present invention. The object recognition system 10 performs object recognition using, for example, the recognition dictionary 15-1 and detects people P1 and P2. At this time, the first index value for person P1 is CV=80 and the first index value for person P2 is CV=60. In Figure 3 and below, CV stands for Confidence Value, and with an upper limit of 100, a higher value indicates higher reliability. Based on these first index values, the object recognition system 10 determines whether or not to change the recognition dictionary. For example, as sunset approaches and the image from camera 20 becomes darker, the CV decreases. The object recognition system 10 determines to change the recognition dictionary when the average CV falls below a predetermined value. The object recognition system 10 then acquires time information as a second index value and switches to a recognition dictionary for nighttime. This improves the accuracy of subsequent object recognition.

[0020] Furthermore, whether or not to modify the recognition dictionary based on the first index value CV described above can be determined using various criteria. One example is shown below. • If the average CV is less than or equal to a predetermined threshold A, the recognition dictionary is changed. • If the CV of one or more moving objects is below a predetermined threshold B, the recognition dictionary is changed. If the CV of two or more moving objects is below a predetermined threshold C, the recognition dictionary is changed. • If the CV of a moving object with a specific attribute is below a predetermined threshold D, the recognition dictionary is changed.

[0021] According to the object recognition system 10 operating as described above, it is possible to detect a decline in the recognition performance of the object recognition means 11 at an early stage, change the recognition dictionary, and restore the recognition performance.

[0022] [First Embodiment] Next, a first embodiment that focuses on maintaining the function of detecting moving objects located within a predetermined position range will be described in detail with reference to the drawings. Figure 4 is a diagram showing the configuration of the object recognition system 100 of the first embodiment of the present invention. Referring to Figure 4, the object recognition system 100 is shown comprising an object recognition means 101, a first acquisition means 102, a second acquisition means 103, a control means 104, and a recognition dictionary storage means 105.

[0023] The object recognition means 101 performs object recognition on moving objects captured by the camera 20 using a classifier to which a recognition dictionary has been applied. In this embodiment, the object recognition means 101 is described as recognizing people and vehicles captured by the camera and outputting the results to a predetermined output destination.

[0024] The first acquisition means 102 acquires a first index value indicating the reliability of the object recognition result of the moving object and sends it to the control means 104. In the following description, this first index value will be referred to as "CV". Hereinafter, the first acquisition means 102 will be described as acquiring mAP and IoU calculated during the object recognition process from the object recognition means 101 and calculating CV. In this embodiment, "CV" will be described with an upper limit of 100, and a larger value indicates a higher reliability of the object recognition result. Of course, the first index value does not need to be a system of values ​​like "CV" in this embodiment, as long as it allows the control means 104 to determine whether or not to change the recognition dictionary.

[0025] The second acquisition means 103 acquires a second index value representing the shooting environment of the camera 20. In this embodiment, the second acquisition means 103 is described as determining the distinction between day and night and the weather from the image of the camera 20 in response to a request from the control means 104 and returning it to the control means 104.

[0026] Based on the CV received from the first acquisition means 102, the control means 104 determines whether or not it is necessary to change the recognition dictionary used by the object recognition means 101. If the determination results in a decision to change the recognition dictionary, the control means 104 selects a recognition dictionary from the recognition dictionary storage means 105 based on the second index value and sends it to the object recognition means 101.

[0027] The recognition dictionary storage means 105 stores the recognition dictionary used by the object recognition means 101 for object recognition. Figure 5 shows the set of recognition dictionaries stored by the recognition dictionary storage means 105. In this embodiment, the recognition dictionary storage means 105 is described as holding recognition dictionaries that can be selected by combinations of weather, distinguishing between day and night, such as recognition dictionary 1051 for daytime and sunny weather, recognition dictionary 1052 for daytime and rainy weather, recognition dictionary 105m for nighttime and sunny weather, and recognition dictionary 105n for nighttime and rainy weather. In the example in Figure 5, recognition dictionaries for sunny and rainy weather are provided as weather-specific recognition dictionaries, but recognition dictionaries for other weather conditions such as fog and snow may also be provided. Furthermore, regarding time, instead of dividing it into two categories, daytime and nighttime, recognition dictionaries for time periods of any length, such as morning, evening, aeon, and afternoon, may also be provided. Even under the same sunny conditions, the position of the sun and the way shadows fall differ between morning, evening, aeon, and afternoon, so separating the recognition dictionaries may improve recognition accuracy. Therefore, a recognition dictionary corresponding to combinations of time of day and weather conditions may be prepared, such as sunny-morning, sunny-forenoon, sunny-afternoon, sunny-evening, and sunny-night. Of course, the recognition dictionary storage means 105 may also store recognition dictionaries for use in situations other than those mentioned above, as well as even more subdivided recognition dictionaries.

[0028] Next, the operation of the object recognition system 100 of this embodiment will be described in detail with reference to the drawings. Figure 6 is a flowchart showing the operation of the object recognition system of the first embodiment of the present invention. First, the object recognition system 100 performs object recognition on a moving object captured by the camera (step S101).

[0029] Next, the object recognition system 100 acquires the CV of the moving object detected by object recognition (step S102). Figure 7 shows an example of a moving object detected by the object recognition system 100 and its CV.

[0030] Next, the object recognition system 100 determines whether it is necessary to change the recognition dictionary applied to the object recognition means 101 based on the CV of the moving object (step S103). At this time, the control means 104 of the object recognition system 100 selects one or more moving objects located within a predetermined distance range from the camera 20 and uses their CV to determine whether it is necessary to change the recognition dictionary.

[0031] For example, suppose that moving objects MO1 to MO4 are detected as shown in Figure 7. In this case, the control means 104 of the object recognition system 100 selects moving objects MO2 to MO4 located within a predetermined distance range from the camera 20 and uses their CVs to determine whether or not to change the recognition dictionary. In the example in Figure 7, the CVs for moving object MO2 (person), moving object MO3 (person), and moving object MO4 (car) are 80, 60, and 70, respectively. The control means 104 of the object recognition system 100 calculates, for example, the average CV from these CVs and compares it with a predetermined threshold to determine whether or not to change the recognition dictionary. For example, if the predetermined threshold is 60, in the example in Figure 7, the control means 104 of the object recognition system 100 determines that it is not necessary to change the recognition dictionary.

[0032] On the other hand, the recognition performance of the object recognition system 100 may deteriorate as the sun sets or the weather changes. Figure 8 shows the CV when the recognition performance has deteriorated. In the example in Figure 8, the CVs for moving object MO2 (person), moving object MO3 (person), and moving object MO4 (car) are 80, 40, and 30, respectively. At this time, the average CV is 50, and if the predetermined threshold is 60, the control means 104 of the object recognition system 100 determines that the recognition dictionary needs to be changed.

[0033] In this way, if it is determined that the recognition dictionary should be changed (Yes in step S103), the object recognition system 100 obtains a second index value representing the shooting environment of the camera (step S104), and based on this second index value, selects and switches to the recognition dictionary to be used for object recognition from among the multiple recognition dictionaries (step S105). For example, if the current situation in which the camera is placed is nighttime and rainy, the recognition dictionary for nighttime-rainy weather is selected and the recognition dictionary is switched. This will help to restore the performance of the object recognition process in subsequent operations.

[0034] Furthermore, as explained in Figures 7 and 8 above, the object recognition system 100 of this embodiment selects moving objects MO2 to MO4 located within a predetermined distance range from the camera 20 and uses their CVs to determine whether or not a change in the recognition dictionary is necessary. For this reason, the CV of moving object MO4, which is far from the camera 20, is not used to determine whether or not a change in the recognition dictionary is necessary. In this embodiment, because moving objects are selected in this way, it is possible to quickly identify and take countermeasures against any deterioration in recognition performance that may affect the system's performance.

[0035] Furthermore, by selecting moving objects in this manner, it becomes possible to correctly determine that no change to the recognition dictionary is necessary, even if, for example, as shown in Figure 9, there are many moving objects outside a predetermined distance and their CV is low. Conversely, as shown in Figure 10, even if the overall CV is high, if the CV of moving objects within a predetermined distance is low, it becomes possible to determine early on that a change to the recognition dictionary is necessary.

[0036] In the explanation above, the determination was made by comparing the average CV with the threshold value, but the method for determining whether or not to change the recognition dictionary is not limited to this. For example, the determination may be made using the maximum CV, minimum CV, median CV, or other statistical values.

[0037] [Second Embodiment] Next, a second embodiment will be described in which the object recognition system takes into account the importance of the moving object and determines whether or not it is necessary to change the recognition dictionary. Figure 11 is a block diagram showing the configuration of the object recognition system 100a of the second embodiment of the present invention. The difference from the first embodiment shown in Figure 3 is the operation of the control means 104a in determining whether or not it is necessary to change the recognition dictionary. The other configurations and operations are the same as in the first embodiment, so their explanation will be omitted.

[0038] Figure 12 is a diagram illustrating the operation of the object recognition system 100a in the second embodiment. Similar to the first embodiment, it selects moving objects MO2 to MO4 from the detected moving objects MO1 to MO4 and uses their CV to determine whether or not to change the recognition dictionary. In this embodiment, the control means 104a of the object recognition system 100a determines whether or not to change the recognition dictionary based on values ​​obtained by weighting each type of moving object.

[0039] The average CV of the moving objects MO2 to MO4 in Figure 12 is (60 + 80 + 71) / 3 = approximately 70.3. If the predetermined threshold is 65, the control means 104a of the object recognition system 100a determines that no change to the recognition dictionary is necessary. However, it is also possible to calculate the average CV of the moving objects MO2 to MO4 after multiplying the CV of the vehicle and pedestrian by different coefficients.

[0040] Table 1 shows the CV values ​​for vehicles (MO4) and pedestrians (MO2, MO3).

[0041] [Table 1]

[0042] For example, if the coefficient multiplied by the pedestrian's CV is set to 0.8 and the coefficient multiplied by the vehicle's CV is set to 1.0, and the average CV is calculated, the corrected average CV will be ((140 × 0.8) + (71 × 1.0)) / 3 = 61. Similarly, if the predetermined threshold is 65, the control means 104a of the object recognition system 100a in this embodiment will determine that the recognition dictionary needs to be changed. This will help to restore the performance of the object recognition process in subsequent operations.

[0043] Note that the weighting example shown in Figure 12 is merely an example. Various modifications can be made. For example, if early detection of pedestrians is required for the application of the object recognition system 100a, a smaller value can be set as the weighting coefficient multiplied by the pedestrian's CV. This will lower the average CV after weighting correction, making it possible to prompt an earlier switch of the recognition dictionary. In the embodiment described above, the types of moving objects were explained as being pedestrians and vehicles, but the types of moving objects are not limited to these. For example, the moving object (pedestrian) MO2 and the moving object (pedestrian) MO3 with a cane in Figure 12 could each be treated as different types, and the average CV after weighting correction could be calculated after multiplying each by a weighting coefficient. For example, the moving object (four-wheeled vehicle) MO1 and the moving object (two-wheeled vehicle) MO4 in Figure 12 could each be treated as different types, and the average CV after weighting correction could be calculated after multiplying each by a weighting coefficient.

[0044] As described above, this embodiment makes it possible to detect early on the deterioration of the recognition performance of a specific type of mobile object among the detected mobile objects and prompt a change in the recognition dictionary.

[0045] The above explanation described a method of determining whether a change to the recognition dictionary is necessary by comparing the weighted average CV with the threshold value. However, this is not the only method for determining whether a change to the recognition dictionary is necessary. For example, the need to change the recognition dictionary may be determined using multiple weighted maximum CVs, minimum CVs, median CVs, or other statistical values.

[0046] Furthermore, when determining whether or not to change the recognition dictionary, a method other than weighting may be used to determine whether or not to change the recognition dictionary. Specifically, the control means 104a may determine whether or not to change the recognition dictionary based on the first index value and criteria such as thresholds determined for each type of moving object. For example, by setting different thresholds for the moving object (pedestrian) MO2 and the moving object (pedestrian) MO3 with a cane in Figure 12 and comparing them, the same determination result as in the above example can be obtained.

[0047] [Third Embodiment] Next, a third embodiment will be described in which the object recognition system checks for improvements in recognition performance before switching the recognition dictionary. Figure 13 is a block diagram showing the configuration of the object recognition system 100b of the third embodiment of the present invention. The difference from the first embodiment shown in Figure 3 is the operation of the control means 104b to determine whether or not it is necessary to change the recognition dictionary. The other configurations are the same as in the first embodiment, so their explanation will be omitted.

[0048] In this embodiment, the control means 104b of the object recognition system 100b determines that it is necessary to change the recognition dictionary, causes the object recognition means 101 to perform object recognition using the candidate recognition dictionary, and then switches the recognition dictionary after confirming that the first evaluation value increases.

[0049] Figure 14 is a flowchart illustrating the operation of an object recognition system according to a third embodiment of the present invention. The operation of steps S101 to S104 and S105 in Figure 14 is the same as in the first embodiment, so the differences will be explained below. After determining that a change in the recognition dictionary is to be made and obtaining a second index value representing the shooting environment of the camera, the control means 104b of the object recognition system 100b selects a candidate for switching the recognition dictionary to be used for object recognition from among a plurality of recognition dictionaries based on this second index value. Then, the control means 104b requests the object recognition means 101 to perform object recognition processing using the candidate recognition dictionary (step S205).

[0050] Next, the control means 104b of the object recognition system 100b requests the first acquisition means 102 to acquire the CV of the moving object detected by object recognition, and acquires it (step S206). Then, the control means 104b determines whether the CV acquired in step S206 has improved or not (step S207). This determination of whether the CV has improved or not can be made by comparing it with the CV acquired in step S102. Alternatively, in another variation, a determination equivalent to the determination process in step S103 may be made to determine again whether or not a switch in the recognition dictionary is necessary. If, as a result of object recognition using the candidate recognition dictionary, it is determined that a switch in the recognition dictionary is unnecessary, then that candidate recognition dictionary will be adopted. If, as a result of object recognition using the candidate recognition dictionary, it is determined that a switch in the recognition dictionary is necessary, then it is determined that a switch to that candidate recognition dictionary is unnecessary.

[0051] If the determination in step S207 indicates that the CV has improved, the control means 104b of the object recognition system 100b switches to the recognition dictionary of the switching candidate (step S105). On the other hand, if the determination in step S207 indicates that the CV has not improved, the control means 104b of the object recognition system 100b continues to use the previous recognition dictionary (step S207 No).

[0052] As explained above, according to this embodiment, before switching the recognition dictionary, the improvement of the CV, which indicates the reliability of the object recognition result, is checked. Therefore, compared to the first embodiment, it is possible to prevent a situation in which the accuracy of object recognition deteriorates after switching the recognition dictionary.

[0053] Furthermore, while the above explanation assumes that the CV improves before switching the recognition dictionary, it is also possible to switch the recognition dictionary first, calculate the CV, and then revert to the original recognition dictionary if the CV is low.

[0054] In the description of the third embodiment above, the same method as in the first embodiment was used to confirm whether or not a change to the recognition dictionary is necessary, but it is also possible to combine the second and third embodiments. In this case, the control means 104b of the object recognition system 100b can prioritize the CV of a specific type of moving object among the detected moving objects to determine whether or not a change to the recognition dictionary is necessary, and in the CV improvement in step S207, it can also confirm whether or not the CV of a specific type of moving object has improved.

[0055] [Fourth Embodiment] Next, a fourth embodiment will be described, in which a function for outputting an index value representing the recognition performance of the object recognition system is added. Figure 15 is a block diagram showing the configuration of the object recognition system 100c of the fourth embodiment of the present invention. The difference from the first embodiment shown in Figure 3 is the addition of a performance index output means 106. The other configurations are the same as those of the first embodiment, so their explanation will be omitted.

[0056] The performance indicator output means 106 acquires CV for each type of moving object at predetermined intervals and outputs it to a predetermined output destination as a performance indicator of the object recognition system 100c.

[0057] Figure 16 is a flowchart illustrating additional operations to the object recognition system according to a fourth embodiment of the present invention. Referring to Figure 16, first, the object recognition system 100c performs object recognition processing on a moving object captured by the camera (step S401). This object recognition processing may also serve as the normal object recognition processing performed in step S101, or it may be performed for the purpose of outputting a performance indicator of the object recognition system 100c.

[0058] Next, the object recognition system 100c acquires the CV of the moving object detected by object recognition (step S402).

[0059] Next, the object recognition system 100c creates a screen or report that displays the acquired CV of the moving object by type of moving object and distance, and outputs it to a predetermined output destination. Figure 17 is an example of a screen created by the object recognition system 100c. In this example, the CV of the moving object is displayed by type of moving object and distance. By referring to such a screen, the user of the object recognition system 100c can easily and visually confirm for which types and distances of moving objects the object recognition system 100c maintains accuracy, or conversely, for which accuracy has deteriorated.

[0060] For example, in the example shown in Figure 17, the CVs of elderly individuals P3 and P4 among the moving objects (pedestrians) are 30 and 40 respectively, indicating deterioration. A user of such an object recognition system 100c can understand that it is necessary to improve the CVs of elderly individuals P3 and P4 within the first distance range. For example, a user of the object recognition system 100c can implement improvement measures such as applying an existing recognition dictionary that can improve the CVs of elderly individuals P3 and P4 within the first distance range, or creating a new recognition dictionary. As a result, the recognition accuracy of elderly individuals by the object recognition system 100c will improve thereafter.

[0061] In the example in Figure 17, the distance range is divided into two categories: a first distance range and a second distance range. However, the distance range may be further subdivided. For example, if the primary use of the object recognition system 100c is to monitor elderly people crossing a pedestrian crossing 15m to 20m away from the camera 20, the distance range may be divided into that section and the sections before and after it, with each representing a different CV. Also, in the example in Figure 17, the type of moving object is divided into two categories: elderly and non-elderly. However, the type of moving object may be further subdivided. This makes it easier to understand which types of moving objects the currently applied recognition dictionary is good at or not good at.

[0062] In the example shown in Figure 17, the screen is described as displaying both the type of mobile object and the distance range, but it is not necessary to use both the type of mobile object and the distance range. For example, it is also possible to output the CV for the type of mobile object selected by the user, or to output the CV for each distance range selected by the user. Of course, these can be easily switched using a dropdown list on the screen or a hardware key.

[0063] (Regarding hardware configuration) In each embodiment of this disclosure, each component of each device represents a functional unit block. Some or all of each component of each device is realized by any combination of an information processing device 900 and a program, for example, as shown in Figure 18. Figure 18 is a block diagram showing an example of the hardware configuration of the information processing device 900 that realizes each component of each device. The information processing device 900 includes, as an example, the following configuration. ·CPU(Central Processing Unit)901 • ROM (Read Only Memory) 902 ·RAM(Random Access Memory)903 Program 904 is loaded into RAM903. • Storage device 905 for storing program 904 • Drive device 907 for reading and writing recording medium 906 • Communication interface 908 connected to communication network 909 • Input / output interface 910 for data input and output. • Bus 911 connecting each component

[0064] Each component of each device in each embodiment is realized by the CPU 901 acquiring and executing a program 904 that realizes these functions. That is, the CPU 901 in Figure 18 executes a program that detects an object and acquires its CV (first index value), and a program that determines whether or not it is necessary to change the recognition dictionary based on the CV, and then performs the update process of each calculation parameter held in RAM 903, storage device 905, etc. The program 904 that realizes the functions of each component of each device is, for example, stored in advance in storage device 905 or ROM 902, and read by the CPU 901 as needed. The program 904 may be supplied to the CPU 901 via a communication network 909, or it may be stored in advance in a recording medium 906, and the drive device 907 may read the program and supply it to the CPU 901.

[0065] Furthermore, this program 904 can display its processing results, including intermediate states, step by step via a display device, or communicate with the outside world via a communication interface, as needed. This program 904 can also be recorded on a computer-readable (non-transitive) storage medium.

[0066] There are various variations in how each device is implemented. For example, each device may be implemented by any combination of a separate information processing device 900 and a program for each component. Alternatively, the multiple components of each device may be implemented by any combination of a single information processing device 900 and a program. That is, these devices can be implemented by a computer program that causes the processor mounted on these devices, as shown in the first to fourth embodiments above, to execute the above-described processes using its hardware.

[0067] Furthermore, some or all of the components of each device are realized by other general-purpose or dedicated circuits, processors, etc., or combinations thereof. These may consist of a single chip or multiple chips connected via a bus.

[0068] Some or all of the components of each device may be realized by a combination of the circuits and programs described above.

[0069] When some or all of the components of each device are implemented by multiple information processing devices or circuits, these multiple information processing devices or circuits may be centrally located or distributed. For example, the information processing devices or circuits may be implemented in a form in which each is connected via a communication network, such as a client-and-server system or a cloud computing system.

[0070] The embodiments described above are preferred embodiments of this disclosure and do not limit the scope of this disclosure to these embodiments alone. That is, a person skilled in the art can modify or substitute the embodiments described above to construct various modified forms without departing from the gist of this disclosure.

[0071] For example, although the above embodiments were described as changing the recognition dictionary, it is also possible to adopt a form in which a classifier is changed from among multiple classifiers.

[0072] For example, in the embodiments described above, the determination of whether or not to change the recognition dictionary was made based on a first index value. However, the control means 104 and 104a may also refer to other information in addition to the first index value to determine whether or not to change the recognition dictionary. For example, the control means 104 and 104a may use a second index value in addition to the first index value to determine whether or not to change the recognition dictionary. In this case, the control means 104 and 104a may determine to change the recognition dictionary when the CV is below a threshold and the second index value indicates that the area around the camera 20 is dark. In addition to illuminance, the second index value can also be simply the aperture opening information of the camera 20, the shutter speed, the ISO value, etc.

[0073] Some or all of the above embodiments may also be described as follows, but are not limited to these.

[0074] [Note 1] An object recognition means that performs object recognition on moving objects captured by a camera using a recognition dictionary, A first acquisition means for obtaining a first index value indicating the reliability of the object recognition result of the moving object, A second acquisition means for acquiring a second index value representing the shooting environment of the camera, The system includes a control means that determines whether or not it is necessary to change the recognition dictionary used for object recognition based on the first index value, When the control means determines that it is necessary to change the recognition dictionary, it selects a recognition dictionary to be used for object recognition from among a plurality of recognition dictionaries based on the second index value. Object recognition system. [Note 2] The control means of the object recognition system described above can be configured to determine whether or not it is necessary to change the recognition dictionary used for object recognition based on the first index value of one or more moving objects located within a predetermined distance range from the camera among the moving objects captured by the camera. [Note 3] The control means of the object recognition system described above can be configured to determine whether or not to change the recognition dictionary based on a value obtained by weighting the first index value according to the type of moving object. [Note 4] The control means of the object recognition system described above can be configured to determine whether or not to change the recognition dictionary based on the first index value and criteria defined for each type of moving object. [Note 5] The control means of the object recognition system described above can be configured to check whether the first index value improves when it switches to a recognition dictionary selected based on the second index value, and to change the recognition dictionary if the first index value improves. [Note 6] The second index value acquired by the object recognition system described above includes at least weather information and information indicating the time of day. The control means may be configured to select a recognition dictionary from among a plurality of recognition dictionaries that corresponds to a combination of weather and time of day. [Note 7] The object recognition system described above further, A configuration can be adopted that includes a performance indicator output means for acquiring the first indicator value for each type of moving object and outputting it to a predetermined output destination as a performance indicator of the object recognition system. [Note 8] Using a recognition dictionary, object recognition is performed on moving objects captured by the camera. A first index value indicating the reliability of the object recognition result for the moving object is obtained, Based on the first index value, it is determined whether or not it is necessary to change the recognition dictionary used for object recognition. If it is determined that the recognition dictionary should be changed, a second index value representing the camera's shooting environment is obtained, and based on this second index value, a recognition dictionary to be used for object recognition is selected from among multiple recognition dictionaries. Object recognition method. [Note 9] The process involves performing object recognition on moving objects captured by the camera using a recognition dictionary, and A process to obtain a first index value indicating the reliability of the object recognition result of the moving object, A process to determine whether or not it is necessary to change the recognition dictionary used for object recognition based on the first index value, If it is determined that the recognition dictionary should be changed, a second index value representing the camera's shooting environment is obtained, and based on the second index value, a recognition dictionary to be used for object recognition is selected from among multiple recognition dictionaries. A recording medium that contains a program that causes a computer to execute a program.

[0075] Furthermore, the forms described in appendices 8 to 9 above can be expanded into the forms described in appendices 2 to 7, similar to appendice 1.

[0076] Furthermore, each disclosure in the above-mentioned patent documents is incorporated into this document by reference and may be used as the basis or part of the present invention as necessary. Within the framework of the full disclosure of the present invention (including the claims), further modifications and adjustments to the embodiments or examples are possible based on the basic technical concept. Also, within the framework of the disclosure of the present invention, various combinations or selections (including partial deletions) of various disclosure elements (including each element of each claim, each element of each embodiment or example, each element of each drawing, etc.) are possible. In other words, the present invention naturally includes the full disclosure, including the claims, and various modifications and alterations that a person skilled in the art could make in accordance with the technical concept. In particular, with respect to the numerical ranges described in this document, any numerical value or sub-range included within that range should be interpreted as being specifically described, even if not otherwise stated. Furthermore, each disclosure in the above-mentioned cited documents may, as necessary, be used in part or in whole as part of the disclosure of the present invention, in accordance with the spirit of the present invention, and this is also considered to be included in the disclosure of this application. [Explanation of Symbols]

[0077] 10, 100, 100a, 100b, 100c Object Recognition System 11, 101 Object recognition means 12, 102 First means of acquisition 13, 103 Second means of acquisition 14, 104, 104b Control means 15-1~15-2 Recognition Dictionary 20 cameras 105 Recognition dictionary storage means 106 Performance index output means 1051, 1052, 105m, 105n recognition dictionary P1, P2 people P3, P4 people (elderly) MO1~MO4 Mobile Units 900 Information Processing Equipment 901 CPU(Central Processing Unit) 902 ROM (Read Only Memory) 903 RAM (Random Access Memory) 904 Program 905 Storage device 906 Recording media 907 Drive unit 908 Communication Interface 909 Communication Network 910 Input / Output Interface 911 Bus

Claims

1. An object recognition means that performs object recognition on moving objects captured by a camera using a recognition dictionary, A first acquisition means for obtaining a first index value indicating the reliability of the object recognition result of the moving object, A second acquisition means for acquiring a second index value representing the shooting environment of the camera, The system includes a control means for determining whether or not to change the recognition dictionary used for object recognition, based on the average value of values ​​obtained by weighting the first index value of each of the recognized moving objects according to the type of moving object, When the control means determines that it is necessary to change the recognition dictionary, it selects a recognition dictionary to be used for object recognition from among the plurality of recognition dictionaries based on the second index value. Object recognition system.

2. The object recognition system according to claim 1, wherein the control means determines whether or not it is necessary to change the recognition dictionary used for object recognition based on the average value of a value obtained by weighting the first index value of a plurality of moving objects located within a predetermined distance range from the camera among the moving objects captured by the camera, according to the type of moving object.

3. The object recognition system according to claim 1, wherein the control means checks whether the first index value improves when the recognition dictionary selected based on the second index value is switched, and if the first index value improves, it changes the recognition dictionary.

4. The second index value includes at least weather information and time-of-day information, The object recognition system according to claim 1, wherein the control means selects a recognition dictionary from a plurality of recognition dictionaries that corresponds to a combination of weather and time of day.

5. Furthermore, the object recognition system according to claim 1 or 2 includes a performance indicator output means for acquiring the first indicator value for each type of moving object and outputting the acquired first indicator value to a predetermined output destination as a performance indicator of the object recognition system.

6. Computer equipment, Using a recognition dictionary, object recognition is performed on moving objects captured by the camera. A first index value indicating the reliability of the object recognition result for the moving object is obtained, Based on the average value obtained by weighting the first index value of each of the recognized moving objects according to the type of the moving object, it is determined whether or not it is necessary to change the recognition dictionary used for object recognition. If it is determined that the aforementioned recognition dictionary should be changed, a second index value representing the camera's shooting environment is obtained, and based on this second index value, a recognition dictionary to be used for object recognition is selected from among a plurality of recognition dictionaries. Object recognition method.

7. The process involves performing object recognition on moving objects captured by the camera using a recognition dictionary, and A process to obtain a first index value indicating the reliability of the object recognition result of the moving object, A process to determine whether or not to change the recognition dictionary used for object recognition, based on the average value obtained by weighting the first index value of each of the recognized multiple moving objects according to the type of moving object, If it is determined that the aforementioned recognition dictionary should be changed, a second index value representing the camera's shooting environment is obtained, and based on this second index value, a process is performed to select a recognition dictionary to be used for object recognition from among multiple recognition dictionaries. A program that causes a computer to execute something.

Images