Information processing method and information processing apparatus
The information processing method addresses the inefficiency of switching between daytime and nighttime learning models by integrating identification labels and confidence levels from both models, resulting in robust and stable image identification across different time zones.
Patent Information
- Application Number
- JP2023208016
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-12-08
- Publication Date
- 2025-06-19
AI Technical Summary
Existing methods for image identification require switching between daytime and nighttime learning models based on time zones, which can be inefficient and may lead to incorrect results if switching is not properly executed.
An information processing method that stores both daytime and nighttime identifiers, applies them to captured images, integrates identification labels and confidence levels from both identifiers into a feature vector, and uses this integrated identifier to perform robust and stable identification regardless of day or night.
This method enables robust and stable image identification by learning the tendencies of multiple identifiers and integrating their opinions, thereby improving identification accuracy and efficiency across different time zones.
Smart Images
Figure 2025092252000001_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to an information processing method and an information processing apparatus.
Background Art
[0002] Conventionally, a method of performing identification using a daytime learning model for daytime images and a nighttime learning model for nighttime images has been known (see, for example, Non-Patent Documents 1 and 2). The learning model used is switched according to the time zone.
Prior Art Documents
Non-Patent Documents
[0003]
Non-Patent Document 1
Non-Patent Document 2
Summary of the Invention
Problems to be Solved by the Invention
[0004] In the method of the above background art, it is necessary to switch between the daytime learning model and the nighttime learning model according to the time zone, and there is room for improvement in terms of efficiency. Further, if the switching is not performed appropriately, there is a possibility that correct results cannot be obtained.
[0005] In view of such circumstances, an object of the present disclosure is to perform robust and stable identification regardless of day or night on a captured image.
Means for Solving the Problems
[0006] 〔1〕The information processing method according to an embodiment of the present disclosure is an information processing method executed by an information processing apparatus, wherein the information processing apparatus includes a control unit, a communication unit, and a storage unit, and is communicable with a network via the communication unit, storing a first identifier used for identifying an identification target in a captured image captured in a first time zone in the storage unit; storing a second identifier used for identifying an identification target in a captured image captured in a second time zone in the storage unit; acquiring a captured image and applying the first identifier and the second identifier to the captured image; acquiring, from each of the first identifier and the second identifier, one or more identification labels indicating the state of the identification target in the captured image and a confidence level for each identification label; setting the identification label and the confidence level as a feature vector and learning the tendency of the feature vector in an integrated identifier; acquiring a final identification label indicating the state of the identification target in the captured image from the integrated identifier; and includes. With this configuration, the tendency of the previous-stage identifier group is learned, and the identification opinions from multiple viewpoints are integrated, so that robust and stable identification can be performed regardless of day or night.
[0007] 〔2〕The information processing method according to an embodiment of the present disclosure is in the information processing method described in the above 〔1〕, Preferably, the first time period is at night and the second time period is during the day. With this configuration, more robust and stable identification can be achieved.
[0008] 〔3〕The information processing method according to an embodiment of the present disclosure is In the information processing method described in the above 〔1〕 or 〔2〕, Preferably, the object to be identified includes a road surface. With this configuration, the identification accuracy for the road surface can be further stabilized.
[0009] 〔4〕The information processing method according to an embodiment of the present disclosure is In the information processing method described in any one of the above 〔1〕 to 〔3〕, acquiring a captured image, acquiring, from the captured image, a region effective for identifying the object to be identified by gradient load class activation mapping, acquiring, from the captured image, presence information of an object or a space different from the object to be identified by an object detection algorithm, setting an analysis window in the captured image from the acquired region and the presence information, analyzing the state of the object to be identified in the analysis window, and preferably including With this configuration, the identification accuracy can be improved.
[0010] 〔5〕The information processing apparatus according to an embodiment of the present disclosure is an information processing apparatus including a control unit, a communication unit, and a storage unit, and being capable of communicating with a network via the communication unit, wherein the control unit stores, in the storage unit, a first identifier used for identifying an object to be identified in a captured image captured in a first time period, stores, in the storage unit, a second identifier used for identifying an object to be identified in a captured image captured in a second time period, Obtain a captured image and apply the first identifier and the second identifier to the captured image; Obtain, from each of the first identifier and the second identifier, one or more identification labels indicating the state of the object to be identified in the captured image and the confidence level for each identification label; Set the identification label and the confidence level as a feature vector, and have an integrated identifier learn the trend of the feature vector; Obtain a final identification label indicating the state of the object to be identified in the captured image from the integrated identifier; Execute an operation including the above. With this configuration, since the trends of the previous-stage identifier group are learned and the identification opinions from multiple viewpoints are integrated, robust and stable identification can be performed regardless of day or night.
Advantages of the Invention
[0011] According to the present disclosure, robust and stable identification can be performed on a captured image regardless of day or night.
Brief Description of the Drawings
[0012]
Figure 1
Figure 2A
Figure 2B
Figure 2C
Figure 3A
Figure 3B
Figure 4
Figure 5
Figure 6
Modes for Carrying Out the Invention
[0013] The information processing apparatus 1 shown in FIG. 1 may be a server or a cloud server. The information processing apparatus 1 may be installed, for example, in a dedicated facility for an operator or a shared facility including a data center. As an alternative example, the information processing apparatus 1 may be installed in a moving vehicle. The processing executed by the information processing apparatus 1 may be executed by a plurality of information processing apparatuses 1 arranged in a distributed manner.
[0014] The information processing apparatus 1 includes a control unit 11, a communication unit 12, and a storage unit 13. Each component of the information processing apparatus 1 is communicably connected to each other.
[0015] The control unit 11 includes, for example, one or more general-purpose processors including a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control unit 11 may include one or more dedicated processors specialized for specific processing. Instead of including a processor, the control unit 11 may include one or more dedicated circuits. The dedicated circuit may be, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit 11 may include an ECU (Electronic Control Unit). The control unit 11 transmits and receives any information via the communication unit 12.
[0016] The communication unit 12 includes a communication module that conforms to one or more wired or wireless LAN (Local Area Network) standards for connecting to a network. The network may be, for example, the Internet, a mobile communication network, a fixed communication network, a LAN, or a combination thereof. The communication unit 12 may include a module that conforms to one or more mobile communication standards including LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation). The communication unit 12 may include a communication module or the like that conforms to one or more short-range communication standards or specifications including Bluetooth (registered trademark), AirDrop (registered trademark), IrDA, ZigBee (registered trademark), Felica (registered trademark), or RFID. The communication unit 12 transmits and receives any information via the network.
[0017] The storage unit 13 includes, for example, a semiconductor memory, a magnetic memory, an optical memory, or a combination of at least two of these, but is not limited thereto. The semiconductor memory is, for example, a RAM or a ROM. The RAM is, for example, an SRAM or a DRAM. The ROM is, for example, an EEPROM. The storage unit 13 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 13 may store the information of the result analyzed or processed by the control unit 11. The storage unit 13 may store various information related to the operation or control of the information processing apparatus 1. The storage unit 13 may store a system program, an application program, a database, and embedded software, etc. The storage unit 13 may be provided outside the information processing apparatus 1 and accessed from the information processing apparatus 1.
[0018] The information processing method executed by the information processing apparatus 1 of the present embodiment will be described in detail.
[0019] [Setting of analysis window] The control unit 11 of the information processing apparatus 1 acquires a captured image captured by an in-vehicle camera via the communication unit 12. The captured image here is an image of the front of the traveling vehicle, as shown in the captured image 21 of FIG. 2A. The vehicle 22 is a vehicle traveling forward. The control unit 11 acquires (for example, extracts) an area effective for identifying the identification target from the captured image by gradient-weighted class activation mapping such as Grad-Cam. The identification target here is, as an example, a road surface. The acquired area is shown in FIG. 2B. As an additional example, the control unit 11 may limit the range of the area using any other arbitrary rules. For example, as a rule, the control unit 11 may limit the area to the lower half within the captured image.
[0020] The control unit 11 acquires information on the presence of an object (for example, a road shoulder, a pedestrian) or a space different from the identification target from the captured image by an object detection algorithm. The object detection algorithm may be, for example, YOLO (You Only Look Once).
[0021] As shown in FIG. 2C, the control unit 11 dynamically sets an analysis window 23 effective for identifying the identification target in the captured image by using the acquired area and the presence information together. For example, the control unit 11 sets the analysis window 23 in an area within the acquired area excluding the object or space indicated by the presence information. As an additional example or an alternative example, when the control unit 11 detects a road shoulder at the right end of the captured image, the control unit 11 may exclude the area corresponding to the road shoulder and also exclude the area at the left end of the captured image. As an additional example or an alternative example, the control unit 11 may update the position or size of the analysis window in real time or at a predetermined time interval.
[0022] Another example of the dynamic setting of the analysis window according to the present embodiment is shown in FIG. 3A. As shown in the figure, the set analysis window 31 includes the identification target 33 (here, the road surface) but does not include the surrounding vehicle 32 that has no relation to the identification target. In contrast, in the conventional method shown in FIG. 3B, the conventional analysis window 34 includes not only the road surface 36 but also a part of the surrounding vehicle 35.
[0023] In FIG. 4, a flowchart of an information processing method by the information processing apparatus 1 is described.
[0024] In S1, the control unit 11 of the information processing apparatus 1 acquires a captured image. In S2, the control unit 11 acquires, from the captured image, an area effective for identifying an identification target by gradient load class activation mapping. In S3, the control unit 11 acquires, from the captured image, presence information of an object or space different from the identification target by an object detection algorithm. S2 and S3 may be executed in the reverse order. In S4, the control unit 11 sets an analysis window in the captured image from the acquired area and presence information.
[0025] [Classifier for learning the tendencies of multiple classifiers] As shown in FIG. 5, the control unit 11 acquires a captured image 41. The captured image 41 may be an image in which an analysis window is set by the above method, or may be another image captured by an in-vehicle camera.
[0026] The control unit 11 applies each of a first classifier 42 and a second classifier 43 to the captured image. The first classifier 42 here is used for identifying any identification target in a captured image captured at night, which is the first time period. The second classifier 43 here is used for identifying any identification target in a captured image captured during the day, which is the second time period. The first classifier 42 and the second classifier 43 may include existing learning models. The identification target here is the road surface. When an analysis window is set in the captured image 41, identification of the identification target may be performed in the analysis window.
[0027] The control unit 11 acquires, from the first classifier 42, identification labels L1 to L6 and the confidence level (e.g., a numerical value) for each identification label. The identification label indicates the state of the identification target in the captured image. The number of identification labels is arbitrary. The control unit 11 acquires, from the second classifier 43, identification labels L7 to L12 and the confidence level for each identification label.
[0028] The control unit 11 sets the identification labels L1 to L12 and their respective confidence levels as a 12-dimensional feature vector 44.
[0029] The control unit 11 inputs the feature vector 44 together with the correct data into the integrated discriminator 45, and the integrated discriminator 45 learns the tendency of the feature vector 44. The integrated discriminator 45 may be any algorithm for performing classification or the like, and may include, for example, ELM (Extreme Learning Machine) or random forest.
[0030] The control unit 11 obtains one final identification label indicating the state of the object to be identified in the captured image from the integrated discriminator 45. When an analysis window is set in the captured image, the control unit 11 may analyze the state of the object to be identified in the analysis window.
[0031] In FIG. 6, a flowchart of the information processing method by the information processing apparatus 1 is described.
[0032] In S11, the control unit 11 of the information processing apparatus 1 stores the first discriminator 42 in the storage unit 13 and stores the second discriminator 43 in the storage unit 13. In S12, the control unit 11 acquires a captured image and applies the first discriminator 42 and the second discriminator 43 to the captured image. In S13, the control unit 11 acquires one or more identification labels indicating the state of the object to be identified in the captured image and the confidence level for each identification label from each of the first discriminator 42 and the second discriminator 43. In S14, the control unit 11 sets the identification label and the confidence level as a feature vector and learns the tendency of the feature vector in the integrated discriminator 45. In S15, the control unit 11 obtains a final identification label indicating the state of the object to be identified in the captured image from the integrated discriminator 45.
[0033] Although the present disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art can make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these modifications and corrections are included in the scope of the present disclosure. For example, the configurations or functions included in each embodiment can be rearranged so as not to be logically contradictory. In addition, the configurations or functions included in each embodiment can be used in combination with other embodiments, and it is possible to combine, divide, or omit a part of a plurality of configurations or functions into one.
[0034] Also, for example, an embodiment in which a general-purpose computer functions as the information processing apparatus 1 according to the above-described embodiment is also possible. Specifically, a program describing the processing contents for realizing each function of the information processing apparatus 1 according to the above-described embodiment is stored in the memory of a general-purpose computer, and the program is read out and executed by a processor. Therefore, the present disclosure can also be realized as a program executable by a processor or a non-temporary computer-readable medium storing the program. Non-temporary computer-readable media include, for example, magnetic recording devices, optical disks, magneto-optical recording media, or semiconductor memories.
Industrial Applicability
[0035] According to the present disclosure, robust and stable identification can be performed on the captured image regardless of day or night.
[0036] [Contribution to the Sustainable Development Goals (SDGs) Led by the United Nations] Towards the realization of a sustainable society, the SDGs have been proposed. One embodiment of the present invention can be a technology that contributes to "No. 9 - Build the foundation for industry and technological innovation", "No. 12 - The responsibility to create, the responsibility to use", and "No. 13 - Take specific measures against climate change", etc.
Explanation of Signs
[0037] 1: Information processing apparatus
Claims
1. An information processing method executed by an information processing apparatus, The information processing apparatus includes a control unit, a communication unit, and a storage unit, and can communicate with a network via the communication unit, Storing a first identifier used for identifying an identification target in a captured image captured in a first time period in the storage unit; Storing a second identifier used for identifying an identification target in a captured image captured in a second time period in the storage unit; Acquiring a captured image and applying the first identifier and the second identifier to the captured image; Obtaining, from each of the first identifier and the second identifier, one or more identification labels indicating the state of the identification target in the captured image and a confidence level for each identification label; Setting the identification label and the confidence level as a feature vector and training the trend of the feature vector in an integrated identifier; Obtaining a final identification label indicating the state of the identification target in the captured image from the integrated identifier; An information processing method including the above.
2. In the information processing method according to Claim 1, The first time period is night and the second time period is day. An information processing method.
3. In the information processing method according to Claim 1, The identification target includes a road surface. An information processing method.
4. In the information processing method according to Claim 1, Acquiring a captured image; Obtaining, from the captured image, a region effective for identifying the identification target by gradient-weighted class activation mapping; Obtaining, from the captured image, presence information of an object or space different from the identification target by an object detection algorithm; Setting an analysis window in the captured image from the acquired area and the presence information; Analyzing the state of the object to be identified in the analysis window; An information processing method including the above.
5. An information processing apparatus including a control unit, a communication unit, and a storage unit, and capable of communicating with a network via the communication unit, wherein the control unit Stores a first identifier used for identifying an object to be identified in a captured image captured in a first time period in the storage unit; Stores a second identifier used for identifying an object to be identified in a captured image captured in a second time period in the storage unit; Acquires a captured image and applies the first identifier and the second identifier to the captured image; Obtains one or more identification labels indicating the state of the object to be identified in the captured image and a confidence level for each identification label from each of the first identifier and the second identifier; Sets the identification label and the confidence level as a feature vector, and learns the trend of the feature vector in an integrated identifier; Obtains a final identification label indicating the state of the object to be identified in the captured image from the integrated identifier; An information processing apparatus that executes operations including the above.