Information processing method and information processing device
The information processing method dynamically sets an analysis window in in-vehicle camera images to exclude irrelevant objects, thereby improving the identification accuracy of road surface conditions, addressing the limitations of existing methods.
Patent Information
- Application Number
- JP2023208017
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-12-08
- Publication Date
- 2025-06-19
AI Technical Summary
Existing methods for analyzing images from in-vehicle cameras do not effectively improve identification accuracy by avoiding areas such as preceding vehicles, which are irrelevant to road surface condition identification.
An information processing method that dynamically sets an analysis window in captured images by excluding irrelevant objects or spaces, using gradient-weighted class activation mapping and object detection algorithms to enhance identification accuracy.
The method significantly improves the identification accuracy of road surface conditions in real-time by focusing on relevant areas within the captured images, thereby enhancing the reliability of image analysis.
Smart Images
Figure 2025092253000001_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to an information processing method and an information processing apparatus.
Background Art
[0002] Conventionally, a technique of using data obtained from a fixed analysis window as input to Mobile Nets is known (for example, Non-Patent Documents 1 and 2). Specifically, as parameters set during the experiment, an analysis window with 700 pixels horizontally and 280 pixels vertically is set. At this time, the X coordinate of the upper left coordinate of the analysis window is 500, and the Y coordinate is 700. Note that the number of pixels of the original image captured by the in-vehicle camera is 1920×1080.
Prior Art Documents
Non-Patent Documents
[0003]
Non-Patent Document 1
Non-Patent Document 2
Summary of the Invention
Problems to be Solved by the Invention
[0004] When assuming real-time analysis of an image captured by an in-vehicle camera, from the viewpoint of improving the identification accuracy, it is preferable to determine the road surface condition while avoiding in real time the area including the preceding vehicle and the like. However, this is not considered in the method of the above-described background art, and there is room for improvement.
[0005] An object of the present disclosure made in view of such circumstances is to improve the identification accuracy in a captured image.
Means for Solving the Problem
[0006] 〔1〕An information processing method according to an embodiment of the present disclosure is an information processing method executed by an information processing apparatus, the information processing apparatus includes a control unit, a communication unit, and a storage unit, and is communicable with a network via the communication unit, acquiring a captured image, acquiring, from the captured image, an area effective for identifying an object to be identified by gradient load class activation mapping, acquiring, from the captured image, presence information of an object or space different from the object to be identified by an object detection algorithm, setting an analysis window in the captured image from the acquired area and the presence information, and including. With this configuration, the identification accuracy in the captured image can be improved.
[0007] 〔2〕An information processing method according to an embodiment of the present disclosure is in the information processing method described in the above 〔1〕, it is preferable that the object to be identified includes a road surface. With this configuration, the identification accuracy of the road surface can be further improved.
[0008] 〔3〕An information processing method according to an embodiment of the present disclosure is In the information processing method described in the above [1] or [2], including setting the analysis window in an area obtained by excluding the object or space indicated by the presence information within the acquired area. With this configuration, the identification accuracy can be further improved.
[0009] 〔4〕The information processing method according to an embodiment of the present disclosure is in the information processing method according to any one of the above [1] to [3], storing a first identifier used for identifying an identification target in a captured image captured in a first time zone in the storage unit; storing a second identifier used for identifying an identification target in a captured image captured in a second time zone in the storage unit; applying the first identifier and the second identifier to the captured image; acquiring, from each of the first identifier and the second identifier, one or more identification labels indicating the state of the identification target in the captured image and a confidence level for each identification label; setting the identification label and the confidence level as a feature vector, and learning the tendency of the feature vector in an integrated identifier; acquiring a final identification label indicating the state of the identification target in the captured image from the integrated identifier; including. With this configuration, robust and stable identification can be performed on the captured image.
[0010] 〔5〕The information processing apparatus according to an embodiment of the present disclosure is an information processing apparatus including a control unit, a communication unit, and a storage unit, and capable of communicating with a network via the communication unit, wherein the control unit acquires a captured image; acquires, from the captured image, an area effective for identifying an identification target by gradient-weighted class activation mapping; Using an object detection algorithm, obtaining presence information of an object or a space different from the object to be identified from the captured image, Setting an analysis window in the captured image from the obtained area and the presence information, Performing an operation including this. With this configuration, the identification accuracy in the captured image can be improved.
Advantages of the Invention
[0011] According to the present disclosure, the identification accuracy in the captured image can be improved.
Brief Description of the Drawings
[0012]
Figure 1
Figure 2A
Figure 2B
Figure 2C
Figure 3A
Figure 3B
Figure 4
Figure 5
Figure 6
Embodiments for Carrying Out the Invention
[0013] The information processing apparatus 1 shown in FIG. 1 may be a server or a cloud server. The information processing apparatus 1 may be installed, for example, in a dedicated facility for a business operator or a shared facility including a data center. As an alternative, the information processing apparatus 1 may be installed in a traveling vehicle. The processing executed by the information processing apparatus 1 may be executed by a plurality of distributed information processing apparatuses 1.
[0014] The information processing apparatus 1 includes a control unit 11, a communication unit 12, and a storage unit 13. Each component of the information processing apparatus 1 is communicably connected to each other.
[0015] The control unit 11 includes, for example, one or more general-purpose processors including a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control unit 11 may include one or more dedicated processors specialized for specific processing. Instead of including a processor, the control unit 11 may include one or more dedicated circuits. The dedicated circuit may be, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit 11 may include an ECU (Electronic Control Unit). The control unit 11 transmits and receives arbitrary information via the communication unit 12.
[0016] The communication unit 12 includes a communication module corresponding to one or more wired or wireless LAN (Local Area Network) standards for connecting to a network. The network may be, for example, the Internet, a mobile communication network, a fixed communication network, a LAN, or a combination thereof. The communication unit 12 may include a module corresponding to one or more mobile communication standards including LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation). The communication unit 12 may include a communication module or the like corresponding to one or more short-range communication standards or specifications including Bluetooth (registered trademark), AirDrop (registered trademark), IrDA, ZigBee (registered trademark), Felica (registered trademark), or RFID. The communication unit 12 transmits and receives arbitrary information via the network.
[0017] The storage unit 13 includes, for example, a semiconductor memory, a magnetic memory, an optical memory, or a combination of at least two of these, but is not limited thereto. The semiconductor memory is, for example, a RAM or a ROM. The RAM is, for example, an SRAM or a DRAM. The ROM is, for example, an EEPROM. The storage unit 13 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 13 may store the information of the result analyzed or processed by the control unit 11. The storage unit 13 may store various information related to the operation or control of the information processing apparatus 1. The storage unit 13 may store a system program, an application program, a database, and embedded software, etc. The storage unit 13 may be provided outside the information processing apparatus 1 and accessed from the information processing apparatus 1.
[0018] The information processing method executed by the information processing apparatus 1 of the present embodiment will be described in detail.
[0019] [Setting of Analysis Window] The control unit 11 of the information processing apparatus 1 acquires the captured image captured by the in-vehicle camera via the communication unit 12. The captured image here is an image in front of the traveling vehicle, as shown in the captured image 21 of FIG. 2A. The vehicle 22 is a vehicle traveling forward. The control unit 11 acquires (for example, extracts) an area effective for the identification of the identification target from the captured image by gradient-weighted class activation mapping such as Grad-Cam. The identification target here is, as an example, a road surface. The acquired area is shown in FIG. 2B. As an additional example, the control unit 11 may limit the range of the area using any other arbitrary rule. For example, the control unit 11 may limit the area to the lower half within the captured image as a rule.
[0020] The control unit 11 acquires the presence information of an object (for example, a road shoulder, a pedestrian) or a space different from the identification target from the captured image by an object detection algorithm. The object detection algorithm may be, for example, YOLO (You Only Look Once).
[0021] As shown in FIG. 2C, the control unit 11 dynamically sets, in the captured image, an analysis window 23 effective for identifying an object to be identified by using together the acquired region and the presence information. For example, the control unit 11 sets the analysis window 23 in a region within the acquired region excluding the object or space indicated by the presence information. As an additional example or an alternative example, when the control unit 11 detects a road shoulder at the right end of the captured image, the control unit 11 may exclude the region corresponding to the road shoulder and also exclude the region at the left end of the captured image. As an additional example or an alternative example, the control unit 11 may update the position or size of the analysis window in real time or at a predetermined time interval.
[0022] Another example of the dynamic setting of the analysis window according to the present embodiment is shown in FIG. 3A. As shown in the figure, the set analysis window 31 includes the object to be identified 33 (here, the road surface) but does not include the surrounding vehicles 32 that have no relation to the object to be identified. In contrast, in the conventional method shown in FIG. 3B, the conventional analysis window 34 includes not only the road surface 36 but also a part of the surrounding vehicles 35.
[0023] In FIG. 4, a flowchart of the information processing method by the information processing apparatus 1 is described.
[0024] In S1, the control unit 11 of the information processing apparatus 1 acquires a captured image. In S2, the control unit 11 acquires, from the captured image, a region effective for identifying the object to be identified by gradient load class activation mapping. In S3, the control unit 11 acquires, from the captured image, the presence information of an object or space different from the object to be identified by an object detection algorithm. S2 and S3 may be executed in the reverse order. In S4, the control unit 11 sets an analysis window in the captured image from the acquired region and the presence information.
[0025] [Classifier for learning the tendencies of multiple classifiers] As shown in FIG. 5, the control unit 11 acquires a captured image 41. The captured image 41 may be an image in which the analysis window is set by the above method, or may be another image captured by an in-vehicle camera.
[0026] The control unit 11 applies each of the first discriminator 42 and the second discriminator 43 to the captured image. Here, the first discriminator 42 is used to identify any object to be identified in the captured image taken at night, which is the first time period. Here, the second discriminator 43 is used to identify any object to be identified in the captured image taken during the day, which is the second time period. The first discriminator 42 and the second discriminator 43 may include existing learning models. Here, the object to be identified is the road surface. When an analysis window is set in the captured image 41, the identification of the object to be identified may be performed in the analysis window.
[0027] The control unit 11 obtains from the first discriminator 42 the identification labels L1 to L6 and the confidence level (e.g., a numerical value) for each identification label. The identification label indicates the state of the object to be identified in the captured image. The number of identification labels is arbitrary. The control unit 11 obtains from the second discriminator 43 the identification labels L7 to L12 and the confidence level for each identification label.
[0028] The control unit 11 sets the identification labels L1 to L12 and their respective confidence levels as a 12-dimensional feature vector 44.
[0029] The control unit 11 inputs the feature vector 44 together with the correct data into the integrated discriminator 45, and the integrated discriminator 45 learns the tendency of the feature vector 44. The integrated discriminator 45 may be any algorithm for performing classification or the like, and may include, for example, ELM (Extreme Learning Machine) or random forest.
[0030] The control unit 11 obtains one final identification label indicating the state of the object to be identified in the captured image from the integrated discriminator 45. When an analysis window is set in the captured image, the control unit 11 may analyze the state of the object to be identified in the analysis window.
[0031] In FIG. 6, a flowchart of the information processing method by the information processing apparatus 1 is described.
[0032] In S11, the control unit 11 of the information processing apparatus 1 stores the first identifier 42 in the storage unit 13 and stores the second identifier 43 in the storage unit 13. In S12, the control unit 11 acquires a captured image and applies the first identifier 42 and the second identifier 43 to the captured image. In S13, the control unit 11 acquires, from each of the first identifier 42 and the second identifier 43, one or more identification labels indicating the state of the object to be identified in the captured image and the confidence level for each identification label. In S14, the control unit 11 sets the identification label and the confidence level as a feature vector and learns the tendency of the feature vector in the integrated identifier 45. In S15, the control unit 11 acquires the final identification label indicating the state of the object to be identified in the captured image from the integrated identifier 45.
[0033] Although the present disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art can make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these modifications and corrections are included in the scope of the present disclosure. For example, the configurations or functions included in each embodiment can be rearranged so as not to be logically inconsistent. In addition, the configurations or functions included in each embodiment can be used in combination with other embodiments, and it is possible to combine, divide, or omit a part of a plurality of configurations or functions into one.
[0034] Also, for example, an embodiment in which a general-purpose computer functions as the information processing apparatus 1 according to the above-described embodiment is also possible. Specifically, a program describing the processing content for realizing each function of the information processing apparatus 1 according to the above-described embodiment is stored in the memory of a general-purpose computer, and the program is read out and executed by a processor. Therefore, the present disclosure can also be realized as a program executable by a processor or a non-temporary computer-readable medium storing the program. The non-temporary computer-readable medium includes, for example, a magnetic recording device, an optical disk, a magneto-optical recording medium, or a semiconductor memory.
Industrial Applicability
[0035] According to the present disclosure, robust and stable identification can be performed on a captured image.
[0036] [Contribution to the Sustainable Development Goals (SDGs) led by the United Nations] Towards the realization of a sustainable society, the SDGs have been proposed. One embodiment of the present invention can be a technology that contributes to "No. 9 - Build the foundation of industry and technological innovation" and "No. 12 - The responsibility to produce and the responsibility to consume" and "No. 13 - Specific measures against climate change", etc.
Description of reference numerals
[0037] 1: Information processing apparatus
Claims
1. An information processing method executed by an information processing apparatus, The information processing apparatus includes a control unit, a communication unit, and a storage unit, and is capable of communicating with a network via the communication unit, obtaining a captured image, obtaining, from the captured image, an area effective for identifying an object to be identified by gradient load class activation mapping, obtaining, from the captured image, presence information of an object or space different from the object to be identified by an object detection algorithm, setting an analysis window in the captured image from the obtained area and the presence information, An information processing method including the above.
2. In the information processing method according to Claim 1, The object to be identified includes a road surface, and an information processing method.
3. In the information processing method according to Claim 1, The information processing method includes setting the analysis window in an area obtained by excluding the object or space indicated by the presence information within the obtained area.
4. In the information processing method according to Claim 1, storing, in the storage unit, a first identifier used for identifying an object to be identified in a captured image captured in a first time zone, storing, in the storage unit, a second identifier used for identifying an object to be identified in a captured image captured in a second time zone, applying the first identifier and the second identifier to the captured image, obtaining, from each of the first identifier and the second identifier, one or more identification labels indicating the state of the object to be identified in the captured image and a confidence level for each identification label, setting the identification label and the confidence level as a feature vector, and learning the tendency of the feature vector in an integrated identifier, Obtaining, from the integrated identifier, a final identification label indicating the state of the object to be identified in the captured image; An information processing method including this.
5. An information processing apparatus including a control unit, a communication unit, and a storage unit, and capable of communicating with a network via the communication unit, The control unit: Obtaining a captured image; Obtaining, from the captured image, a region effective for identifying the object to be identified by gradient-weighted class activation mapping; Obtaining, from the captured image, presence information of an object or space different from the object to be identified by an object detection algorithm; Setting an analysis window in the captured image from the obtained region and the presence information; An information processing apparatus that executes operations including this.