Image processing system, image processing method, and program

Abstract

The present invention is capable of providing images that enable appropriate object recognition. This image processing system comprises: an extraction unit that extracts a region including an object, from a travel region in a travel image captured by an imaging device mounted on a movable body; an identification unit that identifies the object existing in the extracted region; and a compression unit that compresses the travel image so that the region of the object identified as an unknown object by the identification unit has higher image quality than a region other than the region of the object identified as the unknown object in the travel image.

Description

Image processing system, image processing method, and program

[0001] The present disclosure relates to an image processing system, an image processing method, and a program.

[0002] As related art, Patent Document 1 discloses an image transmission system. In the image transmission system described in Patent Document 1, an in-vehicle device acquires an image of a road on which a vehicle is traveling from a camera mounted on the vehicle. The in-vehicle device determines whether an important object is included in the acquired image. When it is determined that an important object is included in the image, the in-vehicle device non-uniformly compresses the image so that the important object is compressed at a lower compression rate than the non-important object. The in-vehicle device transmits the compressed image to a server.

[0003] Japanese Patent Application Laid-Open No. 2021-128532

[0004] In Patent Document 1, the amount of information to be reduced for important parts of an image is suppressed, while the amount of information for unimportant parts is further reduced. However, an object determined to be an important object is not necessarily important. In Patent Document 1, for example, when there is some object on the road that is not any distinguishable object, the area of an object not identified as a specific object is compressed at a high compression rate as an unimportant area. In that case, there is a possibility that the objects in the transmitted video cannot be properly recognized.

[0005] One exemplary object of the present disclosure is to provide an image processing system, an image processing method, and a program that can provide an image enabling appropriate object recognition.

[0006] An image processing system according to a first aspect of the present disclosure includes an extraction unit that extracts an area including an object in a driving area of a driving image captured by an imaging device mounted on a moving body, an identification unit that identifies an object existing in the extracted area, and a compression unit that compresses the driving image so that the area of an object identified as an unknown object in the identification unit has higher image quality than an area other than the area of the object identified as the unknown object in the driving image.

[0007] An image processing method according to a second aspect of the present disclosure includes extracting a region containing an object in the driving region of a driving image captured by an imaging device mounted on a moving body, identifying the object present in the extracted region, and compressing the driving image such that the region of the object identified as an unknown object has higher image quality than the region of the driving image other than the region of the object identified as an unknown object.

[0008] A program according to a third aspect of this disclosure causes a computer to perform a process that includes extracting a region containing an object in the driving region of a driving image captured by an imaging device mounted on a moving body, identifying the object present in the extracted region, and compressing the driving image such that the region of the object identified as an unknown object has higher image quality than the region of the driving image other than the region of the object identified as an unknown object.

[0009] The image processing system, image processing method, and program relating to this disclosure can provide images that enable appropriate object recognition.

[0010] This is a block diagram showing a schematic configuration example of an image processing system related to this disclosure. This is a block diagram showing a configuration example of a first image processing system related to this disclosure. This is a block diagram showing a configuration example of an image processing device. This is a flowchart showing the operating procedure of an image processing device. This is a block diagram showing a configuration example of a second image processing system related to this disclosure. This is a block diagram showing a configuration example of an image processing device. This is a block diagram showing a configuration example of a computer device.

[0011] Prior to describing the embodiments of this disclosure, the inventors will explain the matters they have considered. Here, we consider a system for collecting images or videos taken from one or more vehicles using cameras mounted on the vehicles. The collected videos are labeled and stored in a video storage unit. Users, such as developers of autonomous driving artificial intelligence (AI), search for driving videos in which a desired object is visible using search keywords that include the desired object. The retrieved driving videos are used for training object recognition models, etc.

[0012] Training and evaluating an image recognition model requires not only a large amount of information on normal driving but also information on corner cases. Here, a corner case can refer to an unusual case that does not usually occur when a vehicle is driving normally. An image that constitutes a corner case is, for example, an image in which an object that does not normally exist is present on the road, and it is thought that objects that do not normally exist on the road are not given high image quality as important objects in compression. If the areas other than the area of ​​important objects that can be recognized in compression are given low image quality, the collected image cannot be used as an image that constitutes a corner case. The inventors of the present invention came up with the present invention as a result of considering the above matters.

[0013] The outline of this disclosure will now be explained. Figure 1 is a block diagram showing a schematic configuration example of an image processing system according to this disclosure. The image processing system 10 has an extraction unit 11, an identification unit 12, and a compression unit 13.

[0014] The extraction unit 11 acquires a driving image captured by an imaging device mounted on the moving vehicle. The extraction unit 11 extracts the region containing an object from the driving region of the acquired driving image. The identification unit 12 identifies the object present in the region extracted by the extraction unit 11. The compression unit 13 compresses the driving image so that the region of the object identified as an unknown object by the identification unit 12 has higher image quality than the region of the driving image other than the region of the object identified as an unknown object.

[0015] In this disclosure, the compression unit 13 compresses the video such that the region containing the unknown object has relatively high image quality, and the region without the unknown object has relatively low image quality. By making the region containing the unknown object relatively high quality, the compression unit 13 can reduce the overall data size of the video while providing a clear image in the region containing the unknown object. In this disclosure, the image processing system 10 can provide an image that enables appropriate object recognition. The image provided by the image processing system 10 can be used, for example, as corner case video in the development of autonomous driving AI.

[0016] The embodiments of this disclosure will be described in detail below with reference to the drawings. Note that the following description and drawings have been omitted and simplified as appropriate for clarity of explanation. Furthermore, in the following drawings, the same elements and similar elements are denoted by the same reference numerals, and redundant explanations have been omitted where necessary.

[0017] Figure 2 is a block diagram showing an example configuration of a first image processing system according to the present disclosure. The first embodiment will be described with reference to Figure 2. The image processing system 100 shown in Figure 2 includes an image processing device 110, an image storage unit 130, an image acquisition device 140, and one or more cameras 210.

[0018] Each of the one or more cameras 210 is, for example, an imaging device mounted on a mobile body 200. The mobile body 200 is configured as, for example, a road-traveling land vehicle such as a car, bus, taxi, or truck. The mobile body 200 may also be configured as heavy machinery such as a forklift, an Automatic Guided Vehicle (AGV), an autonomous mobile robot (AMR), or a drone. The cameras 210 capture images or videos of the area in front of the mobile body, for example. The number of cameras 210 mounted on each mobile body 200 is not limited to one. Each mobile body 200 may be equipped with multiple cameras 210.

[0019] In this embodiment, the image processing device 110 is mounted on a mobile body 200. The image processing device 110 is connected to the image acquisition device 140 via a network 150. The network 150 includes, for example, a wireless communication network using a communication line standard such as Long Term Evolution (LTE). The network 150 may also include a wireless communication network such as Wi-Fi® or a fifth-generation mobile communication system. The image processing device 110 compresses images captured by the camera 210. The image acquisition device 140 collects images or videos compressed by the image processing device 110 from one or more mobile bodies 200 via the network 150. The image processing device 110 corresponds to the image processing system 10 shown in Figure 1.

[0020] The image acquisition device 140 stores the acquired images or videos in the image storage unit 130. The image storage unit 130 is configured as a storage device such as a solid-state drive (SSD) or a hard disk drive (HDD). The image storage unit 130 may also be cloud storage. The image processing device 110 does not necessarily have to be mounted on the mobile body 200. The image processing device 110 may be included in the image acquisition device 140. If the image processing device 110 is included in the image acquisition device 140, the image processing device 110 may receive images or videos transmitted from multiple mobile bodies 200 via the network 150 and compress the received images or videos.

[0021] Figure 3 is a block diagram showing an example configuration of the image processing device 110. The image processing device 110 includes an image acquisition unit 111, an extraction unit 112, an identification unit 113, an image compression unit 114, an image transmission unit 115, and a label list 120. The image processing device 110 is configured as a computer device having, for example, one or more memories and one or more processors. At least some of the functions of each part in the image processing device 110 can be realized by the processor operating according to a program read from memory.

[0022] The image acquisition unit 111 acquires images or videos captured by the camera 210. In this embodiment, the image acquisition unit 111 acquires driving images captured by the camera 210 of the mobile body 200 on which the image processing device 110 is mounted. In this disclosure, the terms "image" and "video" are not necessarily distinguished and are used interchangeably, and a driving image may mean a driving video. The extraction unit 112 extracts the driving region from the acquired video. Here, the driving region indicates the region in the driving video that corresponds to the location where the mobile body 200 is driving. For example, if the mobile body 200 is a vehicle driving on a road, the driving region may mean the road region, i.e., the region of the road on which the mobile body 200 is driving. The extraction unit 112 also extracts regions in the driving region where objects exist. The extraction unit 112 extracts the driving region from the driving image, for example, by semantic segmentation. The extraction unit 112 further extracts regions containing some object from the driving image. The extraction unit 112 extracts, for example, the regions in the driving area where objects exist for each frame of the acquired video.

[0023] The extraction unit 112 may extract an area containing an object from a specific area within the driving area, depending on the driving state of the moving body 200. In other words, the extraction unit 112 may limit the driving area from which an area containing an object is extracted, depending on the driving state of the moving body 200. For example, depending on the driving state of the moving body 200, the extraction unit 112 extracts an area containing an object from at least one of the following areas: the area of ​​the lane in which the moving body 200 is traveling, the area of ​​the lane adjacent to the lane in which the moving body 200 is traveling, and the area of ​​the lane opposite to the lane in which the moving body 200 is traveling. The driving state of the moving body includes, for example, turning right or left at an intersection, and changing lanes.

[0024] The extraction unit 112 may extract areas where objects exist when the travel area on which the mobile body 200 travels satisfies predetermined conditions. For example, the extraction unit 112 may determine whether the travel area on which the mobile body 200 travels satisfies predetermined conditions based on the position information of the mobile body 200. The extraction unit 112 may extract areas where objects exist from the travel area when, for example, the mobile body 200 is located at an intersection. Alternatively, the extraction unit 112 may extract areas where objects exist from the travel area at a designated specific intersection, for example, an intersection where traffic accidents frequently occur. The extraction unit 112 may extract areas where objects exist from the travel area in locations where the road shape is a specific shape. For example, the extraction unit 112 may extract areas where objects exist from the travel area at an irregular intersection such as a three-lane road or a five-way intersection. The extraction unit 112 corresponds to the extraction unit 11 shown in Figure 1.

[0025] The identification unit 113 identifies objects present in the region extracted as an area where objects exist. The identification unit 113 identifies objects using, for example, a label list 120. The label list 120 is a list containing specific objects (object names). The label list 120 includes, for example, objects important to traffic such as vehicles, people, bicycles, and traffic lights, or objects frequently detected on roads. In each frame, the identification unit 113 identifies whether or not an object present in the region extracted as an area where objects exist is an object included in the label list 120. If an object present in the region extracted as an area where objects exist is not included in the label list 120, the identification unit 113 identifies the object present in the extracted region as an unknown object.

[0026] The identification unit 113 generates an index value indicating object-likeness for each of one or more identifiable objects. For example, the identification unit 113 generates an index value indicating object-likeness, i.e., an object score, for each object included in the label list 120. If the object score for any object is greater than a determination threshold, the identification unit 113 identifies the object in the extracted region as an object whose object score is greater than the determination threshold. If the object score for any object is less than or equal to a predetermined threshold, the identification unit 113 identifies the object in the extracted region as an unknown object. The threshold for determining an object as unknown may be a smaller value than the threshold for determining a specific object. The identification unit 113 corresponds to the identification unit 12 shown in Figure 1.

[0027] The image compression unit 114 compresses the image acquired by the image acquisition unit 111 using the identification result from the identification unit 113. In image compression, the image compression unit 114 sets the area of ​​the object identified as an unknown object by the identification unit 113 as the gaze area. The image compression unit 114 compresses the image so that the image quality of the gaze area is higher than that of the areas other than the gaze area. As a result of image compression, the unknown object becomes relatively high quality in the compressed image, while other areas become low quality. By compressing the image so that the unknown object becomes relatively high quality and other areas become low quality, it is possible to reduce the image data size to about 1 / 10 of that of the uncompressed image. The image compression unit 114 transmits the compressed image to the image acquisition device 140 via the network 150. The image acquisition device 140 can collect rare case driving images, including unknown objects, from multiple moving objects 200. The image compression unit 114 corresponds to the compression unit 13 shown in Figure 1.

[0028] Next, the operation procedure will be explained. Figure 4 is a flowchart of the operation procedure of the image processing device 110. The operation procedure of the image processing device 110 corresponds to the image processing method. The image acquisition unit 111 acquires an image captured by the camera 210 (step S1). The extraction unit 112 extracts a region containing an object from the image acquired in step S1 within the travel area of ​​the mobile body 200 (step S2).

[0029] The identification unit 113 identifies objects present in the region extracted in step S2 (step S3). For example, in step S2, the identification unit 113 generates an object score for each object included in the label list 120. If no object in the label list 120 has an object score greater than a predetermined threshold, the identification unit 113 identifies the object present in the extracted region as an unknown object. In other words, if the object scores of all objects in the label list 120 are below a predetermined threshold, the identification unit 113 identifies the object present in the extracted region as an unknown object.

[0030] The image compression unit 114 compresses the image containing the region identified as an unknown object (step S4). In step S4, the image compression unit 114 compresses the image such that the image quality of the region containing the unknown object is higher than that of other regions. The image compression unit 114 transmits the compressed image to the image acquisition device 140 (step S5). The image acquisition device 140 stores the image received from the image processing device 110 in the image storage unit 130. The image stored in the image storage unit 130 can be used as a rare case image for training the object recognition model. The image stored in the image storage unit 130 can also be used for evaluating the trained object recognition model.

[0031] In addition to identifying that an object in the extracted region is an unknown object, the identification unit 113 may also identify which of the objects in the extracted region is included in the label list 120. The image compression unit 114 may compress the image so that the image quality of the region containing the objects in the label list 120 is higher than that of other regions. The image compression unit 114 may transmit the image containing the objects in the label list 120 as a normal driving image to the image acquisition device 140 or another image acquisition device. Normal driving images may be separated from rare case driving images and stored in the image storage unit 130. Normal driving images can be used for image retrieval.

[0032] In this embodiment, in the image processing device 110 mounted on the mobile body 200, the identification unit 113 identifies unknown objects present in the driving area. The image compression unit 114 compresses the image so that the image quality of the area containing the unknown object on the road is higher than that of other areas. The image transmission unit 115 transmits the compressed image to the image acquisition device 140. The image acquisition device 140 collects images from multiple mobile bodies 200 in which the area containing the unknown object has been relatively enhanced in image quality, and stores the collected images in the image storage unit 130. In this embodiment, the image storage unit 130 can store images with reduced overall data volume while maintaining high image quality for the area containing the unknown object on the road, thereby reducing storage costs during image storage. The area containing the unknown object is compressed with relatively high image quality, and it is considered that the area containing the unknown object in the compressed image has image quality that can be recognized by AI.

[0033] For example, when it comes to video footage used in the development of autonomous driving AI, the probability of an unknown object being present on the road is low, so vehicle manufacturers need to collect a vast amount of driving footage to capture images of unknown objects. For example, a scene of a helicopter on the road is extremely rare. In this embodiment, objects not included in the label list 120 are important, and the image processing device 110 transmits high-resolution video footage containing objects not included in the label list 120 to the image acquisition device 140. In this embodiment, the image acquisition device 140 can acquire corner case information as high-quality video from a moving object 200, which is a general vehicle such as a passenger car.

[0034] Figure 5 is a block diagram showing an example configuration of a second image processing system according to the present disclosure. The second embodiment will be described using Figure 5. The image processing system 100a shown in Figure 5 includes an image processing device 110a, an image storage unit 130, and one or more cameras 210. In this embodiment, the image processing device 110a receives images or videos captured by the cameras 210 from a plurality of mobile bodies 200 via a network 150. The image processing device 110a compresses the images or videos received from the plurality of mobile bodies 200 and stores the compressed images or videos in the image storage unit 130. The images or videos captured by the cameras 210 may be stored in a portable storage device on the mobile body 200. If the images or videos are stored in a portable storage device, the image processing device 110a may acquire the images or videos from the portable storage device removed from the mobile body 200.

[0035] Figure 6 is a block diagram showing an example configuration of the image processing apparatus 110a. The image processing apparatus 110a includes an image acquisition unit 111, an extraction unit 112, an identification unit 113, an image compression unit 114, an object model generation unit 116, an image generation unit 117, and a label list 120. The operation from image acquisition to image compression in the image processing apparatus 110a may be the same as the operation from image acquisition to image compression in the image processing apparatus 110 shown in Figure 3.

[0036] The object model generation unit 116 generates a three-dimensional object model of an unknown object based on the image of the region identified as an unknown object by the identification unit 113 in the driving image. In other words, the object model generation unit 116 converts the two-dimensional image of the unknown object identified by the identification unit 113 into a three-dimensional asset. Known methods can be used to generate a three-dimensional model from a two-dimensional image.

[0037] The image generation unit 117 generates a composite image or video in which an unknown object exists on the travel path of the moving body 200, using the 3D model generated by the object model generation unit 116. The image generation unit 117 generates a composite image by combining the 3D model generated by the object model generation unit 116 with the image of the travel path that will be used as the source for synthesis. The image generation unit 117 uses, for example, image generation AI to reconstruct the 3D model onto real-world image data. The image generation unit 117 stores the generated composite image in the image storage unit 130. The composite image can be used for training and evaluation of an object recognition model, similar to the image compressed by the image compression unit 114.

[0038] In this embodiment, the object model generation unit 116 converts objects identified as unknown objects by the identification unit 113 into 3D assets. The image generation unit 117 synthesizes the 3D assets of the unknown objects into driving images. In this embodiment, the image generation unit 117 can synthesize unknown objects into a variety of road images. Furthermore, the image generation unit 117 can synthesize unknown objects into road images at various angles and positions. The synthesized images can be used, for example, to reproduce diverse scenarios in the training and evaluation of object recognition models. This embodiment generates diverse corner case data from valuable corner case data by converting unknown objects into 3D assets. The generated diverse corner case data can be used as training data in a robust environment.

[0039] In the first and second embodiments, the threshold value for the object score used by the identification unit 113 to determine an object as an unknown object does not need to be a fixed value. The identification unit 113 may dynamically change or adjust the threshold value for determining an object as an unknown object based on the object identification result. For example, the identification unit 113 may adjust the threshold value based on the number of times an object in the region extracted by the extraction unit 112 has been identified as an unknown object. For example, if the identification unit 113 identifies an object as an unknown object many times, the number of images containing unknown objects stored in the image storage unit 130 will increase. If the identification unit 113 identifies an object as an unknown object many times, it may lower the threshold value for determining an object as an unknown object. When the threshold value is lowered, the conditions for determining an object as an unknown object in the identification unit 113 become stricter, and the number of images containing unknown objects stored in the image storage unit 130 can be reduced compared to when the threshold value is not lowered. Therefore, by lowering the threshold value, it is possible to suppress the storage of an excessive number of images in the image storage unit 130.

[0040] Furthermore, in the first and second embodiments, the identification unit 113 may prompt the user to update the label list 120 if a predetermined number or more images of the same object identified as an unknown object are acquired. When a large number of images of an object identified as an unknown object are acquired, the images containing such objects are considered to be normal driving images, not rare case images. The identification unit 113 prompts the user, for example, a vehicle or autonomous driving AI developer, to update the label list 120. When the user updates the label list 120 and an object that was previously an unknown object is added to the label list, the images containing such objects can be used as normal driving images.

[0041] In the first and second embodiments, the image processing apparatus 110 and 110a do not necessarily have to be configured as a single physical device. In this disclosure, the image processing apparatus 110 and 110a may be configured using a plurality of physically separated devices. For example, the image processing apparatus 110a shown in Figure 6 may be separated into a first device having an image acquisition unit 111, an extraction unit 112, an identification unit 113, and an image compression unit 114, and a second device having an object model generation unit 116 and an image generation unit 117. When the image processing apparatus 110a is separated into a first device and a second device, the first device may be mounted on each mobile body 200. The first device and the second device may be connected via a network 150. When the first device is mounted on each mobile body 200, the first device compresses images on each mobile body 200 and transmits the compressed images to the second device via the network 150. When images are compressed in each mobile device 200, network traffic 150 can be reduced compared to when images are transmitted without compression.

[0042] In this disclosure, the image processing devices 110 and 110a may be configured as a computer device or a server device, respectively. Figure 7 is a block diagram showing an example configuration of a computer device that can be used as the image processing device 110 or 110a. The computer device 500 includes a processor 510 such as a Central Processing Unit (CPU), a storage unit 520, a Read Only Memory (ROM) 530, a Random Access Memory (RAM) 540, a communication interface (IF) 550, and a user interface 560.

[0043] The communication interface 550 is an interface for connecting the computer device 500 to a communication network via wired communication means or wireless communication means. The user interface 560 includes a display unit, such as a display. The user interface 560 also includes input units such as a keyboard, mouse, and touch panel.

[0044] The storage unit 520 is an auxiliary storage device that can hold various types of data. The storage unit 520 does not necessarily have to be a part of the computer device 500 and may be an external storage device or a cloud storage connected to the computer device 500 via a network.

[0045] The ROM 530 is a non-volatile storage device. A semiconductor storage device such as a flash memory with a relatively small capacity, for example, is used for the ROM 530. The program executed by the processor 510 can be stored in the storage unit 520 or the ROM 530. The storage unit 520 or the ROM 530 stores various programs for realizing the functions of each part in the image processing device 110, for example.

[0046] When the above program is loaded into a computer, it includes a group of instructions (or software code) for causing the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or tangible storage medium includes RAM, ROM, flash memory, SSD, or other memory technologies, Compact Disc (CD), digital versatile disc (DVD), Blu-ray (registered trademark) disc, or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage, or other magnetic storage devices. The program may be transmitted on a transitory computer-readable medium or a communication medium. By way of example and not limitation, the transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0047] The RAM 540 is a volatile memory device. Various semiconductor memory devices such as Dynamic Random Access Memory (DRAM) or Static Random Access Memory (SRAM) are used for the RAM 540. The RAM 540 can be used as an internal buffer for temporarily storing data and the like. The processor 510 loads a program stored in the storage unit 520 or the ROM 530 into the RAM 540 and executes it. By the processor 510 executing the program, the functions of each part in the image processing apparatus 110 or 110a can be realized. The processor 510 may have an internal buffer that can temporarily store data and the like.

[0048] As described above, the embodiments of the present disclosure have been described in detail. However, the present disclosure is not limited to the above-described embodiments, and modifications and changes made to the above embodiments without departing from the spirit of the present disclosure are also included in the present disclosure. Each embodiment can be combined as appropriate.

[0049] Each drawing is merely an example for explaining one or more embodiments. Each drawing is not associated with only one specific embodiment, but may be associated with one or more other embodiments. As can be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with the features or steps shown in one or more other drawings to create, for example, embodiments not explicitly illustrated or described. Not all of the features or steps shown in any one drawing for explaining exemplary embodiments are necessarily essential, and some features or steps may be omitted. The order of the steps described in any drawing may be changed as appropriate.

[0050] For example, some or all of the above embodiments can be described as follows in the appended claims, but are not limited thereto.

[0051] [Note 1] An image processing system comprising: an extraction unit that extracts a region containing an object in the travel area of ​​a travel image captured by an imaging device mounted on a mobile body; an identification unit that identifies an object present in the extracted region; and a compression unit that compresses the travel image such that the region of the object identified as an unknown object by the identification unit has higher image quality than the region of the travel image other than the region of the object identified as an unknown object.

[0052] [Note 2] The image processing system according to Note 1, wherein the extraction unit extracts a region containing the object from at least one region of the travel area, which includes the region of the lane in which the moving body is traveling, the region of the lane adjacent to the lane in which the moving body is traveling, and the region of the lane opposite to the lane in which the moving body is traveling, depending on the travel state of the moving body.

[0053] [Note 3] The image processing system according to Note 1 or 2, further comprising an object model generation unit that generates a three-dimensional object model of the unknown object based on an image of the region identified as the unknown object in the driving image.

[0054] [Appendix 4] The image processing system according to Appendix 3, further comprising an image generation unit that generates a driving image in which the unknown object is present on the driving path of the moving body by compositing the generated three-dimensional object model onto an image of the driving path of the moving body.

[0055] [Note 5] The image processing system according to any one of Notes 1 to 4, wherein the identification unit generates an index value indicating object-likeness for each of one or more identifiable objects, and identifies objects whose index value is less than or equal to a predetermined threshold as the unknown object.

[0056] [Appendix 6] The image processing system according to Appendix 5, wherein the identification unit adjusts the threshold value based on the object identification result.

[0057] [Note 7] The image processing system according to Note 6, wherein the identification unit adjusts the threshold based on the number of times an object present in the extracted region has been identified as an unknown object.

[0058] [Appendix 8] The image processing system according to any one of Appendix 1 to 7, wherein the identification unit identifies an object not included in a label list containing information for identifying one or more known objects as an unknown object.

[0059] [Note 9] The image processing system described in Note 8, wherein the identification unit prompts the user to update the label list when a predetermined number or more images of the same object identified as the unknown object have been acquired.

[0060] [Note 10] The image processing system according to any one of Notes 1 to 9, wherein the extraction unit extracts a region containing the object in the travel area of ​​the travel image when the travel path on which the moving body travels satisfies predetermined conditions.

[0061] [Note 11] An image processing method that extracts a region containing an object in the driving region of a driving image captured by an imaging device mounted on a moving body, identifies the object present in the extracted region, and compresses the driving image such that the region of the object identified as an unknown object has higher image quality than the region of the driving image other than the region of the object identified as an unknown object.

[0062] [Note 12] A program that causes a computer to perform a process including: extracting a region containing an object in the driving area of ​​a driving image captured by an imaging device mounted on a moving body; identifying the object present in the extracted region; and compressing the driving image such that the region of the object identified as an unknown object has higher image quality than the region of the driving image other than the region of the object identified as an unknown object.

[0063] Some or all of the elements (e.g., configuration and function) described in Appendices 2 to 10 that are subordinate to Appendice 1 may also be subordinate to Appendices 11 and 12 in the same manner as those described in Appendices 2 to 10. Some or all of the elements described in any appendice may be applied to various hardware, software, recording means, systems, and methods for recording software.

[0064] This application claims priority based on Japanese Patent Application No. 2024-170010, filed on 30 September 2024, and incorporates all of its disclosures herein.

[0065] 10: Image processing system 11: Extraction unit 12: Identification unit 13: Compression unit 100: Image processing system 110: Image processing device 111: Image acquisition unit 112: Extraction unit 113: Identification unit 114: Image compression unit 115: Image transmission unit 116: Object model generation unit 117: Image generation unit 130: Image storage unit 140: Image collection device 150: Network 200: Mobile device 210: Camera 500: Computer device 510: Processor 520: Storage unit 530: ROM 540: RAM 550: Communication IF 560: User IF

Claims

1. An image processing system comprising: an extraction unit that extracts a region containing an object in the travel area of ​​a travel image captured by an imaging device mounted on a mobile body; an identification unit that identifies an object present in the extracted region; and a compression unit that compresses the travel image such that the region of the object identified as an unknown object by the identification unit has higher image quality than the region of the travel image other than the region of the object identified as an unknown object.

2. The image processing system according to claim 1, wherein the extraction unit extracts a region containing the object from at least one region of the travel area, which includes the region of the lane in which the moving body is traveling, the region of the lane adjacent to the lane in which the moving body is traveling, and the region of the lane opposite to the lane in which the moving body is traveling, depending on the travel state of the moving body.

3. The image processing system according to claim 1 or 2, further comprising an object model generation unit that generates a three-dimensional object model of the unknown object based on an image of the region identified as the unknown object in the driving image.

4. The image processing system according to claim 3, further comprising an image generation unit that generates a driving image in which the unknown object is present on the driving path of the moving object by compositing the generated three-dimensional object model onto an image of the driving path of the moving object.

5. The image processing system according to any one of claims 1 to 4, wherein the identification unit generates an index value indicating object-likeness for each of one or more identifiable objects, and identifies objects whose index value is less than or equal to a predetermined threshold as the unknown object.

6. The image processing system according to claim 5, wherein the identification unit adjusts the threshold value based on the object identification result.

7. The image processing system according to claim 6, wherein the identification unit adjusts the threshold based on the number of times an object present in the extracted region has been identified as an unknown object.

8. The image processing system according to any one of claims 1 to 7, wherein the identification unit identifies an object not included in a label list containing information for identifying one or more known objects as an unknown object.

9. The image processing system according to claim 8, wherein the identification unit prompts the user to update the label list when a predetermined number or more images of the same object identified as the unknown object have been acquired.

10. The image processing system according to any one of claims 1 to 9, wherein the extraction unit extracts a region containing the object in the travel region of the travel image when the travel path on which the moving body travels satisfies predetermined conditions.

11. An image processing method that extracts a region containing an object from the driving region of a driving image captured by an imaging device mounted on a moving body, identifies the object present in the extracted region, and compresses the driving image such that the region of the object identified as an unknown object has higher image quality than the region of the driving image other than the region of the object identified as an unknown object.

12. A program that causes a computer to perform a process including: extracting a region containing an object from the driving area of ​​a driving image captured by an imaging device mounted on a moving object; identifying the object present in the extracted region; and compressing the driving image such that the region of the object identified as an unknown object has higher image quality than the region of the driving image other than the region of the object identified as an unknown object.

Images