Image data transmission device and image analysis device

Abstract

To provide an image data transmission device capable of transmitting and receiving information effective for operation support, by using a communication capacity effectively, and an image analysis device.SOLUTION: The image data transmission device comprises: a determination unit which on the basis of image data including an image of an object and a distance from an imaging sensor having captured the image to the object, determines an attribute capable of determination about the object from the image data; and a transmission unit which adds attribute information including the attribute to the image data and transmits the image data to a transmission destination.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] This disclosure relates to an image data transmission device and an image analysis device. This application claims priority to Japanese Application No. 2020-184132, filed on November 4, 2020, and incorporates by reference all of the contents of that Japanese application. [Background technology]

[0002] Various systems have been proposed to assist drivers in driving automobiles, motorcycles, and the like (hereinafter referred to as vehicles). Such systems include roadside devices equipped with various sensor devices installed on roads and their surrounding areas. The sensor devices include, for example, cameras and radar. A server computer (hereinafter referred to as the "server") collects sensor information from these roadside devices, analyzes the collected information, and generates traffic-related information. The traffic-related information includes, for example, information about accidents, congestion, and the like. The server provides this information to the vehicle as dynamic driving assistance information.

[0003] It has also been proposed that servers collect information from sensors installed in vehicles, not just roadside equipment, and use the collected information for driving assistance. For example, there is a standardization project called 3GPP (registered trademark: Third Generation Partnership Project). This project promotes standardization of third-generation mobile communication systems and subsequent generations of mobile communication systems. 3GPP has proposed a standard called Cellular V2X. V stands for vehicle, and X stands for everything, including vehicles. In other words, V2X encompasses vehicle-to-cellular network (V2N), vehicle-to-vehicle (V2V), vehicle-to-roadside infrastructure (V2I), and vehicle-to-pedestrian (V2P). This standard assumes that communication between vehicles and other devices will be performed using high-speed, low-latency wireless communications such as 4G (fourth-generation mobile communication system) and 5G (fifth-generation mobile communication system).

[0004] A vehicle is essentially an organic collection of mechanical elements, such as the engine, transmission, airbags, brakes, and steering. Traditionally, these have been controlled by mechanical mechanisms to reflect the driver's operations. However, in recent years, vehicles have become increasingly electronic, and are now equipped with a variety of ECUs (Electronic Control Units) that electronically control each part. These ECUs include, for example, engine control ECUs, stop-start control ECUs, transmission control ECUs, airbag control ECUs, power steering control ECUs, and hybrid control ECUs. Among these, autonomous driving ECUs have attracted particular attention, with many companies competing to develop them. Remote monitoring technology has also been attracting attention recently.

[0005] As the name suggests, an autonomous driving ECU is designed to drive a vehicle automatically. Humans gather information about the vehicle's surroundings through their five senses and decide how to control the vehicle. However, the vehicle itself does not have a mechanism for gathering such information. Therefore, modern vehicles are equipped with multiple sensors to gather information. These sensors include cameras, lidars, millimeter-wave radars, and more. However, in the case of a vehicle, it is not easy to make appropriate decisions for autonomous driving based on the information gathered from these sensors.

[0006] Vehicles can travel at very high (fast) speeds. Therefore, it is difficult to achieve proper autonomous driving without high-speed processing of information collected from on-board sensors. In order for the vehicle to perform this processing, it is necessary to equip the vehicle with a computer for this purpose. However, high-performance computers have problems such as a large installation space, high power consumption, cost, and heat generation. Therefore, it is difficult to install them in vehicles for general use at least. Therefore, the following method is currently being adopted. In this method, a relatively low-performance computer is installed in the vehicle. This computer sends information obtained from the on-board sensors to a server, just like roadside devices. The server processes the information at high speed, generates driving assistance information, and distributes it to the vehicle. This is also the case with remote monitoring.

[0007] However, as the number of vehicles adopting autonomous driving increases and as onboard sensors become more sophisticated, the amount of data transmitted from vehicles to servers is expected to increase. Cameras, in particular, that can capture high-resolution color images are becoming cheaper. Therefore, it is predicted that more and more cameras will be installed in more vehicles in the future. As a result, even if high-speed wireless communication technology further develops and becomes more widespread, resulting in faster and larger-capacity wireless communication lines, the increase in communication traffic may exceed this, potentially resulting in a shortage of communication capacity.

[0008] A proposal to solve these problems is disclosed in Patent Document 1 listed below. The system disclosed in Patent Document 1 relates to communication between a vehicle and an emergency call center. When the vehicle transmits images of the interior and exterior of the vehicle taken by a camera mounted on the vehicle to an emergency call center, the vehicle checks the communication status (e.g., communication capacity) and changes the type of image data to be transmitted (e.g., video or still image), resolution (e.g., low / high), and frame rate (e.g., low / medium / high) according to the communication status. Because the transmission content and frame rate are changed according to the communication status, it is said that even when communication conditions such as communication speed fluctuate, communication capacity can be maximized and the desired content of the transmitted information can be accurately transmitted to the outside. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-263580 Summary of the Invention [Means for solving the problem]

[0010] The image data transmission device according to this disclosure includes a determination unit that determines attributes that can be determined about an object from image data based on image data including an image of the object and the distance from the imaging sensor that captured the image to the object, and a transmission unit that adds attribute information including the attributes to the image data and transmits the image data to a destination.

[0011] The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the invention taken in conjunction with the accompanying drawings. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram showing the configuration of a driving assistance system. [Figure 2] FIG. 2 is a schematic diagram showing a situation in which a vehicle images a nearby pedestrian. [Figure 3] FIG. 3 is a schematic diagram showing an image captured by a vehicle in the situation shown in FIG. [Figure 4] FIG. 4 is a schematic diagram showing a situation in which a vehicle captures an image of a pedestrian in the distance. [Figure 5] FIG. 5 is a schematic diagram showing an image captured by a vehicle in the situation shown in FIG. [Figure 6] FIG. 6 is a diagram showing a resolution selection table. [Figure 7] FIG. 7 is a diagram showing a detected attribute output table. [Figure 8] FIG. 8 is a functional block diagram of the edge server according to the first embodiment. [Figure 9] FIG. 9 is a block diagram showing a schematic configuration of the vehicle according to the first embodiment. [Figure 10] FIG. 10 is a functional block diagram of the in-vehicle device. [Figure 11] FIG. 11 is a sequence diagram showing communication between the edge server and the vehicle. [Figure 12] FIG. 12 is a flowchart showing a control structure of a program for generating the detected attribute output table. [Figure 13] FIG. 13 is a flowchart showing a control structure of a program realizing the functions of the in-vehicle apparatus. [Figure 14] FIG. 14 is an external view of the edge server according to the first embodiment. [Figure 15] FIG. 15 is a hardware block diagram of the edge server shown in FIG. [Figure 16] FIG. 16 is a hardware block diagram of the in-vehicle device according to the first embodiment. [Figure 17] FIG. 17 is a functional block diagram of the edge server according to the second embodiment. [Figure 18] FIG. 18 is a functional block diagram of an in-vehicle device according to the second embodiment. [Figure 19] FIG. 19 is a sequence diagram of communication between the edge server and the in-vehicle device according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] [Problem this disclosure aims to solve] The technology disclosed in Patent Document 1 changes the amount of information to be transmitted depending on the communication capacity, even when the communication conditions change. However, from the perspective of the server receiving this information, it is inconvenient to receive a large amount of information that is not useful for driving assistance, even if the communication conditions are good. It is also inconvenient to receive a large amount of information that is useful for normal driving assistance but is not used in the driving assistance processing performed by a certain server. In other words, the technology disclosed in Patent Document 1 does not make the most efficient use of communication capacity to send information that is useful to the server.

[0014] An object of this disclosure is to provide an image data transmission device and an image analysis device that can transmit and receive information that is useful for driving assistance by effectively utilizing communication capacity.

[0015] [Description of the embodiments of the present disclosure] In the following description and drawings, the same parts are designated by the same reference numerals, and therefore detailed descriptions thereof will not be repeated.

[0016] (1) An image data transmission device according to a first aspect of this disclosure includes a determination unit that determines attributes that can be determined about an object from image data containing an image of the object based on the image data and the distance from the imaging sensor that captured the image to the object, and a transmission unit that adds attribute information containing the attributes to the image data and transmits the image data to a destination.

[0017] Based on image data containing an image of the object and the distance from the imaging sensor to the object, determinable attributes of the object are determined from the image data. Attribute information containing the attributes is added to the image data and transmitted to a destination. At the destination, the image data can be analyzed to obtain the attributes indicated by the attribute information added to the image data. There is no need for the destination to determine what attributes to obtain, and the image data can be analyzed immediately. As a result, the image data transmission device can transmit and receive information useful for driving assistance by effectively utilizing communication capacity.

[0018] (2) In the above (1), the determination unit may include an attribute determination unit that determines the attribute based on a resolution of the image and the distance.

[0019] The attributes that can be determined vary depending on the resolution of the image containing the object and the distance from the imaging sensor to the object. The destination of the image does not need to check what attributes to determine, but can determine the attributes included in the attribute information added to the image, thereby reducing the processing load of the destination of the image analysis.

[0020] (3) In (2) above, the attribute determination unit may include a detection attribute output table storage unit that stores a pre-prepared detection attribute output table for determining attributes that can be determined based on a combination of the resolution of the image and the distance, and a table reference unit that determines the attributes by referring to the detection attribute output table based on the resolution of the image and the distance.

[0021] By preparing a detection attribute table in advance, attributes that can be determined from an image can be immediately read from the detection attribute table based on the image resolution and the distance from the imaging sensor to the object. Therefore, the image data transmission device can determine attributes that can be determined from an image through simple processing.

[0022] (4) In the above (3), the detected attribute output table may store the most detailed attribute that can be determined by a combination of resolution and distance.

[0023] With this configuration, the destination device can determine not only the most detailed attribute but also less detailed attributes as attributes indicated by the attribute information. This reduces the amount of attribute information added to the image, reduces the processing load in the image data transmission device, and also reduces the amount of data to be transmitted.

[0024] (5) An image analysis device according to a second aspect of this disclosure includes a receiving unit that receives image data and attribute information about the image that is attached to the image, and an analyzing unit that analyzes the image based on the attribute information.

[0025] The image analysis device can analyze the image to find the attributes indicated by the attribute information added to the image data. This eliminates unnecessary processing, such as finding attributes that cannot be obtained by image analysis. As a result, the processing load of the image analysis device can be reduced.

[0026] (6) In the above (5), the analysis unit may determine whether to analyze the image based on the attribute information.

[0027] The attribute information may include attributes that cannot be determined from the image data, and in such cases, the image analysis device can be prevented from performing unnecessary analysis processing.

[0028] The above and other objects, features, aspects and advantages of the present disclosure will become apparent from the following detailed description of the disclosure taken in conjunction with the accompanying drawings.

[0029] [Effect of this disclosure] As described above, according to this disclosure, it is possible to provide an image data transmission device, an image data transmission method, a computer program, and a storage medium that are capable of transmitting and receiving information that is effective for driving assistance by effectively utilizing communication capacity.

[0030] [Details of the embodiments of the present disclosure] Specific examples of an image data transmission device, an image data transmission method, a computer program, and a storage medium according to embodiments of the present disclosure will be described below with reference to the drawings. Note that the present disclosure is not limited to these examples, but is defined by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

[0031] First Embodiment <composition> Conceptual explanation FIG. 1 shows a conceptual configuration of this traffic assistance system 50. Referring to FIG. 1, the traffic assistance system 50 includes an edge server 60 that performs processing for driving assistance, a number of vehicles such as a vehicle 62, and roadside equipment (not shown). The vehicle 62 has one or more sensors. These sensors are equipped with a camera, which is a type of imaging sensor, and a lidar and millimeter-wave radar, which are types of distance measuring sensors. The vehicle 62 may have one or more of these various sensors. In the following embodiment, a case will be described in which there is one camera, one lidar, and one millimeter-wave radar. Even when there are multiple of each of these, the present disclosure can be realized with a configuration similar to that described below.

[0032] Traffic participants such as pedestrians 66 and surrounding objects such as trees 68 may enter the imaging area 64 of the camera of the vehicle 62. For example, as shown in FIG. 2, consider a case where the pedestrian 66 crosses in front of the vehicle 62 in an area relatively close to the vehicle 62 within the imaging area 64.

[0033] An example of an image 100 captured by the camera of vehicle 62 in this case is shown in Figure 3. As shown in Figure 3, image 100 includes an image 102 of pedestrian 66 shown in Figure 2 and a tree image 104, which is an image of tree 68 shown in Figure 2. In the example shown in Figure 3, tree image 104 on the left and image 102 are approximately the same size.

[0034] 4, consider a case where the camera of vehicle 62 captures an image of pedestrian 110 within imaging area 64 but at a position farther away than pedestrian 66 in FIG. 3, i.e., farther away from tree 68. In this case, as shown in FIG. 5, image 122 of pedestrian 110 will be small in the resulting image 120. For example, image 122 is expected to be significantly smaller than image 104 of the tree to the left of image 102, as compared to the case in FIG. 3.

[0035] As such, the greater the distance between the object and the camera, the smaller its image in the captured image. Specifically, the area of ​​the object in the image is inversely proportional to the square of the distance between the object and the camera. In other words, the number of vertical and horizontal pixels becomes smaller. This leads to the following problems:

[0036] For example, consider a case where the edge server 60 shown in Figure 1 determines the attribute values ​​of a subject, such as a pedestrian, through image recognition. An attribute is a "characteristic or property possessed by an object" (Iwanami Shoten Kojien, 6th Edition). For example, typical attributes for a person are "height" and "weight." Each person is assigned a unique value for these attributes (165 cm, 180 cm, 45 kg, 67 kg, etc.). In other words, in this specification, an attribute refers to a characteristic that is considered to be common to objects, and the same attribute may have different values ​​depending on the object. In this specification, "determining an attribute" means "determining (or estimating) an attribute value."

[0037] In this specification, five types of attributes are considered: simple attributes, detailed attributes, behavioral attributes, and, in the case of a person, body orientation and face orientation. In this embodiment, all of these attributes are information that indicates the characteristics or properties of subjects that are traffic participants such as vehicles and pedestrians, and are information that should be extracted from images as information that is useful for driving assistance.

[0038] Simple attributes are rough attributes that can be recognized from a distance, such as attributes that indicate the type of object. For example, distinguishing between pedestrians, cars, motorcycles, and guardrails. Detailed attributes are often more detailed than the type of object and cannot be distinguished without getting closer to the object. For example, in the case of cars, this would be distinguishing between trucks and passenger cars, and then between large, medium, and small vehicles, and in the case of people, it would be distinguishing between adults, children, and elderly people. Behavioral attributes are not related to the appearance of an object, such as its type, but rather to its movement. For example, in the case of cars and people, this would be movement speed and direction. These attributes require detailed attributes and are determined based on their time series information. Therefore, behavioral attributes can be considered more detailed than detailed attributes. The last two attributes, body orientation and face orientation, are both unique to people. These are useful for predicting future human behavior. Body orientation cannot be determined without knowing not only the movement of an object but also the details of the image. Therefore, body orientation can be considered more detailed than behavioral attributes. Furthermore, the face is only a part of the body. Therefore, face orientation can be considered more detailed than body orientation.

[0039] In this way, the attributes of an object have different levels of detail as information necessary for traffic assistance. In this specification, the level of detail of an object's attribute is referred to as the attribute detail level, and the value of the attribute is referred to as the attribute value. In the following description, determining the attribute value may also be simply referred to as determining the attribute.

[0040] When a person drives a car, these attributes are judged instantaneously based on information received through the five senses. Humans can easily make such judgments. However, when autonomous driving is performed by computer or remote control is performed using only sensor data without using human senses, it is not so easy to determine these attribute values.

[0041] In general, the number of image pixels required to determine the attribute value of a subject varies depending on the level of detail of the attribute. That is, for simple and rough attributes such as simple attributes, the attribute value can be determined even if the image of the subject consists of a relatively small number of pixels. However, particularly detailed attributes such as a person's body orientation and facial orientation cannot be determined unless the image of the subject consists of a large number of pixels. Therefore, for example, if a pedestrian is far from the camera, it is difficult to determine attribute values ​​such as the body orientation or facial orientation from the image of the subject without using a high-resolution image.

[0042] However, high-resolution images have a problem in that they require a large amount of data, resulting in a large amount of data to be transmitted from the vehicle to the edge server. Large amounts of data not only take time to transmit, but also pose the risk of wireless communication congestion, preventing the edge server from quickly collecting the necessary information. Therefore, it is not desirable to use only high-resolution images as image data to be transmitted to the edge server. Furthermore, if the edge server is performing processing that does not require detailed information, sending high-resolution images is not efficient in the first place.

[0043] Therefore, in this disclosure, when transmitting an image to an edge server, attention is paid to the distance between the object to be imaged and the camera (vehicle). That is, if the distance between the object to be imaged and the camera is large, the image of the object becomes small. Therefore, the resolution of the image transmitted from the vehicle to the edge server is increased. Conversely, if the distance between the object to be imaged and the camera is small, the resolution of the image transmitted from the vehicle to the edge server is decreased. By adjusting the image resolution in this manner, the edge server can obtain attributes with a similar level of detail regardless of the distance between the object to be imaged and the camera mounted on the vehicle. Furthermore, in this disclosure, the image resolution is adjusted according to the level of detail of the attribute that the server is attempting to determine. If it is considered that the level of detail is determined by the type of attribute, it can also be considered that the resolution of the image transmitted to the edge server is adjusted according to the type of attribute that the edge server is attempting to determine based on the image.

[0044] To this end, this embodiment uses a resolution selection table 150 shown in FIG. 6. The resolution selection table 150 shows the correspondence between the combination of the distance between the camera and an object, the type of attribute to be determined from the object, and the predetermined image resolution that allows the attribute value of the object to be determined for that combination. The resolution selection table 150 shown in FIG. 6 shows the attribute to be determined by the edge server on the horizontal axis, the distance between the subject and the camera on the vertical axis, and the minimum image resolution required to determine the attribute value of the object using an image of the object located at that distance in the cell where these axes intersect. On the horizontal axis of the resolution selection table 150, the attributes are arranged with increasing detail from left to right. On the vertical axis, the distance increases from top to bottom. Note that "5m" on the vertical axis indicates a distance greater than 0m and less than or equal to 5m. Similarly, "150m" refers to the previous row and indicates a distance greater than 50m and less than or equal to 150m. The "150m-" in the last row indicates a distance greater than 150m.

[0045] In Figure 6, "HD" stands for "High Definition," "FHD" stands for "Full High Definition," and "QHD" stands for "Quad High Definition." "4K" means a resolution of around 4,000 horizontal pixels.

[0046] Using the resolution selection table 150, the minimum resolution required for transmitting an image to the edge server can be determined by providing the distance between the subject and the camera and the attribute that the edge server is attempting to determine about the subject. For example, referring to FIG. 6, assume that the distance between the subject and the camera is greater than 15 m and less than 50 m, and the attribute that the edge server is attempting to determine is the orientation of a person's body. In this case, the minimum resolution required is "FHD" as determined by the cell in the resolution selection table 150 where the horizontal axis is "body orientation" and the vertical axis is "50 m."

[0047] When the object is far from the camera and its image is small, the resolution is increased as much as necessary so that the edge server can determine the attribute values ​​it is trying to determine about the object. On the other hand, when the object is close to the camera and its image is relatively large, the resolution is reduced as much as possible within the range in which the edge server can determine the attributes it is trying to determine about the object. In this way, when the edge server tries to determine the attribute values ​​of the object, the resolution is relatively increased if a detailed image is required for the determination, and reduced if a simpler image is sufficient.

[0048] In this way, by transmitting the minimum necessary information from the vehicle to the edge server according to the attributes that the edge server is trying to determine, appropriate processing can be performed in the edge server while saving communication capacity.

[0049] On the other hand, there may be cases where an image cannot be transmitted to the edge server at the resolution indicated in the resolution selection table 150 due to limitations in communication capacity, camera performance, etc. In such cases, in this embodiment, an image with a lower resolution than that determined by the resolution selection table 150 is transmitted to the edge server. At the same time, information on attributes that can be determined at that resolution is also transmitted from the vehicle to the edge server. In this way, the edge server can know the attributes that can be determined using the received image, and can effectively use the image for driving assistance. For this reason, in this embodiment, the detected attribute output table 160 shown in FIG. 7 is used when transmitting image data from the vehicle to the edge server.

[0050] 7, the detected attribute output table 160 shows, in a tabular format, the image resolution on the horizontal axis and the distance on the vertical axis, with the most detailed attribute that can be determined at that resolution and distance in the cell where they intersect. The meaning of the vertical axis is the same as in FIG. 6.

[0051] For example, referring to Figure 7, when the distance is more than 15 m but less than 50 m and only FHD resolution is available, the edge server can determine up to three attributes (body orientation). That is, the edge server can determine simple attributes, detailed attributes, behavioral attributes, and even body orientation for the subject. However, it cannot determine face orientation. Similarly, when the distance is more than 50 m but less than 150 m and only HD is available, the edge server can only determine simple attributes. When the distance exceeds 150 m and the resolution is HD, the edge server cannot determine any attributes. When the resolution is FHD, the edge server can determine detailed attributes, body orientation for QHD, and face orientation for 4K.

[0052] The detected attribute output table 160 shown in FIG. 7 can be calculated from the resolution selection table 150 shown in FIG.

[0053] Edge Server 60 FIG. 8 shows a functional block diagram of the edge server 60. Referring to FIG. 8, the edge server 60 includes a communication device 180 for communicating with vehicles and roadside devices (hereinafter referred to as "vehicles, etc.") via wireless or wired communication. The edge server 60 further includes a driving assistance analysis unit 182 connected to the communication device 180, generating information for driving assistance based on sensor data including image data received by the communication device 180 from the vehicles, etc., and transmitting the information to each vehicle, etc. The edge server 60 further includes a vehicle management unit 188 for storing and managing vehicle information received by the communication device 180 from the vehicles, etc., including the vehicle type, position, moving speed, sensor placement status, etc. of each vehicle. The driving assistance analysis unit 182 communicates with each vehicle, etc., based on the vehicle information stored in the vehicle management unit 188.

[0054] The edge server 60 further includes a resolution selection table creation unit 184 connected to the driving assistance analysis unit 182 for creating the resolution selection table 150 shown in FIG. 6. When creating the resolution selection table 150, the resolution selection table creation unit 184 calculates the image resolution required to determine each attribute based on constraints imposed when the communication device 180 analyzes sensor data (particularly image data). The edge server 60 further includes a resolution selection table storage unit 186 for storing the resolution selection table 150 created by the resolution selection table creation unit 184. The edge server 60 further includes a resolution selection table transmission unit 190 for transmitting the resolution selection table 150 via the communication device 180 to vehicles to which the latest resolution selection table 150 has not yet been transmitted, based on vehicle information stored in the vehicle management unit 188. At this time, the resolution selection table transmission unit 190 selects a destination vehicle from among vehicles identified by the vehicle information stored in the vehicle management unit 188.

[0055] Vehicle 62 9 shows a functional block diagram of the portions of the vehicle 62 relevant to this disclosure. Referring to Fig. 9, the vehicle 62 includes a millimeter-wave radar 200 which is a distance measurement sensor, a camera 202 which is an imaging sensor, and a lidar 204 which is a distance measurement sensor. The vehicle 62 further includes an on-board device 210 which receives sensor data from these sensors and transmits it to the edge server 60 via wireless communication, and which receives information for driving assistance from the edge server 60 via wireless communication and performs driving assistance processing.

[0056] 10 , the in-vehicle device 210 is connected to the millimeter-wave radar 200, the camera 202, and the lidar 204, and includes an I / F unit 230 for receiving sensor data from these devices and converting the data into a digital format that can be processed within the in-vehicle device 210. Note that I / F stands for Interface. The in-vehicle device 210 further includes an image acquisition unit 232 for acquiring images captured by the camera 202 via the I / F unit 230. The in-vehicle device 210 further includes a wireless communication device 236 for wireless communication with the edge server 60. The in-vehicle device 210 further includes a driving assistance processing device 238 that performs processing to assist the driver in driving, based on driving assistance information received by the wireless communication device 236 from the edge server 60.

[0057] The on-board device 210 further includes an image data transmission unit 234 for transmitting, to the edge server 60 via the wireless communication device 236, transmission data obtained by converting the image data to a resolution as necessary to the minimum required for determining the attribute, based on the distance between the object detected in the image data and the camera 202 and the attribute that the edge server 60 is to determine. At this time, the image data transmission unit 234 calculates the distance between the camera 202 and the position of the real object detected in the image data, based on distance measurement data (a point cloud in the case of the LIDAR 204) between the object and the image data received from the millimeter-wave radar 200 and the LIDAR 204 via the I / F unit 230, and image data received from the camera 202 by the image acquisition unit 232. In this calculation, if the camera 202 and the LIDAR 204 are located close to each other, the distance measurement data from the LIDAR 204 to the object is used as is. When the distance between the camera 202 and the LIDAR 204 is large enough to be significant compared to the distance between the camera 202 and the object, the following procedure is performed. That is, the distance from the camera 202 to the object is calculated using the principle of triangulation using the distance between the camera 202 and the lidar 204, the distance from the lidar 204 to the object, and the distance between the line segment connecting the camera 202 and the lidar 204 and the half line extending from the lidar 204 in the direction of the object.

[0058] The image data transmission unit 234 includes a table and target attribute receiving unit 254 for receiving, from the edge server 60 via the wireless communication device 236, information indicating the resolution selection table 150 and the target attributes, which are the attributes that the edge server 60 is to determine. The table and target attribute receiving unit 254 receives the resolution selection table 150 when the vehicle 62 first communicates with the edge server 60, although this embodiment does not particularly limit the timing for receiving the table and target attribute. The target attributes are also not particularly limited and may be received when the vehicle 62 first communicates with the edge server 60, or may be received from the edge server 60 at any time. The image data transmission unit 234 further includes a resolution selection table storage unit 256 for storing the resolution selection table 150 received by the table and target attribute receiving unit 254. The image data transmission unit 234 further includes a target attribute storage unit 266 for storing the target attributes received by the table and target attribute receiving unit 254 and providing the target attributes to a requester in response to a target attribute read request. The edge server 60 may transmit the resolution selection table 150 to each vehicle each time the resolution selection table 150 is changed.

[0059] The image data transmission unit 234 further includes an object / distance detection unit 250 for calculating an object in an image and a distance from the camera to the object using outputs from the millimeter-wave radar 200 and the LIDAR 204 and image data from the camera 202. In this case, the object / distance detection unit 250 receives outputs from the millimeter-wave radar 200 and the LIDAR 204 via the I / F unit 230, and receives image data from the camera 202 via the image acquisition unit 232. The image data transmission unit 234 further includes a detected attribute output table generation unit 258 for generating a detected attribute output table 160 from the resolution selection table 150 in response to the resolution selection table 150 being stored in the resolution selection table storage unit 256, and a detected attribute output table storage unit 260 for storing the detected attribute output table 160 generated by the detected attribute output table generation unit 258.

[0060] The image data transmission unit 234 further includes a resolution determination unit 252 that determines the resolution of an image to be transmitted to the edge server 60 based on the distance detected for each object in the image by the object / distance detection unit 250 and the object attributes stored in the object attribute storage unit 266. The resolution determination unit 252 then determines the image resolution for each image by looking up the resolution selection table 150 stored in the resolution selection table storage unit 256 based on the distance detected for each object in the image and the object attributes. The image data transmission unit 234 further includes a communication status determination unit 262 that measures the communication bandwidth available for wireless communication between the wireless communication device 236 and the edge server 60. The image data transmission unit 234 further includes a transmission data generation unit 264 that generates a transmission image by changing the resolution of the image acquired by the image acquisition unit 232 based on the resolution determined by the resolution determination unit 252 and the communication status determined by the communication status determination unit 262, and transmits the transmission image to the vehicle 62 via the wireless communication device 236. The transmission data generation unit 264 operates as described above when it can transmit to the vehicle 62 an image with a resolution sufficient to determine the target attribute. However, when it cannot transmit an image with the required resolution, the transmission data generation unit 264 refers to the detected attribute output table storage unit 260 to obtain information specifying attributes that can be determined at the available resolution. The transmission data generation unit 264 further adds this information to the image to be transmitted and transmits it to the vehicle 62 via the wireless communication device 236.

[0061] <Timing diagram> 11 shows the timing of communication between the edge server 60 and the vehicle 62. Referring to FIG. 11, if the vehicle 62 has not received the resolution selection table 150 from the edge server 60 in step 280, the vehicle 62 transmits vehicle information to the edge server 60. In step 300, the edge server 60 compares the received vehicle information with vehicle information stored therein to determine whether the vehicle 62 is a new vehicle. If the vehicle 62 is a new vehicle, the edge server 60 transmits to the vehicle 62 in step 302 the resolution selection table 150 it holds and attributes (target attributes) determined based on an image from the vehicle 62. If the vehicle 62 corresponds to vehicle information already stored in the edge server 60, the edge server 60 skips the processing of step 302. If the vehicle 62 has already received the resolution selection table 150 from the edge server 60 in step 280, the vehicle 62 does not transmit vehicle information to the edge server 60. In this case, the vehicle 62 does not execute steps 282 and 284, but executes step 286, which will be described later.

[0062] The vehicle 62 receives the resolution selection table 150 and target attributes transmitted from the edge server 60 in step 282 and stores them in the resolution selection table storage unit 256 and target attribute storage unit 266 shown in Fig. 10. Furthermore, when the vehicle 62 has received and stored the resolution selection table 150, the vehicle 62 generates a detected attribute output table 160 from the resolution selection table 150 in step 284. The vehicle 62 stores the detected attribute output table 160 in the detected attribute output table storage unit 260 in Fig. 10. If the vehicle 62 does not receive the target attributes and resolution selection table 150 from the edge server 60 within a predetermined time, the vehicle 62 does not execute steps 282 and 284, and instead executes the next step 286.

[0063] Next, in step 286, the vehicle 62 captures an image using the camera 202 shown in Fig. 10. In step 286, the object / distance detection unit 250 detects objects from the captured image and measures the distance from the camera 202 to each object using the outputs of the lidar 204 and the millimeter-wave radar 200. In step 288, the resolution of the transmitted image is determined.

[0064] If the resolution determined in step 288 is not the resolution required to determine the attribute value of the object, the vehicle 62 determines the determinable attributes in step 289. Otherwise, the vehicle 62 does not execute the process of step 289. Finally, the vehicle 62 transmits the image to the edge server 60. At this time, the vehicle 62 converts the image to the resolution determined in step 288, if necessary, in step 290. If the process of step 289 is executed, the vehicle 62 adds attribute information to the image that specifies the attributes determinable from the image. The edge server 60 receives this image in step 304. If the resolution of the received image is sufficient to determine the attribute value of the object, the edge server 60 performs a process to determine the attribute value of the object from the received image. If the resolution of the received image is not sufficient to determine the attribute value of the object, the edge server 60 determines the attribute value of each object determined by the attribute information added to the image and uses the attribute value for driving assistance.

[0065] <<Generation Program for Detection Attribute Output Table 160>> 12 shows the control structure of a program in which the on-board device 210 of the vehicle 62 generates the detected attribute output table 160 based on the resolution selection table 150. In this embodiment, the detected attribute output table 160 is generated by the vehicle 62. However, this disclosure is not limited to such an embodiment. The edge server 60 may generate the detected attribute output table 160 from the resolution selection table 150 and transmit it to the vehicle 62 together with the resolution selection table 150.

[0066] Referring to Figure 12, this program includes step 330 in which the on-board device 210 prepares a memory area for the detected attribute output table 160 and initializes its contents, step 332 in which the on-board device 210 arranges distance on the vertical axis and resolution on the horizontal axis of the detected attribute output table 160, and step 334 in which the on-board device 210 repeats step 336 for all cells of the detected attribute output table 160.

[0067] In step 336, the cell to be processed in the detected attribute output table 160 prepared by the in-vehicle device 210 in step 330 is determined to be the most detailed attribute whose value can be determined using the image at that distance and resolution.

[0068] More specifically, in step 336, the on-board device 210 determines the value of the cell corresponding to the target distance on the vertical axis and resolution on the horizontal axis as follows. That is, the on-board device 210 examines the horizontal axis values ​​corresponding to the distance being processed in the resolution selection table 150, starting from the left, and determines the most detailed attribute whose value can be determined at the target resolution. The on-board device 210 then assigns a pre-assigned value indicating the type of attribute to the cell in the detected attribute output table 160. For example, a simple attribute is assigned a value of 0, a detailed attribute a value of 1, a behavior attribute a value of 2, a body orientation a value of 3, and a face orientation a value of 4. Depending on the combination of distance and resolution corresponding to a cell, it may be impossible to determine any of the attributes. In such cases, although not limited thereto, in this embodiment, the value "-1" is stored in the corresponding cell. Note that in this embodiment, a numerical value indicating the most detailed attribute that can be determined for the combination of distance and resolution for that cell is used as the value of each cell in the detected attribute output table 160. As a result, the following effects are obtained.

[0069] When a certain distance and a certain resolution are given, a numerical value (0, 1, 2, 3, or 4) indicating the most detailed attribute obtained by that combination is obtained from the detected attribute output table 160. The larger the value, the higher the level of detail of the attribute. Therefore, once this numerical value is obtained, it is also clear that attributes corresponding to smaller numerical values ​​can be obtained. Therefore, instead of listing and storing in each cell the attributes that can be determined for the combination of distance and resolution of that cell, it is possible to know all the determinable attributes by storing only one value. As a result, the size of each cell can be minimized, thereby saving storage space.

[0070] <Program of the in-vehicle device 210 of the vehicle 62> FIG. 13 is a flowchart showing the control structure of a program for implementing processing performed by vehicle 62 with on-board device 210. This program is repeatedly executed at a predetermined interval. The interval is typically a predetermined first interval, but as described below, this program can change the interval to a second interval longer than the first interval. The first interval is, for example, 100 milliseconds. The second interval is, for example, 200 milliseconds. The second interval can be selected from a plurality of intervals. Referring to FIG. 13, this program includes step 350 of receiving image data and distance data from a camera, lidar, or millimeter-wave radar; step 352 of detecting objects in the image data; and step 354 of determining the distance from camera 202 for each detected object.

[0071] This program further includes step 356 of determining an image resolution using the resolution selection table 150 based on the distance determined in step 354 and the target attributes received from the edge server 60, step 357 of measuring the currently available communication bandwidth, and step 358 of branching the flow of control depending on whether the amount of image data at the image resolution determined in step 356 is greater than the available communication bandwidth measured in step 357. This program further includes step 360 of transmitting image data at the resolution determined in step 356 to the edge server 60 and terminating the process when the determination in step 358 is negative, i.e., when the amount of image data is equal to or less than the available communication bandwidth.

[0072] This program further includes step 362 for reducing the amount of transmitted data when the determination in step 358 is positive, i.e., when the amount of image data exceeds the available communication bandwidth. In this embodiment, in step 362, the transmission frame rate is reduced within the range permitted by the edge server 60. This program further includes step 364 for branching the control flow depending on whether the amount of transmitted data when image data is transmitted using the transmission frame rate determined in step 362 exceeds the available communication bandwidth. When the determination in step 364 is negative, i.e., when the amount of transmitted data is equal to or less than the available communication bandwidth, control proceeds to step 360. By reducing the transmission frame rate in step 362, the data transmission period changes from the first period to a longer second period.

[0073] This program further includes step 366 for extracting a high-priority area from within the image when the determination in step 364 is affirmative, and step 368 for reconstructing a reduced version of the image using the image extracted in step 366. The functions of steps 366 and 368 are also intended to reduce the amount of data transmitted.

[0074] This program further includes step 370, which determines whether the amount of data transmitted by the processing up to step 368 exceeds the available communication bandwidth and branches the flow of control in accordance with the determination. If the determination in step 370 is negative, i.e., if the amount of data transmitted is equal to or less than the available communication bandwidth, control proceeds to step 360.

[0075] This program further includes, when the determination in step 370 is positive, step 372 of reducing the resolution of the original image so that the amount of data is equal to or less than the available bandwidth, and step 374 of determining attributes that can be determined from the image data by table lookup in the detected attribute output table 160, based on the resolution determined in step 372 and the distance determined in step 354. This program further includes step 376 of adding the attributes determined in step 374 as attribute information to the image with the resolution determined in step 372, transmitting the attribute information to the edge server 60, and ending the process.

[0076] In this embodiment, if there are multiple objects detected in step 352, the image resolution is determined based on the attribute that requires the highest resolution among them. However, this disclosure is not limited to such an embodiment. For example, the processes from step 356 to step 376 may be performed for each individual object (excluding step 357). Alternatively, images with different resolutions may be generated for each region of the image of each object, and these images may be sent together to the edge server 60. The processes performed in step 366 and step 368 correspond to such processes.

[0077] In the process of FIG. 13 , steps 362, 366, and 368, and steps 372 and 374 are performed in order, with a determination regarding the data amount intervening. However, this disclosure is not limited to such an embodiment. If the determination regarding the data amount is positive, only steps 362, 366 and 368, or 372 and 374 may be performed, followed by step 376. Any combination of these steps may also be performed in any order, with a determination regarding the data amount intervening. For example, an embodiment may be considered in which, when the determination in step 358 is positive, steps 362 through 370 are not performed, and the processes from step 372 onward are performed directly. Alternatively, an embodiment may be possible in which the order of the process of reducing the transmission frame rate in step 362 and the process of reconstructing an image to reduce the data amount in steps 366 and 368 is reversed.

[0078] Computer Systems In the following explanation, CPU stands for Central Processing Unit, DVD stands for Digital Versatile Disc, GPU stands for Graphics Processing Unit, ROM stands for Read-Only Memory, RAM stands for Random Access Memory, GPS stands for Global Positioning System, and RF stands for Radio Frequency.

[0079] -Edge Server 60- As shown in FIG. 14, the edge server 60 includes a computer 440 having a DVD drive 450 capable of reading and writing to a DVD 452 shown in FIG. 15, and a monitor 442, a keyboard 446, and a mouse 448, all connected to the computer 440.

[0080] Referring to FIG. 15, the computer 440 of the edge server 60 includes a CPU 460 and a bus 462 connected to the CPU 460 and serving as a communication path for data and commands between the CPU 460 and other modules of the computer 440 .

[0081] The computer 440 further includes a GPU 464, a ROM 466, a RAM 468, a hard disk drive 470 which is a non-volatile auxiliary storage device, the aforementioned DVD drive 450 to which a network 454 can be attached, a network I / F 472 which provides a connection to the network 454 for the CPU 460, and a semiconductor memory port 474 which is connected to the bus 462 and to which a semiconductor memory 456 can be attached or detached, all of which are connected to the bus 462.

[0082] The edge server 60 used in the above embodiment can be realized by computer hardware, its peripheral devices, and programs executed thereon, as shown in Figures 14 and 15. The in-vehicle device 210 used in the above embodiment can be realized by computer hardware, its peripheral devices and peripheral modules, and programs executed on the computer hardware, as shown in Figure 16.

[0083] A computer program for causing the computer 440 to function as each functional unit, such as the edge server 60 according to each embodiment, is stored and distributed on a DVD 452 mounted on the DVD drive 450 or a semiconductor memory 456 mounted on the semiconductor memory port 474, and then transferred from there to the hard disk drive 470. Alternatively, the program may be transmitted to the computer 440 via the network 454 and the network I / F 472 and stored in the hard disk drive 470. The program is loaded into the RAM 468 when executed. The program may also be loaded directly into the RAM 468 from the network 454 or via the network 454 and the network I / F 472. The ROM 466 stores a program for starting the computer 440. The RAM 468 and the hard disk drive 470 are used to store data such as sensor data, analysis results, and vehicle information. The GPU 464 is used to perform numerical calculations in parallel at high speed and is used when analyzing sensor data from multiple sensors. The monitor 442 , keyboard 446 and mouse 448 are used by the administrator of the edge server 60 when operating the edge server 60 .

[0084] This program includes a sequence of instructions including a plurality of instructions for causing the computer 440 to function as the edge server 60 and each of its functional units according to the above-described embodiments. Some of the basic functions required for the computer 440 to perform these operations are provided by an operating system or third-party program running on the computer 440, or various dynamically linkable programming toolkits or program libraries installed on the computer 440. Therefore, the program itself does not necessarily include all of the functions required to realize the system, apparatus, and method of this embodiment. The program may include only instructions that implement the functions of the above-described system, apparatus, or method by dynamically calling appropriate functions or appropriate programs in a programming toolkit or program library at runtime using a controlled manner to achieve the desired results. Of course, all of the necessary functions may be provided by the program alone.

[0085] -In-vehicle device 210- Referring to FIG. 16, the in-vehicle device 210 includes a controller 500, a GPS module 502 connected to the controller 500, a memory 504 connected to the controller 500, a power supply circuit 506 for the controller 500, and an audio circuit 508 and a camera 202 connected to the controller 500.

[0086] The in-vehicle device 210 further includes a monitor 510 including an LCD (Liquid Crystal Display), all of which are connected to the controller 500; a touch panel 512; various sensors 514 including an acceleration sensor, a tilt sensor, a temperature sensor, a humidity sensor, a pressure sensor, an illuminance sensor, etc.; an RF baseband circuit 516 for providing wireless communication functions using mobile phone lines such as 5G; and a wireless communication module 518 for providing wireless communication functions such as Wi-Fi communication.

[0087] The controller 500 is essentially a computer, and includes a CPU 550 and a bus 552 that serves as a transmission path for data and commands between the CPU 550 and each unit within the controller 500. The controller 500 further includes a memory controller 554 connected to the bus 552 that controls the memory 504 to write and read data in accordance with commands from the CPU 550, a power supply management circuit 556 that manages the power supply circuit 506 in accordance with control by the CPU 550, and a system management circuit 558 that manages the operation timing of each unit within the controller 500.

[0088] The controller 500 further includes a media processing circuit 560 connected to the bus 552 and serving as an interface with the audio circuit 508 and the camera 202, a display controller 562 for controlling the monitor 510 in accordance with commands and parameters transmitted from the CPU 550 via the bus 552, and an input / output I / F 564 connected to the bus 552 and external modules such as a touch panel 512, various sensors 514, and an RF / baseband circuit 516 and serving as an interface between the CPU 550, the memory 504, and the external modules. The controller 500 further includes a GPU 566 connected to the bus 552 and performing processes such as graphic processing and parallel calculations delegated by the CPU 550 and returning results to the CPU 550 via the bus 552, and a network I / F 568 for connecting the controller 500 to an in-vehicle network 570, etc. The RF / baseband circuit 516 and the wireless communication module 518 in FIG. 16 constitute the wireless communication device 236 shown in FIG. 10.

[0089] As with the edge server 60, in this embodiment, a computer program for causing the controller 500 shown in FIG. 16 to function as each functional unit of the in-vehicle device 210 according to the above-described embodiments is transmitted to the controller 500 from an external network via wireless communication using the RF baseband circuit 516 or the wireless communication module 518, and stored in the memory 504 via the memory controller 554. If the in-vehicle device 210 has a processing module on a memory card, the program can be transferred from the memory card to the memory 504. The program stored in the memory 504 is read, interpreted, and executed by the CPU 550 during execution. The execution result is transferred to an address determined by the program. The data is stored in the memory specified by this address, or transferred to a predetermined module for processing. The program may also be executed directly from the memory card, etc.

[0090] This program includes an instruction sequence including a plurality of instructions for causing the controller 500 to function as the in-vehicle device 210 and each of its functional units according to the above-described embodiments. Some of the basic functions required for the controller 500 to perform these operations are provided by an operating system or third-party program running on the controller 500, or by various dynamically linkable programming toolkits or program libraries installed on the controller 500. Therefore, the program itself does not necessarily include all of the functions required to realize the system, device, and method of this embodiment. The program need only include instructions that implement the functions of the above-described system, device, or method by dynamically calling appropriate functions or appropriate programs in a programming toolkit or program library at runtime in a controlled manner to achieve the desired results. Of course, all of the necessary functions may be provided by the program alone.

[0091] The operation of the computer 440 shown in Figure 15 and the controller 500 shown in Figure 16 and their peripherals is well known or not particularly relevant to this disclosure, and therefore, detailed configurations and operations thereof will not be further described in this specification.

[0092] <Operation> The edge server 60 and vehicle 62 described above operate as follows. The following explanation of the operation will be divided into three cases: (A) initial processing, (B) a case where an image with a resolution sufficient to determine the attributes specified by the edge server 60 is obtained, and (C) a case where an image with a resolution sufficient to determine the attributes specified by the edge server 60 is not obtained.

[0093] <<Initial Processing>> -Processing on Edge Server 60- 8, the resolution selection table creation unit 184 generates the resolution selection table 150 based on the analysis capabilities of the driving assistance analysis unit 182 and the content of the analysis processing. The resolution selection table 150 is stored in the resolution selection table storage unit 186. This processing is performed, for example, when the driving assistance analysis unit 182 is first installed in the edge server 60, when a new function is added to the driving assistance analysis unit 182, or when one of the functions of the driving assistance analysis unit 182 is enhanced. For example, if the driving assistance analysis unit 182 uses an existing analysis engine, the provider of that analysis engine may also provide the resolution selection table 150 to the user. Note that this disclosure relates to image resolution, and the content of the resolution selection table 150 will vary depending, for example, on whether the driving assistance analysis unit 182 has the ability to perform super-resolution processing.

[0094] Assume that a new vehicle 62 enters the jurisdiction of an edge server 60. The vehicle 62 detects that it is now able to communicate with a different edge server 60 than before, and transmits its own vehicle information to the edge server 60. The edge server 60 stores this vehicle information in the vehicle management unit 188 of FIG. 8. In response to the vehicle management unit 188 storing the new vehicle information, the resolution selection table transmission unit 190 reads out the resolution selection table 150 from the resolution selection table storage unit 186. The resolution selection table transmission unit 190 transmits the resolution selection table 150 to the vehicle 62 via the communication device 180 together with attributes (target attributes) required for the driving assistance processing performed by the driving assistance analysis unit 182.

[0095] -Disposal of vehicle 62- 10 , the table and target attribute receiving unit 254 receives the resolution selection table 150 and the target attributes from the edge server 60 via the wireless communication device 236. The table and target attribute receiving unit 254 stores the resolution selection table 150 in the resolution selection table storage unit 256 and the target attributes in the target attribute storage unit 266. In response to the resolution selection table 150 being stored in the resolution selection table storage unit 256, the detected attribute output table generating unit 258 generates the detected attribute output table 160. The generated detected attribute output table 160 is stored in the detected attribute output table storage unit 260. This completes preparations for image transmission from the vehicle 62 to the edge server 60.

[0096] (A) When images with sufficient resolution are available -Vehicle 62- The millimeter-wave radar 200, camera 202, and lidar 204 each periodically output sensor data relating to the surrounding environment and provide it to the I / F unit 230. The I / F unit 230 provides image data to the image acquisition unit 232 and other ranging data to the object / distance detection unit 250. The image output by the camera 202 is image data with the maximum resolution within its capabilities.

[0097] The millimeter-wave radar 200 outputs information such as the distance to the detected object and its relative speed with respect to the camera. Meanwhile, the lidar 204 outputs a set of three-dimensional positions (point cloud) of points of the laser beam that have returned after being reflected from the object. The millimeter-wave radar 200 and the lidar 204 function as a distance measurement sensor that measures the distance from the vehicle to the object.

[0098] The object / distance detection unit 250 receives the outputs of the millimeter-wave radar 200 and the lidar 204 together with the image data received by the image acquisition unit 232 (step 350 in FIG. 13 ) and analyzes them together. The object / distance detection unit 250 then detects an object in the image (step 352) and calculates the distance from the camera to the object (step 354). For simplicity's sake, the following description will be given assuming that only one object is detected. However, if multiple objects are detected, the object requiring the highest resolution among them may be used as the basis for the following processing. The object farthest from the camera may also be used as the basis. Alternatively, the object farthest from the camera within a distance range specified by the edge server 60 may be used as the basis. Distant objects are of little or no urgency in terms of driving assistance processing. This is because the attributes of distant objects can often be determined the next time an image is captured.

[0099] The resolution determination unit 252 determines the resolution of the image data by referring to the resolution selection table 150 stored in the resolution selection table storage unit 256 using the distance between the object detected by the object / distance detection unit 250 and the camera 202 and the object attributes stored in the object attribute storage unit 266 (step 356 in FIG. 13 ). The transmission data generation unit 264 calculates the amount of data after converting the image acquired by the image acquisition unit 232 to the determined resolution. The transmission data generation unit 264 further determines whether the amount of image data is larger than the available communication bandwidth, taking into account the size of the communication bandwidth measured by the communication status determination unit 262 in step 357 in FIG. 13 (step 358). Here, this determination is assumed to be negative. Since there is no problem in transmitting image data at the determined resolution, the transmission data generation unit 264 in FIG. 10 converts the image to the determined resolution and transmits it to the edge server 60 via the wireless communication device 236 (step 360 in FIG. 13 ).

[0100] -Edge Server 60- 8, the communication device 180 receives this image data and provides it to the driving assistance analysis unit 182. This image data has a resolution sufficient for the driving assistance analysis unit 182 to determine the expected attributes. Therefore, the driving assistance analysis unit 182 analyzes this image data and performs driving assistance processing. The results of the analysis processing are stored in the vehicle management unit 188 and transmitted to each vehicle within the jurisdiction of the edge server 60.

[0101] (B) When an image of sufficient resolution cannot be obtained or the amount of data transmitted exceeds the communication bandwidth The processing from step 350 to step 357 in Figure 13 is the same as when an image of sufficient resolution is obtained. However, if an image of sufficient resolution is not obtained or the amount of transmission data exceeds the communication bandwidth, the determination in step 358 is positive. That is, the amount of image data exceeds the communication bandwidth. Therefore, in step 362, the vehicle 62 calculates the amount of transmission data when the transmission frame rate is lowered within the range allowed by the server. The image resolution is maintained at the value determined in step 356.

[0102] After this, the process branches at step 364 and step 370. These steps will be explained in order below.

[0103] (B1) When the amount of transmitted data is less than or equal to the available communication bandwidth In this case, the determination is negative in step 364. The vehicle 62 converts the image to this resolution in step 360 and transmits the image data to the edge server 60.

[0104] The process performed by the edge server 60 upon receiving this image data is the same as the process performed in "(A) When an image with sufficient resolution is obtained."

[0105] (B2) When the amount of transmitted data is greater than the available communication bandwidth In this case, the determination in step 364 is affirmative. At this time, an area with a high transmission priority is extracted from the image (step 366 in FIG. 13). The transmission priority is information shared in advance between the edge server 60 and the vehicle 62 as a priority for transmitting a detected object. Examples of objects with a low transmission priority include objects other than dynamic objects, objects whose reflection area is below a threshold and whose attributes are clearly difficult to determine, and objects that exist at a location farther than the distance specified by the edge server 60. An area with a high transmission priority is an area smaller than the original image that includes at least the detected object.

[0106] In step 368, a reduced image is reconstructed using the image of this extracted region (step 368). When there are multiple objects, the multiple regions containing them are arranged at their positions on the screen. Here, the resolution of each of the extracted regions is the resolution determined in step 356. In the regions other than these, blank images, for example, are arranged. Blank images are efficiently compressed when transmitted. Therefore, by arranging images in this manner, a reduced image with a small amount of transmission data can be reconstructed. Instead of blank images, images with a particularly low resolution may be arranged in the regions other than the object regions, or images with a specific pattern recorded therein that require a small amount of transmission data may be arranged.

[0107] In step 370, it is determined whether the amount of data of the image thus reconstructed is equal to or less than the available communication bandwidth.

[0108] (B2-1) When the data volume is less than or equal to the available communication bandwidth In this case, the determination in step 370 is negative. As a result, the vehicle 62 transmits the image data reconstructed as described above to the edge server 60 in step 360. The processing performed by the edge server 60 is the same as the processing performed in the case of "(A) When an image of sufficient resolution can be obtained."

[0109] (B2-2) When data volume > available communication bandwidth In this case, the determination in step 370 is affirmative. That is, in step 370, it is determined that the amount of data is still greater than the communication bandwidth. This is the case, for example, when there are many objects with high transmission priority in the image and they are widely included. The vehicle 62 then performs the following process.

[0110] That is, regardless of the resolution value determined in step 356, the vehicle 62 reduces the resolution of the original image so that the amount of data is equal to or less than the amount that can be transmitted using the available communication bandwidth. If the resolution is reduced in this way, the edge server 60 cannot use this image to determine the target attribute, but it may be possible to determine other attributes using this image. However, unless the edge server 60 actually performs a process to determine the attributes of an object from the image, the edge server 60 cannot determine which attributes can be determined and which attributes cannot be determined.

[0111] Therefore, in this embodiment, the vehicle 62 adds information to the image regarding what attributes can be determined using an image with a resolution lower than the required level. Specifically, in step 374, the vehicle 62 determines the extent to which attributes can be determined by referring to the detected attribute output table 160 stored in the detected attribute output table storage unit 260 shown in Fig. 10 based on the distance to the object and the resolution determined in step 372 (step 374). By referring to the detected attribute output table 160, a value indicating the most detailed type of attribute that can be determined from the image data obtained in step 372 is determined.

[0112] The vehicle 62 adds attribute information indicating the type of attribute determined in step 374 to the image data converted to low resolution in step 372. The vehicle 62 transmits the image data with the attribute information added to the edge server 60, and the process ends.

[0113] The edge server 60, which receives this data, detects that attribute information has been added to the received image. In this case, the edge server 60 determines the attributes specified by the attribute information and determinable attributes from the received image, and uses them for processing to assist driving.

[0114] Effects of the First Embodiment As described above, according to this embodiment, the edge server 60 transmits the desired attribute (target attribute) and the resolution selection table 150 to the vehicle 62. The vehicle 62 converts the image to the lowest resolution within the range in which the edge server 60 can determine the target attribute and transmits the converted image to the edge server 60. The vehicle 62 does not need to transmit the image to the edge server 60 using a resolution higher than necessary. As a result, the amount of transmission data required for the driving assistance processing by the edge server 60 can be reduced without degrading the quality of the processing. If the image data is larger than the available communication bandwidth, the vehicle 62 reduces the transmission frame rate when transmitting the image. In this case, the resolution of the image data received by the edge server 60 is the desired resolution. Therefore, the impact on the driving assistance processing is minimized.

[0115] If the image data cannot be transmitted to the edge server 60 even after lowering the transmission frame rate, the vehicle 62 reconstructs the image so that the amount of transmission data is further reduced and transmits it to the edge server 60. If the amount of transmission data is still greater than the available communication bandwidth, the vehicle 62 reduces the resolution of the original image to a level that can be transmitted using that communication bandwidth and transmits it to the edge server 60. At this time, the vehicle 62 adds attribute information to the image data that indicates the types of attributes that can be determined using an image of that resolution. Upon receiving this image data, the edge server 60 can determine the attributes that can be determined from the image data by examining the attribute information added to the image data and can immediately perform processing for that purpose. Compared to when attribute information is not added, the edge server 60 can avoid unnecessary processing to extract attributes that cannot be determined from the image.

[0116] As a result, the vehicle 62 can transmit information necessary for driving assistance to the edge server 60 while effectively utilizing the communication band.

[0117] Second Embodiment <composition> In the first embodiment, the edge server 60 creates the resolution selection table 150 and transmits it to the vehicle 62. The vehicle 62 creates the detected attribute output table 160, and when the resolution of the transmitted image is low, attribute information obtained from the detected attribute output table 160 is added to the image data. However, this disclosure is not limited to such an embodiment. For example, the detected attribute output table 160 may be in a database (DB) rather than in a so-called table format. If a DB is used, the vehicle 62 can obtain the required attribute information using the DB's functions without creating the detected attribute output table 160. This second embodiment is such an embodiment.

[0118] The overall configuration is the same as in Fig. 1. Here, the edge server 60 in Fig. 8 is replaced with an edge server 600 shown in Fig. 17. Similarly, the in-vehicle device 210 in Fig. 10 is replaced with an in-vehicle device 650 shown in Fig. 18. These hardware configurations are the same as in the first embodiment. The functional configurations of these will be described below.

[0119] Edge Server 600 17, edge server 600 includes the same communication device 180 and driving assistance analysis unit 182 as those shown in Fig. 8, and a vehicle management unit 618 for storing and managing vehicle information received by communication device 180 from vehicles and the like, including the vehicle model, location, moving speed, sensor placement status, etc. of each vehicle. Edge server 600 further includes a resolution selection table DB creation unit 608 for extracting information similar to the information for creating resolution selection table 150 from the configuration of driving assistance analysis unit 182 and performing a process of registering the information in the DB, and a resolution selection table DB 610 for storing information similar to the resolution selection table 150 and providing a function for searching using various search methods.

[0120] The resolution selection table DB 610 is a so-called relational DB, and includes a resolution selection table in the DB sense.

[0121] The record format of this resolution selection table is, for example, <record identifier, lower distance limit, upper distance limit, attribute identifier, resolution>. The record identifier is a unique identifier for identifying each record and is always used in normal DB records. The lower distance limit and upper distance limit indicate the distance range to which the record is applicable. The attribute identifier is an identifier for distinguishing the attribute to be determined. As in the first embodiment, the attributes include simple attributes, detailed attributes, behavioral attributes, body direction, and face direction. The attribute identifier distinguishes these attributes and a unique value is assigned to each attribute. As in the first embodiment, the resolution also includes HD, FHD, QHD, and 4K.

[0122] For example, if the distance to the object detected in the image is 30m and the target attribute is body orientation, the following query is sent to the database.

[0123] Select distinct resolution from resolution selection table where distance lower limit < 30m and distance upper limit = > 30m and attribute identifier = body orientation (identifier) Referring to Figure 6, this query returns a list of resolutions {FHD, HD} that satisfy this condition. To reduce the amount of data transmitted, the smallest resolution in this list can be selected.

[0124] The edge server 600 further includes a resolution selection table dump processing unit 612 that outputs the dump file from the resolution selection table in the resolution selection table DB 610 in response to the vehicle management unit 618 receiving vehicle information from a new vehicle, a resolution selection table dump file memory unit 614 that stores the dump file obtained by the resolution selection table dump processing unit 612, and a dump file transmission unit 616 that transmits the dump file stored in the resolution selection table dump file memory unit 614 to the vehicle corresponding to the newly received vehicle information by the vehicle management unit 618 via the communication device 180.

[0125] 《In-vehicle device 650》 18, an in-vehicle device 650 according to the second embodiment is different from in-vehicle device 210 shown in Fig. 10 in that it includes an image data transmission unit 660 instead of image data transmission unit 234 of Fig. 10. Image data transmission unit 660 does not use resolution selection table 150 and detected attribute output table 160 as in Fig. 10, but uses a resolution selection table DB.

[0126] The image data transmission unit 660 includes the object / distance detection unit 250, the communication state determination unit 262, and the object attribute storage unit 266, similar to the image data transmission unit 234 in FIG.

[0127] The image data transmission unit 660 further includes a table and target attribute receiving unit 684 for receiving a dump file of the resolution selection table 150 and target attributes indicating attributes that the vehicle is to determine from the edge server 600 via the wireless communication device 236 when the vehicle first communicates with the edge server 600, a resolution selection table DB 688, and a resolution selection table DB restore unit 686 for restoring the resolution selection table DB 688 using the dump file received by the table and target attribute receiving unit 684.

[0128] The image data transmission unit 660 further includes a resolution determination unit 682 that determines a list of resolutions of images to be transmitted to the edge server 600 based on the distance detected for each object in the image by the object / distance detection unit 250 and the object attributes stored in the object attribute storage unit 266. Specifically, the resolution determination unit 682 obtains the list of resolutions by issuing a query to a resolution selection table DB 688. The image data transmission unit 660 further includes a transmission data generation unit 692 that generates a transmission image by changing the resolution of the image acquired by the image acquisition unit 232 based on the resolution determined by the resolution determination unit 682 and the communication state determined by the communication state determination unit 262, and transmits the transmission image to the vehicle 62 via the wireless communication device 236.

[0129] The image data transmission unit 660 further includes a resolution selection table DB search unit 690 for providing the transmission data generation unit 692 with a list of attributes that can be determined using an image of a given resolution when the transmission data generation unit 692 cannot obtain a resolution at which the target attribute can be determined when generating an image for transmission and decides to lower the image resolution. Specifically, the resolution selection table DB search unit 690 searches the resolution selection table DB 688 for a list of attributes that can be determined at a given resolution.

[0130] The structure (record configuration) of the resolution selection table DB 688 is as explained for the resolution selection table DB 610 in Fig. 17. As already mentioned, examples of queries issued by the resolution determination unit 682 when determining the resolution of an image are as follows:

[0131] Select distinct resolution from resolution selection table where distance lower limit < 30m and distance upper limit = > 30m and attribute identifier = body orientation (identifier) This query will return a list of image resolutions {FHD, HD} at which the target attribute can be determined at a distance of 30m.

[0132] In response to this, the resolution selection table DB search unit 690 issues the following query:

[0133] Select attribute identifier from resolution selection table where distance lower limit < 30 m and distance upper limit = > 30 m and resolution = resolution determined by the transmission data generation unit 692 When the resolution selection table DB search unit 690 issues this query to the resolution selection table DB 688, a list of attributes that can be determined from an image of an object located at a specified distance (e.g., 30 m) at a resolution determined by the transmission data generation unit 692 is obtained from the resolution selection table DB 688. In the above query, the attribute {3 (body orientation)} is obtained, which satisfies the conditions that the distance to the object is 30 m and the resolution determined by the transmission data generation unit 692 is FHD or lower. Therefore, it can be seen that the edge server 600 can use this image to determine body orientation and coarser (less detailed) attributes, namely, simple attributes, detailed attributes, and behavioral attributes (see FIG. 7). Therefore, the vehicle can transmit a list of these attributes or the attribute name indicating the most detailed attribute to the edge server 600 as attribute information.

[0134] <Operation> The edge server 600 and the in-vehicle device 650 according to the second embodiment operate as follows.

[0135] 17, the resolution selection table DB creation unit 608 registers each record of the resolution selection table in the resolution selection table DB 610 based on the functions of the driving assistance analysis unit 182. The resolution selection table DB creation unit 608 creates or updates the resolution selection table DB 610 when the driving assistance analysis unit 182 is started for the first time, when a new function is added to the driving assistance analysis unit 182, or when any modification is made to the functions of the driving assistance analysis unit 182.

[0136] When the on-vehicle device 650 enters the jurisdiction of the edge server 600, the on-vehicle device 650 transmits its own vehicle information to the edge server 600 (step 280), as shown in Fig. 19. In response to receiving the vehicle information, the edge server 600 checks whether the vehicle information has already been registered in the vehicle management unit 618 (step 300). If the vehicle has already been registered in the vehicle management unit 618, the edge server 600 waits until it receives an image from the vehicle (the path from step 300 to step 304 in Fig. 19).

[0137] 17, when the vehicle has not yet been registered in the vehicle management unit 618, the resolution selection table dump processing unit 612 of the edge server 600 creates a dump file of the resolution selection table DB 610 and saves it in the resolution selection table dump file storage unit 614 (step 730 in FIG. 19). In response to the dump file being saved in the resolution selection table dump file storage unit 614, the dump file transmission unit 616 transmits the dump file together with the attribute (target attribute) to be determined by the edge server 600 to the in-vehicle device 650 via the communication device 180 of the edge server 600 (step 732).

[0138] 19, the table and target attribute receiving unit 684 of the in-vehicle device 650 shown in Fig. 18 receives this dump file and the target attributes, outputs the dump file to the target attribute storage unit 266, and stores the target attributes in the target attribute storage unit 266. The resolution selection table DB restore unit 686 restores the resolution selection table DB 688 from this dump file. As a result, the resolution selection table DB 688 becomes able to respond to queries from the resolution determination unit 682 and queries from the resolution selection table DB search unit 690. Note that if the resolution selection table dump file has already been received, the in-vehicle device 650 does not perform the process of step 710, but instead performs the process of step 286.

[0139] In step 286, an image is captured using camera 202, and the distance from camera 202 to each object in the image is measured. This process is performed by object / distance detection unit 250 in Fig. 18. In the following step 711, the query described above is issued to transmission data generation unit 692 based on the distance measured in step 286 and the target attribute stored in target attribute storage unit 266, thereby obtaining a list of corresponding resolutions. Resolution determination unit 682 determines the lowest resolution from this list of resolutions as the resolution of the image to be transmitted.

[0140] 13, a determination is made as to whether an image of this resolution can be transmitted, and if so, the in-vehicle device 650 converts the image to that resolution and transmits it to the edge server 600 (step 290). This corresponds to the case where the determination is negative in any of steps 358, 364, and 370 of FIG.

[0141] If it is not possible to transmit an image of that resolution (if the determinations at steps 358, 364, and 370 in FIG. 13 are all positive), the in-vehicle device 650 converts the image to a resolution with a data amount that can be transmitted to the edge server 600 based on the available communication bandwidth in step 712. Furthermore, the in-vehicle device 650 determines attributes that can be determined based on the image of the resolution after such conversion. Specifically, the in-vehicle device 650 issues a query to the resolution selection table DB 688 (FIG. 18) using the distance and resolution as keys to search for attributes that can be determined from the image at that distance and resolution. The in-vehicle device 650 determines the attributes obtained as a result of this query as attribute information to be transmitted to the edge server 600.

[0142] Thereafter, in step 290, the in-vehicle device 650 adds attribute information indicating the most detailed attributes that can be determined from the image data to the image data after the resolution conversion, if necessary, and transmits the image data to the edge server 600.

[0143] The edge server 600 receives this image in step 304. The subsequent operation of the edge server 600 is the same as in the first embodiment.

[0144] Effect of the Second Embodiment As described above, according to this embodiment, it is possible to realize the functions of both the resolution selection table 150 and the detected attribute output table 160 by using a DB, without creating the detected attribute output table 160. There is no need to generate the detected attribute output table 160, and the configuration of the in-vehicle device 650 can be further simplified.

[0145] <Modification> In the first embodiment described above, the resolution selection table 150 is transmitted from the edge server 60 to the vehicle 62, and the vehicle 62 generates the detected attribute output table 160. However, this disclosure is not limited to such an embodiment. For example, both the resolution selection table 150 and the detected attribute output table 160 may be generated by the edge server 60 and transmitted to the vehicle 62.

[0146] In the first and second embodiments, when only an image with a resolution that does not allow the target attribute to be determined can be transmitted, the in-vehicle device reduces the image resolution and transmits information on attributes that can be determined using the image with that resolution to the edge server 60, 600. However, this disclosure is not limited to such embodiments. For example, in such a case, information indicating that an image with sufficient resolution cannot be transmitted may be transmitted to the edge server 60, 600, etc. In this case, there is no need to use the detected attribute output table 160 or the resolution selection table DB 688, and the configuration of the in-vehicle device can be further simplified.

[0147] When there are multiple vehicles capable of transmitting images to the edge server 60, 600, etc., the following processing sequence may be adopted. That is, each vehicle transmits information about attributes that can be determined even at the resolution of the image that it transmits to the edge server 60, 600, etc. The edge server 60, 600, etc. selects the vehicle that will actually transmit the image based on this information. The edge server 60, 600, etc. instructs the selected vehicle to transmit the image. The vehicle that receives the instruction transmits an image to the edge server 60, 600, etc. at the resolution used to determine the determinable attribute. By using this processing sequence, it is possible to obtain an image with a resolution that allows the edge server 60, 600, etc. to determine the target attribute. Because only the selected vehicle needs to transmit images, the amount of transmitted data can be reduced without degrading the quality of the driving assistance processing in the edge server 60, 600, etc.

[0148] Furthermore, in the first and second embodiments, the resolution is expressed using a symbol such as "HD." This is because this is a typical resolution used in current image processing. However, this disclosure is not limited to such an embodiment. For example, the resolution may be specified directly using a numerical value (vertical resolution x horizontal resolution). Alternatively, the vertical and horizontal resolution may each be defined by a continuous function of distance and object attribute. In this case, the vertical and horizontal ratio may be set to match the HD ratio or may be set to match another value.

[0149] In the above embodiment, the distance between the vehicle (camera) and the target object is measured or calculated using a distance measurement sensor such as a lidar. However, this disclosure is not limited to such an embodiment. For example, the distance to the target may be calculated using a stereo camera, or the distance to the target may be calculated using image processing of multiple images from a monocular camera.

[0150] Furthermore, in the above-described embodiment, when attribute information is attached to an image, the edge server 60 and the edge server 600 determine the attributes specified by the attribute information from the image. However, this disclosure is not limited to such an embodiment. It is also possible to determine whether or not to actually perform image analysis by examining the value of the attribute information. In this case, the computational resources of the edge server 60 or the edge server 600 can be saved.

[0151] Furthermore, in the above embodiment, the vehicle 62 and the in-vehicle device 650 add attribute information to an image when they can only transmit images with a resolution that makes it impossible to determine the target attribute. However, this disclosure is not limited to such an embodiment. For example, attribute information may always be added to an image. In this case, there is no need for the edge server 60 and the in-vehicle device 650 to transmit the target attribute to the vehicle 62 and the in-vehicle device 650. The edge server 60 and the in-vehicle device 650 may determine what attribute to determine based on the image information added to the image.

[0152] The above embodiment uses a vehicle and a server as an example, but this disclosure is not limited to such an embodiment. This disclosure can also be applied to a so-called vehicle-to-vehicle system.

[0153] In the above embodiment, after the vehicle 62 or the in-vehicle device 650 starts communication with the edge server 60 or the edge server 600, the edge server 60 or the edge server 600 transmits the resolution selection table 150 and the like to the vehicle 62 or the in-vehicle device 650. However, this disclosure is not limited to such an embodiment. For example, a server such as the edge server 60 may send the resolution selection table 150 to a server adjacent to itself and request that the server distribute the table in advance to vehicles entering its jurisdiction. The identifier of the edge server 60 or the edge server 600 may also be added to the resolution selection table 150. In this case, the resolution selection table 150 does not need to be deleted even when the vehicle leaves the jurisdiction of the edge server 60 or the edge server 600. When the vehicle enters the jurisdiction of the edge server 60 or the edge server 600, the vehicle and the server may communicate to check whether the resolution selection table 150 has been updated, and only if an update has been made, a new resolution selection table 150 may be transmitted to the vehicle. In this case, too, it is possible to transmit only the updated items rather than transmitting the entire resolution selection table 150.

[0154] The embodiments disclosed herein should be considered in all respects as illustrative and not restrictive. The scope of the present disclosure is not defined by the detailed description of the disclosure, but by the claims of the appended claims, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0155] 50 Traffic Assistance System 60, 600 Edge Server 62 vehicles 64 imaging area 66, 110 Pedestrians 68 Trees 100, 120 images 102, 122 statue 104 Tree Statue 150 Resolution Selection Table 160 Detected Attributes Output Table 180 Communication Equipment 182 Driving Assistance Analysis Department 184 Resolution selection table creation section 186, 256 Resolution selection table storage section 188, 618 Vehicle Management Department 190 Resolution selection table transmitter 200 mm wave radar 202 Camera 204 Rider 210, 650 In-vehicle equipment 230 I / F section 232 Image acquisition unit 234, 660 Image data transmission unit 236 Wireless communication equipment 238 Operational support processing equipment 250 Object and distance detection unit 252, 682 Resolution determination unit 254, 684 Table and target attribute receiver 258 Detection attribute output table generation unit 260 Detection attribute output table storage unit 262 Communication status determination unit 264, 692 transmission data generation unit 266 Target Attribute Memory Unit 280, 282, 284, 286, 288, 289, 290, 300, 302, 304, 330, 332, 334, 336, 350, 352, 354, 356, 357, 358, 360, 362, 364, 366, 368, 370, 372, 374, 376, 710, 711, 712, 730, 732 steps 440 Computer 442, 510 monitor 446 keyboard 448 Mouse 450 DVD drive 452 DVD 454 Network 456 Semiconductor Memory 460, 550 CPU Buses 462 and 552 464, 566 GPUs 466 ROM 468 RAM 470 hard disk drive 472 Network Interface 474 Semiconductor Memory Port 500 Controller 502 GPS module 504 memory 506 Power supply circuit 508 Audio Circuit 512 Touch Panel 514 Various sensors 516 RF / Baseband Circuit 518 Wireless Communication Module 554 Memory Controller 556 Power management circuit 558 System Management Circuit 560 Media Processing Circuit 562 Display Controller 564 Input / Output Interface 568 Network Interface 570 In-Vehicle Network 608 Resolution Selection Table DB Creation Department 610 Resolution Selection Table DB 612 Resolution selection table dump processing section 614 Resolution selection table dump file storage section 616 Dump File Transmission Unit 686 Resolution Selection Table DB Restore Section 688 Resolution Selection Table DB 690 Resolution Selection Table DB Search Section

Claims

1. a determination unit that determines determinable attributes of an object from image data based on image data including an image of the object and a distance from an imaging sensor that captured the image to the object; a transmitting unit that adds attribute information including the attribute to the image data and transmits the data to a destination.

2. 2. The image data transmission device according to claim 1, wherein said determination unit includes an attribute determination unit that determines said attribute based on a resolution of said image and said distance.

3. The attribute determination unit a detection attribute output table storage unit that stores a detection attribute output table prepared in advance for determining attributes that can be determined based on a combination of the image resolution and the distance; 3. The image data transmission device according to claim 2, further comprising a table reference section for determining said attribute by referring to said detected attribute output table based on the resolution of said image and said distance.

4. 4. The image data transmission device according to claim 3, wherein said detected attribute output table stores the most detailed attribute that can be determined by a combination of resolution and distance.

5. a receiving unit that receives image data and attribute information related to the image that is added to the image; and an analysis unit that analyzes the image based on the attribute information.

6. The image analyzing device according to claim 5 , wherein the analyzing unit determines whether or not to analyze the image based on the attribute information.

Images