Information processing system, information processing method, and program

Abstract

To provide an information processing system or the like capable of recognizing that an object having a different size is erroneously extracted as a target.SOLUTION: An image processing device includes an extraction unit that scans an image by a detection area and extracts a target, a distance calculation unit that calculates a distance to the target extracted by the extraction unit, and a determination unit that determines whether or not the target is erroneously extracted based on a result of comparing the distance calculated by the distance calculation unit with a size of the detection area in which the target is extracted. The extraction unit may switch a range of a size of the detection area for extracting the target according to an area of the image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] A method for extracting an object included in a detection target image using a bounding box is known (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2024-082924 Summary of the Invention [Problem to be solved by the invention]

[0004] However, since the size of an object in an image changes depending on the distance from the camera to the object, there is a possibility that an object of a different size may be mistakenly extracted as the object.

[0005] Therefore, in one aspect, the present invention aims to provide an information processing system or the like that can recognize that an object of a different size has been erroneously extracted as a target. [Means for solving the problem]

[0006] In one embodiment, an extraction unit that scans the image using a detection area to extract an object; a distance calculation unit that calculates a distance to the object extracted by the extraction unit; a determination unit that determines whether the object has been erroneously extracted based on a result of comparing the distance calculated by the distance calculation unit with the size of the detection area from which the object has been extracted; An information processing system is provided. [Effects of the Invention]

[0007] According to one aspect, the present invention can recognize that objects of different sizes have been erroneously extracted as targets. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating a configuration of an information processing system according to an embodiment of the present invention. [Figure 2] 10 is a flowchart illustrating an example of the operation of the information processing system. [Figure 3] FIG. 10 is a diagram showing an example of extracting a face by scanning an image using a bounding box. [Figure 4] FIG. 10 is a diagram illustrating an appropriate size of a bounding box. [Figure 5] FIG. 10 is a diagram illustrating a method for limiting the size of a bounding box by associating it with a region of an image. DETAILED DESCRIPTION OF THE INVENTION

[0009] FIG. 1 is a diagram showing the configuration of an information processing system according to this embodiment.

[0010] As shown in FIG. 1, the information processing system 10 of this embodiment includes an extraction unit 11 that scans an image using a bounding box as a detection area to extract an object, a distance calculation unit 12 that calculates the distance to the object extracted by the extraction unit 11, and a determination unit 13 that determines whether the object has been erroneously extracted based on the result of comparing the distance calculated by the distance calculation unit 12 with the size of the bounding box from which the object is extracted.

[0011] The distance calculation unit 12 may calculate the distance by any method, and for example, a method using laser distance measurement or a method using triangulation based on a stereo camera or the like can be applied.

[0012] The information processing system 10 also includes a storage unit 14 that stores the relationship between the range of bounding box sizes and the area of ​​the image.

[0013] The information processing system 10 can be configured using one or more computers on which a predetermined program is installed.

[0014] Next, the operation of the information processing system 10 will be described.

[0015] In this embodiment, an example is shown in which the information processing system 10 is applied to a safety device that controls the operation of a work machine such as a heavy machine.

[0016] FIG. 2 is a flowchart showing an example of the operation of the information processing system, FIG. 3 is a diagram showing an example of scanning an image using a bounding box to extract a face, and FIG. 4 is a diagram showing an appropriate size of the bounding box.

[0017] The flowchart in Figure 2 shows the process of extracting the face, upper body, and entire body of a person contained in an image captured by a camera 20 attached to a work machine. Each time the image to be processed is updated, the process in Figure 2 is repeated.

[0018] 2, extraction unit 11 sets the size of the bounding box to an initial value. Note that the shape of the bounding box (the aspect ratio of the rectangle) can be defined for each extraction target (face, upper body, and whole body).

[0019] In step S104, the extraction unit 11 scans the image with a bounding box of the set size.

[0020] In step S106, the determination unit 13 determines whether or not a face was extracted in step S104, and if the determination is affirmative, the process proceeds to step S108, and if the determination is negative, the process proceeds to step S112.

[0021] In step S108, the distance calculation unit 12 calculates the distance from the camera 20 to the face extracted in step S104.

[0022] In step S110, determination unit 13 compares the distance calculated by distance calculation unit 12 in step S108 with the size of the bounding box from which the face was extracted in step S104. Here, determination unit 13 determines whether the calculated distance indicates a value appropriate for the size of the bounding box, and if the determination is affirmative, proceeds to step S130, or if the determination is negative, proceeds to step S112.

[0023] In step S112, the determination unit 13 determines whether or not the upper body was extracted in step S104, and if the determination is affirmative, the process proceeds to step S114, and if the determination is negative, the process proceeds to step S118.

[0024] In step S114, the distance calculation unit 12 calculates the distance from the camera 20 to the upper body extracted in step S104.

[0025] In step S116, determination unit 13 compares the distance calculated by distance calculation unit 12 in step S112 with the size of the bounding box from which the upper body was extracted in step S104. Here, determination unit 13 determines whether the calculated distance indicates a value appropriate for the size of the bounding box, and if the determination is affirmative, proceeds to step S130, or if the determination is negative, proceeds to step S118.

[0026] In step S118, the determination unit 13 determines whether or not the whole body has been extracted in step S104, and if the determination is affirmative, the process proceeds to step S120, and if the determination is negative, the process proceeds to step S124.

[0027] In step S120, the distance calculation unit 12 calculates the distance from the camera 20 to the whole body extracted in step S104.

[0028] In step S122, the determination unit 13 compares the distance calculated by the distance calculation unit 12 in step S120 with the size of the bounding box from which the entire body was extracted in step S104. Here, the determination unit 13 determines whether the calculated distance indicates a value appropriate for the size of the bounding box, and if the determination is affirmative, the process proceeds to step S130, and if the determination is negative, the process proceeds to step S124.

[0029] In step S124, extraction unit 11 determines whether scanning of the image using the bounding box has finished, and if the determination is affirmative, ends the process, and if the determination is negative, proceeds to step S 126. If the processes of steps S104 to S122 have been performed for all sizes of the bounding box, the determination in step S124 is affirmative.

[0030] In step S126, the extraction unit 11 sets the size of the bounding box to a specific size (changes the size), and the process proceeds to step S104.

[0031] In step S128, the determination unit 13 calculates the distance between the extracted target and the work machine. The distance can be calculated based on the distance calculated by the distance calculation unit 12 (step S108, step S114, or step S120).

[0032] In step S130, the determination unit 13 determines whether the distance calculated in step S128 is within a range in which an alarm should be issued. If the determination is affirmative, the determination unit 13 proceeds to step S132, and if the determination is negative, the determination unit 13 proceeds to step S134.

[0033] In step S132, the determination unit 13 issues an alarm (for example, an alarm sound) to the surroundings of the work machine or to the operator of the work machine.

[0034] In step S134, the determination unit 13 determines whether the distance calculated in step S128 is within a range in which a stop signal should be output. If the determination is affirmative, the determination unit 13 proceeds to step S136, and if the determination is negative, the determination unit 13 proceeds to step S124.

[0035] In step S136, the determination unit 13 outputs a stop signal and ends the process. When a stop signal is output, the work machine receives the stop signal and stops.

[0036] In this way, in the processing of steps S130 to S138, an alarm is issued or a stop signal is output depending on the distance between the extracted object and the work machine. The distance range for issuing an alarm and the distance range for outputting a stop signal can each be set in advance. For example, the distance range may be set so that an alarm is issued before a stop signal is output when an object (person) approaches. Alternatively, the distance ranges for both may be the same. In this way, in this embodiment, when a person near the work machine is captured by camera 20, an alarm is issued to the surrounding area or to the operator of the work machine, or the work machine is brought to an emergency stop, thereby effectively ensuring safety.

[0037] In this embodiment, the determination unit 13 compares the distance calculated by the distance calculation unit 12 with the size of the bounding box from which the object (face, upper body, or whole body) was extracted in step S104 (steps S110, S116, and S122). As shown in FIGS. 3 and 4, the size of the face 51 in the image varies depending on the distance from the camera 20 to the face 51. Therefore, if the face 51 is correctly extracted, the face 51 should be extracted using a bounding box of a size corresponding to the distance from the camera 20 to the face 51. For example, a face 51 located a certain distance from the camera 20 (FIG. 4) is extracted using the bounding box 6a. On the other hand, a face 51 located 10 meters further away is extracted using the smaller bounding box 6b. The same applies to the upper body and the whole body. For example, the whole body located a certain distance from the camera 20 is extracted using the bounding box 7a. On the other hand, the whole body, which is located 10 m further away, is extracted using a smaller bounding box 7b.

[0038] In this embodiment, the determination unit 13 determines that the object has been correctly extracted if the size of the bounding box used to extract the object is within a reasonable range corresponding to the distance from the camera 20 to the object. On the other hand, if the size of the bounding box used to extract the object is not within that range, the determination unit 13 determines that the object has been erroneously extracted. The reasonable range may be, for example, a range that allows for a deviation of approximately 10 percent from the range of the actual size of the object. Note that the allowable deviation may be further increased to prevent missed detection of objects included in images, such as in an application to a safety device.

[0039] The correspondence between the distance from the camera 20 to the object and the appropriate range of bounding box sizes can be stored in the storage unit 14. The correspondence is defined for each of the face 51, the upper body, and the whole body. The correspondence can be calculated based on the imaging magnification of the camera 20, or the like, or the correspondence may be acquired by performing calibration based on an image of the object captured by the camera 20.

[0040] As described above, according to this embodiment, by comparing the distance calculated by the distance calculation unit 12 with the size of the bounding box from which the object is extracted, it is possible to recognize that the object has been erroneously extracted. Furthermore, it is possible to effectively suppress malfunctions (output of a stop signal in step S130) caused by erroneous extraction of the object. For example, there is a possibility that an object larger or smaller than the object may be erroneously extracted as the object. However, in such a case, according to this embodiment, it is possible to identify the size of the extracted object based on the distance and the size of the bounding box. Therefore, it is possible to recognize erroneous extraction of the object and avoid the occurrence of inconveniences such as malfunctions.

[0041] FIG. 5 is a diagram showing a method for limiting the range of bounding box sizes in association with image regions.

[0042] In the example of FIG. 5, the image is vertically divided into four regions 8A to 8D. As shown in the example of FIG. 5, when an image is taken with camera 20 from diagonally above a horizontal ground, the face 51 of a standing person is positioned higher in the image and becomes smaller in size as the face moves away from camera 20. For example, face 51 in region 8C can be extracted using bounding box 6a (FIG. 4), and face 51 in region 8D can be extracted using bounding box 6b (FIG. 4). In other words, the size of the bounding box required to extract face 51 becomes smaller as it moves higher in the image. The same applies to the sizes of the bounding boxes for extracting the upper body and the whole body.

[0043] Therefore, by limiting the range of bounding box sizes (maximum and minimum size values) for each of regions 8A to 8D, the probability of erroneously extracting an object can be reduced. Also, since the range of bounding box sizes is reduced for the entire image, the processing load when extracting an object can be reduced. In the example of Figure 5, the range of bounding box sizes can be shifted in the direction of successively smaller sizes for regions 8A to 8D.

[0044] In this case, the relationship between the regions 8A-8D and the distance from the camera 20 to the target may be determined by calibration. For example, the distance from the camera 20 to the target in each of the regions 8A-8D can be measured, and a range of bounding box sizes can be associated with the measured distance. The relationship between each of the regions 8A-8D and the range of bounding box sizes can be stored in the storage unit 14. By using the relationship stored in the storage unit 14, the range can be switched depending on the region 8A-8D when the extraction unit 11 extracts the target.

[0045] Although the embodiments have been described in detail above, the present disclosure is not limited to the specific embodiments and various modifications and changes are possible within the scope of the claims. For example, the detection area is not limited to a rectangular (box-shaped) shape. Furthermore, the application of the present disclosure is arbitrary and is not limited to application to safety devices. [Explanation of symbols]

[0046] 10 Information Processing Systems 11 Extraction part 12 Distance calculation unit 13 Judgment section 14 Storage section

Claims

1. an extraction unit that scans the image using a detection area to extract an object; a distance calculation unit that calculates a distance to the object extracted by the extraction unit; a determination unit that determines whether the object has been erroneously extracted based on a result of comparing the distance calculated by the distance calculation unit with the size of the detection area from which the object has been extracted; An information processing system comprising:

2. The information processing system according to claim 1 , wherein the extraction unit switches a range of sizes of the detection region for extracting the object depending on the region of the image.

3. a storage unit that stores a relationship between a size range of the detection area and an area of ​​the image; The information processing system according to claim 2 , wherein the extraction unit switches a range of the size of the detection region for each region of the image using the relationship stored in the storage unit.

4. an extraction step of scanning the image with the detection region to extract the object; a distance calculation step of calculating a distance to the object extracted by the extraction step; a determination step of determining whether the object has been erroneously extracted based on a result of comparing the distance calculated in the distance calculation step with the size of the detection area from which the object has been extracted; An information processing method comprising:

5. an extraction step of scanning the image with the detection region to extract the object; a distance calculation step of calculating a distance to the object extracted by the extraction step; a determination step of determining whether the object has been erroneously extracted based on a result of comparing the distance calculated in the distance calculation step with the size of the detection area from which the object has been extracted; A program that causes a computer to execute the above.

Images