Image processing apparatus, method for image processing, and program

Abstract

To maintain the accuracy of processing to generate information relating to an object by using point cloud data.SOLUTION: An image processing apparatus includes a data acquisition unit, a feature amount calculation unit, and a region correction unit. The data acquisition unit acquires point cloud data generated by scanning a space of a measurement target. The feature amount calculation unit sets a plurality of partial regions with reference to a first boundary of a processing target region in the point cloud data and calculates the feature amount of each of the plurality of partial regions. The region correction unit corrects the processing target region by setting a new second boundary on the basis of positions of partial regions satisfying a criterion relating to the feature amounts.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an image processing method, and a program. [Background technology]

[0002] Patent Document 1 below discloses a method for determining an estimated volume of an object captured by a three-dimensional point cloud. The method of Patent Document 1 includes, for example, the following steps (1) to (3). (1) Receiving a three-dimensional point cloud including a plurality of three-dimensional points and a reference plane spatially related to the three-dimensional point cloud. (2) For each two-dimensional bin having a length and width extending along the reference plane, determine the number of three-dimensional points in each bin and the height of the bin above the reference plane. (3) determining an estimated volume of the object captured by the 3D point cloud based on the number of 3D points in each bin and the height of each bin; [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2023-525538 Summary of the Invention [Problem to be solved by the invention]

[0004] In a technology for generating information about an object based on point cloud data (for example, an estimated volume of an object as exemplified in Patent Document 1), if a point cloud that indicates important features related to the object is missing from a processing target area, the accuracy of the processing may be reduced. A technology for maintaining the accuracy of the processing for generating information about an object using point cloud data is desired. [Means for solving the problem]

[0005] The image processing device in the present disclosure includes: a data acquisition means for acquiring point cloud data generated by scanning a space to be measured; a feature amount calculation means for setting a plurality of partial areas based on a first boundary of a processing target area in the point cloud data and calculating a feature amount of each of the plurality of partial areas; an area correcting means for correcting the processing target area by setting a new second boundary based on the position of a partial area that satisfies a criterion related to the feature amount; Equipped with.

[0006] The image processing method in the present disclosure includes: The computer Acquire point cloud data generated by scanning the space to be measured, setting a plurality of partial regions based on a first boundary of a processing target region in the point cloud data, and calculating feature amounts of each of the plurality of partial regions; modifying the processing target region by setting a new second boundary based on the position of a partial region that satisfies a criterion related to the feature amount; This includes:

[0007] The program in this disclosure is Computer, a data acquisition means for acquiring point cloud data generated by scanning a space to be measured; a feature amount calculation means for setting a plurality of partial areas based on a first boundary of a processing target area in the point cloud data and calculating a feature amount of each of the plurality of partial areas; an area correcting means for correcting the processing target area by setting a new second boundary based on the position of a partial area that satisfies a criterion related to the feature amount; Function as. [Effects of the Invention]

[0008] According to the present disclosure, a technique is provided for maintaining the accuracy of a process for generating information about an object using point cloud data. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 2 is a diagram illustrating a first example of a functional configuration of an image processing device according to the present disclosure. [Figure 2] FIG. 1 is a diagram illustrating an example of a hardware configuration of a computer that realizes an image processing device according to the present disclosure. [Figure 3] 1 is a flowchart showing a first example of processing executed by an image processing device according to the present disclosure. [Figure 4] 1 is a diagram illustrating an example of a configuration of a system including an image processing device according to the present disclosure. [Figure 5] FIG. 10 is a diagram illustrating an example of definition information. [Figure 6] 10A to 10C are diagrams illustrating an example of a process for setting a plurality of partial regions in point cloud data along the depth direction of a space to be measured. [Figure 7] 10A to 10C are diagrams illustrating an example of a process for setting a plurality of partial regions in point cloud data along the width direction of a space to be measured. [Figure 8] FIG. 10 is a diagram illustrating a first example of a flow for calculating a feature amount for each of a plurality of set partial regions. [Figure 9] FIG. 10 is a diagram illustrating a second example of the flow of calculating the feature amount of each of a plurality of set partial regions. [Figure 10] 10 is a flowchart illustrating a processing flow for correcting (setting) the boundary of a processing target area along the depth direction of a space to be measured. [Figure 11] 10 is a flowchart illustrating a processing flow for correcting (setting) the boundary of a processing target area along the depth direction of a space to be measured. [Figure 12] 10A and 10B are diagrams illustrating an example of a setting operation of a second boundary by an area correction unit. [Figure 13] 10 is a flowchart illustrating a process flow for correcting (setting) the boundary of a processing target area along the width direction of a space to be measured. [Figure 14] 10 is a flowchart illustrating a process flow for correcting (setting) the boundary of a processing target area along the width direction of a space to be measured. [Figure 15]10A and 10B are diagrams showing an example of information for determining how to set the boundary of the processing target area in the width direction. [Figure 16] 10A and 10B are diagrams showing a specific example of an operation for moving the boundary of a processing target area in the width direction. [Figure 17] FIG. 10 is a diagram illustrating a second example of the functional configuration of an image processing device according to the present disclosure. [Figure 18] FIG. 10 is a diagram illustrating a third example of the functional configuration of an image processing device according to the present disclosure. [Figure 19] 10 is a flowchart illustrating a flow of processing executed by an object region identifying unit in the present disclosure. [Figure 20] FIG. 10 is a diagram illustrating an example of information output by an object region specifying unit. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. Note that in this disclosure, the drawings relate to one or more embodiments. Furthermore, in all drawings, similar components are given similar reference numerals, and descriptions thereof will be omitted as appropriate. Furthermore, unless otherwise specified, in each block diagram, each block represents a functional configuration, not a hardware configuration. Furthermore, when elements such as blocks in a diagram are linked by arrows, the direction of the arrows is merely intended to make the flow of information easier to understand. In this case, unless otherwise specified, the direction of the arrows does not limit the direction of communication (one-way communication / two-way communication).

[0011] In the following description, "acquisition" includes at least one of the following: a device going to retrieve data or information stored in another device or storage medium (active acquisition), and inputting data or information output from another device into the device (passive acquisition). Examples of active acquisition include making a request or inquiry to another device and receiving a reply, and accessing and reading information from another device or storage medium. Examples of passive acquisition include receiving information that is distributed (or transmitted, push notification, etc.). Furthermore, "acquisition" may also mean selecting and acquiring data or information from received data or information, or selecting and receiving distributed data or information.

[0012] ·overview The image processing device described in the present disclosure has a function for grasping with high accuracy the state of a space to be measured from a scan image (point cloud data) generated by scanning the space using a scanning device. Below, several exemplary embodiments will be given to explain each function of the image processing device in the present disclosure. Note that the image processing device in the present disclosure is not limited to the following embodiments.

[0013] First embodiment <Example of functional configuration> 1 is a diagram showing a first example of the functional configuration of an image processing device according to the present disclosure. The image processing device 10 shown in the diagram includes a data acquisition unit 110, a feature calculation unit 120, and an area correction unit .

[0014] The data acquisition unit 110 acquires point cloud data. The point cloud data is generated by scanning a space to be measured using a scanning device (not shown). The data acquisition unit 110 is configured to read out the target point cloud data at any timing, for example, from a database (not shown) that stores the point cloud data generated by the scanning device (not shown). Alternatively, the data acquisition unit 110 may be configured to communicate with the scanning device (not shown) and acquire the generated point cloud data.

[0015] The feature amount calculation unit 120 sets a plurality of partial areas based on a first boundary of the processing target area in the point cloud data. As will be described in detail later, the "processing target area" is provisionally set using predefined information according to the range corresponding to the point cloud data (the space that is the measurement target). Furthermore, the "processing target area" can be appropriately corrected by the area correction unit 130, which will be described later. Furthermore, the feature amount calculation unit 120 calculates the feature amount of each of the set plurality of partial areas.

[0016] The area correction unit 130 corrects the processing target area as needed using the feature amounts calculated for each of the multiple partial areas. The area correction unit 130 identifies a partial area that satisfies a criterion related to the feature amounts based on the feature amounts of each of the multiple partial areas. The area correction unit 130 corrects the processing target area by setting a new boundary (second boundary) based on the position of the identified partial area.

[0017] <Hardware configuration example> FIG. 2 is a diagram illustrating an example of the hardware configuration of a computer that realizes the image processing device according to the present disclosure.

[0018] The computer 1000 illustrated in this figure includes a bus 1010 , a processor 1020 , a memory 1030 , a storage device 1040 , an input / output interface 1050 , and a network interface 1060 .

[0019] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0020] The processor 1020 is a central processing unit (CPU) or a graphics processing unit (GPU).

[0021] The memory 1030 is a main storage device realized by a random access memory (RAM) or the like.

[0022] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read only memory (ROM), etc. The storage device 1040 stores at least program modules that realize the functions of the image processing device 10 described above (the data acquisition unit 110, the feature amount calculation unit 120, and the area correction unit 130). In addition, the storage device 1040 may be used as a storage location for a database that stores measurement results (point cloud data) obtained by the scanning device.

[0023] The processor 1020 loads a program module read from the storage device 1040 onto the memory 1030 and executes the program module, thereby realizing the function corresponding to the program module. For example, the processor 1020 reads a program module corresponding to the data acquisition unit 110 onto the memory 1030 and executes the program module, thereby realizing the function of the data acquisition unit 110 described in this disclosure. Similarly, the processor 1020 reads a program module corresponding to the feature calculation unit 120 onto the memory 1030 and executes the program module, thereby realizing the function of the feature calculation unit 120 described in this disclosure. Similarly, the processor 1020 reads a program module corresponding to the region correction unit 130 onto the memory 1030 and executes the program module, thereby realizing the function of the region correction unit 130 described in this disclosure. This operation of the processor 1020 is common to each embodiment included in this disclosure.

[0024] The above-described program modules may be recorded on a recording medium other than the storage device 1040. The recording medium on which the program modules are recorded may include any non-transitory, tangible medium usable by the computer 1000. The recording medium on which the program modules are recorded may have embedded therein program code that can be read by the computer 1000 (processor 1020).

[0025] The input / output interface 1050 is an interface for connecting the computer 1000 (image processing device 10) with various devices. The "various devices" referred to here are not particularly limited, but include, for example, input devices such as operation buttons, keyboards, touch panels, mice, and microphones, and output devices such as displays and speakers.

[0026] The network interface 1060 is an interface for connecting the computer 1000 (image processing device 10) to a network. This network may be, for example, a local area network (LAN) or a wide area network (WAN). The network interface 1060 supports various wireless and wired communication standards and establishes communication between the computer 1000 (image processing device 10) and other devices (not shown) on the network. Although not particularly limited, the "other devices" may include, for example, a server device having a database that stores measurement results (point cloud data) obtained by a scanning device. The data acquisition unit 110 can communicate with the server device via the network interface 1060 and acquire the target point cloud data.

[0027] Furthermore, the computer 1000 (image processing device 10) may be connected to the above-mentioned "various devices" via a network interface 1060.

[0028] <Example of operation of image processing device> An example of the operation of the image processing device according to the present disclosure will be described below with reference to the drawings.

[0029] FIG. 3 is a flowchart showing a first example of processing executed by the image processing device according to the present disclosure.

[0030] The data acquisition unit 110 acquires point cloud data (step S102). The data acquisition unit 110 accesses a database provided in a server device (not shown), for example, and reads out target point cloud data from the point cloud data stored in the database.

[0031] The feature calculation unit 120 then sets multiple partial regions based on the boundary (first boundary) of the processing target region in the point cloud data acquired in step S102 (step S104). Here, the processing target region of the point cloud data is provisionally set using predefined information according to the range corresponding to the point cloud data (the space to be measured). For example, the feature calculation unit 120 identifies the space corresponding to the range of the point cloud data based on information such as the position and orientation of the scanning device at the time when the scanning operation to generate the point cloud data was performed. Then, the feature calculation unit 120 recognizes the processing target region in the point cloud data using definition information of the processing target region that is pre-associated with the identified space (for example, information specifying the position coordinates of the processing target region in the coordinate system of the point cloud data). Here, the definition information for identifying the processing target region is pre-stored in a storage area accessible by the feature calculation unit 120, such as the storage device 1040 or an external device (not shown). Then, the feature amount calculation unit 120 sets a plurality of partial regions in the point cloud data using the boundary (first boundary) of the processing target region in the point cloud data as a reference. For example, the feature amount calculation unit 120 sets a plurality of rectangular regions each having a predetermined size using the boundary of the processing target region as a reference.

[0032] Then, the feature calculation unit 120 calculates a feature based on the point cloud for each of the set partial regions (step S106). For example, the feature calculation unit 120 calculates a feature indicating the likelihood of an object being a surface from the point cloud data. The "likelihood of an object being a surface" can be estimated from the arrangement of the point cloud, such as the number and density of points in the point cloud data. The feature calculation unit 120 calculates at least one of the number of points and the density of points included in each of the multiple partial regions as a feature indicating the "likelihood of an object being a surface."

[0033] Based on the feature values ​​calculated for each of the multiple partial regions, the region modification unit 130 identifies partial regions that satisfy a predetermined criterion for the feature values ​​(step S108). The "predetermined criterion" here refers to a criterion for determining the "likeliness of an object's surface." For example, given the nature of point cloud data, a partial region corresponding to the position of an object's surface will contain many points representing the object's surface, while a partial region corresponding to a position other than the object's surface will contain relatively few points. Therefore, the region modification unit 130 can identify a region corresponding to the position of an object's surface based on the variation in the feature values ​​between partial regions. For example, the region modification unit 130 sequentially compares the feature values ​​between two adjacent partial regions along a predetermined direction. If a large variation in the feature values ​​is observed for a partial region newly selected for comparison, the region modification unit 130 can infer that the object's surface is likely located in that partial region or an adjacent partial region. The region modification unit 130 can identify a partial region in which a large variation in the feature values ​​is observed as a "partial region that satisfies the criterion."

[0034] Then, the area correction unit 130 executes a process of correcting the processing target area based on the position of the partial area identified as the "partial area satisfying the criteria" as follows: First, the area correction unit 130 determines whether the partial area identified as the "partial area satisfying the criteria" is included in the processing target area (step S110).

[0035] If the partial area identified as the "partial area satisfying the criteria" is included in the processing target area (step S110: YES), the area correction unit 130 determines that the processing target area has been set appropriately and ends the processing.

[0036] On the other hand, if the partial area identified as the "partial area satisfying the criteria" is not included in the processing target area (step S110: NO), the area correction unit 130 sets a new boundary (second boundary) for the processing target area based on the position of the identified partial area (step S112). This "setting a new boundary" process may be, for example, a process of shifting the position of the processing target area set at that time without changing its size, or a process of changing (enlarging or reducing) the size of the processing target area set at that time.

[0037] As described above, the image processing device according to the present disclosure identifies a partial area including a point cloud corresponding to the surface of an object in the acquired point cloud data, and determines whether the position of the default processing target area is appropriate based on the position of the partial area. Furthermore, if the position of the default processing target area is not appropriate, the image processing device according to the present disclosure corrects the processing target area by setting a new boundary (second boundary) of the processing target area based on the position of the identified partial area.

[0038] This operation enables subsequent information processing to be performed on a region containing important features (point cloud) related to the object to be identified. In other words, the image processing device according to the present disclosure solves the problem of maintaining the accuracy of processing to generate information about an object using point cloud data. The present disclosure also provides an image processing method executed by the image processing device (computer) and a program for realizing each function of the image processing device (computer). The present disclosure also provides a computer-readable recording medium on which the program is recorded.

[0039] A detailed example of the image processing device according to the present disclosure will be described below.

[0040] <System configuration> 4 is a diagram illustrating the configuration of a system including an image processing device according to the present disclosure. The system illustrated in this diagram is configured to include a scanning device 22 in addition to the image processing device 10 described above. In the example of this diagram, the scanning device 22 is provided in an autonomously mobile robot 20.

[0041] The scanning device 22 is a device that generates point cloud data that three-dimensionally indicates the positions and shapes of objects present in the surroundings by scanning the surroundings using a pulsed laser, such as a Light Detection and Ranging (LiDAR).

[0042] The robot 20 moves autonomously along a preset route while estimating its own position based on the outputs of various sensors (not shown). The robot 20 also controls the scanning device 22 while moving autonomously to scan a space 30 including an object 32. As a result, point cloud data 222 relating to the space to be measured is generated. The generated point cloud data 222 may be transmitted to the image processing device 10 at any timing, or may be temporarily stored in a server device (not shown).

[0043] The data acquisition unit 110 of the image processing device 10 acquires the point cloud data 222 generated by the scanning device 22. The data acquisition unit 110 of the image processing device 10 can communicate with the scanning device 22 via a network (not shown), for example, and acquire the point cloud data 222 generated by the scanning operation of the scanning device 22. Alternatively, if the scan results (point cloud data 222) generated by the scanning device 22 are temporarily stored in a database (not shown), the data acquisition unit 110 of the image processing device 10 may access the database (not shown) to acquire the point cloud data 222.

[0044] Then, the feature amount calculation unit 120 of the image processing device 10 recognizes a processing target region in the point cloud data 222 by using definition information associated with the acquired point cloud data 222 and the corresponding space.

[0045] In the example of FIG. 4, the definition information is stored in advance in a predetermined database 40. The definition information stored in the database 40 is created in advance based on map information of each space generated by, for example, Simultaneous Localization and Mapping (SLAM). The definition information includes information that is associated with information indicating a space corresponding to point cloud data and specifies the region in the space where an object is located, i.e., the position of a processing target region (position coordinates in the coordinate system of the point cloud data). FIG. 5 is a diagram showing an example of the definition information. In the example of this figure, the definition information includes information that is associated with information identifying each space (space identification information) and information that indicates the position of a processing target region in point cloud data generated by scanning the space (region position designation information). The feature amount calculation unit 120 of the image processing device 10 can identify and read the corresponding definition information based on the information identifying the space corresponding to the point cloud data 222. This "information identifying the space" may be linked to the point cloud data 222 in advance or may be manually input by the user of the image processing device 10.

[0046] In the example of Fig. 4, the feature amount calculation unit 120 of the image processing device 10 extracts point cloud data 224 corresponding to a portion of the panoramic view from point cloud data 222 corresponding to the panoramic view of the space to be measured. For example, the feature amount calculation unit 120 of the image processing device 10 identifies a predetermined range including the recognized processing target area and its surroundings, and extracts point cloud data included in the predetermined range from the point cloud data 222. This generates point cloud data 224 as shown in Fig. 4. Note that in the example of Fig. 4, the feature amount calculation unit 120 of the image processing device 10 recognizes area A surrounded by a white solid line in the point cloud data 224 as the processing target area.

[0047] The feature amount calculation unit 120 of the image processing device 10 then sets a plurality of partial regions based on the recognized processing target region. An example of the flow of setting a plurality of partial regions will be described with reference to Figs. 6 and 7.

[0048] FIG. 6 is a diagram illustrating the flow of setting multiple partial regions in point cloud data 222 along the depth direction of the space of the measurement target. The x-axis in the diagram indicates the width direction of the space of the measurement target, and the y-axis in the diagram indicates the depth direction of the space of the measurement target. The solid line in FIG. 6 indicates the boundary (first boundary) of the processing target region set at that time. The feature amount calculation unit 120 of the image processing device 10 slices the point cloud at a predetermined thickness along the width direction (x-axis in the diagram) as shown by the dotted lines in the diagram, using the boundary (first boundary) of the processing target region as a reference. This sets multiple regions with a predetermined thickness in the depth direction (y-axis in the diagram). In the example of FIG. 6, five partial regions (P -2 , P -1 , P0, P1, P2) are set. Here, the number of partial areas is set to be, for example, slightly more than twice the estimated error in the self-position estimation of the robot 20 as a reference.

[0049] FIG. 7 is a diagram illustrating the flow of setting multiple partial regions in point cloud data 222 along the width direction of the space to be measured. The x-axis in the diagram indicates the width direction of the space to be measured, and the y-axis in the diagram indicates the depth direction of the space to be measured. The solid line in FIG. 7 indicates the boundary (first boundary) of the processing target region that is set at that time. The feature amount calculation unit 120 of the image processing device 10 slices the point cloud at a predetermined thickness along the depth direction (y-axis in the diagram) as shown by the dotted lines in the diagram, using the boundary (first boundary) of the processing target region as a reference. This sets multiple regions with a predetermined thickness in the width direction (x-axis in the diagram). In the example of FIG. 7, five partial regions (P -2 , P -1 , P0, P1, P2) are set. Here, the number of partial areas is set to be, for example, slightly more than twice the estimated error in the self-position estimation of the robot 20 as a reference.

[0050] The feature amount calculation unit 120 of the image processing device 10 then calculates the feature amount for each of the plurality of partial regions as described above. Specifically, the feature amount calculation unit 120 of the image processing device 10 calculates the number of points included in each partial region and the density of the points as the feature amount. The flow of calculating the feature amount for each of the plurality of partial regions will be described with reference to Figs. 8 and 9.

[0051] FIG. 8 is a diagram showing a first example of the flow of calculating the feature quantities of each of a plurality of set partial regions. In the example of FIG. 8, in point cloud data corresponding to the space of the measurement target, the region A surrounded by a solid line is set as the processing target region. In the example of this figure, as explained using FIG. 6, a plurality of partial regions are set along the depth direction of the space of the measurement target. Specifically, as shown in the enlarged portion (center of the figure) of the region surrounded by a dashed line B, a plurality of partial regions (P -3 , P -2 , P -1 , P0, P1, P2, P3) are set. Here, the feature amount calculation unit 120 of the image processing device 10 uses the number of points included in each partial region as a parameter to calculate a predetermined function g(P n ) is used. The histogram in FIG. 8 is calculated by a predetermined function g(P n ) are shown.

[0052] FIG. 9 is a diagram showing a second example of the flow for calculating the feature quantities of each of the set plurality of partial regions. In the example of FIG. 9, in the point cloud data corresponding to the space of the measurement target, the region A surrounded by a solid line is set as the processing target region. In the example of this figure, as explained using FIG. 7, a plurality of partial regions are set along the width direction of the space of the measurement target. Specifically, as shown in the enlarged portion (center of the figure) of the region surrounded by a dashed line B, a plurality of partial regions (P -3 , P -2 , P -1, P0, P1, P2, P3) are set. Here, the feature amount calculation unit 120 of the image processing device 10 uses the number of points included in each partial region as a parameter to calculate a predetermined function g(P n ) is used. The histogram in FIG. 9 is calculated by a predetermined function g(P n ) are shown.

[0053] The area correction unit 130 of the image processing device 10 then appropriately corrects the processing target area based on the feature amounts calculated for each of the multiple partial areas. The flow of correcting the processing target area based on the feature amounts calculated for each of the multiple partial areas will be described using the following flowchart.

[0054] 10 and 11 are flowcharts illustrating the flow of processing for correcting (setting) the boundary of the processing target area along the depth direction of the space to be measured.

[0055] First, the area correction unit 130 selects a partial area that serves as a reference (element number n=0) using the boundary of the processing target area (step S202). For example, assume that multiple partial areas are set as shown in FIG. 8. In this case, the area correction unit 130 selects partial area P0, which uses the boundary of the processing target area that is set at that time (first boundary) as a reference partial area (first partial area). Then, starting from the partial area (first partial area) selected in the processing of step S202, a search process is executed to search for two adjacent partial areas whose difference in feature amount is equal to or greater than a reference value.

[0056] Here, two directions are determined starting from a reference partial region (first partial region). These directions are a first direction extending outward from the processing target region along the depth direction, and a second direction extending inward from the processing target region along the depth direction (opposite to the first direction). The region correction unit 130 performs the search process in either the first direction or the second direction. In the flowcharts illustrated in FIGS. 13 and 14, the search process in the first direction is prioritized. This is to prevent important features (features indicating the surface of the object) of the point cloud data generated by the scanning device 22 from being located outside the processing target region. For example, a discrepancy may occur between the position of the definition information and the position of the point cloud data due to factors such as an error in the robot's estimated self-position. In such a case, the important features may not be included in the processing target region recognized in the point cloud data based on the definition information. Therefore, it is preferable to prioritize the first direction extending outward from the processing target region in the depth direction of the space to be measured.

[0057] 10 and 11, the area correction unit 130 selects a partial area located one area outside (element number n+1) the partial area selected in the process of step S202 (step S204). For example, if the partial area P0 shown in Fig. 8 is selected, the area correction unit 130 selects the partial area P1 with element number n+1, i.e., element number "1".

[0058] That is, in the processes of steps S202 and S204, the area modification unit 130 selects two partial areas adjacent to each other in the depth direction. Furthermore, in the processes of steps S202 and S204, the area modification unit 130 acquires the feature amounts calculated for each of the two selected partial areas. In the initial stage, the area modification unit 130 calculates the feature amounts calculated for the partial area P0 selected as the "reference partial area" by using the predetermined function g(P n Similarly, the region correction unit 130 obtains the feature quantity g(P0) calculated using the predetermined function g(P n) to obtain the feature quantity g(P1). In the following description, the feature quantity calculated for the partial region corresponding to the element number n is referred to as "F n " can also be written as ".

[0059] Next, the region correction unit 130 calculates the difference between the feature amounts of the two selected partial regions (F n -F n+1 ) and calculate the difference between the calculated features (F n -F n+1 ) exceeds a predetermined reference value δ (step S206). In other words, the area correction unit 130 determines whether the feature amount in the next outer partial area has decreased by more than a reference value. If the feature amount in two adjacent partial areas has changed by more than a reference value, it means that the two partial areas contain important features (features that indicate the likelihood of the surface of an object). Specifically, if a decrease in the feature amount that is more than a reference value is observed in the "subpartial area located one area outside," it can be said that "features that suggest the presence of the surface of an object" appear as a point cloud in the other partial area. In step S206, the area correction unit 130 can also be said to be determining partial areas that contain such features.

[0060] The difference between the calculated features (F n -F n+1 If the difference (F) between the calculated feature amounts exceeds the predetermined reference value δ (step S206: YES), the area correction unit 130 ends the search process and executes the process of step S224 described later. n -F n+1 If n does not exceed the predetermined reference value δ (step S206: NO), the region correction unit 130 increments the element number n for the partial region (step S208). This operation shifts the partial region that is the target of the search process one region outward.

[0061] Furthermore, the area correction unit 130 determines whether the incremented element number n has reached the upper limit N based on the partial area set by the feature calculation unit 120 (step S210). In other words, the area correction unit 130 determines in step S210 whether the outermost partial area in the first direction (i.e., the partial area whose element number n is "N") has been selected as the comparison target.

[0062] If the element number n has not reached the upper limit N (step S210: YES), the processes of steps S204 and S206 are repeated for the above-mentioned "subregion one region further out" and "subregion one region further out." On the other hand, if the element number n has reached the upper limit N (step S210: NO), the region correction unit 130 executes a search process in the second direction (the direction toward the inside of the processing target region).

[0063] When starting the search process in the second direction (the direction toward the inside of the processing target area), the area correction unit 130 sets the element number n to "0" (step S212). This initializes the target partial area. In other words, the state returns to one in which partial area P0 is selected.

[0064] Then, the region correction unit 130 selects a partial region located one region inside (element number n-1) of the selected partial region (partial region P0) (step S214). At this time, the region correction unit 130 also acquires a feature amount corresponding to the selected partial region. For example, when the partial region P0 illustrated in FIG. 8 is selected, the region correction unit 130 selects the partial region P with element number n-1, i.e., the element number "-1". -1 In addition, the region correction unit 130 selects the partial region P -1 Regarding the above-mentioned predetermined function g(P n ) is calculated using the feature g(P -1 The process in step S214 is the same as the process in step S204.

[0065] Next, the region correction unit 130 calculates the difference between the feature amounts of the two selected partial regions (F n -Fn-1 ) and calculate the difference between the calculated features (F n -F n-1 ) exceeds a predetermined reference value δ (step S216). In other words, the region correction unit 130 determines whether the feature amount in the next partial region on the inside has decreased by more than a reference value. As with the process for moving outward, if the feature amount in two adjacent partial regions has changed by more than a reference value, it means that the two partial regions contain important features (features that indicate the surface of an object). Specifically, if a decrease in the feature amount by more than a reference value is observed in the "partial region located one region on the inside," it can be said that "features suggesting the presence of the surface of an object" appear as a point cloud in the other partial region. In step S216, the region correction unit 130 can also be said to be determining partial regions that contain such features.

[0066] The difference between the calculated features (F n -F n-1 If the difference (F) between the calculated feature amounts exceeds the predetermined reference value δ (step S216: YES), the area correction unit 130 ends the search process and executes the process of step S224 described later. n -F n-1 If n does not exceed the predetermined reference value δ (step S216: NO), the region correction unit 130 decrements the element number n for the partial region (step S218). This operation shifts the partial region that is the target of the search process one position inward.

[0067] Furthermore, the area correction unit 130 determines whether the decremented element number n has reached the upper limit −N based on the partial area set by the feature calculation unit 120 (step S220). In other words, the area correction unit 130 determines in step S220 whether the outermost partial area in the second direction (i.e., the partial area whose element number n is “−N”) has been selected as the comparison target.

[0068] If the element number n has not reached the upper limit -N (step S220: NO), the processes of steps S214 and S216 are repeated for the above-mentioned "subregion one region inward" and "subregion one region further inward." On the other hand, if the element number n has reached the upper limit -N (step S220: YES), the region correction unit 130 sets the element number n to "0," i.e., the initial value (S222). This ends the search process.

[0069] When the search process is completed, the area correction unit 130 sets a boundary based on the partial area corresponding to the element number n that was set at the time of completion of the search process (step S224). As an example, if the element number n that was set at the time of completion of the search process is "2," the area correction unit 130 sets a new boundary (second boundary) of the processing target area based on the partial area P2 that corresponds to this element number. Here, the element number n is "2" when the feature amount of the partial area P2 is greater than the feature amount of the partial area P3 by a reference value or more. In other words, when two partial areas are found whose difference in feature amount is greater than the reference value, the area correction unit 130 sets a new boundary (second boundary) of the processing target area based on the partial area (second partial area) that has the greater feature amount of the two partial areas.

[0070] As shown in FIG. 12, for example, the area correction unit 130 moves the boundary (first boundary) of the processing target area along one of the multiple sides constituting the partial area P2 (second partial area) that extends in the direction intersecting the depth direction (width direction) of the space to be measured. FIG. 12 is a diagram illustrating the operation of setting the second boundary by the area correction unit. In the example of FIG. 12, the area correction unit 130 selects side S2 of the partial area P2 that extends in the width direction (x-axis direction in the figure) and that includes the partial area P2 in the processing target area, and moves the boundary (first boundary) of the processing target area to the position of side S2. By this operation, a new boundary (second boundary) of the processing target area is set.

[0071] 13 and 14 are flowcharts illustrating the flow of processing for correcting (setting) the boundary of the processing target area along the width direction of the space to be measured.

[0072] First, the area correction unit 130 selects a partial area to be a reference (element number n=0) using the boundary of the processing target area (step S302). For example, assume that multiple partial areas are set as shown in FIG. 9. In this case, the area correction unit 130 selects partial area P0, which uses the boundary of the processing target area set at that time (first boundary) as a reference partial area (first partial area). Then, starting from the partial area (first partial area) selected in the processing of step S302, a search process is executed to search for two adjacent partial areas whose difference in feature amount is equal to or greater than a reference value.

[0073] Here, two directions are determined starting from a reference partial region (first partial region). That is, a third direction extending inward along the width direction toward the inside of the processing target region, and a fourth direction extending inward along the width direction toward the outside of the processing target region (opposite to the third direction). The region correction unit 130 executes search processing in the third and fourth directions, as described below. In the flowcharts illustrated in FIGS. 13 and 14, the search processing in the third direction is executed first, followed by the search processing in the fourth direction. However, the order in which the search processing in the width direction is executed is not limited to this. The region correction unit 130 may execute the search processing in the fourth direction first, or may execute the search processing in the third direction and the search processing in the fourth direction in parallel.

[0074] 13 and 14, the region correction unit 130 selects a partial region located one region inside (element number n-1) of the partial region selected in the process of step S302 (step S304). For example, when the partial region P0 illustrated in FIG. 9 is selected, the region correction unit 130 selects the partial region P0 having element number n-1, i.e., the element number "-1". -1 Select .

[0075] That is, in the processes of steps S302 and S304, the area correction unit 130 selects two partial areas adjacent to each other in the width direction. Furthermore, in the processes of steps S302 and S304, the area correction unit 130 acquires the feature amounts calculated for each of the two selected partial areas. In the initial stage, the area correction unit 130 calculates the feature amounts calculated for the partial area P0 selected as the "reference partial area" by using the predetermined function g(P n Similarly, the region correction unit 130 obtains the feature quantity g(P0) calculated using the predetermined function g(P n ) to obtain the feature quantity g(P1). In the following description, the feature quantity calculated for the partial region corresponding to the element number n is referred to as "F n " can also be written as ".

[0076] Next, the region correction unit 130 calculates the absolute value of the difference between the feature amounts of the two selected partial regions (|F n -F n-1 |) and calculate the absolute value of the difference between the calculated features (|F n -F n-1 It is then determined whether or not the difference (|) exceeds a predetermined reference value δ (step S306). The reason for using the "absolute value of the difference in feature amounts" in the widthwise search process is that different types of objects may be arranged in the widthwise direction. For example, at a location where the type of object changes in the widthwise direction, a certain tendency for the feature amounts of the point cloud data (the number of points or the density of points in the point cloud data) to increase or decrease appears. In step S306, the area correction unit 130 can be said to determine a partial area that includes such features.

[0077] The absolute value of the difference between the calculated features (|F n -F n-1 If |) exceeds the predetermined reference value δ (step S306: YES), the region correction unit 130 ends the search process on the inside (third direction) and saves the result (step S312). As an example, the region correction unit 130 calculates the increase / decrease tendency ΔF of the feature amount on the inside (third direction). in As "F n -Fn-1 Alternatively, the area correction unit 130 sets the increase / decrease tendency ΔF of the feature amount on the inside (third direction). in As "F n -F n-1 The area correction unit 130 may store the sign (+, -) based on the calculation of "." Also, the area correction unit 130 may store the element number n in The element number n at that time is stored as n = 1. Thereafter, the area correction unit 130 executes a search process on the outside (fourth direction), as will be described later.

[0078] On the other hand, the absolute value of the difference between the calculated features (|F n -F n-1 If |) does not exceed the predetermined reference value δ (step S306: NO), the area correction unit 130 decrements the element number n for the partial area (step S308). This operation shifts the partial area that is the target of the search process one area inward.

[0079] Furthermore, the area correction unit 130 determines whether the decremented element number n has reached the upper limit −N based on the partial area set by the feature calculation unit 120 (step S310). In other words, the area correction unit 130 determines in step S310 whether the outermost partial area in the third direction (i.e., the partial area whose element number n is “−N”) has been selected as the comparison target.

[0080] If the element number n has not reached the upper limit -N (step S310: NO), the processes of steps S304 and S306 are repeated for the above-mentioned "subregion one region inward" and "subregion one region further inward." On the other hand, if the element number n has reached the upper limit -N (step S310: YES), the region correction unit 130 ends the search process on the inside (third direction) and saves the results (step S314). This is the case where two adjacent subregions whose absolute value of the difference in feature amount exceeds the reference value have not been found. In this case, the region correction unit 130 calculates the increase / decrease tendency ΔF of the feature amount on the inside (third direction). inFor example, "0" indicating that there is no sign is set as the element number n in An initial value (0) is set as the area correction unit 130. Thereafter, the area correction unit 130 executes a search process on the outside (fourth direction), as will be described later.

[0081] To start the search process in the outer direction (fourth direction), the area correction unit 130 sets the element number n to "0" (step S316). This initializes the target partial area. In other words, the state returns to one in which the partial area P0 is selected.

[0082] Then, the area correction unit 130 selects a partial area located one area (element number n+1) outside the selected partial area (partial area P0) (step S318). At this time, the area correction unit 130 also acquires a feature amount corresponding to the selected partial area. For example, when the partial area P0 illustrated in FIG. 9 is selected, the area correction unit 130 selects the partial area P1 with element number n+1, i.e., element number "1". Furthermore, the area correction unit 130 calculates the above-mentioned predetermined function g(P n ) to obtain the feature quantity g(P1). The process of step S318 is the same as the process of step S304.

[0083] Next, the region correction unit 130 calculates the absolute value of the difference between the feature amounts of the two selected partial regions (|F n -F n+1 |) and calculate the absolute value of the difference between the calculated features (|F n -F n+1 It is determined whether or not | exceeds a predetermined reference value δ (step S320).

[0084] The absolute value of the difference between the calculated features (|F n -F n+1If |) exceeds the predetermined reference value δ (step S320: YES), the region correction unit 130 ends the search process in the outer side (fourth direction) and saves the result (step S326). As an example, the region correction unit 130 calculates the increase / decrease tendency ΔF of the feature amount in the outer side (fourth direction). out As "F n -F n+1 Alternatively, the area correction unit 130 sets the increase / decrease tendency ΔF of the feature amount on the outside (fourth direction). out As "F n -F n+1 The area correction unit 130 may store the sign information (+, -) based on the calculation of "." Also, the area correction unit 130 may store the element number n out The element number n at that time is saved as n. Then, a process is executed to set the boundary of the processing area based on a combination of the results of the search process on the inside (third direction) and the outside (fourth direction).

[0085] On the other hand, the absolute value of the difference between the calculated features (|F n -F n+1 If |) does not exceed the predetermined reference value δ (step S320: NO), the area correction unit 130 increments the element number n for the partial area (step S322). This operation shifts the partial area that is the target of the search process one area outward.

[0086] Furthermore, the area correction unit 130 determines whether the incremented element number n has reached the upper limit N based on the partial area set by the feature calculation unit 120 (step S324). In other words, the area correction unit 130 determines in step S324 whether the outermost partial area on the inside (third direction) (i.e., the partial area whose element number n is "N") has been selected as the comparison target.

[0087] If the element number n has not reached the upper limit N (step S324: YES), the processes of steps S318 and S320 are repeated for the "subregion one region further out" and the "subregion one region further out." On the other hand, if the element number n has reached the upper limit N (step S324: NO), the region correction unit 130 ends the search process on the outside (fourth direction) and saves the results (step S328). This is the case where two adjacent subregions whose absolute value of the difference in feature amount exceeds the reference value have not been found. In this case, the region correction unit 130 calculates the increase / decrease tendency ΔF of the feature amount on the outside (fourth direction). out For example, "0" indicating that there is no sign is set as the element number n out An initial value (0) is set as . Then, a process is executed to set the boundary of the processing target area based on a combination of the results of the search process on the inside (third direction) and the outside (fourth direction).

[0088] When the search process in the third and fourth directions is completed, the area correction unit 130 sets the boundary of the processing target area based on a combination of the results of the search process inside (third direction) and the results of the search process outside (fourth direction) (step S330).

[0089] As an example, the area correction unit 130 uses the table shown in Fig. 15 to determine whether to set a new boundary (second boundary) for the processing target area, and how to move the current boundary (first boundary) in the width direction if a new boundary (second boundary) is to be set. Fig. 15 is a diagram showing an example of information for determining how to set the boundary of the processing target area in the width direction. A specific example of the operation of moving the boundary of the processing target area in the width direction using the decision table exemplified in Fig. 15 will be described with reference to another diagram (Fig. 16).

[0090] Fig. 16 is a diagram showing a specific example of an operation for moving the boundary of the processing target area in the width direction. In the specific example of Fig. 16, the following results are obtained as the results of the search process in the inner direction (third direction) and the outer direction (fourth direction), respectively.

[0091] - Results of the search process in the inner (third direction): No change above the reference value (ΔF in is "0") Outer (fourth direction) search result: n out = 1, there is a change greater than the reference value (ΔF out teeth"-")

[0092] In this case, the area correction unit 130 calculates the increase / decrease tendency of the feature amount ("ΔF in " is "0") and the increase / decrease tendency of the feature quantity identified in the search results of the outer (fourth direction) ("ΔF out 15 is "-"). Specifically, the area correction unit 130 specifies the content surrounded by the dashed line in FIG. 15 as the content corresponding to the combination. Then, based on the specified content, the area correction unit 130 determines that "a new boundary (second boundary) needs to be set for the processing target area." Furthermore, based on the same content, the area correction unit 130 determines that "element number n out It is determined that the boundary of the processing target area needs to be moved outward (in the fourth direction) so as to include the partial area corresponding to the partial area (i.e., partial area P1). As a result, a new boundary (second boundary) of the processing target area is set as shown in the figure.

[0093] As described above, the image processing device disclosed herein identifies a partial area from a point cloud included in multiple partial areas set near the processing target area, where features indicating the surface resemblance of the measurement object appear, and appropriately modifies the boundary of the processing target area based on the partial area. This configuration allows the processing target area to include features important for object identification processing using point cloud data. As a result, it is expected to have the effect of improving the accuracy of object identification processing performed in a subsequent process.

[0094] <Modification> 17 is a diagram showing a second example of the functional configuration of an image processing device according to the present disclosure. In the image processing device 10 shown in this diagram, the data acquisition unit 110 further includes a coordinate system setting unit 112.

[0095] The coordinate system setting unit 112, for example, aligns the coordinate system of the acquired point cloud data with the coordinate system set in the definition information. As an example, the coordinate system setting unit 112 converts the coordinate system of the acquired point cloud data to match the coordinate system of the definition information associated with the space corresponding to the acquired point cloud data.

[0096] For example, the coordinate system setting unit 112 acquires, as additional information related to the point cloud data, information related to the self-position of the robot 20 at the time the point cloud data was acquired. The information related to the self-position includes, for example, information indicating the self-position (estimated position) of the robot 20 when the point cloud data was acquired and the posture (orientation of the robot 20 or the scanning device 22) when the point cloud data was acquired. The coordinate system setting unit 112 calculates a coordinate system for the acquired point cloud data based on the information related to the self-position. Then, the coordinate system setting unit 112 calculates a matrix that converts the coordinate system of the point cloud data into the coordinate system of the definition information. As an example, the coordinate system setting unit 112 can compare a reference part (for example, a support part at the end of a shelf) between two coordinate systems (the coordinate system of the point cloud data and the coordinate system of the definition information) and calculate a matrix that converts coordinate values ​​in one coordinate system into coordinate values ​​in the other coordinate system.

[0097] In the configuration of this modification, the point cloud data is converted so that the coordinate system between the point cloud data and the definition information is the same, which is expected to have the effect of suppressing positional deviation when setting the processing target area using the definition information.

[0098] Second embodiment <Example of functional configuration> 18 is a diagram illustrating a third example of the functional configuration of an image processing device according to the present disclosure. The image processing device 10 illustrated in this diagram includes a data acquisition unit 110, a feature amount calculation unit 120, and a region correction unit 130, as well as an object region identification unit 140.

[0099] The object region identification unit 140 processes the processing target region to identify the region in which the object exists. The processing target region processed by the object region identification unit 140 includes the processing target region after being corrected by the region correction unit 130 as described in the first embodiment.

[0100] <Example of operation> FIG. 19 is a flowchart illustrating the flow of processing executed by the object region identifying section 140 in the present disclosure.

[0101] The object region identifying unit 140 acquires point cloud data in which a processing target region is set (step S402). As an example, the object region identifying unit 140 acquires point cloud data 224 cut out as illustrated in Fig. 4 and information indicating the processing target region in the point cloud data 224. Here, the processing target region set in the point cloud data 224 may have been corrected as described using Figs. 10 to 16.

[0102] The object region identification unit 140 processes the processing target region set in the acquired point cloud data to generate information about objects located in the space of the measurement target (step S404). For example, the object region identification unit 140 identifies the surface position of the object (the position where the object exists) based on the point cloud of the processing target region. Then, the object region identification unit 140 calculates the filling rate of the object (e.g., the total volume of the object) based on the surface position of the identified object. Specifically, the object region identification unit 140 calculates how much of the area allocated to the object is filled with the object, depending on the surface position of the identified object. Alternatively, the object region identification unit 140 may be configured to calculate the number of objects based on the filling rate and the size of each object.

[0103] Then, the object region identification unit 140 performs a predetermined output based on the information generated in the processing of step S404 (step S406). As an example, the object region identification unit 140 determines whether the filling rate or the number of displayed objects calculated in the processing of step S404 is equal to or less than a predetermined number, and outputs notification information (e.g., FIG. 20) that prompts the user to replenish the objects according to the determination result.

[0104] FIG. 20 is a diagram showing an example of information output by the object region identification unit 140. In the example of FIG. 20, the object region identification unit 140 identifies out-of-stock status of each product on a retail store's product shelf and outputs notification information regarding the out-of-stock status as appropriate. This configuration for notifying information is expected to facilitate management operations in places where goods are managed, such as product sales floors in retail stores and inventories in transportation warehouses. Furthermore, the point cloud data used by the object region identification unit 140 is appropriately corrected by the region correction unit 130, as exemplified in the first embodiment. This makes it possible to improve the accuracy of the information exemplified in FIG. 20.

[0105] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0106] In addition, although the flowcharts used in the above description show multiple steps (processes) in a sequential order, the order of the steps executed in each embodiment is not limited to the order shown. In each embodiment, the order of the steps shown in the drawings can be changed as long as it does not cause any problems in terms of the content.

[0107] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes. 1. a data acquisition means for acquiring point cloud data generated by scanning a space to be measured; a feature amount calculation means for setting a plurality of partial areas based on a first boundary of a processing target area in the point cloud data and calculating a feature amount of each of the plurality of partial areas; an area correcting means for correcting the processing target area by setting a new second boundary based on the position of a partial area that satisfies a criterion related to the feature amount; An image processing device comprising: 2. further comprising an object identification means for identifying an area in which an object exists by processing the processing target area; 1. An image processing device according to claim 1. 3. The feature indicates the likelihood of an object being a surface. 1. An image processing device according to 1. or 2. 4. the feature amount includes at least one of the number of points included in each partial region and the density of the points; 3. An image processing device according to claim 3. 5. the feature amount calculation means sets the plurality of partial regions along a depth direction of the space, The area correction means selecting a first subregion based on the first boundary; a process of searching for two adjacent partial regions in which the difference in the feature amount is equal to or greater than a reference value is performed in a first direction along the depth direction or a second direction opposite to the first direction, starting from the first partial region; If two partial regions are found in the process in which the difference is equal to or greater than the reference value, the second boundary is set based on the second partial region of the two partial regions having the larger feature amount. 4. An image processing device according to claim 4. 6. The area correction means the second boundary is set by moving the first boundary along any one of a plurality of sides constituting the second partial region, the side extending in a direction intersecting with the depth direction of the space. 5. An image processing device according to claim 5. 7. the feature amount calculation means sets the plurality of partial regions along a width direction of the space, The area correction means selecting a first subregion based on the first boundary; a process of searching for two adjacent partial regions in which the absolute value of the difference between the feature amounts is equal to or greater than a reference value is executed in a third direction along the width direction and a fourth direction opposite to the third direction, starting from the first partial region; setting the second boundary by moving the first boundary based on the search result in the third direction and the search result in the fourth direction; 4. An image processing device according to claim 4. 8. The area correction means determining whether to set the second boundary and, if the second boundary is to be set, in which direction, the third direction or the fourth direction, the first boundary should be moved, based on a combination of the increase / decrease tendency of the feature amount in the third direction and the increase / decrease tendency of the feature amount in the fourth direction; 7. An image processing device according to claim 7. 9. The computer Acquire point cloud data generated by scanning the space to be measured, setting a plurality of partial regions based on a first boundary of a processing target region in the point cloud data, and calculating feature amounts of each of the plurality of partial regions; modifying the processing target region by setting a new second boundary based on the position of a partial region that satisfies a criterion related to the feature amount; An image processing method comprising: 10. Computer, a data acquisition means for acquiring point cloud data generated by scanning a space to be measured; a feature amount calculation means for setting a plurality of partial areas based on a first boundary of a processing target area in the point cloud data and calculating a feature amount of each of the plurality of partial areas; an area correcting means for correcting the processing target area by setting a new second boundary based on the position of a partial area that satisfies a criterion related to the feature amount; A program to function as a

[0108] Furthermore, some or all of the configurations described in Supplementary Notes 2 to 8 that are subordinate to the above-mentioned Supplementary Note (image processing device) may also be subordinate to Supplementary Note 9 (image processing method) and Supplementary Note 10 (program) in the same subordinate relationship as Supplementary Note 2 to Supplementary Note 8. Furthermore, not limited to Supplementary Note 1, Supplementary Note 9, and Supplementary Note 10, some or all of the configurations described as Supplements may be subordinate to various hardware, software, various recording means for recording software, or systems, within the scope of each of the above-mentioned embodiments. [Explanation of symbols]

[0109] 10 Image processing device 1000 computers 1010 Bus 1020 processor 1030 memory 1040 Storage Device 1050 Input / Output Interface 1060 Network Interface 110 Data Acquisition Unit 112 Coordinate system setting section 120 Feature calculation unit 130 Area correction part 140 Object area identification part 20. Robot 22 Scanning Device 222 point cloud data 224 point cloud data 30 space 32 Object 40 databases

Claims

1. a data acquisition means for acquiring point cloud data generated by scanning a space to be measured; a feature amount calculation means for setting a plurality of partial areas based on a first boundary of a processing target area in the point cloud data and calculating a feature amount of each of the plurality of partial areas; an area correcting means for correcting the processing target area by setting a new second boundary based on the position of a partial area that satisfies a criterion related to the feature amount; An image processing device comprising:

2. further comprising an object identification means for identifying an area in which an object exists by processing the processing target area; The image processing device according to claim 1 .

3. The feature indicates the likelihood of an object being a surface.

3. The image processing device according to claim 1.

4. the feature amount includes at least one of the number of points included in each partial region and the density of the points; The image processing device according to claim 3 .

5. the feature amount calculation means sets the plurality of partial regions along a depth direction of the space, The area correction means selecting a first subregion relative to the first boundary; a process of searching for two adjacent partial regions in which a difference between the feature amounts is equal to or greater than a reference value, the process being performed in a first direction along the depth direction from the first partial region as a starting point or in a second direction opposite to the first direction; If two partial regions are found in the process in which the difference is equal to or greater than the reference value, the second boundary is set based on the second partial region of the two partial regions having the larger feature amount. The image processing device according to claim 4 .

6. The area correction means the second boundary is set by moving the first boundary along any one of a plurality of sides constituting the second partial region, the side extending in a direction intersecting with the depth direction of the space. The image processing device according to claim 5 .

7. the feature amount calculation means sets the plurality of partial regions along a width direction of the space, The area correction means selecting a first subregion relative to the first boundary; a process of searching for two adjacent partial regions in which the absolute value of the difference between the feature amounts is equal to or greater than a reference value is executed in a third direction along the width direction and a fourth direction opposite to the third direction, starting from the first partial region; setting the second boundary by moving the first boundary based on the search result in the third direction and the search result in the fourth direction; The image processing device according to claim 4 .

8. The area correction means determining whether to set the second boundary and, if the second boundary is to be set, in which direction, the third direction or the fourth direction, the first boundary should be moved, based on a combination of the increase / decrease tendency of the feature amount in the third direction and the increase / decrease tendency of the feature amount in the fourth direction; The image processing device according to claim 7 .

9. The computer Acquire point cloud data generated by scanning the space to be measured, setting a plurality of partial regions based on a first boundary of a processing target region in the point cloud data, and calculating feature amounts of each of the plurality of partial regions; modifying the processing target region by setting a new second boundary based on the position of a partial region that satisfies a criterion related to the feature amount; An image processing method comprising:

10. Computer, a data acquisition means for acquiring point cloud data generated by scanning a space to be measured; a feature amount calculation means for setting a plurality of partial areas based on a first boundary of a processing target area in the point cloud data and calculating a feature amount of each of the plurality of partial areas; an area correcting means for correcting the processing target area by setting a new second boundary based on the position of a partial area that satisfies a criterion related to the feature amount; A program to function as a

Images