Object recognition device, object recognition system, and object recognition method

Abstract

To accurately recognize the internal structure of an object.SOLUTION: The object recognition apparatus includes a recognition unit arranged upstream of a path on which a target object is conveyed, and a processing unit arranged downstream of the recognition unit on the path. The processing unit includes a second three dimensional information acquisition unit that acquires three dimensional information indicating an outer shape of the target object at a point on a downstream side of the path from a point at which the three dimensional information is acquired by the recognition unit, and a processing control unit that performs coordinate transformation of the feature point based on a transformation matrix obtained as a result of performing matching between the three dimensional information acquired by the first three dimensional information acquisition unit and the three dimensional information acquired by the second three dimensional information acquisition unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an object recognition device, an object recognition system, and an object recognition method. [Background technology]

[0002] Conventionally, there has been a technology in meat processing systems that uses feature values ​​extracted from fluoroscopic images taken using an X-ray imaging device or the like to understand the internal structure of meat and perform processes such as cutting. According to this conventional technology, a device for taking fluoroscopic images, such as an X-ray imaging device, is not installed in the processing section where processing is performed, but is instead concentrated in an upstream recognition section, thereby building a safe and inexpensive system. In this type of system, since the posture of the meat to be processed changes during transportation, the recognition section and the processing section grasp the correlation between the posture of the meat to be processed in order to use the feature values ​​obtained from the fluoroscopic images obtained by the recognition section in the processing section (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Republished WO2020 / 246288 Summary of the Invention [Problem to be solved by the invention]

[0004] According to a conventional method such as that described in Patent Document 1, the recognition unit and processing unit each perform three-dimensional external measurement, detect the same three feature points on the workpiece, and perform coordinate conversion. Of the three detected feature points, one is a reference point used for translational movement, and the other two are points used for rotational movement. This configuration makes it possible to calculate the amount of movement of the recognition unit's coordinate system so that it matches the processing unit's coordinate system, and to convert the feature values ​​extracted by the recognition unit into the processing unit's coordinate system.

[0005] When the recognition unit and processing unit detect the same feature points, errors can occur due to errors in 3D external measurement or mistakes in image processing. In particular, when the positions of three feature points are close to each other, even a slight error can become a large error when converted into the amount of movement in a coordinate system. Another issue is that the farther the feature point is from the reference point, the larger the error when converted into the amount of movement in a coordinate system. The larger the error when converted into the amount of movement in a coordinate system, the more likely it is to have a negative impact on the success rate and yield during processing.

[0006] Furthermore, the three feature points to be detected must be able to be extracted from the external shape information and must not deform while the processing object is being transported. Therefore, when processing meat or other foods, the feature points to be detected are defined on exposed bones or in positions based on the bones. However, if there are no exposed bones on the processing object, there is an issue in that the three feature points cannot be defined.

[0007] In view of the above circumstances, an object of the present invention is to provide an object recognition device, an object recognition system, and an object recognition method that are capable of accurately recognizing the internal structure of an object. [Means for solving the problem]

[0008] [1] One aspect of the present invention is an object recognition device comprising: a recognition unit arranged upstream of a route along which a target object is transported; and a processing unit arranged downstream of the recognition unit along the route. The recognition unit comprises a perspective image acquisition unit that acquires a perspective image of the internal structure of the target object; a first three-dimensional information acquisition unit that acquires three-dimensional information indicating the outline of the target object; and a recognition control unit that extracts feature points from the perspective image. The processing unit comprises a second three-dimensional information acquisition unit that acquires three-dimensional information indicating the outline of the target object at a point downstream of the route from the point where the three-dimensional information was acquired by the recognition unit; and a processing control unit that performs coordinate transformation of the feature points based on a transformation matrix obtained as a result of matching the three-dimensional information acquired by the first three-dimensional information acquisition unit with the three-dimensional information acquired by the second three-dimensional information acquisition unit.

[0009] According to the above-described aspect, the upstream recognition unit and the downstream processing unit each acquire three-dimensional information, and perform coordinate transformation of feature points based on the transformation matrix obtained as a result of matching the three-dimensional information. Therefore, according to this aspect, three-point detection is not performed as in conventional methods. Since changes in the posture of the target object are grasped by matching the entire three-dimensional information without detecting three feature points, the problem of larger errors occurring for features farther from the reference point can be solved. Therefore, according to this aspect, it is possible to accurately recognize the internal structure of an object.

[0010] Furthermore, according to this aspect, since the detection of three feature points based on bone parts is not performed as in the conventional technology, it is possible to apply it to target objects where bones are not exposed. Furthermore, according to this aspect, it is not necessary to develop an algorithm for detecting three feature points, and it is possible to realize a system with a small number of steps. Note that the conventional method may be used as the algorithm for matching the three-dimensional information itself.

[0011] [2] In one aspect of the present invention, in the object recognition device of [1] described above, the three-dimensional information acquired by the first three-dimensional information acquisition unit and the three-dimensional information acquired by the second three-dimensional information acquisition unit are both point cloud data, and the processing control unit performs matching based on the distance and relative direction of the point clouds.

[0012] According to the above-described aspect, point cloud data is used as three-dimensional information, and matching is performed based on the distance and relative direction of the point cloud. That is, according to this aspect, point cloud data representing the entire target object is used, and matching is performed based on the distance and relative direction of the point cloud. Therefore, according to this aspect, the problem of errors increasing as the feature is farther from the reference point does not occur, and it is possible to accurately recognize the internal structure of the object.

[0013] [3] In one aspect of the present invention, in the object recognition device described in [2] above, the processing control unit performs matching based on the distance and relative direction of a point cloud obtained by thinning out information from the acquired point cloud data.

[0014] That is, according to this aspect, the obtained three-dimensional information is not used as is, but the amount of information is first reduced so that it is suitable for the matching process, and then the processing is performed. Therefore, according to this aspect, the efficiency of the matching process can be improved (the amount of processing can be reduced).

[0015] [4] In one aspect of the present invention, in the object recognition device of [3] described above, the processing control unit performs a first matching based on the distance and relative direction of point cloud data, which is a point cloud obtained by thinning out information from the acquired point cloud data and has a first amount of information, and then performs a second matching based on the distance and relative direction of point cloud data, which is a point cloud obtained by thinning out information from the acquired point cloud data and has a second amount of information that is greater than the first amount of information.

[0016] That is, according to this aspect, the obtained three-dimensional information is not used as is, but the amount of information is first reduced so as to be suitable for the matching process, and then a first matching process is performed. The first matching can also be expressed as sparse fine adjustment. Furthermore, according to this aspect, after the sparse fine adjustment, a second matching process is performed using point cloud data with an amount of information greater than the amount of information used in the first matching process. The second matching can also be expressed as dense fine adjustment. In dense fine adjustment, matching can be performed at a more detailed resolution. Therefore, according to this aspect, it is possible to accurately recognize the internal structure of an object while increasing the efficiency of the matching process (while reducing the amount of processing).

[0017] [5] In one aspect of the present invention, in the object recognition device described in any one of [1] to [4] above, in the recognition unit, a perspective image capturing device that captures the perspective image and a first three-dimensional information capturing device that captures three-dimensional information showing the outline of the target object are arranged at different positions on the path, and the recognition control unit aligns the coordinates of the perspective image and the three-dimensional information according to the distance the target object has been transported along the path.

[0018] According to the above-described aspect, if the recognition unit can capture the perspective image and the three-dimensional information at the same position on the path, it is not necessary to align the coordinates of the perspective image and the three-dimensional information. However, this is difficult in reality. Therefore, according to this aspect, the recognition unit can simply regard the perspective image and the three-dimensional information as being in the same coordinate system, even though they are actually located at different positions. By adopting such a configuration, the perspective image and the three-dimensional information can be obtained in corresponding coordinate systems through simple processing.

[0019] [6] In one aspect of the present invention, in the object recognition device according to any one of [1] to [5] above, the perspective image is an X-ray image.

[0020] According to the above-described aspect, the plurality of processing units do not require an X-ray camera for capturing X-ray images, which reduces the overall system cost and the installation area of ​​the entire system.

[0021] [7] In one aspect of the present invention, in the object recognition device described in any of [1] to [6] above, the target object is meat, and the path is a belt conveyor that transports the meat.

[0022] According to the above-described aspect, in a system in which meat is transported on a belt conveyor and processed such as cutting, the processing unit can recognize the internal structure of the meat, such as bones, without taking a fluoroscopic image, and can perform cutting processes and the like in an appropriate manner.

[0023] [8] In one aspect of the present invention, in the object recognition device described in [7] above, the feature points are cutting points that serve as landmarks when cutting the meat.

[0024] According to the above-described aspect, in a system in which meat is transported on a belt conveyor and subjected to processes such as cutting, the meat can be suitably cut based on the obtained cutting points.

[0025] [9] One aspect of the present invention is an object recognition system including an object recognition device described in any one of [1] to [8] above, comprising one recognition unit and a plurality of processing units, wherein the path branches to each of the plurality of processing units located downstream when the recognition unit is considered to be upstream, and the target object is transported from the recognition unit to one of the processing units.

[0026] According to the above-described aspect, by providing one recognition unit and multiple processing units, it is sufficient that only the one recognition unit has a configuration for capturing a fluoroscopic image. In other words, the multiple processing units can perform processing using the fluoroscopic image captured by the one recognition unit. Therefore, according to this aspect, it is possible to reduce the cost of the entire system and the installation area of ​​the entire system.

[0027]

[10] One aspect of the present invention is the object recognition system described in [9] above, wherein the plurality of processing units perform cutting of the target object based on the feature points.

[0028] According to the above-described aspect, a system is provided with one recognition unit and multiple processing units, and cutting is performed by the multiple processing units. In other words, this aspect is a system that includes multiple processing units that perform cutting processing. By providing one recognition unit, the internal structure of the target object is recognized upstream and the internal structure is reused downstream. By adopting such a configuration, this aspect can reduce the cost of the entire system and the installation area of ​​the entire system.

[0029]

[11] One aspect of the present invention is an object recognition method comprising: a recognition step performed upstream of a route along which a target object is transported; and a processing step performed downstream of the recognition step along the route. The recognition step comprises a perspective image acquisition step of acquiring a perspective image of the internal structure of the target object, a first three-dimensional information acquisition step of acquiring three-dimensional information indicating the outline of the target object, and a recognition control step of extracting feature points from the perspective image. The processing step comprises a second three-dimensional information acquisition step of acquiring three-dimensional information indicating the outline of the target object at a point downstream of the route from the point where the three-dimensional information was acquired in the recognition step, and a processing control step of performing coordinate transformation of the feature points based on a transformation matrix obtained as a result of matching the three-dimensional information acquired by the first three-dimensional information acquisition step with the three-dimensional information acquired by the second three-dimensional information acquisition step. [Effects of the Invention]

[0030] According to the present invention, it is possible to provide an object recognition device, an object recognition system, and an object recognition method that are capable of accurately recognizing the internal structure of an object. [Brief explanation of the drawings]

[0031] [Figure 1] FIG. 1 is a diagram illustrating an example of an overview of an object processing system according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of an overview of a recognition unit according to the present embodiment. [Figure 3]FIG. 2 is a functional configuration diagram illustrating an example of the functional configuration of a recognition unit according to the present embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of an overview of a processing unit according to the present embodiment. [Figure 5] FIG. 2 is a functional configuration diagram showing an example of the functional configuration of a processing unit according to the present embodiment. [Figure 6] FIG. 10 is a diagram for explaining an example of alignment according to the present embodiment. [Figure 7] 1 is a flowchart showing an example of a series of processing steps performed by an object processing system according to the present embodiment. [Figure 8] FIG. 10 is a diagram for explaining a modified example of the outline of the object processing system according to the present embodiment. [Figure 9] FIG. 2 is a block diagram showing the internal configuration of each device included in the object processing system according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0032] [Embodiment] Preferred embodiments of an object recognition device, an object recognition system, and an object recognition method according to aspects of the present invention will be described in detail below with reference to the accompanying drawings. Note that the aspects of the present invention are not limited to these embodiments and include various modifications or improvements. In other words, the components described below include those that would be easily conceivable to a person skilled in the art or that are substantially identical, and the components described below can be combined as appropriate. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the present invention. Furthermore, in the drawings below, the scale and number of components may differ from the scale and number of the actual structures to make each configuration easier to understand.

[0033] [Object Processing System 1] 1 is a diagram illustrating an example of an outline of an object processing system according to an embodiment. First, an outline of the object processing system 1 will be described with reference to the diagram. The object processing system 1 includes one recognition unit 10 and at least one or more processing units 20. In the illustrated example, one processing unit 20 is provided for one recognition unit 10.

[0034] The object processing system 1 processes a target object OBJ. Specifically, the object processing system 1 performs object recognition of the target object OBJ and then processes it. Within the object processing system 1, a device configured to perform object recognition may be referred to as an object recognition device.

[0035] The recognition unit 10 and the processing unit 20 are connected by a belt conveyor CB. The belt conveyor CB may also be referred to as a path along which a target object OBJ is transported. The transport direction of the belt conveyor CB is indicated by an arrow AR. For example, a target object OBJ is placed on the belt conveyor CB and transported along the arrow AR. The transport of the belt conveyor CB may be for transporting a placed target object OBJ or for transporting a suspended target object OBJ.

[0036] Here, the target object OBJ is an object that is the target of object recognition in this embodiment. Specifically, the target object OBJ may be meat that is to be processed. The target object OBJ preferably has an internal structure that can be grasped using a perspective image and cannot be grasped from the outside. The internal structure that can be grasped using a perspective image may be, for example, bones present inside the meat that can be imaged using X-rays. However, this embodiment is not limited to this example, and various objects can be applied as the processing target.

[0037] As shown in the figure, a recognition unit 10 is installed upstream of the belt conveyor CB. A target object OBJ transported by the belt conveyor CB always passes through the recognition unit 10. The recognition unit 10 acquires three-dimensional information and an internal structure of the target object OBJ. After passing through the recognition unit 10, the target object OBJ is transported to a processing unit 20.

[0038] The target object OBJ is always conveyed to the processing unit 20 after passing through the recognition unit 10. The processing unit 20 can also be said to be located downstream of the recognition unit 10.

[0039] The path from the recognition unit 10 to the processing unit 20 may not be a straight line depending on the design of the object processing system 1. In the example shown in the figure, the direction changes by 90 degrees when the belt conveyor CB branches off to the processing unit 20. When the target object OBJ is transported along such a path, the orientation grasped by the processing unit 20 may have changed. Therefore, it is necessary to grasp the orientation of the object again before processing is performed by the processing unit 20.

[0040] When the target object OBJ is transported to the processing unit 20, first, three-dimensional information of the target object OBJ is acquired. Based on the three-dimensional information acquired by the processing unit 20 and the three-dimensional information and perspective image acquired by the recognition unit 10, the internal structure in the orientation at the processing unit 20 is estimated.

[0041] Here, in order to compare information about the target object OBJ recognized by the recognition unit 10 with information about the target object OBJ recognized by the processing unit 20, an identification number may be assigned to the target object OBJ. The identification number may be assigned, for example, to a tray (not shown) on which the target object OBJ is placed, or a predetermined feature of the target object OBJ may be used as identification information using object recognition technology. Furthermore, identification may be performed according to the transport distance of the belt conveyor CB, route branch instruction information, etc.

[0042] [Recognition unit 10] 2 is a diagram illustrating an example of an overview of the recognition unit according to this embodiment. With reference to the same figure, an example of an overview of the recognition unit 10 will be described. The recognition unit 10 includes an X-ray camera 11, a 3D camera 12, and a control unit 13. Note that the illustrated example is a schematic diagram for explanation purposes and does not limit the arrangement of the components.

[0043] The target object OBJ is placed on a belt conveyor CB and is transported in the direction of the arrow AR. When the target object OBJ is transported to a point P11, an X-ray image is taken by an X-ray camera 11.

[0044] The X-ray camera 11 is an example of a configuration for capturing an image of the internal structure described above. Specifically, the X-ray camera 11 captures an X-ray image as an example of a fluoroscopic image of the internal structure. The X-ray camera 11 can also be called a fluoroscopic image capturing device. The X-ray camera 11 outputs the captured X-ray image to the control unit 13.

[0045] After a perspective image of the target object OBJ is captured at point P11, the target object OBJ is transported to point P12. At the point when the target object OBJ is transported to point P12, three-dimensional information is acquired by the 3D camera 12.

[0046] The 3D camera 12 is an example of a configuration for acquiring the above-mentioned three-dimensional information. Specifically, the 3D camera 12 acquires a distance image having distance information at each coordinate in an xy coordinate system as an example of the three-dimensional information. The three-dimensional information acquired by the 3D camera 12 may be point cloud data. In the following description, the 3D camera 12 may be referred to as a first three-dimensional information imaging device.

[0047] The control unit 13 acquires fluoroscopic images from the X-ray camera 11 and acquires three-dimensional information from the 3D camera 12. The acquired information is associated with each other, and information in which the fluoroscopic images and the three-dimensional information are associated with each other is stored in a predetermined storage unit. An example of the functional configuration of the control unit 13 will be described below with reference to FIG. 3.

[0048] 3 is a functional configuration diagram showing an example of the functional configuration of the recognition unit according to this embodiment. With reference to the same figure, an example of the functional configuration of the control unit 13 will be described. The control unit 13 has, as its functional configuration, a perspective image acquisition unit 131, a first three-dimensional information acquisition unit 132, and a recognition control unit 133.

[0049] The perspective image acquisition unit 131 acquires a perspective image capturing an internal structure of the target object OBJ. Specifically, the perspective image may be an X-ray image captured by the X-ray camera 11. Note that in this embodiment, the perspective image is not limited to an X-ray image. The perspective image may be any image that includes information about the internal structure of the object that is difficult to grasp from its external appearance.

[0050] The first three-dimensional information acquisition unit 132 acquires three-dimensional information indicating the outer shape of the target object OBJ. Specifically, the three-dimensional information may be information captured by the 3D camera 12. The three-dimensional information acquired by the first three-dimensional information acquisition unit 132 may also be referred to as first three-dimensional information.

[0051] The recognition control unit 133 first extracts feature points from the perspective image acquired by the perspective image acquisition unit 131. Feature points are points used when processing the target object OBJ. For example, if the target object OBJ is meat, the feature points may be cutting points that serve as landmarks when cutting the meat. The recognition control unit 133 associates the first three-dimensional information with the feature points and stores them in the storage unit 30. The recognition control unit 133 may further associate the first three-dimensional information with identification information of the target object OBJ that has been measured and store them in the storage unit 30.

[0052] Here, in the recognition unit 10, it is preferable to place the X-ray camera 11 and the 3D camera 12 at positions on the path where they can be considered to be the same, but this may be difficult due to restrictions on installation positions. That is, the X-ray camera 11 and the 3D camera 12 may be placed at different positions on the path. The recognition control unit 133 may simply consider the X-ray camera 11 and the 3D camera 12 to be in the same coordinate system even when they are placed at different positions on the path. This is because, within the recognition unit 10, the path is a straight line and the orientation of the target object OBJ is likely to remain unchanged. In other words, the recognition control unit 133 may simply consider the X-ray camera 11 and the 3D camera 12 to be in the same coordinate system if the distance the target object OBJ has traveled on the path is short. Furthermore, if the distance the target object OBJ has traveled on the path is long, the recognition control unit 133 may convert the coordinates into the same coordinate system by performing a predetermined coordinate transformation. That is, the recognition control unit 133 may align the coordinates of the perspective image and the three-dimensional information depending on the distance the target object OBJ has traveled on the path.

[0053] [Processing section 20] 4 is a diagram illustrating an example of the outline of the processing unit according to this embodiment. An example of the outline of the processing unit 20 will be described with reference to the same figure. The processing unit 20 includes a 3D camera 21, a control unit 23, and a robot processing unit 25. Note that the illustrated example is a schematic diagram for explanation purposes and does not limit the arrangement of the components. Note that, although not illustrated, the processing unit 20 may include an RGB camera capable of capturing a visible image of the target object OBJ in addition to the 3D camera 21.

[0054] The target object OBJ is placed on a belt conveyor CB and is transported in the direction of the arrow AR. When the target object OBJ is transported to a point P21, three-dimensional information is captured by a 3D camera 21.

[0055] The 3D camera 21 is an example of a configuration for acquiring the above-mentioned three-dimensional information. Specifically, the 3D camera 21 acquires a distance image having distance information at each coordinate in an xy coordinate system as an example of the three-dimensional information. The three-dimensional information acquired by the 3D camera 21 may be point cloud data. In the following description, the 3D camera 21 may also be referred to as a second three-dimensional information imaging device.

[0056] The control unit 23 acquires three-dimensional information from the 3D camera 21. The control unit 23 also acquires information obtained by the recognition unit 10 (specifically, the first three-dimensional information and feature points described with reference to FIG. 3). Based on the acquired information, the control unit 23 corrects the positions of the feature points to match the posture of the target object OBJ at point P21. The control unit 23 outputs information about the corrected feature points to the robot processing unit 25.

[0057] After the three-dimensional information of the target object OBJ is acquired at point P21, the target object OBJ is transported to point P22. At the time when the target object OBJ is transported to point P22, processing is performed by the robot processing unit 25. At this time, the robot processing unit 25 performs processing on the target object OBJ using information on the feature points acquired from the control unit 23. For example, if the target object OBJ is meat, the processing on the target object OBJ may be a cutting process of the meat. Hereinafter, an example of the functional configuration of the control unit 23 will be described with reference to FIG. 5.

[0058] 5 is a functional configuration diagram showing an example of the functional configuration of a processing unit according to this embodiment. With reference to the same figure, an example of the functional configuration of the control unit 23 will be described. The control unit 23 has, as its functional configuration, a second three-dimensional information acquisition unit 231, a recognition information acquisition unit 232, and a processing control unit 233.

[0059] The second three-dimensional information acquisition unit 231 acquires three-dimensional information indicating the outer shape of the target object OBJ. Specifically, the three-dimensional information may be information captured by the 3D camera 21. Furthermore, the three-dimensional information is information acquired at point P21, which is a point downstream on the route from point P12, which is the point at which the first three-dimensional information was acquired by the processing unit 20. The three-dimensional information acquired by the second three-dimensional information acquisition unit 231 may also be referred to as second three-dimensional information.

[0060] The recognition information acquisition unit 232 acquires first three-dimensional information and information about feature points of the target object OBJ (transported to the processing unit 20) from the storage unit 30. Here, the recognition information acquisition unit 232 may acquire identification information about the target object OBJ transported to the processing unit 20 using a predetermined method, and use the acquired identification information to search for the first three-dimensional information and information about feature points corresponding to the identification information. The predetermined method for acquiring the identification information may be based on the three-dimensional information acquired by the second three-dimensional information acquisition unit 231, or may be based on some member that moves along the path with the target object OBJ, such as a tray on which the target object OBJ is placed. Furthermore, the identification information may be affixed to the target object OBJ itself by, for example, engraving. In this case, the identification information may be obtained by analyzing an RGB image captured by, for example, an RGB camera (not shown).

[0061] The processing control unit 233 matches the three-dimensional information acquired by the first three-dimensional information acquisition unit 132 with the three-dimensional information acquired by the second three-dimensional information acquisition unit 231. As a specific example of the matching process, for example, a known algorithm may be used. An example of a known algorithm is a surface matching algorithm. As a result of the matching process, a transformation matrix is ​​obtained. The transformation matrix converts the position and orientation of the first three-dimensional information into the position and orientation of the second three-dimensional information. The processing control unit 233 performs coordinate transformation of the feature points based on the transformation matrix obtained as a result of the matching. The processing control unit 233 outputs information on the transformed feature points to the robot processing unit 25.

[0062] Specifically, the processing control unit 233 includes a matching unit 234 and a coordinate conversion unit 235, and thereby performs matching between the three-dimensional information acquired by the first three-dimensional information acquisition unit 132 and the three-dimensional information acquired by the second three-dimensional information acquisition unit 231. The matching unit 234 performs matching processing and outputs a transformation matrix as a result of the matching processing. The coordinate conversion unit 235 performs coordinate conversion of the feature points acquired by the recognition information acquisition unit based on the transformation matrix.

[0063] Here, the processing control unit 233 may perform matching processing based on, for example, distance and relative direction. For example, the three-dimensional information acquired by the first three-dimensional information acquisition unit 132 and the three-dimensional information acquired by the second three-dimensional information acquisition unit 231 may both be point cloud data. In such a case, the processing control unit 233 may perform matching based on the distance and relative direction of each point cloud.

[0064] Here, point cloud data has a large capacity, and matching all points may require a huge amount of processing. Therefore, the processing control unit 233 may perform a thinning process on the three-dimensional information acquired by the first three-dimensional information acquisition unit 132 to create data with a small capacity. Similarly, the processing control unit 233 may perform a similar thinning process on the three-dimensional information acquired by the second three-dimensional information acquisition unit 231 to create data with a small capacity. Furthermore, the processing control unit 233 may perform a matching process based on the distance and relative direction of the point cloud from which the information has been thinned.

[0065] [Example of matching] 6 is a diagram for explaining an example of matching according to this embodiment. With reference to the same figure, an example of matching performed by the above-mentioned process control unit 233 will be explained. Note that a specific example of the matching process can be surface matching processing.

[0066] First, the process control unit 233 performs a thinning process on the first three-dimensional information (step S11). As a result of the thinning process, the amount of information in the first three-dimensional information is reduced to the first amount of information. Furthermore, the process control unit 233 performs a thinning process on the second three-dimensional information to reduce the amount of information to the first amount of information (step S12). The thinning process is a process of reducing the volume by sampling at a predetermined interval based on the information of the point cloud data. The sampling intervals in steps S11 and S12 are assumed to be the same.

[0067] Next, the process control unit 233 performs a matching process based on the information with the first amount of information (step S13). In other words, the process control unit 233 performs the first matching based on the distance and relative direction of the point cloud data with the first amount of information, which is a point cloud obtained by thinning out information from the acquired point cloud data. The first matching process can also be considered as a fine adjustment of sparseness.

[0068] Next, the process control unit 233 performs a thinning process on the first three-dimensional information (step S14). As a result of the thinning process, the amount of information in the first three-dimensional information is reduced to a second amount of information. Furthermore, the process control unit 233 performs a thinning process on the second three-dimensional information to reduce the amount of information to the second amount of information (step S15). Here, the sampling intervals in steps S14 and S15 are assumed to be the same. The second amount of information is greater than the first amount of information. In other words, the sampling intervals in steps S14 and S15 are denser than the sampling intervals in steps S11 and S12.

[0069] Furthermore, the process control unit 233 performs a matching process based on the information with the second amount of information (step S16). In other words, the process control unit 233 can be said to perform a second matching process based on the distance and relative direction of the point cloud data with the second amount of information, which is a point cloud obtained by thinning out information from the acquired point cloud data. Because the second amount of information is greater than the first amount of information, the second matching process can also be said to be a fine adjustment of the density.

[0070] By performing the coarse fine adjustment before the fine fine adjustment, the two pieces of three-dimensional information are aligned to suitable positions, and the alignment process can be easily performed in the fine fine adjustment that is performed later.

[0071] [Processing flow of object processing system 1] 7 is a flowchart showing an example of the flow of a series of processes in the object processing system according to this embodiment, which will be described with reference to the drawing.

[0072] (Step S21) First, the perspective image acquisition unit 131 acquires a perspective image, and the first three-dimensional information acquisition unit 132 acquires three-dimensional information.

[0073] (Step S22) Next, the recognition control unit 133 performs a point extraction process based on the acquired perspective image and three-dimensional information. The point extraction process is a process of extracting feature points.

[0074] (Step S23) Next, the recognition control unit 133 creates a model based on the acquired three-dimensional information. Specifically, the model is obtained by performing a sampling process on the three-dimensional information, which is point cloud data, to reduce the amount of information. The created model is stored in the storage unit. Note that step S22 and step S23 may be performed in parallel.

[0075] (Step S24) Next, the recognition control unit 133 stores in the storage unit 30 information about the feature points extracted in step S22 and the model created in step S23.

[0076] The processes of steps S21 to S24 described so far are performed in the recognition unit 10. Therefore, these steps may be referred to as recognition steps. The processes of steps S31 to S37 described below are performed in the respective processing units 20. Therefore, these steps may be referred to as processing steps.

[0077] (Step S31) When each processing unit 20 detects that a target object OBJ has been transported (i.e., step S31; YES), the process proceeds to step S32. A standby state is maintained until the target object OBJ is transported. Here, the transport of the target object OBJ may be detected, for example, by a sensor or the like provided at the entrance of the processing unit 20.

[0078] (Step S32) When the target object OBJ is transported to the processing unit 20, first, the second three-dimensional information acquisition unit 231 acquires three-dimensional information.

[0079] (Step S33) Next, the process control unit 233 creates a model based on the acquired three-dimensional information. This model is a model of the same format as the model created in the above-mentioned step S23. The same format means the same data format and the same sampling interval.

[0080] (Step S34) Furthermore, based on the ID (identification information) of the transported target object OBJ, the process control unit 233 reads the model created in step S23 and information on the feature points extracted in step S22 from the storage unit 30. Here, the storage unit 30 stores information on a plurality of target objects OBJ.

[0081] (Step S35) Furthermore, the process control unit 233 performs a matching process based on the model created in step S33 and the model read out in step S34. As a result of the matching process, a transformation matrix can be obtained.

[0082] (Step S36) Based on the obtained transformation matrix, the process control unit 233 moves the feature points extracted in step S22. The movement of the feature points can also be considered as coordinate transformation.

[0083] (Step S37) The robot processing unit 25 performs robot control based on the new feature points obtained as a result of the coordinate transformation (feature points corresponding to the posture of the target object OBJ in the processing unit 20). An example of robot control is meat cutting processing.

[0084] [Variations] Fig. 8 is a diagram illustrating a modified example of the outline of the object processing system according to this embodiment. With reference to Fig. 8, an object processing system 1A, which is a modified example of the object processing system 1 described with reference to Fig. 1, will be described. Note that in the description with reference to Fig. 8, components already described with reference to Fig. 1 may be omitted by assigning the same reference numerals.

[0085] The object processing system 1 can also be said to include one recognition unit 10 and at least one processing unit 20. In other words, a plurality of processing units 20 may be provided for one recognition unit 10. In the example shown in the figure, processing units 20-1 and 20-2 are shown as examples of the plurality of processing units 20. In the following description, when there is no need to distinguish between processing units 20-1 and 20-2, they may be simply referred to as processing units 20.

[0086] As shown in the figure, a recognition unit 10 is installed upstream of the belt conveyor CB. A target object OBJ transported by the belt conveyor CB always passes through the recognition unit 10. After passing through the recognition unit 10, the target object OBJ is transported to each processing unit 20. The processing unit 20 to which the object is to be transported may be a processing unit 20 that is not currently performing processing (is empty), or the destination may be selected (assigned) in order.

[0087] The target object OBJ is always transported to the processing unit 20 after passing through the recognition unit 10. It can also be said that each processing unit 20 is located downstream of the recognition unit 10. When the recognition unit 10 is considered to be upstream, the path for transporting the target object OBJ can also be said to branch off to each of the multiple processing units 20 located downstream. The target object OBJ is transported along this path, and is transported from the recognition unit 10 to one of the processing units 20.

[0088] Here, a plurality of processing units 20 are provided, and the target object OBJ reaches the processing unit 20 from the recognition unit 10 by branching the path. That is, the path from the recognition unit 10 to the processing unit 20 may not be a straight line. When the target object OBJ is transported along such a path, the orientation grasped by the processing unit 20 may have changed. Therefore, it is necessary to grasp the orientation of the object again before processing is performed by the processing unit 20.

[0089] When multiple target objects OBJ are processed consecutively, information about the multiple target objects OBJ is stored in the storage unit 30. The target object OBJ whose information is stored in the storage unit 30 is transported to one of the multiple processing units 20 included in the object processing system 1. The processing control unit 233 provided in the processing unit 20 may then identify the target object OBJ that has been transported, and read out information about the identified target object OBJ from the storage unit 30.

[0090] [Internal structure] FIG. 9 is a block diagram showing the internal configuration of each device included in the object processing system according to this embodiment. At least some of the functions of the recognition unit 10 or the processing unit 20 can be implemented using a computer. As shown in the figure, the computer includes a central processing unit 901, a RAM 902, an input / output port 903, input / output devices 904 and 905, and a bus 906. The computer itself can be implemented using existing technology. The central processing unit 901 executes instructions included in a program read from the RAM 902 or the like. In accordance with each instruction, the central processing unit 901 writes data to the RAM 902, reads data from the RAM 902, and performs arithmetic and logical operations. The RAM 902 stores data and programs. Each element included in the RAM 902 has an address and can be accessed using the address. RAM stands for "random access memory." The input / output port 903 is a port through which the central processing unit 901 exchanges data with external input / output devices. The input / output devices 904 and 905 are input / output devices. The input / output devices 904 and 905 exchange data with the central processing unit 901 via the input / output port 903. The bus 906 is a common communication path used within the computer. For example, the central processing unit 901 reads and writes data from and to the RAM 902 via the bus 906. Also, for example, the central processing unit 901 accesses the input / output port via the bus 906. Furthermore, all or part of the functional units provided in the recognition unit 10 or the processing unit 20 may be realized using hardware such as an ASIC, a PLD, or an FPGA. Furthermore, all or part of the functional units may be realized by a combination of software and hardware.

[0091] Note that all or part of the functions of each unit of each device included in the object processing system 1 in the above-described embodiment may be realized by recording a program for realizing these functions on a computer-readable recording medium, and reading and executing the program recorded on the recording medium into a computer system. Note that the term "computer system" here includes hardware such as an OS and peripheral devices.

[0092] Furthermore, "computer-readable recording media" refers to portable media such as optical magnetic disks, ROMs, and CD-ROMs, as well as storage units such as hard disks built into computer systems. Furthermore, "computer-readable recording media" may also include devices that dynamically store programs for a short period of time, such as communication lines used when transmitting programs over a network such as the Internet, and devices that store programs for a fixed period of time, such as volatile memory within a computer system that serves as a server or client in such a case. Furthermore, the program may be one that realizes part of the aforementioned functions, or may be one that can realize the aforementioned functions in combination with a program already stored in the computer system.

[0093] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the spirit of the present invention. [Explanation of symbols]

[0094] 1...object processing system, 10...recognition unit, 20...processing unit, 11...X-ray camera, 12...3D camera, 13...control unit, 131...fluoroscopic image acquisition unit, 132...first three-dimensional information acquisition unit, 133...recognition control unit, 30...memory unit, 21...3D camera, 23...control unit, 25...robot processing unit, 231...second three-dimensional information acquisition unit, 232...recognition information acquisition unit, 233...processing control unit, 234...matching unit, 235...coordinate conversion unit, OBJ...target object, CB...belt conveyor

Claims

1. The system includes a recognition unit disposed upstream of a route along which a target object is transported, and a processing unit disposed downstream of the recognition unit along the route, The recognition unit a perspective image acquisition unit for acquiring a perspective image of an internal structure of the target object; a first three-dimensional information acquisition unit that acquires three-dimensional information indicating an outer shape of the target object; a recognition control unit that extracts feature points from the perspective image; Equipped with The processing unit a second three-dimensional information acquisition unit that acquires three-dimensional information indicating the outline of the target object at a point downstream of the path from the point at which the three-dimensional information was acquired by the recognition unit; a processing control unit that performs coordinate transformation of the feature points based on a transformation matrix obtained as a result of matching the three-dimensional information acquired by the first three-dimensional information acquisition unit with the three-dimensional information acquired by the second three-dimensional information acquisition unit; An object recognition device comprising:

2. the three-dimensional information acquired by the first three-dimensional information acquisition unit and the three-dimensional information acquired by the second three-dimensional information acquisition unit are both point cloud data; the processing control unit performs matching based on distances and relative directions of the point clouds; The object recognition device according to claim 1 .

3. the processing control unit performs matching based on distances and relative directions of point clouds obtained by thinning out information from the acquired point cloud data; The object recognition device according to claim 2 .

4. the processing control unit performs a first matching based on distances and relative directions of point cloud data obtained by thinning out information from the acquired point cloud data and having a first amount of information, and then performs a second matching based on distances and relative directions of point cloud data obtained by thinning out information from the acquired point cloud data and having a second amount of information greater than the first amount of information; The object recognition device according to claim 3 .

5. in the recognition unit, a perspective image capturing device that captures the perspective image and a first three-dimensional information capturing device that captures three-dimensional information indicating an outline of the target object are disposed at different positions on the path, the recognition control unit performs coordinate alignment between the perspective image and the three-dimensional information according to the distance the target object has been transported on the path. The object recognition device according to claim 1 .

6. The fluoroscopic image is an X-ray image. The object recognition device according to claim 1 .

7. the target object is meat, The path is a belt conveyor that transports the meat. The object recognition device according to claim 1 .

8. The characteristic points are cutting points that serve as landmarks when cutting the meat. The object recognition device according to claim 7 .

9. An object recognition system including the object recognition device according to any one of claims 1 to 8, One of the recognition units; a plurality of the processing units; When the recognition unit is considered to be upstream, the path branches off to each of the plurality of processing units located downstream, The target object is transported from the recognition unit to any one of the processing units. Object recognition system.

10. In the plurality of processing units, cutting of the target object is performed based on the feature points. The object recognition system of claim 9.

11. a recognition step performed upstream of a route along which a target object is transported, and a processing step performed downstream of the recognition step along the route; The recognition step includes: a perspective image acquisition step of acquiring a perspective image of an internal structure of the target object; a first three-dimensional information acquisition step of acquiring three-dimensional information indicating an outer shape of the target object; a recognition control step of extracting feature points from the perspective image; and The processing step comprises: a second three-dimensional information acquisition step of acquiring three-dimensional information indicating the outline of the target object at a point downstream of the path from the point at which the three-dimensional information was acquired in the recognition step; a processing control step of performing coordinate transformation of the feature points based on a transformation matrix obtained as a result of matching the three-dimensional information acquired in the first three-dimensional information acquisition step with the three-dimensional information acquired in the second three-dimensional information acquisition step; An object recognition method comprising:

Images