Program, information processing method, and information processing apparatus.

Abstract

To provide a program, or the like, for specifying a plurality of measurement points based on image data of an object.SOLUTION: A program causes a computer to execute the processing of: acquiring image data of an object; extracting segmentation data in a measurement target region of the target, from the acquired image data; acquiring template data of the measurement target region and a plurality of reference measurement points of the object in the template data; and specifying the measurement points in the extracted segmentation data on the basis of the acquired template data and the reference measurement points.SELECTED DRAWING: Figure 4

Description

Technical Field

[0006] , , ,

[0005] , ,

[0001] The present invention relates to a program, an information processing method, and an information processing apparatus.

Background Art

[0002] In recent years, there is a technology for detecting moving objects in an image by image analysis. Patent Document 1 discloses a learning data generation apparatus that detects a moving object from an image including an individual (for example, a dairy cow), extracts a body region image of a body region including the entire body of the individual among the detected moving objects, and generates learning data for learning an identification model from the extracted body region image.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the invention according to Patent Document 1 has a problem that it is impossible to identify a plurality of measurement points used for measuring the body size of an individual (object).

[0005] In one aspect, it is to provide a program or the like capable of identifying a plurality of measurement points based on image data of an object.

Means for Solving the Problems

[0006] A program according to one aspect acquires image data of an object, extracts segmentation data of a measurement target region of the object from the acquired image data, acquires template data of the measurement target region and a plurality of reference measurement points of the object in the template data, and identifies a plurality of measurement points in the extracted segmentation data based on the acquired template data and the plurality of reference measurement points Based on the values ​​of the first coordinate axis of the reference measurement point, the values ​​of the first coordinate axis of the measurement point are determined. Based on the values ​​of the second coordinate axis of the reference measurement point and the maximum and minimum values ​​of the second coordinate axis of the template data, a division ratio is calculated. Based on the calculated division ratio, the values ​​of the second coordinate axis of the measurement point are determined.Have the computer perform the process. [Effects of the Invention]

[0007] In one respect, it becomes possible to identify multiple measurement points based on image data of the object. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing an example of a computer configuration. [Figure 2] This is an explanatory diagram showing an example of the record layout for the template data database and the measurement results database. [Figure 3] This is an explanatory diagram showing an example of multiple reference measurement points for cattle. [Figure 4] This is an explanatory diagram illustrating the process of identifying multiple measurement points on a cow. [Figure 5] This is an explanatory diagram illustrating the process of identifying measurement points. [Figure 6] This is an explanatory diagram showing an example of a display screen for measurement results. [Figure 7] This flowchart shows the processing procedure for identifying multiple measurement points. [Figure 8] This flowchart shows the processing steps of the subroutine that corrects the mask image. [Figure 9] This flowchart shows the processing steps of the subroutine that adjusts the size of the mask image. [Figure 10] This flowchart shows the processing procedure of the subroutine for identifying measurement points. [Modes for carrying out the invention]

[0009] The present invention will be described in detail below with reference to the drawings illustrating its embodiments.

[0010] (Embodiment 1) Embodiment 1 relates to a method for identifying multiple measurement points based on image data and depth data (depth information) of an object. The object includes objects, humans, or animals other than humans. Objects include manufactured goods such as motorcycles, automobiles, or railway vehicles, equipment, utility poles, or components. Animals include elephants, cattle, horses, sheep, pigs, chickens, ducks, dogs, cats, mice, or tuna. The object also includes the skeleton, internal organs, or areas of interest during surgery of humans or animals. In the following description, a cattle will be used as the object, but the method can be applied similarly to other types of objects.

[0011] The system of this embodiment includes an information processing device 1. The information processing device 1 is an information processing device that processes, stores, and transmits various types of information. The information processing device 1 is an information processing device such as a smartphone, mobile phone, wearable device such as Apple Watch (registered trademark), wearable camera, tablet, or personal computer. The information processing device 1 may also be a server device or a cloud server device that provides functions as a cloud service. For simplicity, the information processing device 1 will be read as computer 1 below.

[0012] In this embodiment, Computer 1 acquires image data of the target cow. From the acquired image data of the cow, Computer 1 extracts segmentation data of the measurement target area of ​​the cow. Computer 1 acquires template data of the measurement target area of ​​the cow and multiple reference measurement points of the cow in the template data. Based on the acquired template data and multiple reference measurement points, Computer 1 identifies multiple measurement points in the extracted segmentation data.

[0013] Figure 1 is a block diagram showing an example configuration of computer 1. Computer 1 includes a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, a display unit 15, a reading unit 16, and a large-capacity storage unit 17. Each component is connected by bus B.

[0014] The control unit 11 includes an arithmetic processing unit such as a CPU (Central Processing Unit), MPU (Micro-Processing Unit), GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), or quantum processor, and performs various information processing, control processing, etc. related to the computer 1 by reading and executing the control program 1P (program product) stored in the storage unit 12.

[0015] Note that the control program 1P can be deployed to be executed on a single computer, or arranged at one site, or distributed over multiple sites and executed on multiple computers interconnected by a communication network. In FIG. 1, the control unit 11 is described as a single processor, but it may also be a multi-processor.

[0016] The storage unit 12 includes memory elements such as RAM (Random Access Memory) and ROM (Read Only Memory), and stores the control program 1P or data etc. necessary for the control unit 11 to execute processing. Also, the storage unit 12 temporarily stores data etc. necessary for the control unit 11 to execute arithmetic processing. The communication unit 13 is a communication module for performing communication-related processing.

[0017] The input unit 14 may be a keyboard, a mouse, or a touch panel integrated with the display unit 15. The display unit 15 is a liquid crystal display or an organic EL (electroluminescence) display etc., and displays various information according to the instructions of the control unit 11.

[0018] The reading unit 16 reads a portable storage medium 1a, including a CD (Compact Disc)-ROM or DVD (Digital Versatile Disc)-ROM. The control unit 11 may read the control program 1P from the portable storage medium 1a via the reading unit 16 and store it in the large-capacity storage unit 17. Alternatively, the control unit 11 may download the control program 1P from another computer via a network N or the like and store it in the large-capacity storage unit 17. Furthermore, the control unit 11 may also read the control program 1P from the semiconductor memory 1b.

[0019] The large-capacity storage unit 17 includes a recording medium such as an HDD (Hard disk drive) or an SSD (Solid State Drive). The large-capacity storage unit 17 includes a template data DB (database) 171 and a measurement results DB 172. The template data DB 171 stores template data for the measurement target area of ​​a cow. The measurement results DB 172 stores the measurement results of multiple measurement points in the measurement target area of ​​a cow.

[0020] In this embodiment, the storage unit 12 and the large-capacity storage unit 17 may be configured as a single storage device. Furthermore, the large-capacity storage unit 17 may be composed of multiple storage devices. Moreover, the large-capacity storage unit 17 may be an external storage device connected to the computer 1.

[0021] Computer 1 may perform various information processing and control processing tasks on its own, or it may perform them in a distributed manner across multiple computers. Furthermore, Computer 1 may be implemented using multiple virtual machines located within a single computer.

[0022] Figure 2 is an explanatory diagram showing an example of the record layout for template data DB171 and measurement results DB172. The template data DB171 includes columns for template ID (Identifier), image data, measurement area, reference measurement point name, and reference measurement point coordinates. The template ID column stores a uniquely identified template data ID to identify the template data for the measurement area of ​​each cow. The image data column stores reference image data for the measurement area of ​​the cow.

[0023] The measurement site column stores the names of the measurement sites for cattle. These measurement sites include body height, body length, horizontal body length, chest circumference, or cross-section height. The reference measurement point name column stores the names of the reference measurement points for cattle in the template data. The reference measurement point coordinate column stores the coordinate values ​​of the reference measurement points for cattle in the template data. The template data will be described later.

[0024] The measurement results DB172 includes columns for Image ID, Image Data, Measurement Location, Measurement Point Name, and Measurement Point Coordinates. The Image ID column stores the ID of the image data, which is uniquely identified to identify the image data of each cow. The Image Data column stores the image data of the cow. The Measurement Location column stores the name of the measurement location of the cow. The Measurement Point Name column stores the name of the measurement point in the cow's image data. The Measurement Point Coordinates column stores the coordinate values ​​of the measurement point in the cow's image data.

[0025] The storage configurations described above for each database are merely examples; other storage configurations are also acceptable as long as the relationships between the data are maintained.

[0026] Figure 3 is an explanatory diagram showing an example of multiple reference measurement points for a cow. Multiple reference measurement points are pre-established in the measurement area of ​​the cow in order to measure the cow's body size. As shown in the figure, there is a measurement point 11a indicating the apex of the withers (the point where the line connecting the apex of the left and right scapulae intersects with the spine), a measurement point 11b indicating the shoulder end, a measurement point 11c indicating the ischium end, a measurement point 11d indicating the upper part of the scapula, a measurement point 11e indicating the upper part of the chest circumference, a measurement point 11f indicating the lower part of the chest circumference, and a measurement point 11g indicating the cross (the point where the line connecting the two hip horns of the cow intersects with the spine).

[0027] The body dimensions of a cow can be measured based on the multiple reference measurement points described above. Specifically, the height of a cow can be measured based on measurement point 11a (the top of the withers). The height is the vertical distance from measurement point 11a to the ground measurement point 11h for measuring the height when the cow is standing upright. The body length of a cow can be measured based on measurement point 11b (the shoulder) and measurement point 11c (the ischium). The horizontal body length of a cow can be measured based on measurement point 11c (the ischium) and measurement point 11d (the upper part of the scapula). The chest circumference of a cow can be measured based on measurement point 11e (the upper part of the chest) and measurement point 11f (the lower part of the chest). The cruciate height of a cow can be measured based on measurement point 11g (the cruciate). The cruciate height is the vertical distance from measurement point 11g to the ground measurement point 11i for measuring the cruciate height when the cow is standing upright. The ground measurement points 11h and 11i may be recognized from the image of the cow using well-known image recognition technology, or they may be recognized using pattern matching technology or the like.

[0028] The number or location of the reference measurement points shown in Figure 3 can be adjusted according to actual needs and are not limited to those described in this embodiment.

[0029] Figure 4 is an explanatory diagram illustrating the process of identifying multiple measurement points on a cow. First, computer 1 acquires image data of the cow to be measured. The image data of the cow is image data taken from the side of the cow. For example, computer 1 may acquire image data of the cow taken by an external information processing terminal with a camera function. If computer 1 is a wearable camera or a terminal with a camera function, computer 1 may directly acquire the image data of the cow that has been taken.

[0030] Next, computer 1 extracts segmentation data (hereinafter referred to as mask image) of the measurement target area of ​​the cow from the acquired image data of the cow. The measurement target area is, for example, the cow's torso excluding the legs, tail, and head. For example, computer 1 may extract the mask image of the measurement target area of ​​the cow using an image segmentation model constructed by an instance segmentation network.

[0031] The image segmentation model is trained using image data of a cow photographed from the side. The image segmentation model is configured to output an image in which the cow's body is white and other areas are black, and it is trained to make the output image approximate the correct image. Computer 1 inputs the acquired cow image data into the image segmentation model and outputs a mask image of the area to be measured. In the output mask image, the area corresponding to the cow's body (foreground pixels) is white (pixel value 1), and the other areas (background pixels) are black (pixel value 0).

[0032] The image segmentation model may be a SegNet model, or a semantic segmentation model such as U-NET (Convolutional Networks for Biomedical Image Segmentation). U-NET is a type of FCN (Fully Convolutional Networks) and includes an encoder that performs downsampling and a decoder that performs upsampling. U-NET is a neural network that does not have fully connected layers and consists only of convolutional layers and pooling layers.

[0033] In addition to the image segmentation models described above, methods such as background subtraction can also be used. By applying background subtraction, a mask image is generated as a result of background subtraction.

[0034] Next, if the extracted mask image contains white areas in the background pixels, computer 1 corrects the mask image based on the 3D data. As shown in the figure, since there is a white region 20a in the background pixels, it is necessary to correct the mask image using the 3D data.

[0035] Specifically, computer 1 acquires 3D data using a depth sensor that measures depth (distance) information to the cow. 3D data is data (3D point cloud data) where each pixel of an image has 3D coordinate values. The depth sensor measures the depth to an object, for example, by projecting a laser onto the object and receiving the reflected light with a photodetector. In addition to a depth sensor, an infrared sensor or a laser sensor such as LiDAR may also be used. By using a depth sensor, 3D data including depth information to the cow can be generated.

[0036] Computer 1 corrects the extracted mask image based on the acquired 3D data. As shown in the figure, the 3D data is data containing 3D coordinate values ​​including the X-axis (first coordinate axis), Y-axis (second coordinate axis), and Z-axis. The X-axis indicates the cow's lateral direction (left-right direction), with the right direction on the paper being positive. The Y-axis indicates the cow's vertical direction (up-down direction), with the bottom direction on the paper being positive. The Z-axis indicates the viewpoint direction (perpendicular to the paper plane in Figure 4) of the pixels (white pixels) that represent the cow's body.

[0037] For example, computer 1 calculates a depth difference value, which indicates the difference in depth between the depth (Z-axis value) of the pixel representing the cow's body (white pixel) in the viewing direction and the depth of the pixel representing the target (white area 20a) in the viewing direction. If the calculated depth difference value is greater than or equal to a predetermined threshold, computer 1 sets the pixel value of the target pixel to 0, thereby making that pixel black. If the depth difference value is less than the predetermined threshold, the pixel value of the target pixel is left unchanged.

[0038] The threshold for the depth difference value may be, for example, 50 centimeters (cm). The threshold for the depth difference value should be set appropriately according to the distance between the camera and the cow during shooting. Alternatively, a trained model that outputs a depth difference value threshold when a cow image and 3D data are input can be used. If the depth difference value is outside the predetermined threshold range (e.g., 30cm to 50cm), the pixel value of that pixel may be set to 0, treating it as a pixel other than the cow's body.

[0039] Computer 1 then identifies multiple measurement points in the mask image. Figure 5 is an explanatory diagram illustrating the process of identifying measurement points.

[0040] Computer 1 adjusts the size of the corrected mask image. Specifically, Computer 1 obtains template data (hereinafter referred to as the template mask image) of the measurement target area (for example, the body of a cow) from the template data DB171. The template mask image is image data of the measurement target area that includes the names and coordinate values ​​of several predetermined reference measurement points. The template mask image may be created manually by the user, or it may be generated using a template mask image output model that has been trained to output multiple reference measurement points in the measurement target area of ​​a cow when cow image data is input.

[0041] Computer 1 obtains a rectangle 12a from the acquired template mask image of the measurement target area, based on the maximum connected component of the measurement target area. The maximum connected component refers to the largest area among the areas with continuity between pixels. As shown in the figure, the maximum connected component is the white area on the cow's body. Specifically, Computer 1 obtains a rectangle 12a based on the maximum and minimum values ​​of the X axis and the Y axis of the maximum connected component of the measurement target area. Computer 1 calculates the size of the obtained rectangle 12a based on the coordinate values ​​(x1, y1) of the top-left pixel and the coordinate values ​​(x2, y2) of the bottom-right pixel.

[0042] Computer 1 identifies the measurement target area where the pixel value of each pixel is 1, based on the pixel value of each pixel in the mask image to be measured. Computer 1 obtains a rectangle 12b based on the maximum connected component of the identified measurement target area. Computer 1 adjusts (changes) the size of the mask image based on the size of the calculated rectangle 12a. Specifically, Computer 1 adjusts the size of the mask image so that the width and height of the obtained rectangle 12b are the same as the width and height of rectangle 12a.

[0043] Computer 1 identifies multiple measurement points in the resized mask image. Specifically, Computer 1 determines the X-axis value of each measurement point in the resized mask image based on the X-axis value of each reference measurement point in the template mask image. As shown in the figure, the X-axis value (x) of the reference measurement point P in the template mask image is set to the X-axis value (x) of the measurement point Q in the mask image.

[0044] Computer 1 calculates the division ratio based on the Y-axis values ​​of each reference measurement point in the template mask image and the maximum and minimum Y-axis values ​​of the maximum connected component of the measurement area. Based on the calculated division ratio, Computer 1 determines the Y-axis values ​​of each measurement point in the resized mask image.

[0045] As shown in the figure, computer 1 calculates distances d1 and d2 at the maximum connected component of the measurement area in the template mask image. Distance d1 is the distance between the Y-axis value (y) of the reference measurement point P and the minimum Y-axis value (ymin) of the maximum connected component of the measurement area corresponding to the reference measurement point P. Distance d2 is the distance between the Y-axis value (y) of the reference measurement point P and the maximum Y-axis value (ymax) of the maximum connected component of the measurement area corresponding to the reference measurement point P. Based on the vertical width (ymax-ymin) of the maximum connected component of the measurement area corresponding to the reference measurement point P and the calculated distances d1 and d2, computer 1 calculates the division ratio at the reference measurement point P. For example, the calculated division ratio is 2:3.

[0046] Computer 1 obtains the maximum value (y'max) and minimum value (y'min) of the Y-axis of the maximum connected component of the measurement area of ​​the mask image, corresponding to the X-axis value (x) of the determined measurement point Q of the mask image. Based on the calculated division ratio and the obtained maximum value (y'max) and minimum value (y'min) of the Y-axis of the maximum connected component of the measurement area of ​​the mask image, Computer 1 determines the Y-axis value (y') of the measurement point Q. As shown in the figure, the coordinate value of the determined measurement point Q is (x, y').

[0047] Next, returning to Figure 4, Computer 1 stores the identified measurement points in the measurement results DB 172. Specifically, Computer 1 obtains the coordinate values ​​of each of the identified measurement points. Computer 1 assigns an image ID to the cow image data. Computer 1 stores the cow image data, the name of the measurement area, the name and coordinate values ​​of each measurement point in the measurement results DB 172, associating them with the assigned image ID. Computer 1 displays the identified measurement points superimposed on the cow image data.

[0048] Figure 6 is an explanatory diagram showing an example of a measurement result display screen. This screen includes an image display area 13a, a measurement point information display area 13b, a measurement result display area 13c, and measurement points 11a to 11g. Although Figure 6 is explained using the case of seven measurement points, the number of measurement points is not limited.

[0049] Image display area 13a is a display area for displaying image data of the cow. Measurement point information display area 13b is a display area for displaying information such as the name and coordinate values ​​of the measurement points. Measurement result display area 13c is a display area for displaying the results of the cow's body size measurement. Measurement points 11a to 11g are icons that indicate the identified measurement points.

[0050] Computer 1 acquires image data of a cow. From the acquired image data of the cow, Computer 1 extracts a mask image of the measurement target area of ​​the cow, for example, using an image segmentation model constructed by an instance segmentation network. Computer 1 corrects the extracted mask image based on the 3D data obtained by the depth sensor.

[0051] Computer 1 obtains a template mask image of the measurement area of ​​the cow from the template data DB171. Based on the obtained template mask image, Computer 1 adjusts the size of the mask image corrected with 3D data. Computer 1 identifies multiple measurement points in the resized mask image. Computer 1 displays the identified multiple measurement points superimposed on the cow image data. As shown in the figure, icons indicating the measurement points (measurement points 11a to 11g) are displayed superimposed on the cow image data in the image display area 13a.

[0052] For example, computer 1 identifies multiple measurement points, including the apex of the withers, the shoulder edge, the ischial edge, the upper part of the scapula, the upper part of the chest circumference, the lower part of the chest circumference, and the cruciate area. Based on the coordinate values ​​of each identified measurement point, computer 1 displays each measurement point on its corresponding measurement point icon.

[0053] As shown in the figure, measurement point 11a indicates the apex of the withers, measurement point 11b indicates the shoulder end, measurement point 11c indicates the ischium end, measurement point 11d indicates the upper part of the scapula, measurement point 11e indicates the upper part of the chest circumference, measurement point 11f indicates the lower part of the chest circumference, and measurement point 11g indicates the cross part.

[0054] When computer 1 receives a touch (click) operation at any of the measurement points 11a to 11g, it displays the name and coordinate values ​​of the corresponding measurement point in the measurement point information display field 13b.

[0055] Computer 1 measures the body dimensions of a cow based on the coordinate values ​​of several identified measurement points. Specifically, Computer 1 calculates the cow's height based on the coordinate value of measurement point 11a, which is the apex of the withers, and the coordinate value of the ground measurement point 11h used for measuring body height, as shown in Figure 3. Computer 1 calculates the cow's body length based on the coordinate values ​​of measurement point 11b, which is the shoulder, and measurement point 11c, which is the ischium.

[0056] Computer 1 calculates the horizontal body length of the cow based on the coordinate values ​​of measurement point 11c, which is the ischial tuberosity, and measurement point 11d, which is the upper part of the scapula. Computer 1 calculates the chest circumference of the cow based on the coordinate values ​​of measurement point 11e, which is the upper part of the chest circumference, and measurement point 11f, which is the lower part of the chest circumference. Computer 1 calculates the cross-section height of the cow based on the coordinate value of measurement point 11g, which is the cross section, and the coordinate value of the ground measurement point 11i for measuring the cross section height shown in Figure 3.

[0057] Computer 1 displays the measurement date and time, along with the calculated height, length, horizontal length, chest circumference, and cross-section height of the cow, in the measurement result display field 13c.

[0058] Figure 7 is a flowchart showing the processing procedure for identifying multiple measurement points. The control unit 11 of computer 1 acquires image data of a cow taken from the side from, for example, an external information processing terminal with a shooting function, via the communication unit 13 (step S101). The control unit 11 extracts a mask image of the measurement target area of ​​the cow using, for example, an image segmentation model constructed by U-NET (step S102).

[0059] The control unit 11 executes a subroutine for correcting the mask image (step S103). The control unit 11 executes a subroutine for adjusting the size of the mask image (step S104). The control unit 11 executes a subroutine for identifying the measurement points (step S105). The subroutines for correcting the mask image, adjusting the mask image size, and identifying the measurement points will be described later.

[0060] The control unit 11 stores the measurement results in the measurement results DB 172 of the large-capacity storage unit 17 (step S106). Specifically, the control unit 11 assigns an image ID to the image data of the cow. The control unit 11 stores the image data of the cow, the name of the measurement area, the name and coordinate values ​​of each measurement point in the measurement results DB 172, corresponding to the assigned image ID. The control unit 11 displays the identified measurement points superimposed on the image data of the cow via the display unit 15 (step S107), and then terminates the process.

[0061] Figure 8 is a flowchart showing the processing procedure of a subroutine for correcting a mask image. The control unit 11 of computer 1 acquires 3D data using a depth sensor that measures depth information to the cow (step S01). From the acquired 3D data, the control unit 11 acquires 3D data of the pixels that represent cows (white pixels) (step S02). These pixels may be, for example, any pixels in the region that represents the cow's body.

[0062] The control unit 11 acquires one pixel from the mask image to be measured (step S03). The control unit 11 determines whether the acquired pixel is black or not (step S04). For example, the control unit 11 determines that the pixel is black if the pixel value is 0. Alternatively, the control unit 11 determines that the pixel is white if the pixel value is 1.

[0063] If the control unit 11 determines that the acquired pixel is black (YES in step S04), it returns to the process in step S03. If the control unit 11 determines that the acquired pixel is not black (NO in step S04), it determines whether or not 3D data for that pixel exists from the 3D data acquired in the process in step S01 (step S05). If the control unit 11 determines that 3D data for that pixel does not exist (NO in step S05), it proceeds to the process in step S08, which will be described later.

[0064] If the control unit 11 determines that 3D data exists for the pixel in question (YES in step S05), it calculates a depth difference between the depth in the viewpoint direction (Z-axis value) of the pixel (white pixel) that represents a cow (for example, the cow's body) acquired in step S02 and the depth in the viewpoint direction of the pixel in question (step S06). The control unit 11 then determines whether the calculated depth difference value is greater than or equal to a predetermined threshold (for example, 50 cm) (step S07).

[0065] If the calculated depth difference value is less than a predetermined threshold (NO in step S07), the control unit 11 proceeds to the process in step S09, which will be described later. If the calculated depth difference value is greater than or equal to a predetermined threshold (YES in step S07), the control unit 11 sets the pixel to black by setting the pixel value of the pixel to 0 (step S08). The control unit 11 determines whether or not the pixel is the last pixel in the mask image (step S09).

[0066] If the control unit 11 determines that the pixel is not the last pixel (NO in step S09), it returns to the processing in step S03. If the control unit 11 determines that the pixel is the last pixel (YES in step S09), it terminates the mask image correction subroutine and returns.

[0067] Figure 9 is a flowchart showing the processing procedure of a subroutine for adjusting the size of the mask image. The control unit 11 of computer 1 obtains a template mask image of the area to be measured (for example, the torso of a cow) from the template data DB 171 of the large-capacity storage unit 17 (step S11).

[0068] As shown in Figure 5, the control unit 11 obtains a first rectangle (rectangle 12a) from the acquired template mask image of the measurement target area based on the maximum connected component of the measurement target area (step S12). The control unit 11 calculates the size of the acquired first rectangle (rectangle 12a) based on the top-left pixel coordinate value (x1, y1) and the bottom-right pixel coordinate value (x2, y2) of the first rectangle (rectangle 12a) (step S13). The control unit 11 identifies the measurement target area where the pixel value of each pixel is 1 based on the pixel value of each pixel in the mask image to be measured (step S14). The control unit 11 obtains a second rectangle (rectangle 12b) based on the maximum connected component of the identified measurement target area (step S15).

[0069] The control unit 11 adjusts the size of the measurement target area in the mask image based on the calculated size of the first rectangle (rectangle 12a) (step S16). Specifically, the control unit 11 adjusts the size of the mask image so that the width and height of the second rectangle (rectangle 12b) obtained based on the maximum connected component of the measurement target area in the mask image are the same as the width and height of the first rectangle (rectangle 12a) obtained based on the maximum connected component of the measurement target area in the template mask image. The control unit 11 finishes the mask image size adjustment subroutine and returns.

[0070] Figure 10 is a flowchart showing the processing procedure of a subroutine for identifying measurement points. The control unit 11 of computer 1 obtains multiple reference measurement points in the corresponding template mask image from the template data DB 171 of the large-capacity storage unit 17 based on the template ID (step S21). The control unit 11 obtains the maximum connected component of the measurement target area (for example, the body of a cow) in the template mask image (step S22).

[0071] The control unit 11 obtains the coordinate values ​​of one reference measurement point from a plurality of acquired reference measurement points (step S23). The control unit 11 determines the value of the first coordinate axis (e.g., the X-axis) of the measurement point to be specified (step S24). Specifically, the control unit 11 sets the value of the first coordinate axis of the acquired reference measurement point to the value of the first coordinate axis of the measurement point.

[0072] The control unit 11 obtains the maximum and minimum values ​​of the second coordinate axis (e.g., Y axis) of the maximum connected component of the measurement target area of ​​the template mask image, corresponding to the value of the first coordinate axis (e.g., X axis) of the reference measurement point (step S25). Based on the value of the second coordinate axis of the reference measurement point and the obtained maximum and minimum values ​​of the second coordinate axis of the maximum connected component, the control unit 11 calculates the division ratio at the reference measurement point (step S26).

[0073] The control unit 11 determines the value of the second coordinate axis of the measurement point to be identified based on the calculated division ratio (step S27). Specifically, the control unit 11 obtains the maximum and minimum values ​​of the second coordinate axis of the maximum connected component of the measurement target area of ​​the mask image, corresponding to the value of the first coordinate axis of the measurement point to be identified, which was determined in the processing of step S24. The control unit 11 determines the value of the second coordinate axis of the measurement point to be identified based on the calculated division ratio and the obtained maximum and minimum values ​​of the second coordinate axis of the maximum connected component of the measurement target area of ​​the mask image.

[0074] The control unit 11 determines whether a given reference measurement point is the last reference measurement point from among multiple reference measurement points in the template mask image (step S28). If the control unit 11 determines that the given reference measurement point is not the last reference measurement point (NO in step S28), it returns to the process in step S23. If the control unit 11 determines that the given reference measurement point is the last reference measurement point (YES in step S28), it outputs the multiple identified measurement points (step S29). The control unit 11 terminates the subroutine for identifying measurement points and returns.

[0075] According to this embodiment, it is possible to identify (automatically extract) multiple measurement points based on image data of the object and template data of the measurement area of ​​the object.

[0076] According to this embodiment, it becomes possible to superimpose and display multiple identified measurement points onto the image data of the object.

[0077] According to this embodiment, the efficiency of measuring the body size or dimensions of an object is improved by automatically extracting multiple measurement points.

[0078] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the invention is indicated by the claims, not in the sense described above, and all modifications are intended to be in the sense and scope equivalent to the claims.

[0079] The matters described in each embodiment can be combined with each other. Furthermore, the independent and dependent claims described in the claims can be combined with each other in any combination, regardless of the form of reference. In addition, the claims use a form in which claims referencing two or more other claims (multi-claim form), but are not limited to this. A form in which multi-claims referencing at least one multi-claim (multi-multi-claim) may also be used. [Explanation of symbols]

[0080] 1. Information processing equipment (computer) 11 Control Unit 12 Storage section 13 Communications Department 14 Input section 15 Display 16 Reading section 17 Mass storage 171 Template Data DB 172 Measurement result DB 1a Portable storage medium 1b Semiconductor memory 1P Control Program

Claims

1. Acquire image data of the object, From the acquired image data, segmentation data of the measurement target area of ​​the object is extracted. The template data of the measurement target area and multiple reference measurement points of the object in the template data are acquired. Based on the acquired template data and multiple reference measurement points, multiple measurement points in the extracted segmentation data are identified. Based on the values ​​of the first coordinate axis of the aforementioned reference measurement point, the values ​​of the first coordinate axis of the measurement point are determined. The division ratio is calculated based on the values ​​of the second coordinate axis of the aforementioned reference measurement point and the maximum and minimum values ​​of the second coordinate axis of the template data. Based on the calculated division ratio, the value of the second coordinate axis of the measurement point is determined. A program that instructs a computer to perform a process.

2. Based on the template data of the measurement target area, adjust the size of the segmentation data. The program according to claim 1.

3. Three-dimensional data is acquired by a depth sensor that measures depth information to the aforementioned object. The segmentation data is corrected based on the acquired 3D data. The program according to claim 1 or 2.

4. The identified multiple measurement points are superimposed onto the image data of the object and displayed. The program according to claim 1 or 2.

5. Acquire image data of the object, From the acquired image data, segmentation data of the measurement target area of ​​the object is extracted. The template data of the measurement target area and multiple reference measurement points of the object in the template data are acquired. Based on the acquired template data and multiple reference measurement points, multiple measurement points in the extracted segmentation data are identified. Based on the values ​​of the first coordinate axis of the aforementioned reference measurement point, the values ​​of the first coordinate axis of the measurement point are determined. The division ratio is calculated based on the values ​​of the second coordinate axis of the aforementioned reference measurement point and the maximum and minimum values ​​of the second coordinate axis of the template data. Based on the calculated division ratio, the value of the second coordinate axis of the measurement point is determined. An information processing method that involves having a computer perform a task.

6. An information processing apparatus comprising a control unit, The control unit, Acquire image data of the object, From the acquired image data, segmentation data of the measurement target area of ​​the object is extracted. The template data of the measurement target area and multiple reference measurement points of the object in the template data are acquired. Based on the acquired template data and multiple reference measurement points, multiple measurement points in the extracted segmentation data are identified. Based on the values ​​of the first coordinate axis of the aforementioned reference measurement point, the values ​​of the first coordinate axis of the measurement point are determined. The division ratio is calculated based on the values ​​of the second coordinate axis of the aforementioned reference measurement point and the maximum and minimum values ​​of the second coordinate axis of the template data. Based on the calculated division ratio, the value of the second coordinate axis of the measurement point is determined. Information processing device.

7. Obtain image data of the object, From the acquired image data, segmentation data of the measurement target area of ​​the object is extracted. From the regions with continuity between pixels in the extracted segmentation data, a rectangle enclosing the measurement target region is obtained based on the largest connected component. The template data of the measurement target area and multiple reference measurement points of the object in the template data are acquired. Based on the size of the rectangle obtained based on the maximum connected component of the measurement target area in the template data, the size of the rectangle surrounding the measurement target area in the acquired segmentation data is adjusted. In the adjusted segmentation data, multiple measurement points corresponding to the reference measurement point are identified. A program that instructs a computer to perform a process.

Images