A three-dimensional modeling method, device and storage medium

Abstract

The application discloses a three-dimensional modeling method, comprising the following steps: acquiring a two-dimensional image of a living organism based on a camera module and a light source; acquiring a coordinate parameter corresponding to the two-dimensional image; determining a three-dimensional model of the living organism based on the coordinate parameter; the application also discloses a three-dimensional modeling device and a storage medium; through the three-dimensional modeling method, device and storage medium, the accuracy and efficiency of three-dimensional sign detection of the living organism are improved.

Description

Technical Field

[0001] This application relates to the field of 3D modeling technology, and in particular to a 3D modeling method, device and storage medium. Background Technology

[0002] In the aquaculture industry, monitoring various vital signs of live organisms is of great significance for risk control and improving economic efficiency. In particular, the three-dimensional vital signs (body shape, posture, etc.) of live organisms are among the most important monitoring parameters. Three-dimensional vital signs are not only the direct application of external physical characteristics, but more importantly, they also contain secondary characteristic information related to three-dimensional body shape characteristics, such as the sub-health state represented by obesity.

[0003] For example, in pig breeding, the thickness of the pig's back tag is an important three-dimensional physical characteristic used to determine the pig's health and productivity. However, current technologies, such as B-scan ultrasonography or manual assessment of the animal's appearance, are not ideal in terms of efficiency and accuracy. Therefore, improving the accuracy and efficiency of three-dimensional physical characteristic detection in live animals is a technical problem that needs to be solved. Summary of the Invention

[0004] This application provides a three-dimensional modeling method, device, and storage medium, which can improve the accuracy and efficiency of three-dimensional vital sign detection of living organisms.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] In a first aspect, embodiments of this application provide a three-dimensional modeling method, including:

[0007] Two-dimensional images of living organisms are acquired using a camera module and a light source;

[0008] Obtain the coordinate parameters corresponding to the two-dimensional image;

[0009] The three-dimensional model of the living organism is determined based on the coordinate parameters.

[0010] In the above scheme, acquiring two-dimensional images of living organisms based on the camera module and light source includes:

[0011] The light source illuminates the living organism via an optical encoding component, forming a projected image;

[0012] The projected image is captured using the camera module to obtain the two-dimensional image.

[0013] In the above scheme, obtaining the coordinate parameters corresponding to the two-dimensional image includes:

[0014] The coordinate system is defined with the light source as the origin and the optical axis of the camera module as the polar axis.

[0015] Based on the coordinate system, obtain the coordinate parameters of the light rays in the two-dimensional image.

[0016] The identification light ray is determined based on an optical encoding component.

[0017] In the above scheme, determining the three-dimensional model of the living organism based on the coordinate parameters includes:

[0018] The coordinate parameters of the light rays in the two-dimensional image are used as the input of the first deep learning model, and the three-dimensional model of the living organism is determined based on the output of the first deep learning model.

[0019] In the above scheme, determining the three-dimensional model of the living organism based on the coordinate parameters includes:

[0020] Obtain the feature points of the two-dimensional image;

[0021] The spinal features of the living organism are determined based on the feature points;

[0022] Obtain the coordinate parameters of the spinal feature in the coordinate system;

[0023] The three-dimensional model of the living organism is determined based on the coordinate parameters of the spinal features.

[0024] In the above scheme, determining the spinal features of the living organism based on the feature points includes:

[0025] Taking the first feature point among the feature points as the vertex, draw a straight line in the direction opposite to the curve to which the first feature point belongs, deflected by a first angle in the normal direction of the curve to which the first feature point belongs.

[0026] If it is confirmed that the length of the line segment between the two intersection points of the straight line and the outline of the living organism is less than a first threshold, then the midpoint of the line segment is determined to be the spine point;

[0027] Alternatively, if it is confirmed that the length of the line segment between the two intersection points of the straight line and the outline of the living organism is greater than or equal to the first threshold, at least one ray is drawn from the first feature point as the endpoint to the opposite direction of the curve to which the first feature point belongs.

[0028] Then, the midpoint of the shortest line segment between the two intersection points of the at least one ray and the outline of the living organism is determined as the spine point;

[0029] The spinal features of the living organism are determined based on at least two vertebral points;

[0030] The spinal point refers to the spinal feature point of the living organism.

[0031] In the above scheme, determining the three-dimensional model of the living organism based on the coordinate parameters includes:

[0032] The coordinate parameters of the spinal feature are used as input to the second deep learning model, and the three-dimensional model of the living organism is determined based on the output of the second deep learning model.

[0033] In the above scheme, the light source is a radial light source, and the angle between the projection direction of the light source and the optical axis direction of the camera module is less than a set angle or parallel.

[0034] In the above scheme, the coordinate system is a polar coordinate system, and correspondingly, the coordinate parameters are polar coordinate parameters.

[0035] Secondly, embodiments of this application provide a three-dimensional modeling device, the device comprising:

[0036] The camera module and intervention module are used to acquire two-dimensional images of living organisms;

[0037] The acquisition module is used to acquire the coordinate parameters corresponding to the two-dimensional image;

[0038] The determination module is used to determine the three-dimensional model of the living organism based on the coordinate parameters.

[0039] Thirdly, embodiments of this application provide a storage medium storing an executable program, which, when executed by a processor, implements the three-dimensional modeling method executed by the aforementioned device.

[0040] Fourthly, embodiments of this application provide a three-dimensional modeling device, which enables a processor to execute the above-described three-dimensional modeling method.

[0041] The three-dimensional modeling method, device, and storage medium provided in this application acquire two-dimensional images of a living organism based on a camera module and a light source; acquire coordinate parameters corresponding to the two-dimensional images; and determine a three-dimensional model of the living organism based on the coordinate parameters. This can improve the accuracy and efficiency of three-dimensional vital sign detection of living organisms. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of an optional process for a three-dimensional modeling method provided in an embodiment of this application;

[0043] Figure 2 A schematic diagram of another optional process for the three-dimensional modeling method provided in the embodiments of this application;

[0044] Figure 3This is a schematic diagram of an optional structure of the 3D modeling device provided in an embodiment of this application;

[0045] Figure 4 This is an optional schematic diagram of the confirmation marker points provided in the embodiments of this application;

[0046] Figure 5 This is a schematic diagram of light rays illuminating a three-dimensional object through a rectangular grid, as provided in an embodiment of this application.

[0047] Figure 6 An optional schematic diagram of the identity pattern provided in an embodiment of this application;

[0048] Figure 7 A schematic diagram of another optional process for the three-dimensional modeling method provided in the embodiments of this application;

[0049] Figure 8 A schematic diagram of the light source provided in this application when it shines on a live pig through a rectangular grid;

[0050] Figure 9 This is a schematic diagram illustrating an application of the 3D modeling device provided in an embodiment of this application.

[0051] Figure 10 This is another application diagram of the 3D modeling device provided in the embodiments of this application;

[0052] Figure 11 This is an optional schematic diagram of obtaining spinal points provided in an embodiment of this application;

[0053] Figure 12 This is another optional schematic diagram for obtaining spinal points according to an embodiment of this application;

[0054] Figure 13 This is a schematic diagram of another optional structure of the 3D modeling device provided in the embodiments of this application. Detailed Implementation

[0055] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the scope of the present application.

[0056] In aquaculture, monitoring various vital signs of live organisms is crucial for risk control and improving economic efficiency. Among these, the three-dimensional vital signs (body shape, physique, etc.) of live organisms are one of the most important monitoring parameters. This is not only reflected in the direct application of external physical characteristics, but more importantly, in the secondary characteristic information related to the three-dimensional physical features, such as the sub-health state represented by obesity.

[0057] For example, in pig breeding, the thickness of the backfat is considered one of the important physical characteristics of breeding pigs and is widely used to guide the assessment of their health and production capacity. Traditionally, to accurately obtain backfat thickness, specialized equipment using ultrasound detection principles is used. This method is costly and inefficient, often leading to high testing costs or reduced testing frequency in large-scale farms. However, if the testing frequency is too low, the reference value of this physical characteristic will be significantly reduced.

[0058] Furthermore, in pig farming, there are highly experienced professionals who can determine a pig's backfat parameters based on its physical characteristics. However, because these are empirical parameters, on the one hand, such professionals are extremely scarce, and their experience is not easily standardized and replicated; on the other hand, even the condition of the personnel can affect the judgment results, and in serious cases, may mislead production.

[0059] With the development of computer and information technology, deep learning technology based on neural networks has been widely applied to complex modeling of image objects. The main types of problems solved by deep learning modeling are those where input and output objects have an inherent and clear relationship, but this relationship is highly complex, cannot be explicitly described, or the description process is extremely complicated. The computational power of computers allows for the acquisition of an empirical model. However, since it is an empirical model, it contains a certain probability of error. Even if it passes all existing tests, it still cannot guarantee its future accuracy.

[0060] Therefore, when using neural networks to build production models, in order to reduce the probability of uncertainty in the future applicability of the model, designing prior features for specific problem models and scenarios, and specifying or actively constructing related feature parameters will be the key to the value of the model.

[0061] Based on the problems existing in current methods for determining the three-dimensional physical characteristics of living organisms, this application proposes a three-dimensional modeling method that can solve the technical problems and shortcomings that cannot be solved in existing technical solutions.

[0062] Figure 1 A schematic diagram of an optional process for a three-dimensional modeling method provided in an embodiment of this application is shown, and the process will be explained step by step.

[0063] Step S101: Acquire a two-dimensional image of the living organism based on the camera module and the light source.

[0064] In some embodiments, a 3D modeling device (hereinafter referred to as the device) acquires two-dimensional images of a living organism based on a camera module and a light source included in the device.

[0065] In a specific implementation, the light source illuminates the living organism through the light encoding component to form a projected image; the camera module captures the projected image to obtain the two-dimensional image.

[0066] The two-dimensional image includes at least one of the following: the outline of the living organism, feature points on the outline of the living organism, and the marker rays formed by the light source after passing through the optical encoding component. The projected image includes the marker rays formed by the light source projecting onto the living organism under the intervention of the optical encoding component, and the outline of the living organism.

[0067] Optionally, if the camera module and the light source are directly above the living organism, the two-dimensional image includes at least one of the following: the top-view outline of the living organism, feature points on the top-view outline of the living organism, and the marker ray formed by the light source after passing through the light encoding component. The projected image includes the marker ray formed by the light source projecting onto the living organism under the intervention of the light encoding component and the top-view outline of the living organism.

[0068] Step S102: Obtain the coordinate parameters corresponding to the two-dimensional image.

[0069] In this embodiment, the polar coordinate system is used as an example for illustration. Those skilled in the art should understand that the technical solutions of this embodiment can also be achieved using other coordinate systems.

[0070] In some embodiments, the device establishes a polar coordinate system with the light source as the origin and the optical axis of the camera module as the polar axis; based on the polar coordinate system, it determines the polar coordinate parameters of the marker rays in the two-dimensional image within the polar coordinate system. The marker rays are determined based on an optical encoding component. The polar coordinate parameters corresponding to the two-dimensional image include at least one of the following: the polar coordinate parameters of the marker rays in the two-dimensional image within the polar coordinate system, the polar coordinate parameters of the marker points in the two-dimensional image within the polar coordinate system, and the polar coordinate parameters of the feature points of a living organism in the two-dimensional image within the polar coordinate system.

[0071] In practice, the light source illuminates the living organism through the light encoding component, and the marker rays in the projected image are not necessarily standard straight lines or standard curves. The marker rays, marker positions, and feature points in the projected image can be simply and directly represented by polar coordinates.

[0072] Optionally, the optical encoding component may further include an identification pattern, which can be distinguished from the shape features of general patterns, such as triangles, crosses, etc. Optionally, at least one identification point can be identified by the identification pattern, and the direction of the spinal feature of the living organism can be determined based on the polar coordinate parameters of the identification point.

[0073] Step S103: Determine the three-dimensional model of the living organism based on the polar coordinate parameters of the identified light rays.

[0074] In some embodiments, the device determines a three-dimensional model of the living organism based on polar coordinate parameters that identify light rays in the two-dimensional image.

[0075] In practice, the device can directly use the polar coordinate parameters of the light rays in the two-dimensional image as the input of the first deep learning model, and determine the three-dimensional model of the living organism based on the output of the first deep learning model.

[0076] Alternatively, in a specific implementation, the device acquires the polar coordinate parameters of the light rays corresponding to the pattern of the light coding component in the two-dimensional image; based on the polar coordinate parameters, it confirms the direction of the spinal feature of the living organism; based on the direction of the spinal feature of the living organism, it uses the polar coordinate parameters of the identified light rays in the two-dimensional image as the input of the first deep learning model, and determines the three-dimensional model of the living organism according to the output of the first deep learning model.

[0077] The first depth model can be a depth model trained by identifying rays, the polar coordinate parameters of the identifying rays, and the three-dimensional model corresponding to the polar coordinate parameters of the identifying rays.

[0078] Before training the deep learning model, confirming the orientation of the spine features of the living organism can confirm the orientation of the living organism in the two-dimensional image, making it easier to obtain the output of the first deep learning model more quickly.

[0079] Thus, the three-dimensional modeling method provided in this application acquires a two-dimensional image of a living organism based on a camera module and a light source; and determines the three-dimensional model of the living organism based on the polar coordinate parameters of the ray markers in the two-dimensional image. This technical solution for dynamically modeling living animals by combining ray markers and a polar coordinate system effectively solves the problem of dynamic synchronization in existing technologies, improving the accuracy and efficiency of three-dimensional vital sign detection of living organisms.

[0080] Figure 2 This paper illustrates another optional flowchart of the three-dimensional modeling method provided in the embodiments of this application, which will be explained step by step.

[0081] Step S201: Acquire a two-dimensional image of the living organism based on the camera module and the light source.

[0082] In some embodiments, a 3D modeling device (hereinafter referred to as the device) acquires two-dimensional images of a living organism based on a camera module and a light source included in the device.

[0083] In a specific implementation, the light source illuminates the living organism through the light encoding component to form a projected image; the camera module captures the projected image to obtain the two-dimensional image.

[0084] The two-dimensional image includes at least the outline of the living organism, feature points on the outline of the living organism, and the corresponding light spot formed by the light source after passing through the light encoding component. The projected image includes the light pattern formed by the light source projecting onto the living organism under the intervention of the light encoding component and the outline of the living organism.

[0085] Optionally, if the camera module and the light source are directly above the living organism, the two-dimensional image can be a two-dimensional top-view image; the two-dimensional image includes at least one of the top-view outline of the living organism, feature points on the top-view outline of the living organism, and the marker rays formed by the light source after passing through the light encoding component. The projected image includes the marker rays formed by the light source projecting onto the living organism under the intervention of the light encoding component and the top-view outline of the living organism.

[0086] Step S202: Obtain feature points from the two-dimensional image, and determine the spinal features of the living organism based on the feature points.

[0087] In some embodiments, the device acquires feature points from the two-dimensional image and determines the spinal features of the living organism based on the feature points. The feature points can be at least one point on the outline of the living organism in the two-dimensional image.

[0088] In specific implementation, the device takes the first feature point among the feature points as the vertex, and deflects by a first angle in the normal direction of the curve to which the first feature point belongs to draw a straight line in the direction opposite to the curve to which the first feature point belongs.

[0089] If the device determines that the length of the line segment between the two intersection points of the straight line and the outline of the living organism is less than a first threshold, then the midpoint of the line segment is determined to be the spine point;

[0090] Alternatively, if the device determines that the length of the line segment between the two intersection points of the straight line and the outline of the living organism is greater than or equal to the first threshold, at least one ray is drawn from the first feature point as the endpoint towards the opposite direction of the curve to which the first feature point belongs; then the midpoint of the shortest line segment between the two intersection points of the at least one ray and the outline of the living organism is determined as the spine point.

[0091] The spinal point refers to the spinal feature point of the living organism.

[0092] In some alternative embodiments, the device can also determine the spinal features of the living organism based on at least two spinal points.

[0093] Step S203: Determine the three-dimensional model of the living organism based on the spinal features.

[0094] In some embodiments, the device acquires the polar coordinate parameters corresponding to the spinal feature; based on the polar coordinate parameters corresponding to the spinal feature, it confirms the orientation of the spinal feature of the living organism.

[0095] In some embodiments, the device determines a three-dimensional model of the living organism based on the orientation of its spinal features, using the spinal features as input to a deep learning model, and based on the output of the deep learning model.

[0096] In other embodiments, the device uses the polar coordinate parameters of the spinal feature as input to a second deep learning model and determines a three-dimensional model of the living organism based on the output of the second deep learning model.

[0097] In some alternative embodiments, the device may also train a second deep learning model based on at least one spinal feature and a three-dimensional model of the body posture corresponding to the at least one spinal feature; and / or, the device may train a second deep learning model based on the polar coordinate parameters of at least one spinal feature and a three-dimensional model of the body posture corresponding to the polar coordinate parameters of the at least one spinal feature.

[0098] Optionally, the device can also train a second deep learning model based on the correspondence between different body postures and a standard body posture. The device can use the spinal features of the living organism as input to the second deep learning model, based on the orientation of the spinal features, and determine a three-dimensional model of the living organism's first body posture according to the output of the second deep learning model; then, based on the three-dimensional model of the first body posture, determine a three-dimensional model of the living organism's standard body posture.

[0099] Thus, the three-dimensional modeling method provided in this application acquires a two-dimensional image of a living organism based on a camera module and a light source; acquires feature points from the two-dimensional image; determines the spinal features of the living organism based on the feature points; and determines a three-dimensional model of the living organism based on the spinal features. This can improve the accuracy and efficiency of three-dimensional vital sign detection of living organisms.

[0100] Figure 3 A schematic diagram of an optional structure of the 3D modeling device provided in an embodiment of this application is shown, and will be described in terms of each part.

[0101] Figure 3The description only refers to the distance between the camera module 301 and the intervention module 302 directly above, but it does not mean that the camera module 301 and the intervention module 302 can only be directly above; optionally, the camera module 301 and the intervention module 302 can also be in front of, behind, to the left or to the right of the living organism.

[0102] pass Figure 3 The 3D modeling device 300 shown can acquire two-dimensional images of living organisms based on the camera module and light source in steps S101 and S201.

[0103] In some embodiments, the 3D modeling device 300 includes a camera module 301 and an intervention module 302.

[0104] The camera module 301 is used to acquire two-dimensional images and / or projected images.

[0105] The intervention module 302 includes a light source; the light source may be a radial point light source that can emit light information that the camera module 301 can recognize and acquire.

[0106] In some embodiments, the projection direction of the light source is consistent with or parallel to the optical axis of the camera module 301. Here, the projection direction of the light source and the optical axis of the camera module 301 are consistent, specifically meaning that the angle between the projection direction of the light source and the optical axis of the camera module 301 is less than a set angle, such as 0.1 degrees, 0.01 degrees, 0.05 degrees, 0.5 degrees, 0.8 degrees, etc. Of course, it is best if the projection direction of the light source and the optical axis of the camera module 301 are parallel.

[0107] In some alternative embodiments, the light information emitted by the light source is different from the ambient light information of the 3D modeling device 300; and / or, the light information emitted by the light source is different from the light information of the living organism. Optionally, the difference between the light information emitted by the light source and the ambient light information of the 3D modeling device 300 is achieved by using different colors of light information; and / or, the difference between the light information emitted by the light source and the light information of the living organism is achieved by using different colors of light information.

[0108] For example, the light source emits green laser light that does not exist in the environment.

[0109] In some optional embodiments, the intervention module 302 is further provided with a light encoding component in the light path projected by the light source. The light encoding component encodes the light emitted by the light source into a corresponding light spot and projects it out. Optionally, the light encoding component may be a light-shielding plate with a specific pattern.

[0110] To facilitate the description of the principle of the 3D modeling device 300, the specific pattern of the optical encoding component is a rectangular grid as an example.

[0111] Figure 3 In this process, when the light beam projected by the light source through the light encoding component is projected onto the plane directly below, it forms a rectangular grid light beam and a projected image of objects on the ground.

[0112] In some embodiments, the intervention module 302 further includes a coordinate system component, which may be a polar coordinate system. This polar coordinate system has a point light source as its origin and the optical axis of the camera module 301 as its polar axis, used to describe the angle of the light rays from the rectangular grid relative to the polar axis.

[0113] The coordinate system component can determine the polar coordinate angle parameters of each marker ray (e.g., edge marker ray and corner marker ray) projected by the light source through the light encoding component.

[0114] Figure 4 A schematic diagram of optional confirmation markers provided in an embodiment of this application is shown.

[0115] Figure 4 The image in the middle is the image acquired when the camera module 301 illuminates a planar object. The light generated by the light source through the rectangular grid is not distorted and is perpendicular or parallel to each other.

[0116] Assume the rectangular region formed by the light source and the rectangular grid, as acquired by the camera module, has a length of L and a width of M. Using horizontal intervals of l and vertical intervals of m to identify marker points, c*d feature points can be constructed within the rectangular region, i.e., feature point M. 11 To feature point M cd .

[0117] Wherein, d can be determined by d*m≤M≤(d+1)*m;

[0118] c can be determined by c*l≤L≤(c+1)*l.

[0119] The marker point is the intersection of the marker rays generated by the light source through the rectangular grid.

[0120] Figure 5 This illustration shows a schematic diagram of light rays illuminating a three-dimensional object through a rectangular grid, as provided in an embodiment of this application.

[0121] When the light source shines onto the plane through the rectangular grid, the resulting light rays are not distorted and are perpendicular or parallel to each other. When the light source shines onto a three-dimensional object through the rectangular grid, the shape of the resulting light rays changes, such as... Figure 5As shown, the original straight line is twisted differently according to the shape of the three-dimensional object. Under the intervention of the light encoding component, the light pattern formed by the light rays projected onto the three-dimensional object has a one-to-one correspondence with the three-dimensional features of the three-dimensional entity itself. Therefore, after obtaining the two-dimensional image, the three-dimensional model of the living organism can be determined based on the polar coordinate parameters of the light rays in the two-dimensional image (which can be achieved through step S102).

[0122] In some alternative embodiments, the optical encoding component may be a replaceable or adjustable structure (such as a mechanical adjustment structure), and the change in the polar coordinate parameters of the feature rays caused by the adjustment is output synchronously.

[0123] In specific implementation, the optical encoding component may include an identity pattern in addition to a general pattern, and the identity pattern may have shape features that distinguish it from the general pattern.

[0124] Figure 6 An optional schematic diagram of the identity pattern provided in an embodiment of this application is shown.

[0125] Figure 6 It includes a cross-shaped double-label identity map, which marks 12 identification points to determine the orientation of the spinal features of the living organism.

[0126] Optionally, the optical encoding component can be a transparent liquid crystal display component (e.g., a liquid crystal display without a backlight). The mature display control of the transparent liquid crystal display allows for arbitrary and flexible adjustment of the projected pattern and identity pattern. Moreover, since the control signal of the liquid crystal, combined with the relative position parameters of the liquid crystal and the light source, can directly solve for the polar coordinate parameters of any light ray, the active solution of the polar coordinates of the light ray is realized. Compared with passive measurement and solution, the amount of computation required to obtain prior parameters can be effectively reduced, and the accuracy is significantly improved.

[0127] Based on a standard 3D model, various interest parameters of a live pig can be obtained, and the data can be mapped to a target (health status, etc.) based on these interest parameters, realizing the economic value of modeling (such as mapping backfat thickness, which currently requires expensive ultrasound to measure). However, due to the living characteristics of the same live pig, it exhibits different states (changing postures) at different times (within seconds or even one second), which brings many problems to the data-to-target mapping. If a specific state is specified for modeling (such as standard standing posture modeling), it is likely necessary to wait for or filter image resources. Therefore, combining steps S101 to S102, steps S201 to S203, Figures 1 to 6The three-dimensional modeling method provided in this application can restore a two-dimensional image into a three-dimensional model based on the characteristic rays generated by the light source projected onto the living organism through the light encoding component, which are different from the ambient light. The technical solution of dynamically modeling living animals by combining the identification rays and the polar coordinate system can effectively solve the problem of dynamic synchronization in the prior art, and improve the accuracy and efficiency of three-dimensional vital sign detection of living organisms.

[0128] In the 3D modeling methods of related technologies, multiple 2D or 3D image information from various angles can be synthesized (3D images can be obtained by scanning depth information). Therefore, these modeling methods are more suitable for static objects. However, in confirming the vital signs of living organisms, due to the living nature of these organisms, 3D modeling methods struggle to achieve accurate modeling because living organisms change constantly over time, making it difficult to simultaneously acquire multiple 2D or 3D images from various angles.

[0129] The following description uses a live pig as an example. It should be understood that the live pig in this embodiment is merely an example and is not intended to limit this application. Optionally, the live organism may be a live organism belonging to the phylum Chordata or the phylum Arthropoda.

[0130] Figure 7 This illustration shows another optional flowchart of the three-dimensional modeling method provided in the embodiments of this application, which will be explained step by step.

[0131] Step S601: Acquire a two-dimensional image of a live pig based on the camera module and the light source.

[0132] Figure 8 This illustration shows a schematic diagram of the light emitted by a light source provided in this application when it shines on a live pig through a rectangular grid. Figure 9 This illustration shows an application diagram of the 3D modeling device provided in an embodiment of this application. Figure 10 This illustration shows another application diagram of the 3D modeling device provided in the embodiments of this application.

[0133] In some embodiments, a 3D modeling device (hereinafter referred to as the device) acquires two-dimensional images of a living organism based on a camera module and a light source included in the device.

[0134] In specific implementation, the light source illuminates the live pig via the optical encoding component to form a projected image (such as...). Figure 8 (As shown); the camera module captures the projected image to obtain the two-dimensional image. Optionally, the two-dimensional image can be a two-dimensional top-view image.

[0135] The two-dimensional image includes at least a top-view outline of the live pig, feature points on the top-view outline of the live pig, and a corresponding light spot formed by the light source after passing through the light encoding component. The projected image includes a light pattern formed by the light source projecting onto the live pig under the intervention of the light encoding component and a top-view outline of the live pig.

[0136] Step S602: Obtain feature points from the two-dimensional image, and determine the spinal features of the live pig based on the feature points.

[0137] In some embodiments, the device acquires feature points from the two-dimensional image and determines the spinal features of the live pig based on the feature points. The feature points in the two-dimensional image are feature points on the top-view outline of the live pig. The feature points can be randomly acquired from the top-view outline of the live pig.

[0138] In specific implementation, the device takes the first feature point among the feature points as the vertex, and deflects by a first angle in the normal direction of the curve to which the first feature point belongs to draw a straight line in the direction opposite to the curve to which the first feature point belongs.

[0139] If the device determines that the length of the line segment between the two intersection points of the straight line and the outline of the live pig is less than a first threshold, then the midpoint of the line segment is determined to be the spine point.

[0140] Figure 11 This illustration shows an optional schematic diagram of obtaining spinal points according to an embodiment of this application.

[0141] like Figure 11 As shown, the first feature point is point C. Taking point C as the vertex, a straight line is drawn in the direction opposite to the curve with the normal direction of the curve to which point C belongs, deflected by a first angle (for example, if the curve is the outline of the right side of the live pig, then a straight line is drawn in the direction to the left side of the live pig). If it is confirmed that the length of the line segment between OC is less than the first threshold, then the midpoint of OC is determined to be the spine point.

[0142] The first threshold can be set according to actual needs; the first angle can be set according to actual needs, such as 1°, 2°, 3°, 4°, 5°, 10°, 15°, etc.

[0143] Alternatively, if the device determines that the length of the line segment between the two intersection points of the straight line and the outline of the living organism is greater than or equal to the first threshold, at least one ray is drawn from the first feature point as the endpoint towards the opposite direction of the curve to which the first feature point belongs; then the midpoint of the shortest line segment between the two intersection points of the at least one ray and the outline of the living organism is determined as the spine point.

[0144] Figure 12This illustration shows another optional schematic diagram of obtaining the spinal point according to an embodiment of this application.

[0145] Figure 12 In the middle, point D is discontinuous with the edge curve. If we follow... Figure 11 The method shown determines the spinal point, then as follows: Figure 12 As shown by the thick solid line, the length of the obtained line segment is greater than the first threshold, and the determined spine point is inaccurate. Therefore, at least one ray is drawn from point D as the endpoint in the direction opposite to the curve to which point D belongs.

[0146] For example, if the curve represents the outline of the right side of the live pig, then at least one ray is drawn towards the left side of the live pig. The at least one ray is as follows: Figure 12 As shown by the medium-thick dashed line, the midpoint of the shortest line segment between the two intersection points of the at least one ray and the outline of the living organism is determined as the spine point.

[0147] The spinal point refers to the characteristic point of the spine of the live pig.

[0148] In some alternative embodiments, the device can also determine the spinal features of the live pig based on at least two spinal points.

[0149] Step S603: Determine a three-dimensional model of the live pig based on the spinal features.

[0150] In some embodiments, the device acquires the polar coordinate parameters corresponding to the spinal feature; based on the polar coordinate parameters corresponding to the spinal feature, it confirms the orientation of the spinal feature of the living organism.

[0151] In some embodiments, the device determines a three-dimensional model of the live pig based on the orientation of the spinal features of the live pig, using the spinal features as input to a second deep learning model, and based on the output of the second deep learning model.

[0152] In some alternative embodiments, the device may also acquire at least one spinal feature and a three-dimensional model of the body posture corresponding to the at least one spinal feature, and train a second deep learning model.

[0153] Optionally, the device can also train a second deep learning model based on the correspondence between different body postures and the standard body posture. The device can determine a three-dimensional model of the first body posture of the live pig based on the orientation of its spinal features, using these features as input to the deep learning model, and then determine a three-dimensional model of the standard body posture of the live pig based on the output of the deep learning model; then, based on the three-dimensional model of the first body posture, it can determine a three-dimensional model of the standard body posture of the live pig.

[0154] Typically, a standard 3D model can be used to obtain various interest parameters of a live pig and map the data to a target (health status, etc.) based on these parameters, realizing the economic value of modeling (e.g., mapping backfat thickness, which currently requires expensive ultrasound measurement). However, due to the living nature of the same live pig, it exhibits different states (changing postures) at different times (even within a second), which brings many problems to the data-to-target mapping. If modeling is performed for a specific state (e.g., standard standing posture modeling), it is highly likely that image resources will need to be waited for or filtered. To address this, the 3D modeling device provided in this application embodiment can acquire a projected image formed by a light source illuminating the live pig through a light encoding component; the camera module captures the projected image to obtain the 2D image. The spinal features of the live pig are acquired, and a mapping relationship library from complete spinal features to the 3D model of the live pig is constructed based on the curvature variation characteristics of the spinal curve. That is, for the same live pig, the model in the standard standing posture is X. Since different postures correspond to different spinal curvature features (which can basically show a one-to-one correspondence in the bodies of single-vertebrate animals such as pigs), n different posture models X1-X of the same live pig can be obtained through the above mapping relationship. n Simultaneously, the device can, based on the constructed model mapping relationship, restore the three-dimensional solid model of a live pig and the spinal curvature feature model obtained at any time to a three-dimensional model of the live pig in its standard posture (or interest posture), so as to realize and simplify the establishment of the "data-to-target mapping" relationship described at the beginning of this paragraph.

[0155] Figure 13 A schematic diagram of another optional structure of the 3D modeling device provided in the embodiments of this application is shown, and will be described in terms of each part.

[0156] In some embodiments, the three-dimensional modeling device 800 includes: a camera module 801, an intervention module 802, an acquisition module 803, and a determination module 804.

[0157] The camera module 801 and the intervention module 802 are used to acquire two-dimensional images of living organisms;

[0158] The acquisition module 803 is used to acquire the coordinate parameters corresponding to the two-dimensional image;

[0159] The determining module 804 is used to determine the three-dimensional model of the living organism based on the coordinate parameters.

[0160] In some embodiments, the 3D modeling device 800 may further include: a modeling module 805.

[0161] The establishment module 805 is used to establish a coordinate system with the light source as the origin and the optical axis of the camera module 801 as the polar axis.

[0162] The determining module 804 is used to confirm the coordinate parameters of the light rays in the two-dimensional image based on the coordinate system.

[0163] The camera module 801 and the intervention module 802 are specifically used to illuminate the living organism through the light source via the light encoding component to form a projected image; and to capture the projected image using the camera module 801 to obtain the two-dimensional image.

[0164] In some embodiments, the determining module 804 is specifically used to use the coordinate parameters of the light rays in the two-dimensional image as input to the first deep learning model, and to determine the three-dimensional model of the living organism based on the output of the first deep learning model.

[0165] In other embodiments, the determining module 804 is specifically used to acquire feature points of the two-dimensional image; determine the spinal features of the living organism based on the feature points; acquire the coordinate parameters corresponding to the spinal features; and determine the three-dimensional model of the living organism based on the coordinate parameters of the spinal features.

[0166] The determining module 804 is specifically used to draw a straight line in the direction opposite to the curve to which the first feature point belongs, with the first feature point as the vertex and the normal direction of the curve to which the first feature point belongs deflected by a first angle.

[0167] If it is confirmed that the length of the line segment between the two intersection points of the straight line and the outline of the living organism is less than a first threshold, then the midpoint of the line segment is determined to be the spine point;

[0168] Alternatively, if it is confirmed that the length of the line segment between the two intersection points of the straight line and the outline of the living organism is greater than or equal to the first threshold, at least one ray is drawn from the first feature point as the endpoint to the opposite direction of the curve to which the first feature point belongs.

[0169] Then, the midpoint of the shortest line segment between the two intersection points of the at least one ray and the outline of the living organism is determined as the spine point;

[0170] The spinal features of the living organism are determined based on at least two vertebral points;

[0171] The spinal point refers to the spinal feature point of the living organism.

[0172] The determining module 804 is specifically used to determine the three-dimensional model of the living organism by using the coordinate parameters of the spinal feature as input to the second deep learning model and based on the output of the second deep learning model.

[0173] In some embodiments, the light source is a radial light source, and the angle between the projection direction of the light source and the optical axis direction of the camera module is less than a set angle or parallel.

[0174] In some embodiments, the coordinate system is a polar coordinate system, and correspondingly, the coordinate parameters are polar coordinate parameters.

[0175] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program commands. The aforementioned program can be stored in a storage medium, including various media capable of storing program code such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0176] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several commands to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0177] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A three-dimensional modeling method, characterized in that, The method includes: Two-dimensional images of living organisms are acquired using a camera module and a light source; Obtain the coordinate parameters corresponding to the two-dimensional image; The three-dimensional model of the living organism is determined based on the coordinate parameters; The step of determining the three-dimensional model of the living organism based on the coordinate parameters includes: Obtain the feature points of the two-dimensional image; The spinal features of the living organism are determined based on the feature points; Obtain the coordinate parameters of the spinal feature in the coordinate system; Based on the coordinate parameters of the spinal features, the orientation of the spinal features of the living organism is determined; Based on the orientation of the spinal features of the living organism, the coordinate parameters of the spinal features are used as input to a second deep learning model. The three-dimensional model of the first body posture of the living organism is determined according to the output of the second deep learning model. Based on the three-dimensional model of the first body posture, the three-dimensional model of the standard body posture of the living organism is determined.

2. The method according to claim 1, characterized in that, The acquisition of two-dimensional images of living organisms based on the camera module and light source includes: The light source illuminates the living organism via an optical encoding component, forming a projected image; The projected image is captured using the camera module to obtain the two-dimensional image.

3. The method according to claim 1 or 2, characterized in that, The step of obtaining the coordinate parameters corresponding to the two-dimensional image includes: The coordinate system is defined with the light source as the origin and the optical axis of the camera module as the polar axis. Based on the coordinate system, obtain the coordinate parameters of the marker rays in the two-dimensional image within the coordinate system; The identification light ray is determined based on the optical encoding component.

4. The method according to claim 3, characterized in that, The process of determining the three-dimensional model of the living organism based on the coordinate parameters includes: The coordinate parameters of the light rays in the two-dimensional image are used as the input of the first deep learning model, and the three-dimensional model of the living organism is determined based on the output of the first deep learning model.

5. The method according to claim 1, characterized in that, Determining the spinal features of the living organism based on the feature points includes: Taking the first feature point among the feature points as the vertex, draw a straight line in the direction opposite to the curve to which the first feature point belongs, deflected by a first angle in the normal direction of the curve to which the first feature point belongs. If it is confirmed that the length of the line segment between the two intersection points of the straight line and the outline of the living organism is less than a first threshold, then the midpoint of the line segment is determined to be the spine point; Alternatively, if it is confirmed that the length of the line segment between the two intersection points of the straight line and the outline of the living organism is greater than or equal to the first threshold, at least one ray is drawn from the first feature point as the endpoint to the opposite direction of the curve to which the first feature point belongs. Then, the midpoint of the shortest line segment between the two intersection points of the at least one ray and the outline of the living organism is determined as the spine point; The spinal features of the living organism are determined based on at least two vertebral points. Wherein, the spine point is the characteristic point of the spine of the living organism; when the curve is the outline of the right side of the living organism, the relative direction of the curve is the left side of the living organism; when the curve is the outline of the left side of the living organism, the relative direction of the curve is the right side of the living organism.

6. The method according to claim 1 or 2, characterized in that, The light source is a radial light source, and the angle between the projection direction of the light source and the optical axis direction of the camera module is less than a set angle or parallel.

7. The method according to claim 1 or 2, characterized in that, The coordinate system is a polar coordinate system, and correspondingly, the coordinate parameters are polar coordinate parameters.

8. A three-dimensional modeling device, characterized in that, The device includes: The camera module and light source module are used to acquire two-dimensional images of living organisms; The acquisition module is used to acquire the coordinate parameters corresponding to the two-dimensional image; A determination module is used to determine the three-dimensional model of the living organism based on the coordinate parameters; The determining module is specifically used for: Obtain the feature points of the two-dimensional image; The spinal features of the living organism are determined based on the feature points; Obtain the coordinate parameters of the spinal feature in the coordinate system; Based on the coordinate parameters of the spinal features, the orientation of the spinal features of the living organism is determined; Based on the orientation of the spinal features of the living organism, the coordinate parameters of the spinal features are used as input to a second deep learning model. The three-dimensional model of the first body posture of the living organism is determined according to the output of the second deep learning model. Based on the three-dimensional model of the first body posture, the three-dimensional model of the standard body posture of the living organism is determined.

9. A storage medium storing an executable program, characterized in that, When the executable program is executed by the processor, it implements the three-dimensional modeling method according to any one of claims 1 to 7.

10. A three-dimensional modeling device, comprising a memory, a processor, and an executable program stored in the memory and executable by the processor, characterized in that, When the processor runs the executable program, it performs the steps of the three-dimensional modeling method as described in any one of claims 1 to 7.

Images