A method and device for detecting posture features of a human back based on images

Abstract

The application relates to the field of scoliosis detection, in particular to a detection method and device for human back posture features based on images, which comprises the following steps: acquiring a back color image; inputting the back color image into a preset feature recognition network model to output coordinates of an even number of back feature points; processing the back color image to obtain a back contour image; fusing the coordinates of the back feature points into the back contour image to determine the position corresponding relationship between each back feature point and each target part in the back contour image, and determining whether target parts of the same category are isohypse by the position corresponding relationship to obtain a back posture detection result of a detected person. The technical scheme of the application is radiation-free throughout the whole process, high in efficiency, and suitable for large-scale and rapid detection of the back scoliosis of a human body.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of scoliosis detection, and in particular to a method and device for detecting human back posture features based on images. BACKGROUND

[0002] Human back posture features are important indicators of scoliosis. According to surveys, scoliosis is more common in children and adolescents. There are currently more than 3 million scoliosis patients, and the number is increasing by 300,000 people per year. The national policy clearly indicates that attention and protection of the healthy development of primary and secondary school students' spines will include scoliosis examination in the health examination of primary and secondary school students. Therefore, it is urgent to use artificial intelligence technology to detect human back posture features to determine the scoliosis condition.

[0003] The representation of human scoliosis on the back mainly includes four aspects: unequal height of both shoulders, unequal height of both scapula corners, asymmetric bilateral waist concave, and unequal height of both iliac crests. Currently, the detection of scoliosis in the physical examination of primary and secondary school students is mainly through professional detection personnel using a scoliosis measuring instrument for scoliosis screening. This method is low in efficiency and requires high skill and proficiency of the detection personnel, so the error of non-professional measurement is large.

[0004] X-ray is low in cost, simple and easy to operate, and accurate in measurement, but it is harmful to the human body of the target primary and secondary school population; EOS is high in cost and still has a small amount of radiation; ultrasound requires high skill of the doctor and has large error, which is not conducive to large-scale screening; and the cloud pattern can only roughly determine the situation of the target. In the present application, the feature point detection algorithm based on deep learning can take a back image of the naked back to calculate the key point position of the current target, and obtain the prediction result of the human posture and scoliosis condition according to the relative position data of the key points. This method realizes a zero-radiation, healthy detection process, and has fast detection speed and can be applied to large-scale screening such as physical examination. SUMMARY

[0005] In view of the problems, the present application is proposed to provide a method and device for detecting human back posture features based on images to overcome the problems or at least partially solve the problems.

[0006] A method for detecting human back posture features based on images, the method comprising:

[0007] obtaining a back color image of a detected person;

[0008] inputting the back color image into a preset feature recognition network, and outputting coordinates of an even number of back feature points from the feature recognition network;

[0009] processing the back color image according to a preset image processing algorithm to obtain a back contour image;

[0010] fusing the coordinates of the even number of back feature points into the back contour image to determine a position correspondence relationship between each back feature point and each target part in the back contour image, and determining whether target parts of the same category are isohypseal according to the position correspondence relationship.

[0011] Preferably, the inputting of the back color image into the preset feature recognition network comprises:

[0012] The feature recognition network is trained in advance, comprising:

[0013] Collecting a back sample image and labeling back sample parts in the back sample image;

[0014] Training the feature recognition network according to the labeled back sample image and using a ResNet50 network structure;

[0015] Verifying the output result of the feature recognition network through a test set, and specifically, when the feature point position output by the feature recognition network is basically consistent with the coordinates based on the back sample image, the training of the feature recognition network is completed.

[0016] Preferably, after the training of the feature recognition network is completed, the method further comprises:

[0017] Initializing the weights of the trained feature recognition network using an Xavier network weight initialization method.

[0018] Preferably, the collecting of the back sample image further comprises:

[0019] Collecting a back sample image set of different body types;

[0020] Performing data normalization processing on the back sample image set using empirical mean and variance data obtained from an ImageNet data set.

[0021] A device for detecting human back posture features based on images, the device comprising:

[0022] An acquisition module configured to acquire a back color image of a detected person;

[0023] An identification module configured to input the back color image into a preset feature recognition network and output coordinates of an even number of back feature points from the feature recognition network;

[0024] A processing module configured to process the back color image according to a preset image processing algorithm to obtain a back contour image;

[0025] a judging module, configured to fuse the coordinates of the even number of back feature points into the back contour image, determine a position correspondence between each of the back feature points and each target part in the back contour image, and determine whether target parts of the same category are of the same height according to the position correspondence.

[0026] Preferably, the recognition module comprises:

[0027] a training submodule, configured to pre-train the feature recognition network, comprising:

[0028] collecting a back sample image, and labeling a back sample part in the back sample image;

[0029] training the feature recognition network according to the labeled back sample image and using a ResNet50 network structure;

[0030] verifying an output result of the feature recognition network by using a test set, and specifically, when a feature point position output by the feature recognition network is correct based on coordinates in the back sample image, the training of the feature recognition network is completed.

[0031] Preferably, the training submodule comprises:

[0032] an initialization unit, configured to perform weight initialization on the trained feature recognition network by using a Xavier network weight initialization method.

[0033] Preferably, the training submodule further comprises

[0034] a sample collection unit, configured to collect a back sample image set of different body types;

[0035] a data processing unit, configured to perform data normalization processing on the back sample image set by using empirical mean and variance data derived from an ImageNet data set.

[0036] An apparatus comprising a processor, a memory, and a computer program stored on the memory and capable of running on the processor, the computer program being executed by the processor to implement the steps of the method for detecting a human back posture feature based on an image as described above.

[0037] A computer readable storage medium, the computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the steps of the method for detecting a human back posture feature based on an image as described above.

[0038] The present application has the following advantages:

[0039] In the embodiment of the present application, a back color image of a detected person is acquired; the back color image is input into a preset feature recognition network, and coordinates of an even number of back feature points are output by the feature recognition network; the back color image is processed according to a preset image processing algorithm to obtain a back contour image; the coordinates of the even number of back feature points are fused into the back contour image to determine a position correspondence relationship between each back feature point and each target part in the back contour image, and whether target parts of the same category are isometric is determined according to the position correspondence relationship to obtain a back posture detection result of the detected person. The technical scheme of the present application can obtain the back posture detection result of the detected person through image processing, is free of radiation throughout the process, is high in efficiency, and is suitable for large-scale and rapid detection of human back scoliosis. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical scheme of the present application, the drawings required to be used in the description of the present application will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and all other drawings obtained by those skilled in the art without creative labor on the basis of these drawings also belong to the protection scope of the present application.

[0041] Figure 1 A step flowchart of a back posture feature detection method based on images is shown;

[0042] Figure 2 A human back part schematic diagram is shown;

[0043] Figure 3 A structure schematic diagram of a back posture feature detection device based on images is shown;

[0044] Figure 4 A structure schematic diagram of a computer device of a back posture feature detection method based on images is shown. DETAILED DESCRIPTION

[0045] In order to make the purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor also belong to the protection scope of the present application.

[0046] Please refer to Figure 1 , a step flowchart of a back posture feature detection method based on images is shown, and the method comprises the following steps:

[0047] S110, acquire a back color image of the detected person;

[0048] S120, input the back color image into a preset feature recognition network, and output coordinates of even-numbered back feature points by the feature recognition network;

[0049] S130, process the back color image according to a preset image processing algorithm to obtain a back contour image;

[0050] S140, fuse the coordinates of the even-numbered back feature points into the back contour image, determine a position correspondence relationship between each back feature point and each target part in the back contour image, and determine whether target parts of the same category are isohypse by the position correspondence relationship.

[0051] In the embodiment of the present application, the back color image of the detected person is acquired, the back color image is input into a preset feature recognition network, and the coordinates of even-numbered back feature points are output by the feature recognition network. The back color image is processed according to a preset image processing algorithm to obtain a back contour image. The coordinates of the even-numbered back feature points are fused into the back contour image, a position correspondence relationship between each back feature point and each target part in the back contour image is determined, and whether target parts of the same category are isohypse is determined by the position correspondence relationship. The back color image is feature-extracted to obtain the coordinates of the back feature points in the back color image, the back contour image of the back color image is extracted, and then the back feature points are determined to correspond to the target parts of the back, so as to determine whether target parts of the same category are isohypse, and the back posture detection result of the detected person is obtained. The back posture detection result of the detected person can be obtained by image processing, and the whole process is non-radiation, high efficiency, and suitable for large-scale rapid detection of human back scoliosis.

[0052] In the following, the above-mentioned image-based human back posture feature detection method will be further described through the following embodiments.

[0053] As described in step S110, the back color image of the detected person is acquired.

[0054] It should be noted that in the embodiment of the present application, a Kinect 2.0 color camera is used for image acquisition. When collecting, the detected person needs to expose the back, the arms are naturally lowered and vertically located in front of the camera, and the back sagittal plane is parallel to the imaging plane of the camera.

[0055] Specifically, the Kinect 2.0 color camera is installed at a fixed position, and a black curtain is vertically hung in front of the camera to reduce the influence of environmental noise on shooting. The person to be detected is located 70 cm in front of the color camera plane, and the back is parallel to the imaging plane of the camera. After standing stably, the camera shooting button is clicked to obtain a back color image with a pixel of 1920*1080. Then, the image is cropped to contain only the black curtain area, and then scaled to make the image pixel input into the subsequent feature recognition network to be 256*256.

[0056] As described in step S120, the back color image is input into the preset feature recognition network, and the coordinates of an even number of back feature points are output by the feature recognition network.

[0057] In an embodiment of the present application, the specific process of "inputting the back color image into the preset feature recognition network" in step S120 can be further described in combination with the following description.

[0058] As described in the following steps:

[0059] The feature recognition network is trained in advance, including:

[0060] Collecting a back sample image, and labeling a back sample part in the back sample image;

[0061] Training the feature recognition network according to the labeled back sample image and using a ResNet50 network structure;

[0062] Verifying the output result of the feature recognition network by a test set. Specifically, when the feature point position output by the feature recognition network is basically consistent with the coordinates in the back sample image, the training of the feature recognition network is completed.

[0063] It should be noted that the ResNet50 network structure, i.e., the deep residual network, uses dozens of convolutional layers to train the feature recognition network to solve the problem of difficult training and low precision of the model.

[0064] In an embodiment of the present application, after the training of the feature recognition network is completed, it further includes:

[0065] The trained feature recognition network is initialized by using the Xavier network weight initialization method.

[0066] Specifically, the weight initialization uses the following calculation formula:

[0067]

[0068] where W is the network weight, n jThe number of output neurons for each layer.

[0069] It should be noted that the formula is applicable to the case with activation function, such as Sigmoid, Tanh, etc. The Xavier network weight initialization method ensures that the gradient of each layer maintains approximate variance, which will allow information to flow smoothly in the reverse direction to update the weights.

[0070] In an embodiment of the present application, the back sample image is collected, comprising:

[0071] Collecting a back sample image set of different body types;

[0072] Using the empirical mean and variance data obtained from the ImageNet dataset to perform data normalization processing on the back sample image set.

[0073] As described in step S140, the coordinates of the even number of back feature points are fused into the back contour image to determine the position correspondence relationship between each back feature point and each target part in the back contour image, and whether the target parts of the same category are isometric is determined according to the position correspondence relationship.

[0074] It should be noted that since the back feature points and the back contour image are both based on the same back color image, the positions of the key back parts can be known from the back contour image, for example Figure 2 As shown in the figure, the left and right two parts of the same category are shown at the shoulders of the detected person, the lower corners of the bilateral scapula, the bilateral waist concave, and the bilateral iliac crest. By fusing the coordinates of all the back feature points into the back contour image, the specific coordinates of each part are obtained; and the conclusion of whether the same category, such as the shoulders, is isometric, is further determined to determine whether the back posture of the detected person, such as the spine, is laterally curved.

[0075] For the device embodiment, it is basically similar to the method embodiment, so the description is relatively simple, and the related parts are referred to the part of the method embodiment.

[0076] Referring to Figure 3 , a structure schematic diagram of a human back posture feature detection device based on image is shown, which is provided by an embodiment of the present application;

[0077] The device comprises:

[0078] The acquisition module 110 is configured to acquire a back color image of a detected person;

[0079] The recognition module 120 is configured to input the back color image into a preset feature recognition network, and output coordinates of an even number of back feature points from the feature recognition network;

[0080] The processing module 130 is configured to process the back color image according to a preset image processing algorithm to obtain a back contour image.

[0081] The judging module 140 is configured to fuse the coordinates of the even number of back feature points into the back contour image, determine a position corresponding relationship between each back feature point and each target part in the back contour image, and determine whether target parts of the same category are of the same height according to the position corresponding relationship.

[0082] In an embodiment of the present application, the identifying module comprises:

[0083] The training submodule is configured to pre-train the feature recognition network, and the training comprises:

[0084] Collecting a back sample image and labeling a back sample part in the back sample image;

[0085] Training the feature recognition network according to the labeled back sample image and using a ResNet50 network structure;

[0086] Verifying the output result of the feature recognition network through a test set, and specifically, when the position of the feature point output by the feature recognition network is basically consistent with the coordinates in the back sample image, the training of the feature recognition network is completed.

[0087] In an embodiment of the present application, the training submodule comprises:

[0088] The initialization unit is configured to perform weight initialization on the feature recognition network that has been trained by using a Xavier network weight initialization method.

[0089] In an embodiment of the present application, the training submodule further comprises

[0090] The sample collection unit collects a back sample image set of different body types;

[0091] The data processing unit is configured to perform data normalization processing on the back sample image set by using empirical mean and variance data obtained from an ImageNet data set.

[0092] Referring to Figure 4 , a structural schematic diagram of a computer device of a human back body posture feature detection method based on images is shown, and specifically can comprise the following:

[0093] The computer device 12 is in the form of a general-purpose computing device, and the components of the computer device 12 can include but are not limited to one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components including the system memory 28 and the processing unit 16.

[0094] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration bus, a processor or local bus using any of a variety of bus architectures including an Industry Standard Architecture (ISA), Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0095] Computer device 12 typically includes a variety of computer system readable media. Such media can be any available media that is located either internally or externally to computer device 12, including both volatile and nonvolatile media, removable and non-removable media.

[0096] System memory 28 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer device 12 can further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 can be used for reading from and writing to non-removable, non-volatile magnetic media (typically called a "hard drive"). Although Figure 4 Although not shown, a magnetic hard disk drive, a magnetic disk drive (e.g., to read from or write to a removable, nonvolatile magnetic disk (e.g., "floppy drive" and / or other magnetic media) and an optical disk drive can be used in some implementations. Such drives can be connected to bus 18 by one or more data media interfaces. The drives and their associated computer system storage media, described above and below, can provide nonvolatile storage of computer-readable instructions, data structures, program modules and other data for computer device 12.

[0097] Program / utility 40, having a set (at least one) of program modules 42, can be stored in, for example, memory (RAM, ROM, etc.) by way of example, such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules 42, and program data, each or a combination of which can provide a

[0098] Computer device 12 can also communicate with one or more external devices 14 such as a keyboard, a pointing device, a display 24, a camera, etc.; one or more devices that enable a user to interact with computer device 12; and / or any devices (e.g., network card, modem, etc.) that enable computer device 12 to communicate with one or more other computing devices. Such communication can occur via Input / Output (I / O) interface(s) 22. Still yet, computer device 12 can communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet) via network adapter 20. As depicted, network adapter 20 communicates with the other components of computer device 12 via bus 18. It should be appreciated that although not shown, other hardware and / or software modules could be used in conjunction with computer device 12. Examples, include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems 34, etc. Figure 4

[0099] Processing unit 16, through running programs stored in system memory 28, performs various function applications and data processing, such as implementing an image-based detection method of human back posture features provided by embodiments of the present application.

[0100] That is, when processing unit 16 executes the above program, it implements: obtaining a back color image of a detected person; inputting the back color image into a preset feature recognition network, and outputting coordinates of an even number of back feature points from the feature recognition network; processing the back color image according to a preset image processing algorithm to obtain a back contour image; fusing the coordinates of the even number of back feature points into the back contour image, determining a position correspondence relationship between each back feature point and each target part in the back contour image, and determining whether target parts of the same category are isometric according to the position correspondence relationship.

[0101] In embodiments of the present application, the present application also provides a computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an image-based detection method of human back posture features as provided by all embodiments of the present application:

[0102] ​That is, the program, when executed by a processor, implements: obtaining a color image of a back of a detected person; inputting the color image of the back into a preset feature recognition network, and outputting coordinates of an even number of back feature points from the feature recognition network; processing the color image of the back according to a preset image processing algorithm to obtain a back contour image; fusing the coordinates of the even number of back feature points into the back contour image to determine a position correspondence relationship between each back feature point and each target part in the back contour image, and determining whether target parts of the same category are isometric according to the position correspondence relationship.

[0103] Any combination of one or more computer readable medium can be employed. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, the computer readable storage medium can be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0104] A computer readable signal medium can include a propagated data signal with computer readable program code embodied therein. The propagated data signal can take any of a variety of forms, including but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium can be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0105] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). The embodiments of the present application described above are implemented in a manner as follows, and each of the embodiments focuses on the differences from other embodiments. The same or similar parts among the embodiments can be mutually referred to.

[0106] While the preferred embodiments of the application have been described above, additional variations and modifications of the embodiments can occur to those skilled in the art once advised of the basic inventive concepts. Therefore, the appended claims are intended to cover all such additional variations and modifications as fall within the scope of the application.

[0107] Finally, it should be noted that the terms "first", "second", and the like, herein do not denote any order, quantity, combination, or importance, but rather are used to distinguish one element from another, and are not intended to denote a particular order or configuration. Also, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0108] The above provides a kind of based on image human body back posture feature detection method and device of the present application, has carried out detailed introduction, the principle and implementation mode of the present application are described in this paper with specific examples, the above example is only for helping understanding the method of the present application and its core idea;For the general technical personnel of the field, according to the idea of the present application, there will be changes in specific implementation mode and application range, the above-mentioned contents of the specification should not be understood as the limitation of the present application.

Claims

1. A method for detecting posture features of a human back based on images, characterized in that, The method comprises: acquire the back color image of the detected person; input the back color image into a preset feature recognition network, and output the coordinates of an even number of back feature points from the feature recognition network; the method comprises: pre-training the feature recognition network, including: collecting back sample images, and labeling back sample parts in the back sample images; training the feature recognition network according to the labeled back sample images and using a ResNet50 network structure; verifying the output result of the feature recognition network through a test set, and specifically, when the feature point position output by the feature recognition network is basically consistent with the coordinates based on the back sample images, the training of the feature recognition network is completed; and initializing the weights of the trained feature recognition network using a Xavier network weight initialization method, and the weight initialization uses the following calculation formula: where W is the network weight and n; is the number of output neurons of each layer. processing the back color image according to a preset image processing algorithm to obtain a back contour image; fuse the coordinates of the even number of back feature points into the back contour image, determine the position correspondence relationship between each back feature point and each target part in the back contour image, and determine whether target parts of the same category are isohypse based on the position correspondence relationship; The computer device used in the detection method comprises one or more processors or processing units (16), a system memory (28), and a bus (18), the system memory (28) comprises a random access memory (RAM) (30) and / or a cache memory (32), the computer device further comprises a storage system (34), a display (24), and a network adapter (20), the network adapter (20) communicates with other modules of the computer device through the bus (18), the system memory (28) has a set of program modules (42), the program modules (42) are configured to execute the program of the image-based human back posture feature detection method, and the processing unit (16) executes the program of the program modules (42) stored in the system memory (28) by running.

2. The method of claim 1, wherein, The collection of back sample images further comprises: collect a set of back sample images of different body types; normalize the data of the set of back sample images using the empirical mean and variance data obtained from the ImageNet dataset.

3. An image-based human back posture feature detection device for implementing the detection method of claim 1, characterized by, The method comprises: an acquisition module, configured to acquire a back color image of a detected person; an identification module, configured to input the back color image into a preset feature recognition network, and output the coordinates of an even number of back feature points from the feature recognition network; a processing module, configured to process the back color image according to a preset image processing algorithm to obtain a back contour image; a judgment module, configured to fuse the coordinates of the even number of back feature points into the back contour image, determine the position correspondence relationship between each back feature point and each target part in the back contour image, and determine whether target parts of the same category are isohypse based on the position correspondence relationship.

4. The apparatus of claim 3, wherein, The identification module comprises: a training submodule, configured to pre-train the feature recognition network, including: Collecting a back sample image, labeling a back sample part in the back sample image; According to the labeled back sample image and using a ResNet50 network structure, the feature recognition network is trained. The output result of the feature recognition network is verified by a test set, and specifically, when the feature recognition network outputs a back sample part based on the coordinates in the back sample image, the feature recognition network training is completed.

5. The apparatus of claim 4, wherein, The training submodule includes: An initialization unit is configured to initialize the weights of the trained feature recognition network using the Xavier network weight initialization method.

6. The apparatus of claim 4, wherein, The training submodule further includes a sample collection unit configured to collect a back sample image set of different body types. A data processing unit is configured to perform data normalization processing on the back sample image set using the empirical mean and variance data obtained from the ImageNet data set.

Images