A method of measuring head and neck mobility and related devices

Abstract

The application discloses a head and neck activity measurement system and method, which can be applied to the field of automatic control technology. A multi-view camera and a sliding module are installed on a collection device. The sliding module is composed of multiple sliding modules with fixed relative positions. The multi-view camera is composed of multiple cameras, and one camera is fixed on one sliding module. The camera parameters of the multi-view camera are pre-calibrated. Since the multiple cameras are installed on the multiple sliding modules with fixed relative positions, and the sliding modules can slide to the specified positions according to the sliding instructions, the application can realize the multiple-angle shooting of multiple head and neck images at the same time through the non-fixed camera position. Moreover, the camera parameters do not need to be recalibrated after the camera is moved, so that the positioning accuracy and flexibility of the head and neck images are improved. The host computer calculates the head and neck activity of the head and neck images shot from multiple angles, so that the accuracy of the measurement result of the head and neck activity is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic control, in particular to a head and neck activity measurement method and related device. BACKGROUND

[0002] The measurement of the activity of the cervical vertebra joint is of great significance for the screening and diagnosis of cervical spondylosis. The traditional X-ray and CT three-dimensional reconstruction measurement method is not only costly, but also easy to cause damage to human health. Although non-invasive measurement devices such as cervical vertebra activity measurement instruments or electric goniometers can avoid damage to human health during measurement, they need to be manually operated by professionals, and the measurement convenience is low.

[0003] The camera-based head and neck activity measurement method has the advantages of low invasiveness and high automation, but the existing camera-based head and neck activity measurement method often has the problem of inaccurate fixed camera shooting angle, which leads to low accuracy of the measurement result of the head and neck activity. Therefore, how to improve the accuracy of the measurement result of the head and neck activity is a technical problem to be solved by the person skilled in the art. SUMMARY

[0004] In view of the above problems, the present application provides a head and neck activity measurement method and related device, and the specific solutions are as follows:

[0005] The first aspect of the present application provides a head and neck activity measurement system, comprising a collection device and a host computer in communication connection with the collection device, a multi-view camera and a sliding module are installed on the collection device, wherein the sliding module is composed of a plurality of sliding modules with fixed relative positions, the multi-view camera is composed of a plurality of cameras, and one camera is fixed on one sliding module, wherein the camera parameters of the multi-view camera are pre-calibrated;

[0006] The sliding module is used to execute the synchronous sliding operation of the plurality of sliding modules in response to a sliding instruction, so that the plurality of sliding modules slide to the position indicated by the sliding instruction;

[0007] The multi-view camera is used to execute the synchronous shooting operation of the plurality of cameras in response to a shooting instruction, to obtain the head and neck images shot by each camera at the current time frame, and to transmit the plurality of head and neck images of the current time frame to the host computer;

[0008] The host is configured to issue the sliding instruction to the sliding module, and issue the shooting instruction to the multi-view camera after the plurality of sliding modules slide to the position indicated by the sliding instruction; receive a plurality of head and neck images corresponding to each time frame, obtain head and neck posture parameters of each time frame, and obtain a head and neck activity degree based on the head and neck posture parameters of each time frame. The head and neck posture parameters include a plurality of rotation angles, and the head and neck activity degree includes an activity range of the plurality of rotation angles.

[0009] In a possible implementation, the sliding module is a programmable screw rod linear sliding module, the screw rod linear sliding module includes four single modules respectively mounted on four beam columns of the collection pavilion of the frame structure, one sliding module is arranged on one single module, and the sliding modules arranged on the four single modules move synchronously up and down. Each camera of the multi-view camera is fixed on a sliding module through a holder.

[0010] In a possible implementation, the host is further configured to obtain a calibration verification result according to the three-dimensional coordinates of each head and neck key point, the calibration verification result is used to indicate whether the multi-view camera needs to be recalibrated, issue a calibration sliding instruction to the sliding module when a preset calibration time arrives, and issue a calibration shooting instruction to the multi-view camera. The calibration time arrives includes a calibration time of a predicted calibration period, and / or the calibration verification result indicates that the multi-view camera needs to be recalibrated.

[0011] The sliding module is further configured to perform a synchronous sliding operation of the plurality of sliding modules in response to the calibration sliding instruction, so that the plurality of sliding modules slide to a calibration position, and the calibration position is pre-configured based on a calibration board on which a preset random texture is displayed.

[0012] The multi-view camera is further configured to perform a synchronous shooting operation of the plurality of cameras in response to the calibration shooting instruction, obtain a random texture image shot by each camera at a current time frame, and transmit a plurality of random texture images of the current time frame to the host, so that the host calibrates camera parameters of the multi-view camera according to the plurality of random texture images, and updates the camera parameters of the multi-view camera.

[0013] The second aspect of the present application provides a head and neck activity degree measurement method applied to a collection device, the collection device is provided with a multi-view camera and a sliding module, the sliding module is composed of a plurality of sliding modules with fixed relative positions, the multi-view camera is composed of a plurality of cameras, and one camera is fixed on one sliding module, wherein the camera parameters of the multi-view camera are pre-calibrated; the head and neck activity degree measurement method includes:

[0014] The sliding module group performs synchronous sliding operation of the plurality of sliding modules in response to a sliding instruction, so that the plurality of sliding modules slide to positions indicated by the sliding instruction.

[0015] The multi-view camera performs synchronous shooting operation of the plurality of cameras in response to a shooting instruction, obtains head and neck images shot by each camera at a current time frame, and transmits a plurality of head and neck images of the current time frame to the host computer, so that the host computer receives the plurality of head and neck images corresponding to each time frame, obtains head and neck pose parameters of each time frame, and obtains head and neck activity based on the head and neck pose parameters of each time frame. The head and neck pose parameters include a plurality of rotation angles, and the head and neck activity includes an activity range of the plurality of rotation angles.

[0016] In a possible implementation, the method for measuring the head and neck activity further includes:

[0017] The sliding module group performs synchronous sliding operation of the plurality of sliding modules in response to a calibration sliding instruction, so that the plurality of sliding modules slide to calibration positions, the calibration positions are preconfigured based on a calibration board, and the calibration board displays preset random textures; the calibration sliding instruction is sent by the host computer when a preset calibration time arrives, the calibration time arrives includes a calibration time point of a predicted calibration period, and / or a calibration check result indicates that the multi-view camera needs to be recalibrated.

[0018] The multi-view camera performs synchronous shooting operation of the plurality of cameras in response to a calibration shooting instruction, obtains random texture images shot by each camera at a current time frame, and transmits a plurality of random texture images of the current time frame to the host computer, so that the host computer calibrates camera parameters of the multi-view camera according to the plurality of random texture images, and updates the camera parameters of the multi-view camera. The calibration shooting instruction is sent by the host computer after the plurality of sliding modules slide to the calibration positions.

[0019] The third aspect of the present application provides a method for measuring head and neck activity, applied to a host computer, the host computer is in communication connection with a collection device, the collection device is installed with a multi-view camera and a sliding module group, wherein the sliding module group is composed of a plurality of sliding modules with fixed relative positions, the multi-view camera is composed of a plurality of cameras, and one camera is fixed on one sliding module, the camera parameters of the multi-view camera are pre-calibrated, and the method for measuring the head and neck activity includes:

[0020] The sliding instruction is sent to the sliding module group, so that the plurality of sliding modules slide to positions indicated by the sliding instruction.

[0021] After the plurality of sliding modules slide to the positions indicated by the instructions, a shooting instruction is sent to the multi-view camera to make the multi-view camera perform a synchronous shooting operation of the plurality of cameras to obtain a plurality of head-neck images shot by the plurality of cameras at a current time frame, and the plurality of head-neck images at the current time frame are transmitted to the host computer.

[0022] For each time frame, three-dimensional coordinates of each preset head-neck key point are obtained according to the plurality of head-neck images corresponding to the time frame.

[0023] For each time frame, the three-dimensional coordinates of each head-neck key point are input into a pre-trained activity estimation model to obtain a head-neck pose parameter output by the activity estimation model, the head-neck pose parameter includes a plurality of rotation angles, and the activity estimation model is constructed based on a neural network.

[0024] Based on the head-neck pose parameters of each time frame, a head-neck activity is obtained, and the head-neck activity includes an activity range of the plurality of rotation angles.

[0025] In a possible implementation, the sliding instruction is sent to the sliding module to make the plurality of sliding modules slide to the positions indicated by the sliding instruction, including:

[0026] Based on the height information of the to-be-tested person, a corresponding to-be-tested head-neck height is obtained; the position sliding instruction is generated based on the to-be-tested head-neck height, the position sliding instruction includes positioning information corresponding to the head-neck height; the position sliding instruction is sent to the sliding module to make the plurality of sliding modules slide to the positions indicated by the positioning information; and the sliding instruction includes the first sliding instruction.

[0027] In a possible implementation, the sliding instruction is sent to the sliding module to make the plurality of sliding modules slide to the positions indicated by the sliding instruction, including:

[0028] At a preset first frequency, at least one to-be-tested image returned by the multi-view camera is received, it is identified whether the to-be-tested image includes a complete head-neck region, if not, a second sliding instruction is generated based on a plurality of preset positioning points in the to-be-tested image; and until the to-be-tested image includes a complete head-neck region, wherein the second sliding instruction includes an upward sliding instruction or a downward sliding instruction, and the positioning points include a plurality of key points located at edges of the head-neck; and the sliding instruction includes the second sliding instruction.

[0029] In a possible implementation, the target time frame is any one time frame, and three-dimensional coordinates of each preset head-neck key point are obtained according to a head-neck image corresponding to the target time frame, including:

[0030] For each of the head and neck key points, a two-dimensional coordinate of the head and neck key point in each of the head and neck images corresponding to the target time frame is obtained, to obtain a two-dimensional coordinate combination of the head and neck key point, the two-dimensional coordinate combination of the head and neck key point including the two-dimensional coordinates of the head and neck key point corresponding to each of the head and neck images;

[0031] The three-dimensional coordinates of the head and neck key points are calculated based on the two-dimensional coordinate combination of the head and neck key points through a triangulation algorithm.

[0032] In a possible implementation, in the step of obtaining the three-dimensional coordinates of each of the preset head and neck key points according to the head and neck images corresponding to the target time frame, the head and neck activity measurement method further includes:

[0033] For each of the head and neck key points, the three-dimensional coordinates of the head and neck key point are mapped to the two-dimensional coordinates corresponding to each of the head and neck images based on an inverse algorithm of the triangulation algorithm, as mapped two-dimensional coordinates.

[0034] The mapped two-dimensional coordinates of each of the head and neck key points are compared with the two-dimensional coordinates corresponding to each of the head and neck images in the two-dimensional coordinate combination of the head and neck key point, to obtain a calibration verification result, the calibration verification result being used to indicate whether the multi-view camera needs to be recalibrated.

[0035] When a preset calibration occasion is reached, a calibration sliding instruction is sent to the sliding module, the calibration sliding instruction being used to control the plurality of sliding modules to slide to a calibration position, the calibration position being preconfigured based on a calibration board, the calibration board having preset random textures displayed thereon.

[0036] When the plurality of sliding modules slide to the calibration position, a calibration shooting instruction is sent to the multi-view camera, the calibration shooting instruction being used to control the multi-view camera to perform a synchronous shooting operation of the plurality of cameras, to obtain random texture images shot by each of the cameras at a current time frame, and to transmit the plurality of random texture images of the current time frame to the host computer.

[0037] The calibration occasion is reached includes a calibration time of a predicted calibration period, and / or the calibration verification result indicates that recalibration is needed.

[0038] Camera parameters of the multi-view camera are calibrated based on the plurality of random texture images, and the camera parameters of the multi-view camera are updated.

[0039] The fourth aspect of the present application provides a computer program product, including computer readable instructions, when the computer readable instructions run on an electronic device, the electronic device implements the head and neck activity measurement method of the first aspect or any implementation manner of the first aspect.

[0040] The fifth aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0041] The memory is configured to store a computer program;

[0042] The processor is configured to execute the computer program, so that the electronic device can implement the head and neck activity measurement method of the first aspect or any implementation manner of the first aspect.

[0043] The fifth aspect of the present application provides a computer storage medium, the storage medium carries one or more computer programs, when the one or more computer programs are executed by an electronic device, the electronic device can implement the head and neck activity measurement method of the first aspect or any implementation manner of the first aspect.

[0044] By the above technical solution, the head and neck activity measurement system and method provided by the present application, the host sends a sliding instruction to the sliding module, so that the plurality of sliding modules slide to the position indicated by the sliding instruction. After the plurality of sliding modules slide to the position indicated by the instruction, a shooting instruction is sent to the multi-camera to make the multi-camera perform a synchronous shooting operation of the plurality of cameras, obtain the head and neck image shot by each camera at the current time frame, and transmit the plurality of head and neck images of the current time frame to the host. For each time frame, the three-dimensional coordinates of each preset head and neck key point are obtained according to the head and neck images corresponding to the plurality of time frames. For each time frame, the three-dimensional coordinates of each head and neck key point are input into the activity estimation model obtained by pre-training to obtain the head and neck pose parameters output by the activity estimation model, the head and neck pose parameters include a plurality of rotation angles, and the activity estimation model is constructed based on a neural network. Based on the head and neck pose parameters of each time frame, the head and neck activity is obtained, and the head and neck activity includes the activity range of the plurality of rotation angles. It can be seen that since the plurality of cameras are installed on the plurality of sliding modules with fixed relative positions, and the sliding modules can slide to the specified position according to the sliding instruction, the present application can realize the shooting of the plurality of head and neck images at the same time through the non-fixed camera position at the plurality of angles, and does not need to recalibrate the camera parameters after the camera moves, thereby improving the positioning accuracy and flexibility of the head and neck image. Further, the head and neck activity calculation is performed on the head and neck images shot at the plurality of angles by the host, thereby improving the accuracy of the measurement result of the head and neck activity. BRIEF DESCRIPTION OF DRAWINGS

[0045] The above and other features, advantages, and aspects of the present disclosure will become more apparent with reference to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals are used to refer to the same or similar elements. It is to be understood that the drawings are schematic, and elements and features do not necessarily appear in proportion to one another.

[0046] Figure 1 A structural schematic diagram of a head and neck activity measurement system provided in the present application;

[0047] Figure 2 An optional hardware structure schematic diagram of a terminal provided in an embodiment of the present application;

[0048] Figure 3 An optional hardware structure schematic diagram of a server provided in an embodiment of the present application;

[0049] Figure 4 A flowchart of a head and neck activity measurement method provided in an embodiment of the present application;

[0050] Figure 5 A specific implementation flowchart of a head and neck activity measurement method provided in an embodiment of the present application;

[0051] Figure 6 A random texture image provided in an embodiment of the present application;

[0052] Figure 7 A head and neck key point distribution provided in an embodiment of the present application;

[0053] Figure 8 A head and neck activity diagram provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The embodiments of the present application are described below in conjunction with the accompanying drawings. The terms used in the embodiment part of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0055] The embodiments of the present application are described below in conjunction with the accompanying drawings. It is known to those skilled in the art that, as technology develops and new scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0056] The terms "first", "second", and the like in the description and in the claims of the present application and above drawings are used for distinguishing between similar objects and not necessarily for describing a specific sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances and are merely employed for descriptive purposes. Furthermore, the terms "comprise", "include", "have" and any variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, system, product or apparatus that comprises, includes or has a list of elements is not necessarily limited to those elements, but can include other elements not expressly listed or inherent to such process, method, system, product or apparatus.

[0057] To overcome the technical defects of low accuracy and flexibility of the conventional head and neck activity measurement method, the embodiments of the present application provide a head and neck activity measurement system, which acquires head and neck movement images by shooting multiple cameras with fixed relative positions but non-fixed shooting positions, processes the head and neck movement images by a host computer to obtain the head and neck activity. Since the multiple cameras have non-fixed shooting positions, the head and neck images of different individuals can be accurately shot, and since the relative positions are fixed, the multiple cameras do not need to be recalibrated after moving, thereby improving the accuracy and flexibility of the head and neck activity measurement.

[0058] Reference Figure 1 , Figure 1 The head and neck activity measurement system provided by the present application has the structure as shown in Figure 1 The head and neck activity measurement system includes a collection device host computer and a host computer in communication connection with the collection device. A multi-view camera and a sliding module are installed on the collection device. The sliding module is composed of multiple sliding modules with fixed relative positions, and the multi-view camera is composed of multiple cameras, and one camera is fixed on one sliding module. That is, the number of sliding modules is equal to the number of cameras of the multi-view camera. The number of sliding modules is greater than 2, and the multiple sliding modules are uniformly arranged at different directions on the same horizontal plane, and the shooting direction of the multiple cameras is the standing position of the person. Figure 1 Taking the number of cameras of the multi-view camera as 4 as an example, a structure of the head and neck activity measurement system is shown.

[0059] As Figure 1As shown, the collection device is installed in a collection booth of a rectangular frame structure, and specifically, the sliding module is a programmable screw linear sliding module, the screw linear sliding module includes a single module on each of the four vertical beams of the collection booth, and one sliding module (also referred to as a sliding block) is arranged on one single module. Each camera in the multi-view camera is fixed on the sliding module through a holder, and moves synchronously with the synchronous up-down movement of the sliding module. The screw linear sliding module has very high repeat positioning accuracy, and the relative positions of each sliding module on the screw linear sliding module are fixed, that is, the up-down movement of each sliding module on the screw linear sliding module is completely synchronous, so that the up-down movement of each camera is completely synchronous.

[0060] In this embodiment, the function of the head and neck activity measurement system is as follows:

[0061] The sliding module is configured to perform synchronous sliding operation of the plurality of sliding modules in response to a sliding instruction, so that the plurality of sliding modules slide to the position indicated by the sliding instruction.

[0062] The multi-view camera is configured to perform synchronous shooting operation of the plurality of cameras in response to a shooting instruction, to obtain a head and neck image shot by each camera at a current time frame, and to transmit the head and neck image corresponding to the current time frame to the host. Wherein, the camera parameters of each camera in the multi-view camera are pre-calibrated.

[0063] The host is configured to issue a sliding instruction to the sliding module, and issue the shooting instruction to the multi-view camera when the plurality of sliding modules slide to the position indicated by the sliding instruction; receive multi-view vision data, obtain head and neck pose parameters of each time frame based on a plurality of head and neck images corresponding to each time frame in the multi-view vision data, and obtain head and neck activity based on the head and neck pose parameters of each time frame.

[0064] In this embodiment, the head and neck pose parameters include a plurality of rotation angles of the head and neck. Optionally, the rotation angles specifically include a first rotation angle, a second rotation angle and a third rotation angle, wherein the first rotation angle is a rotation angle around the x-axis direction of the space rectangular coordinate system, the second rotation angle is a rotation angle around the y-axis direction of the space rectangular coordinate system, and the third rotation angle is a rotation angle around the z-axis direction of the space rectangular coordinate system.

[0065] In this embodiment, the head and neck activity degree includes the activity range of multiple rotation angles of the head and neck. Alternatively, the neck activity degree specifically includes: a first activity degree, a second activity degree, and a third activity degree, wherein the first activity degree is the activity range of a first rotation angle, that is, the maximum rotation angle of the head and neck around the x-axis direction of the space rectangular coordinate system in the first activity degree, the second activity degree is the activity range of a second rotation angle, that is, the maximum rotation angle of the head and neck around the y-axis direction of the space rectangular coordinate system in the second activity degree, and the third activity degree is the activity range of a third rotation angle, that is, the maximum rotation angle of the head and neck around the z-axis direction of the space rectangular coordinate system in the third activity degree.

[0066] It can be seen from the above technical solution that the head and neck activity degree measurement system provided by the embodiment of the application includes a host and a collection device, a multi-view camera and a sliding module are installed on the collection device, the sliding module is composed of multiple sliding modules with fixed relative positions, the multi-view camera is composed of multiple cameras, and one camera is fixed on one sliding module, and the camera parameters of the multi-view camera are pre-calibrated; the host is used to send control instructions (including various types of sliding instructions and shooting instructions) to the collection device to control the sliding module and the multi-view camera; the collection device is used to transmit multi-view visual data to the host; specifically, the sliding module is used to perform a synchronous sliding operation of the multiple sliding modules in response to the sliding instructions, so that the multiple sliding modules slide to the positions indicated by the sliding instructions; the multi-view camera is used to perform a synchronous shooting operation of the multiple cameras in response to the shooting instructions, to obtain head and neck images shot by each camera at the current time frame, and to transmit the multiple head and neck images of the current time frame to the host; and the host is used to receive the multiple head and neck images corresponding to each time frame, to obtain head and neck pose parameters of each time frame, and to obtain the head and neck activity degree based on the head and neck pose parameters of each time frame. It can be seen that, since the multiple cameras are installed on the multiple sliding modules with fixed relative positions, and the sliding modules can slide to the specified positions according to the sliding instructions, the application can realize the shooting of multiple head and neck images at multiple angles at the same time through a non-fixed camera position, and the camera parameters do not need to be recalibrated after the camera is moved, thereby improving the positioning accuracy and flexibility of the head and neck images. Furthermore, the head and neck activity degree is calculated by the host based on the head and neck images shot at multiple angles, thereby improving the accuracy of the measurement result of the head and neck activity degree.

[0067] In Figure 1In a possible implementation shown, a programmable lead screw linear sliding module is installed on the beam column at the four corners of the collection booth, and each camera is stably installed on the slider of the sliding module by a pan-tilt head. With the extremely high repeat positioning accuracy (about 0.02 mm) of the lead screw linear sliding module, the multi-view camera can move up and down completely synchronously after accurate calibration, without the need for recalibration after each movement. Therefore, for different persons to be measured, the height of each camera in the multi-view camera can be adjusted by the sliding instruction issued by the host computer without changing the relative position between the cameras, so that the head and neck of the person to be measured is located at the center of the field of view of each camera. Thus, the synchronous shooting of each camera is controlled by the shooting instruction issued by the host computer, and the head and neck images shot from multiple angles are obtained. It can be seen that the collection of head and neck images by the collection booth has the advantages of non-invasiveness, high head and neck positioning accuracy, full angle, and high flexibility.

[0068] It should be noted that the head and neck activity measurement system provided in the embodiments of the present application can be implemented in various specific structures.

[0069] For example, in a possible implementation, the host computer is further configured to obtain a calibration verification result according to the three-dimensional coordinates of each head and neck key point, the calibration verification result is used to indicate whether the multi-view camera needs to be recalibrated, and the host computer issues a calibration shooting instruction to the multi-view camera when a preset calibration time arrives. After receiving the random texture images, the camera parameters of the multi-view camera are calibrated according to the multiple random texture images, and the camera parameters of the multi-view camera are updated. Wherein, the calibration time includes a predicted calibration time of a calibration period, and / or the calibration verification result indicates that the multi-view camera needs to be recalibrated, that is, when at least one of the calibration time of the calibration period is reached or the calibration verification result indicates that the multi-view camera needs to be recalibrated, the host computer issues a calibration shooting instruction to the multi-view camera.

[0070] The multi-view camera is further configured to perform a synchronous shooting operation of the multiple cameras in response to the calibration shooting instruction, obtain random texture images shot by each camera at a current time frame, and transmit the multiple random texture images of the current time frame to the host computer.

[0071] It can be seen that the head and neck activity measurement system provided in the embodiments of the present application judges whether recalibration is needed by the host computer, and after determining that recalibration is needed, multiple random texture images are shot by the multi-view camera. The camera parameters of the multi-view camera are recalibrated by the host computer according to the multiple random texture images, the accuracy of the camera parameters is improved, and thus the accuracy of the measurement result of the head and neck activity is improved.

[0072] For another example, in a possible implementation, the preset position in the collection kiosk is provided with a calibration board printed with a random texture image, or an electronic screen capable of displaying a random texture image. The host is further configured to send a calibration sliding instruction to the sliding module, so that the sliding module performs a synchronous sliding operation of the plurality of sliding modules, and the plurality of sliding modules slide to the calibration positions indicated by the calibration sliding instruction, so that the multi-view camera can clearly and completely capture the random texture image, and the calibration accuracy is improved.

[0073] For another example, the specific implementation of the function of acquiring the head and neck activity degree includes various implementation manners, and specific implementation manners can be referred to the following embodiments.

[0074] The embodiments of the present application further provide a head and neck activity degree measurement method, which aims to measure the head and neck activity degree through the multi-angle head and neck images captured synchronously, and improve the measurement result accuracy. The head and neck activity degree measurement method provided by the embodiments of the present application is applied to a host, which is in communication connection with a collection device. The collection device is provided with a multi-view camera and a sliding module. The sliding module is composed of a plurality of sliding modules with fixed relative positions. The multi-view camera is composed of a plurality of cameras, and one camera is fixed on one sliding module. The camera parameters of the multi-view camera are pre-calibrated.

[0075] In a possible implementation, the host can be a terminal. The terminal can be provided with a head and neck activity degree measurement application program. The application program and the webpage can provide a head and neck activity degree measurement interface. The terminal can receive the relevant parameters input by a user on the head and neck activity degree measurement interface, such as a collection frequency or information of a person to be measured.

[0076] Next, the product form of the terminal is described.

[0077] The terminal in the embodiments of the present application can be a mobile phone, a tablet computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), and the like. The embodiments of the present application do not make any limitation in this regard.

[0078] Figure 2 An optional hardware structure schematic diagram of the terminal 100 is shown.

[0079] Reference Figure 2As shown, the terminal 100 can include a radio frequency unit 110, a memory 120, an input unit 130, a display unit 140, a camera 150 (optional), an audio circuit 160 (optional), a speaker 161 (optional), a microphone 162 (optional), a headphone jack 163 (optional), a processor 170, an external interface 180, a power supply 190, and the like. Those skilled in the art can understand that Figure 2 The terminal or multifunctional device is merely an example and does not constitute a limitation on the terminal or multifunctional device, and can include more or fewer components than those shown, or combine certain components, or different components.

[0080] The input unit 130 can be used to receive inputted digital or character information, and to generate key signal input related to user settings and function control of the portable multifunctional device. Specifically, the input unit 130 can include a touch screen 131 (optional) and / or other input devices 132. The touch screen 131 can collect touch operations of a user thereon or adjacent thereto (such as operations of the user using a finger, a joint, a stylus, or any suitable object on or adjacent to the touch screen), and drive corresponding connected devices according to a pre-set program. The touch screen can detect touch actions of the user on the touch screen, convert the touch actions into touch signals and send the touch signals to the processor 170, and can receive commands from the processor 170 and execute the commands; the touch signals at least include touch point coordinate information. The touch screen 131 can provide an input interface and an output interface between the terminal 100 and the user. In addition, the touch screen can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch screen 131, the input unit 130 can also include other input devices. Specifically, the other input devices 132 can include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), trackballs, mice, joysticks, and the like.

[0081] Among them, the input device 132 can receive inputted data and the like.

[0082] The display unit 140 can be used to display information inputted by the user or information provided to the user, various menus of the terminal 100, interactive interfaces, file display, and / or playing of any kind of multimedia files. In the embodiments of the present application, the display unit 140 can be used to display interfaces of measurement of head and neck activity, processing results, and the like.

[0083] The memory 120 can be used to store instructions and data. The memory 120 can mainly include a storage instruction area and a storage data area. The storage data area can store various data such as multimedia files, texts, etc. The storage instruction area can store software units such as operating systems, applications, instructions required by at least one function, etc. or their subsets, extended sets. Non-volatile random access memory can also be included. The processor 170 is provided with software and applications that include management of hardware, software and data resources in the computing processing device, support control. It is also used for the storage of multimedia files, as well as the storage of running programs and applications.

[0084] The processor 170 is the control center of the terminal 100, which connects each part of the terminal 100 through various interfaces and lines, executes various functions of the terminal 100 and processes data by running or executing instructions stored in the memory 120 and calling data stored in the memory 120, thereby controlling the terminal as a whole. Optionally, the processor 170 can include one or more processing units. Preferably, the processor 170 can integrate an application processor and a modem processor, wherein the application processor mainly processes operating systems, user interfaces and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 170. In some embodiments, the processor, the memory, can be implemented on a single chip, and in some embodiments, they can also be implemented on separate chips respectively. The processor 170 can also be used to generate corresponding operation control signals to the corresponding components of the computing processing device, read and process data in the software, especially read and process data and programs in the memory 120, so that each functional module therein performs corresponding functions, thereby controlling the corresponding components to act according to the requirements of the instructions.

[0085] The memory 120 can be used to store software codes related to the measurement method of the head and neck activity, and the processor 170 can execute the steps of the measurement method of the head and neck activity, or can also dispatch other units (such as the above-mentioned input unit 130 and display unit 140) to realize corresponding functions.

[0086] The RF unit 110 (optional) can be used to receive and send signals in the process of information or communication, for example, receiving the downlink information of the base station, and processing by the processor 170. In addition, the uplink data is sent to the base station. Generally, the RF circuit includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, etc. In addition, the RF unit 110 can also communicate with network devices and other devices through wireless communication. The wireless communication can use any communication standard or protocol, including but not limited to global system for mobile communication (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), long term evolution (LTE), email, short messaging service (SMS), etc.

[0087] In the embodiments of the present application, the RF unit 110 can send data to the server 200 and receive the processing result sent by the server 200.

[0088] It should be understood that the RF unit 110 is optional, which can be replaced by other communication interfaces, for example, a network interface.

[0089] The terminal 100 further includes a power supply 190 (such as a battery) for supplying power to each component. Preferably, the power supply can be logically connected to the processor 170 through a power management system, so as to realize the functions of power management, such as charge management, discharge management and power consumption management, through the power management system.

[0090] The terminal 100 further includes an external interface 180, which can be a standard Micro USB interface or a multi-pin connector, and can be used for connecting the terminal 100 with other devices for communication, or for connecting a charger to charge the terminal 100.

[0091] Although not shown, the terminal 100 can further include a flash, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which will not be described here. Some or all of the methods described below can be applied in the terminal 100 as shown. Figure 2

[0092] ​In a possible implementation, the host can be a server, the server can be a single-point server, a cloud server or a server cluster, and the product form of the server 200 is described next.

[0093] Figure 3 A structural diagram of the server 200 is provided, as shown in the figure. Figure 3 The server 200 includes a bus 201, a processor 202, a communication interface 203 and a memory 204. The processor 202, the memory 204 and the communication interface 203 communicate through the bus 201.

[0094] The bus 201 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0095] The processor 202 can be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0096] The memory 204 can include a volatile memory, such as a random access memory (RAM). The memory 204 can also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a mechanical hard disk drive (HDD) or a solid state drive (SSD).

[0097] The memory 204 can be used to store software code related to the measurement method of the head and neck activity, and the processor 202 can execute the steps of the measurement method of the head and neck activity of the chip, or can schedule other units to realize the corresponding functions.

[0098] It should be understood that the terminal 100 and the server 200 described above can be centralized or distributed devices, and the processors (for example, the processor 170 and the processor 202) in the terminal 100 and the server 200 can be hardware circuits (for example, an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a general-purpose processor, a digital signal processing (DSP), a microprocessor, a microcontroller, or the like) or a combination of the hardware circuits, for example, the processor can be a hardware system with an instruction execution function, such as a CPU, a DSP, or the like, or a hardware system without an instruction execution function, such as an ASIC, an FPGA, or the like, or a combination of the hardware system without the instruction execution function and the hardware system with the instruction execution function.

[0099] With reference to Figure 4 , Figure 4 A flowchart of a head and neck activity measurement method provided by an embodiment of the present application is shown in FIG. 1. The head and neck activity measurement method provided by the embodiment of the present application can include S401 to S405, which are described in detail as follows. Figure 4

[0100] S401, a sliding instruction is sent to the sliding module, so that the plurality of sliding modules slide to a position indicated by the sliding instruction.

[0101] In the embodiment, the position indicated by the sliding instruction is a position at which the image captured by each camera includes a complete head and neck.

[0102] In a possible implementation, the host obtains the height of the person to be measured through the human-computer interaction interface, and obtains the corresponding head and neck height based on the height information. The first sliding instruction is generated based on the head and neck height. The first sliding instruction includes positioning information corresponding to the head and neck height. The positioning information includes a target height. For example, the height of the person to be measured is 1.8 meters, and the head and neck height is 1.6 meters. The first sliding instruction generated by the host is used to instruct the sliding module to slide to a target height. The target height is the height of the sliding module at which the height of the shooting field of view of the multi-camera is 1.6 meters. It should be noted that the corresponding relationship between the height, the head and neck height, and the target height can be calibrated through experiments in advance.

[0103] ​In a possible implementation, the host receives at least one to-be-detected image returned by the multi-view camera at a preset first frequency, identifies whether the to-be-detected image includes a complete head and neck region, and if not, generates a sliding instruction based on a plurality of preset positioning points in the to-be-detected image until the to-be-detected image includes the complete head and neck region, where the second sliding instruction includes an upward sliding instruction or a downward sliding instruction. It should be noted that the positioning points include a plurality of key points located at the edge of the head and neck region, whether the to-be-detected image includes the complete head and neck region is determined by identifying whether the to-be-detected image includes all the positioning points, and the sliding instruction is generated by the number and type of the positioning points, for example, if the to-be-detected image is missing a positioning point located at the upper edge of the head and neck region, the upward sliding instruction is generated.

[0104] It should be noted that the head and neck region refers to a region including all head and neck key points, and the head and neck key points are human key points that can indicate the activity angle of the head and neck.

[0105] S402, after the position indicated by the sliding instruction of the plurality of sliding modules, the host sends a shooting instruction to the multi-view camera to make the multi-view camera perform a synchronous shooting operation of the plurality of cameras, to obtain a head and neck image of each camera at a current time frame, and transmit the plurality of head and neck images of the current time frame to the host.

[0106] It should be noted that the specific manner in which the host sends control instructions (including but not limited to the sliding instruction and the shooting instruction) to the multi-view camera and the sliding module in the acquisition device and the acquisition device returns the image to the host can refer to the prior art.

[0107] S403, for each time frame, the three-dimensional coordinates of each preset head and neck key point are obtained according to the plurality of head and neck images.

[0108] In this embodiment, the head and neck key points are pre-configured, and the head and neck key points include human body points related to the rotation angle of the head and neck, for example, the top of the head, the center of the eyebrow, the tip of the nose, the chin, the upper edge of the left ear, the upper edge of the right ear, the lower edge of the left ear, the lower edge of the right ear, the left mandibular angle, the right mandibular angle, the left clavicle, the right clavicle, the back of the brain, the spine, the left shoulder, and the right shoulder.

[0109] S404, for each time frame, the three-dimensional coordinates of each head and neck key point are input into a pre-trained activity estimation model to obtain a head and neck pose parameter output by the activity estimation model.

[0110] In this embodiment, the head and neck part pose parameter includes a plurality of rotation angles. The rotation angles specifically include a first rotation angle, a second rotation angle, and a third rotation angle, wherein the first rotation angle is a rotation angle around the x-axis direction of the space rectangular coordinate system, the second rotation angle is a rotation angle around the y-axis direction of the space rectangular coordinate system, and the third rotation angle is a rotation angle around the z-axis direction of the space rectangular coordinate system. The head and neck part pose parameter expresses the plurality of rotation angles of the head and neck part in the form of Euler angles, that is, the head and neck part pose parameter of any time frame is denoted as (rx, ry, rz)t, wherein t represents the time frame, rx is the first rotation angle, that is, the vertical rotation angle of the head, ry is the second rotation angle, that is, the horizontal rotation angle of the head, and rz is the third rotation angle, that is, the rotation around the line-of-sight direction.

[0111] In a possible implementation, the activity degree estimation model is constructed based on a neural network, wherein the neural network is composed of three linear layers and activation layers between the linear layers. The activity degree estimation model is trained based on the head and neck part training data, wherein the head and neck part training data includes a sample three-dimensional coordinate set labeled with the head and neck part pose parameter, and the sample three-dimensional coordinate set includes three-dimensional coordinates of all head and neck part key points.

[0112] S405, based on the head and neck part pose parameter of each time frame, obtaining the head and neck part activity degree.

[0113] In this embodiment, the head and neck part activity degree includes the activity range of the plurality of rotation angles.

[0114] In this embodiment, the head and neck part activity degree includes the activity range of the plurality of rotation angles of the head and neck part. Optionally, the neck part activity degree specifically includes a first activity degree, a second activity degree, and a third activity degree, wherein the first activity degree is the activity range of the first rotation angle, that is, the maximum rotation angle of the head and neck part around the x-axis direction of the space rectangular coordinate system in the first activity degree, the second activity degree is the activity range of the second rotation angle, that is, the maximum rotation angle of the head and neck part around the y-axis direction of the space rectangular coordinate system in the second activity degree, and the third activity degree is the activity range of the third rotation angle, that is, the maximum rotation angle of the head and neck part around the z-axis direction of the space rectangular coordinate system in the third activity degree.

[0115] It can be seen from the above technical solution that the head and neck activity measurement method provided by the embodiment of the application includes the following steps: the host computer sends a sliding instruction to the sliding module, so that the plurality of sliding modules slide to the position indicated by the sliding instruction. After the plurality of sliding modules slide to the position indicated by the instruction, a shooting instruction is sent to the multi-view camera to make the multi-view camera perform a synchronous shooting operation of the plurality of cameras, obtain a head and neck image shot by each camera at a current time frame, and transmit the plurality of head and neck images of the current time frame to the host computer. For each time frame, the three-dimensional coordinates of each preset head and neck key point are obtained according to the plurality of head and neck images corresponding to the time frame. For each time frame, the three-dimensional coordinates of each head and neck key point are input into a pre-trained activity estimation model to obtain a head and neck pose parameter output by the activity estimation model, the head and neck pose parameter includes a plurality of rotation angles, and the activity estimation model is constructed based on a neural network. Based on the head and neck pose parameters of each time frame, the head and neck activity is obtained, and the head and neck activity includes the activity range of the plurality of rotation angles. It can be seen that since the plurality of cameras are installed on the plurality of sliding modules with fixed relative positions, and the sliding modules can slide to the specified position according to the sliding instruction, the application can realize the shooting of the plurality of head and neck images at the same time through the non-fixed camera position at the plurality of angles, and does not need to recalibrate the camera parameters after the camera moves, thereby improving the positioning accuracy and flexibility of the head and neck image. Further, the host computer calculates the head and neck activity of the head and neck image shot at the plurality of angles, thereby improving the accuracy of the measurement result of the head and neck activity.

[0116] Reference Figure 5 , Figure 5 The specific implementation flowchart of the head and neck activity measurement method provided by the embodiment of the application is shown in FIG. 1. Figure 7 The method specifically includes the following steps:

[0117] S501, the host computer receives at least one to-be-measured image returned by the multi-view camera at a preset first frequency, identifies whether the to-be-measured image includes a complete head and neck region, and if not, generates a second sliding instruction based on a plurality of preset positioning points in the to-be-measured image.

[0118] In the embodiment, the second sliding instruction includes an upward sliding instruction or a downward sliding instruction, and the positioning points include a plurality of key points located at the edges of the head and neck, for example, the positioning points include the top of the head, the chin, the left clavicle, the right clavicle, the left shoulder and the right shoulder, and the sliding instruction includes the second sliding instruction.

[0119] In the embodiment, the positioning points are selected from a plurality of key points located at the edges of the head and neck. By identifying the key points of the contour edges of the head and neck, the completeness of the head and neck region can be effectively identified, that is, whether the head and neck of the to-be-measured person is located at the center of the field of view of each camera.

[0120] S502, in response to the second sliding instruction, the sliding module executes a synchronous sliding operation of the plurality of sliding modules, so that the plurality of sliding modules slide to the position indicated by the sliding instruction.

[0121] S503, when the to-be-tested image includes a complete head and neck region, the host computer issues a shooting instruction to the multi-camera.

[0122] S504, the multi-camera executes a synchronous shooting operation of the plurality of cameras, obtains a head and neck image shot by each camera at a current time frame, and transmits a plurality of head and neck images of the current time frame to the host computer.

[0123] S505, for each head and neck key point, the host computer obtains a two-dimensional coordinate of the head and neck key point in each head and neck image corresponding to a time frame, to obtain a two-dimensional coordinate combination of the head and neck key point, at each time frame.

[0124] In this embodiment, the two-dimensional coordinate combination of the head and neck key point includes a two-dimensional coordinate of each head and neck key point corresponding to each head and neck image. In one possible implementation manner, a residual network ResNet is used to extract the two-dimensional coordinate of each head and neck key point in each head and neck image.

[0125] In this embodiment, the head and neck key point distribution is as shown in Figure 6 The head and neck key points include the top of the head, the center of the eyebrow, the tip of the nose, the chin, the upper edge of the left ear, the upper edge of the right ear, the lower edge of the left ear, the lower edge of the right ear, the left mandibular angle, the right mandibular angle, the left clavicle, the right clavicle, the back of the brain, the spine protrusion, the left shoulder, and the right shoulder.

[0126] It should be noted that the present application can more accurately and accurately describe the activity of the head and neck of the human body by pre-configuring dense head and neck key points.

[0127] S506, the host computer calculates the three-dimensional coordinates of the head and neck key points based on the two-dimensional coordinate combination of the head and neck key points by using a triangulation algorithm.

[0128] In this embodiment, based on the camera parameters of the multi-camera and the two-dimensional coordinates of the head and neck key points corresponding to each camera, the three-dimensional coordinates of each head and neck key point are calculated by using a triangulation algorithm. For details of the triangulation algorithm, please refer to the prior art.

[0129] S507, for each head and neck key point, the host computer maps the three-dimensional coordinates of the head and neck key point to the two-dimensional coordinates corresponding to each head and neck image based on the inverse algorithm of the triangulation algorithm, as the mapped two-dimensional coordinates.

[0130] In this embodiment, the calibration timing can be preconfigured, for example, after each camera calibration, the number of times of sliding of the sliding module reaches a preset sliding number threshold, the number of times of shooting reaches a preset shooting number threshold, or the time length reaches a preset time length threshold.

[0131] S508, the host maps the two-dimensional coordinates of each head and neck key point and the two-dimensional coordinates corresponding to each head and neck image in the two-dimensional coordinate combination of the head and neck key points, and obtains a calibration verification result.

[0132] In this embodiment, the calibration verification result is used to indicate whether the multi-camera needs to be recalibrated.

[0133] In one possible implementation, the two-dimensional coordinates of a single head and neck key point of the same camera and the mapped two-dimensional coordinates are compared to obtain a deviation value, and based on the deviation values of multiple head and neck key points, it is determined whether the deviation exceeds a preset deviation threshold based on the deviation ratio to obtain a deviation result. Further, based on the deviation results of multiple cameras, it is determined whether recalibration is needed.

[0134] S509, the host sends a calibration sliding instruction to the sliding module when the calibration verification result indicates that recalibration is needed.

[0135] S510, the sliding module responds to the calibration sliding instruction and performs a synchronous sliding operation of multiple sliding modules, so that the multiple sliding modules slide to a calibration position indicated by the calibration shooting instruction.

[0136] In this embodiment, the calibration sliding instruction is used to control the multiple sliding modules to slide to the calibration position, wherein the calibration position is preconfigured based on a calibration board, and the calibration board displays a preset random texture, for example, a calibration board with a random texture image is installed at a preset position in the collection pavilion, or an electronic screen that can display a random texture image. The host sends a calibration sliding instruction to the sliding module, so that the sliding module performs a synchronous sliding operation of multiple sliding modules, so that the multiple sliding modules slide to the calibration position indicated by the calibration shooting instruction, so that the multi-camera can clearly and completely shoot the random texture image, and improve the calibration accuracy.

[0137] In this embodiment, the random texture image displayed on the calibration board can be replaced periodically, Figure 7 An optional random texture pattern is shown, and the generation algorithm of the random texture can refer to the prior art.

[0138] S511, the host sends a calibration shooting instruction to the multi-camera after the multiple sliding modules slide to the calibration position.

[0139] In this embodiment, the calibration shooting instruction is used to control the multiple cameras to perform synchronous shooting.

[0140] S512, in response to the calibration shooting instruction, the multi-camera performs a synchronous shooting operation of the multiple cameras, obtains random texture images shot by each camera at a current time frame, and transmits the multiple random texture images of the current time frame to the host.

[0141] S513, the host calibrates the camera parameters of the multi-camera according to the multiple random texture images, and updates the camera parameters of the multi-camera.

[0142] In this embodiment, the calibration method is the same each time, that is, the camera parameters of the multi-camera are calibrated based on the current multiple random texture images, and the camera parameters of the multi-camera are updated.

[0143] It should be noted that random texture can provide a large number of feature points even in the case of partial occlusion, thereby improving the accuracy of calibration.

[0144] S514, the host inputs the three-dimensional coordinates of each head and neck key point into the activity estimation model trained in advance for each time frame to obtain the head and neck pose parameters output by the activity estimation model.

[0145] In this embodiment, the head and neck pose parameters include multiple rotation angles, and the activity estimation model is constructed based on a neural network. The neural network is composed of three linear layers and activation layers between the linear layers. The activity estimation model is trained based on head and neck training data, wherein the head and neck training data includes a sample three-dimensional coordinate set labeled with head and neck pose parameters, and the sample three-dimensional coordinate set includes the three-dimensional coordinates of all head and neck key points.

[0146] In a possible implementation, in order to eliminate the influence of the absolute position of the human body, the three-dimensional coordinates of each head and neck key point can be subtracted from the three-dimensional coordinates of the spinous process to obtain standardized three-dimensional coordinates of each head and neck key point, which are input into the neural network to obtain the head and neck pose parameters.

[0147] S515, the host obtains the head and neck activity based on the head and neck pose parameters of each time frame.

[0148] In this embodiment, the head and neck activity includes the activity range of multiple rotation angles. The activity estimation model outputs the head and neck pose parameters of each time frame in the form of Euler angles in the order of time frames. In a possible implementation, a head and neck activity curve is drawn, the head and neck activity curve includes an angle change curve corresponding to each rotation angle, and for each rotation angle, the maximum value and the minimum value on the angle change corresponding to the rotation angle are obtained to obtain the activity range of the rotation angle.

[0149] Figure 8 A schematic diagram of head and neck activity provided by an embodiment of the present application is shown in FIG. 2.Figure 8 As shown, the first activity degree includes the flexion angle and the extension angle of the head and neck, and the first activity degree is used to measure the maximum angle of the head and neck around the positive direction and the reverse direction of the x-axis of the space rectangular coordinate system. The second activity degree includes the left rotation angle and the right rotation angle of the head and neck, and the second activity degree is used to measure the maximum angle of the head and neck around the positive direction and the reverse direction of the y-axis of the space rectangular coordinate system. The third activity degree includes the right lateral flexion angle and the left lateral flexion angle of the head and neck, and the third activity degree is used to measure the maximum angle of the head and neck around the positive direction and the reverse direction of the z-axis of the space rectangular coordinate system. Wherein, the x-axis is a horizontal axis along the left-right direction of the human body, the y-axis is a vertical axis, and the z-axis is a horizontal axis along the front-back direction of the human body.

[0150] It can be seen from the above technical solutions that the measurement method of the head and neck activity degree provided by the embodiments of the present application can realize non-invasive head and neck activity degree estimation compared with the X-ray and CT three-dimensional reconstruction method. Compared with the traditional cervical spine activity degree measuring instrument and other devices, the system almost does not need manual operation when used, and can realize automatic head and neck activity degree estimation. Specifically, the patent does not need to wear a helmet and other additional devices when used, and is more convenient to use. Moreover, the activity degree of the measured person will not be affected by the additional devices. The synchronous motion of the multi-view camera is realized through the acquisition device, thereby reducing the demand for camera calibration of the system and making the multi-view visual data of the measured person collected in the collection booth not disturbed by the background.

[0151] Further, the head and neck pose parameters are obtained based on the pre-configured three-dimensional coordinates of the head and neck key points. Since the head and neck key points focus on the head and neck and have a greater impact on the head and neck pose parameters, the fine reconstruction of the head and neck can be realized, and the accuracy of the head and neck pose parameters is improved.

[0152] Further, the head and neck pose parameters are subjected to regression analysis by using the activity degree estimation model based on the neural network, thereby improving the accuracy of the head and neck pose parameters, i.e., improving the accuracy of the head and neck pose estimation.

[0153] Those skilled in the art can clearly understand that the application can be implemented by means of software plus necessary universal hardware, and of course can also be implemented by means of dedicated hardware including special integrated circuit, special CPU, special memory, special component, etc. Generally, any function completed by computer program can be easily implemented by corresponding hardware, and the specific hardware structure for implementing the same function can also be various, such as analog circuit, digital circuit or special circuit, etc. However, for the application, software program implementation is a better embodiment. Based on such understanding, the technical solution of the application or the part of the application which makes contribution to the prior art can be embodied in the form of software product, which is stored in readable storage medium, such as computer floppy disk, U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a plurality of instructions for making a computer device (which can be personal computer, training device or network device, etc.) execute the method described in various embodiments of the application.

[0154] In the above embodiments, the implementation can be achieved by software, hardware, firmware or any combination thereof, entirely or partially. When implemented by software, the implementation can be achieved in the form of a computer program product, entirely or partially.

[0155] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the flow or function described in the embodiments of the application is generated entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer, training device or data center to another website, computer, training device or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be stored by a computer or a data storage device such as a training device, a data center, etc. integrated with one or more available media sets. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

Claims

1. A system for measuring head and neck mobility, comprising: The application relates to a head and neck activity degree acquisition method and device, and a head and neck activity degree acquisition system. The device comprises a collection device and a host computer connected in communication with the collection device, a multi-view camera and a sliding module are installed on the collection device, the sliding module is composed of a plurality of sliding modules with fixed relative positions, the multi-view camera is composed of a plurality of cameras, and one camera is fixed on one sliding module, and the camera parameters of the multi-view camera are pre-calibrated. The sliding module is used for executing the synchronous sliding operation of the plurality of sliding modules in response to a sliding instruction, so that the plurality of sliding modules slide to the position indicated by the sliding instruction. The multi-view camera is used for executing the synchronous shooting operation of the plurality of cameras in response to a shooting instruction, obtaining the head and neck image shot by each camera at a current time frame, and transmitting the plurality of head and neck images of the current time frame to the host computer. The host computer is used for sending the sliding instruction to the sliding module, sending the shooting instruction to the multi-view camera when the plurality of sliding modules slide to the position indicated by the sliding instruction, receiving the plurality of head and neck images corresponding to each time frame, obtaining the head and neck pose parameters of each time frame, and obtaining the head and neck activity degree based on the head and neck pose parameters of each time frame; the head and neck pose parameters comprise a plurality of rotation angles, and the head and neck activity degree comprises the activity range of the plurality of rotation angles. The host computer is also used for obtaining a calibration verification result according to the three-dimensional coordinates of each head and neck key point, the calibration verification result is used for indicating whether the multi-view camera needs to be recalibrated, sending a calibration sliding instruction to the sliding module when a preset calibration time is reached, and sending a calibration shooting instruction to the multi-view camera; wherein the calibration time comprises a calibration time point of a predicted calibration period, and / or the calibration verification result indicates that the multi-view camera needs to be recalibrated. The sliding module is also used for executing the synchronous sliding operation of the plurality of sliding modules in response to the calibration sliding instruction, so that the plurality of sliding modules slide to a calibration position; the calibration position is pre-configured based on a calibration board, and the calibration board displays a preset random texture.

2. The head and neck range of motion measurement system of claim 1, wherein, The multi-view camera is also used for executing the synchronous shooting operation of the plurality of cameras in response to the calibration shooting instruction, obtaining the random texture image shot by each camera at a current time frame, and transmitting the plurality of random texture images of the current time frame to the host computer, so that the host computer calibrates the camera parameters of the multi-view camera according to the plurality of random texture images, and updates the camera parameters of the multi-view camera. The sliding module is a programmable screw linear sliding module, the screw linear sliding module comprises four single modules respectively installed on four beam columns of a collection pavilion of a frame structure, one sliding module is arranged on one single module, the sliding modules arranged on the four single modules move synchronously up and down, and each camera in the multi-view camera is fixed on each sliding module through a holder.

3. A method of measuring head and neck mobility, characterized in that, The application is applied to a collection device, the collection device is provided with a multi-view camera and a sliding module, the sliding module is composed of a plurality of sliding modules with fixed relative positions, the multi-view camera is composed of a plurality of cameras, and one camera is fixed on one sliding module, wherein the camera parameters of the multi-view camera are pre-calibrated; the head and neck activity measurement method comprises: The sliding module performs synchronous sliding operation of the plurality of sliding modules in response to a sliding instruction, so that the plurality of sliding modules slide to the position indicated by the sliding instruction; The multi-view camera performs synchronous shooting operation of the plurality of cameras in response to a shooting instruction, obtains head and neck images shot by each camera at a current time frame, and transmits a plurality of head and neck images of the current time frame to a host computer, so that the host computer receives a plurality of head and neck images corresponding to each time frame, obtains head and neck pose parameters of each time frame, and obtains head and neck activity based on the head and neck pose parameters of each time frame; the head and neck pose parameters comprise a plurality of rotation angles, and the head and neck activity comprises an activity range of the plurality of rotation angles; The head and neck activity measurement method further comprises: The sliding module performs synchronous sliding operation of the plurality of sliding modules in response to a calibration sliding instruction, so that the plurality of sliding modules slide to a calibration position, the calibration position is pre-configured based on a calibration plate, and the calibration plate displays a preset random texture; the calibration sliding instruction is sent by the host computer when a preset calibration time is reached, the calibration time comprises a calibration time point of a predicted calibration period, and / or a calibration check result indicates that the multi-view camera needs to be recalibrated; The multi-view camera performs synchronous shooting operation of the plurality of cameras in response to a calibration shooting instruction, obtains random texture images shot by each camera at a current time frame, and transmits a plurality of random texture images of the current time frame to the host computer, so that the host computer calibrates the camera parameters of the multi-view camera according to the plurality of random texture images, and updates the camera parameters of the multi-view camera; the calibration shooting instruction is sent by the host computer after the plurality of sliding modules slide to the calibration position.

4. A method of measuring head and neck mobility, characterized in that, The application is applied to a host computer, the host computer is in communication connection with a collection device, the collection device is provided with a multi-view camera and a sliding module, the sliding module is composed of a plurality of sliding modules with fixed relative positions, the multi-view camera is composed of a plurality of cameras, and one camera is fixed on one sliding module, the camera parameters of the multi-view camera are pre-calibrated, and the head and neck activity measurement method comprises: Sliding instruction is sent to the sliding module, so that the plurality of sliding modules slide to the position indicated by the sliding instruction; After the plurality of sliding modules slide to the position indicated by the instruction, a shooting instruction is sent to the multi-view camera, so that the multi-view camera performs synchronous shooting operation of the plurality of cameras, obtains head and neck images shot by each camera at a current time frame, and transmits a plurality of head and neck images of the current time frame to the host computer; For each time frame, three-dimensional coordinates of each preset head and neck key point are obtained according to the head and neck images corresponding to the time frame; For each time frame, the three-dimensional coordinates of each head and neck key point are input into a pre-trained activity estimation model to obtain head and neck pose parameters output by the activity estimation model, the head and neck pose parameters including a plurality of rotation angles, and the activity estimation model being constructed based on a neural network; Based on the head and neck pose parameters of each time frame, a head and neck activity is obtained, the head and neck activity including a plurality of rotation angle activity ranges; Wherein, the target time frame is any one time frame, and the three-dimensional coordinates of each preset head and neck key point are obtained according to the head and neck image corresponding to the target time frame, including: For each of the head and neck key points, two-dimensional coordinates of the head and neck key point in each head and neck image corresponding to the target time frame are obtained to obtain a two-dimensional coordinate combination of the head and neck key point, the two-dimensional coordinate combination of the head and neck key point including two-dimensional coordinates corresponding to each head and neck image of the head and neck key point; The three-dimensional coordinates of the head and neck key point are calculated based on the two-dimensional coordinate combination of the head and neck key point through a triangulation algorithm; Wherein, in the method for measuring the head and neck activity, the method further includes: For each of the head and neck key points, the three-dimensional coordinates of the head and neck key point are mapped to two-dimensional coordinates corresponding to each head and neck image as mapped two-dimensional coordinates based on an inverse algorithm of the triangulation algorithm; The mapped two-dimensional coordinates of each head and neck key point and the two-dimensional coordinates corresponding to each head and neck image in the two-dimensional coordinate combination of the head and neck key point are compared to obtain a calibration verification result, the calibration verification result being used to indicate whether the multi-view camera needs to be recalibrated; When a preset calibration occasion is reached, a calibration sliding instruction is sent to the sliding module, the calibration sliding instruction being used to control the plurality of sliding modules to slide to a calibration position, the calibration position being pre-configured based on a calibration board, and the calibration board having a preset random texture displayed thereon; When the plurality of sliding modules slide to the calibration position, a calibration shooting instruction is sent to the multi-view camera, the calibration shooting instruction being used to control the multi-view camera to perform a synchronous shooting operation of the plurality of cameras to obtain random texture images shot by each camera at a current time frame, and the plurality of random texture images of the current time frame are transmitted to the host computer; Wherein, reaching the calibration occasion includes reaching a calibration time of a predicted calibration period, and / or the calibration verification result indicates that recalibration is needed; According to the plurality of random texture images, camera parameters of the multi-view camera are calibrated, and the camera parameters of the multi-view camera are updated.

5. The method of measuring head and neck mobility of claim 4, wherein, The sliding instruction sent to the sliding module causes the plurality of sliding modules to slide to the position indicated by the sliding instruction, including: Based on the height information of the to-be-tested person, a corresponding to-be-tested head and neck height is obtained; based on the to-be-tested head and neck height, a position sliding instruction is generated, the position sliding instruction including positioning information corresponding to the head and neck height; the position sliding instruction is sent to the sliding module, so that the plurality of sliding modules slide to the position indicated by the positioning information; the sliding instruction includes a first sliding instruction.

6. The method of measuring head and neck mobility of claim 4, wherein, The sliding instruction is sent to the sliding module, so that the plurality of sliding modules slide to the position indicated by the sliding instruction, including: At a preset first frequency, at least one to-be-tested image returned by the multi-view camera is received, and it is identified whether the to-be-tested image includes a complete head and neck region. If not, a second sliding instruction is generated based on a plurality of preset positioning points in the to-be-tested image; until the to-be-tested image includes a complete head and neck region, wherein the second sliding instruction includes an upward sliding instruction or a downward sliding instruction, and the positioning points include a plurality of key points located at the edge of the head and neck; the sliding instruction includes the second sliding instruction.

Images