Image processing method and system for displaying back picture in real time

Abstract

The application discloses an image processing method for displaying a back picture in real time, comprising the following steps: after a camera observes a human body, collecting and processing position information of a user relative to a display mirror, and outputting a distance of the human body from a screen; calling a camera to dynamically extract a back feature value of the human body by using the distance of the human body from the screen, fitting an eye height of the user, so as to determine a specific position of the camera; obtaining back image data of the user by using the camera, filling after human back contour recognition by using a human back recognition algorithm, obtaining processed picture information and transmitting the processed picture information to the display mirror, and displaying the processed back information picture on the screen. The application further discloses an image processing system for displaying a back picture in real time. The application solves the problem that a user is difficult to obtain a back picture in real time when facing the intelligent mirror group by using simple equipment combination and use, and has important production value and application space.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, human back detection, and contour boundary extraction, and in particular to an image processing method and system for displaying a back image in real time. BACKGROUND

[0002] With the development of image processing technology, there are currently many image processing technologies using computer vision that can capture and process real-time images to assist users in obtaining the desired real-time image effects, such as "virtual fitting mirror", "virtual makeup mirror", and other devices. These devices, which combine a camera and a display screen, use computer image processing technology to achieve the desired effect without having to undress or apply makeup directly. The combination of a camera and a display screen, i.e., a "smart mirror set", is a key technology for the above-mentioned application scenarios.

[0003] In recent years, the rapid development of computer vision combined with image processing has given birth to a series of practical application technologies. The key to this application technology lies in the three aspects of "real-time capture of images", "real-time processing of information", and "real-time display of results". The applications listed above all fall into this category and all involve the processing of human images. However, current technology applications pay too much attention to the capture and processing of human front images, and lack the capture and processing of user back information.

[0004] There is currently a lack of a device that allows users to "mirror" their backs, and a lack of an image processing method that allows users to "face the smart mirror set" to process back information and output it in real time. For personal life, whether it is to see the back of the clothes when trying them on in a store, or to see the back of the hair, these are all concerns of users, but so far there is no device and its accompanying implementation technology that can solve this practical need.

[0005] One of the current existing technologies is a virtual mirror device, system and method (CN116797864A), which captures images from a camera and displays a mirror, observes the user's front information, selects the identified elements, selects a new appearance for the selected elements, and renders a new model based on the original model and the selected new appearance for the elements. Thus, the special display effect of the user's front image is achieved, and the image information can be processed to some extent. The disadvantage of this invention is that although it captures and processes images in real time through a camera and a display mirror, users can only turn around and take pictures to display the back information, and cannot "real-time" face the display mirror.

[0006] The second prior art is an auxiliary makeup method, device, equipment and storage medium based on an intelligent mirror (CN105556508A). The application obtains a target picture and a face picture of a user by using an intelligent mirror, performs feature recognition on the target picture and the face picture respectively to obtain a feature set, and performs image alignment on the target picture and the face picture according to the feature set of the target picture and the feature set of the face picture to obtain an overlapping picture combination. The application can combine and overlap the features of the human face and the preset picture to achieve the display effect of the intelligent makeup by “masking” the “user face” with the “makeup picture”. The application has the following disadvantages: the feature recognition technology used in the application can only process the information of the human face and cannot process the information of the back of the user, and the “masking” method used in the application cannot enable the user to obtain the back picture in real time. SUMMARY

[0007] The application aims to overcome the disadvantages of the prior art and provides an image processing method and system for displaying a back picture in real time. The main problems solved by the application are as follows: 1) how to solve the problem that the existing “virtual fitting type” mirror set can only obtain the front picture information of the user due to the same side binding of the camera and the real mirror, and the user cannot “see in real time” when turning around to obtain the back picture; and 2) how to solve the problem that the existing “intelligent auxiliary type” mirror set is also bound on the same side of the camera and the display mirror, cannot face the display mirror to see the back, and although the features of the user and the target picture can be extracted to display the result, the application only focuses on the feature value of the face of the user and does not capture and process the feature value of the back of the human body.

[0008] To solve the above problems, the application provides an image processing method for displaying a back picture in real time, which comprises the following steps:

[0009] Obtaining the position of the user: after the camera is turned on, the initial observation is performed, the human body is observed, the position information of the user relative to the display mirror is calculated and confirmed, the position information of the user relative to the display mirror is processed, the distance between the processed human body and the screen is obtained to represent the distance to be extended in the longitudinal depth axis of the camera and is outputted;

[0010] Adjusting the specific position of the camera: the distance between the processed human body and the screen is used to call the camera to the position of the user and the screen extended by one time, the longitudinal depth coordinate is taken, the feature value of the back of the human body is dynamically extracted, the eye height of the user is fitted, and the specific position of the camera is determined;

[0011] Obtaining the back of the human body picture: using the camera to obtain the image data of the back of the user, extracting the image of the back of the user and the data outside the screen area through the back of the human body recognition algorithm, dividing into available part and filling part, and combining the image RGB matrix of the available part and the image RGB matrix of the preset plane mirror material to obtain the processed picture information;

[0012] Outputting the image processing result: the mirror receives the processed picture information and outputs the result to the screen interface, and the image processing result is the back information picture finally seen by the user.

[0013] Preferably, the back of the human body feature value is dynamically extracted, and the height of the user's eye is fitted to determine the specific position of the camera, specifically:

[0014] The image matrix of the observed human body in real time is recorded, and a "coarse and fine combination" registration strategy is used for image information;

[0015] Firstly, the human body posture point cloud is preprocessed, then the NDT algorithm is used for coarse registration of the processed point cloud to provide an ideal initial pose for fine registration, then the 3D-Harris feature point detection algorithm is used to extract the point cloud feature points, and finally the ICP algorithm is used for fine registration of the point cloud set after extracting the feature points to obtain the real-time human body point cloud information;

[0016] Then, through the human body posture recognition method, the key feature nodes of the back of the human body are obtained according to the real-time human body point cloud information, the fitting values of the left eye and the right eye are x1 and x2 respectively, the horizontal offset positions are y1 and y2 respectively, the user position information z1 is called, the camera to the user and the screen is extended by one time, the longitudinal coordinate is z2, and z2 is twice z1, the center positions of the two eyes are x3 and y3, and the camera center position is determined through the three values of x3, y3 and z2. At this time, the camera is in a horizontal state facing the mirror.

[0017] Preferably, the back of the human body picture: using the camera to obtain the image data of the back of the user, extracting the image of the back of the user and the data outside the screen area through the back of the human body recognition algorithm, dividing into available part and filling part, and combining the image RGB matrix of the available part and the image RGB matrix of the preset plane mirror material to obtain the processed picture information, specifically:

[0018] The human back recognition algorithm is used to process the user back image data, and the user back image is divided into an available part and a filling part, and the image RGB matrix of the available part and the image RGB matrix of the preset plane mirror material are combined, wherein the available part is the information within the human back contour and outside the plane mirror contour observed by the camera when the camera is horizontal to the plane mirror, and the filling part is the picture area formed by the human back contour and the plane mirror contour at this time;

[0019] In the identification of the human body contour, the techniques of flood filling, threshold processing and edge detection are adopted. The flood filling algorithm sets the node value of the image information observed by the camera and the key feature nodes of the human back posture to 1, compares the adjacent pixel values from the key feature nodes of the human back posture, sets the points with a difference exceeding a preset threshold to 0, and finally obtains an area binary matrix about the human back area;

[0020] The edge detection is performed on the area binary matrix to obtain the contour of the human back. Firstly, the horizontal gradient image and the vertical gradient image are obtained by deriving the pixel value of the original image, specifically:

[0021] p ′ [i]=p[i]-p[i-1]

[0022] p ′ [j]=p[j]-p[j-1]

[0023] Wherein, p[i] is the pixel value vector of the image p in the ith row, p[j] is the pixel value vector of the image p in the jth column, the middle value of the adjacent two pixel vectors is taken, and the cross-correlation kernel K is extracted The cross-correlation operation is performed on the cross-correlation kernel K and the area binary matrix, so as to obtain the contour matrix;

[0024] The contour matrix is shifted by two pixels in the horizontal and vertical coordinates, and the redundancy of human dynamic activity is increased to modify the offset;

[0025] After edge detection, the points on the contour boundary are encoded and tracked in the form of chain code, and the specific process is as follows: taking the starting point of the contour matrix as the current point, detecting the nearest pixel point from the current point, if there is also a pixel value detected as a contour edge point in the adjacent pixel point, recording the chain code value corresponding to the relative direction of the current point, that is, the vector of the current point pointing to the next pixel point, recording the vector, and then setting the pointed point as the next contour current point, repeating the operation until no contour point is detected, and finally obtaining a chain code sequence corresponding to the contour, the chain code sequence includes the starting point of the contour and the next direction chain code value of each contour point, so as to realize chain code tracking of the contour, and a set of human body boundary contour chain code values which can be quickly accessed are generated;

[0026] Finally, a pre-stored flat mirror material module matrix is loaded, the matrix is used for displaying the area within the mirror frame, and the difference set is formed by subtracting the closure formed by the chain code sequence corresponding to the contour, so as to cover the image information between the human body back contour boundary and the flat mirror, and the 4*4 pixel points around the human body back contour boundary are modified and recalculated after the two pictures are overlapped, and the edge pixel value matrix of the contour is obtained, and the specific process is as follows:

[0027]

[0028]

[0029] So as to obtain an output image matrix, that is, the picture information after the processing is completed.

[0030] Correspondingly, the application also provides an image processing system for displaying a back picture in real time, comprising:

[0031] A distance obtaining unit is used for obtaining the position of a user: after the camera is turned on, initial observation is performed, after the human body is observed, the position information of the user relative to the display mirror is calculated and confirmed, the position information of the user relative to the display mirror is processed, the distance of the human body from the screen after processing is obtained, which is used to represent the distance to be extended in the longitudinal depth axis of the camera and output the distance;

[0032] A precise positioning unit is used for adjusting the specific position of the camera: the distance of the human body from the screen after processing is used to call the camera to the position of the user and the screen extended by one time, the longitudinal depth coordinate is taken, the feature value of the back of the human body is dynamically extracted, the height of the eye of the user is fitted, so as to determine the specific position of the camera.

[0033] A picture processing unit is configured to acquire a human back picture: acquire image data of the back of the user by using a camera, extract the image of the back of the user and data outside the screen area by using a human back recognition algorithm, divide the data into an available part and a filling part, and combine the image RGB matrix of the available part and the image RGB matrix of the preset plane mirror material to obtain processed picture information;

[0034] A result output unit is configured to output image processing results: the mirror receives the processed picture information, and outputs the results to a screen interface, and the image processing results are the final back information picture seen by the user.

[0035] The present application has the following beneficial effects:

[0036] The present application is based on the pain points of users and the actual needs of users, and a feasible method is explored by using simple device combination and use, so as to solve the problem that it is difficult for users to acquire a back picture in real time when facing the smart mirror group. This has important production value and application space for device production. The device of the present application is attached with a method for solving the observation conflict, that is, a method of contour boundary extraction and preset image mask, which solves the screen cycle problem in the mirror and ensures the normal use of the device. The image processing method used in the present application quickly finds the boundary of the back of the human body by using the methods of flooding filling, threshold processing and edge detection, can respond to human body actions more quickly, and is more timely and accurate for picture output, so that the user can have a better 'back' experience. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a flow chart of an image processing method for real-time display of a back picture according to an embodiment of the present application;

[0038] Figure 2 is a flow chart of human back contour extraction according to an embodiment of the present application;

[0039] Figure 3 is a structure diagram of an image processing system for real-time display of a back picture according to an embodiment of the present application. DETAILED DESCRIPTION

[0040] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0041] Figure 1 is a flow chart of an image processing method for real-time display of a back picture according to an embodiment of the present application, as shown in Figure 1As shown, the method comprises:

[0042] S1, obtaining user position: after the camera is turned on, an initial observation is performed, after a human body is observed, position information of the user relative to the display mirror is calculated and confirmed, the position information of the user relative to the display mirror is processed, a distance of the processed human body from the screen is obtained, which is used to represent a distance to be extended in the longitudinal depth axis of the camera and is outputted;

[0043] S2, adjusting specific position of camera: the distance of the processed human body from the screen is used to call the camera to a position one time longer than the user and the screen, a longitudinal depth coordinate is taken, a feature value of a back of the human body is dynamically extracted, an eye height of the user is fitted, and thus the specific position of the camera is determined;

[0044] S3, obtaining back of human body picture: user back image data is obtained by using the camera, user back image and data outside the screen area are extracted by a human back recognition algorithm, are divided into a usable part and a filling part, and an image RGB matrix of the usable part and an image RGB matrix of a preset plane mirror material are combined to obtain processed picture information;

[0045] S4, outputting image processing result: the display mirror receives the processed picture information, and outputs the result to a screen interface, and the image processing result is a back information picture finally seen by the user.

[0046] Step S2 is specifically as follows:

[0047] S2-1: a human body image matrix observed in real time is recorded, a registration strategy of “coarse and fine combination” is used for image information: first, a human body posture point cloud is preprocessed, then an NDT algorithm is used for coarse registration of the processed point cloud to provide an ideal initial pose for fine registration, then a 3D-Harris feature point detection algorithm is used to extract point cloud feature points, and finally an ICP algorithm is used for fine registration of the point cloud set after the feature points are extracted to obtain real-time human body point cloud information;

[0048] S2-2: through a human body posture recognition method, key feature nodes of a back posture of the human body are obtained according to the real-time human body point cloud information, fitting values of left and right eyes are x1 and x2 respectively, horizontal offset positions are y1 and y2 respectively, a user position information z1 is called to a position one time longer than the user and the screen, a longitudinal depth coordinate z2 is taken, z2 is twice z1, a center position x3 and y3 of the two eyes are taken, and the camera center position is determined through the three values x3, y3 and z2, at this time, the camera is in a horizontal direction to the display mirror.

[0049] Step S3 is specifically as follows:

[0050] S3-1: processing the user back image data using the human back recognition algorithm, dividing the user back image into an available part and a filling part, and combining the image RGB matrix of the available part and the image RGB matrix of the preset plane mirror material, wherein the available part is the information within the human back contour and outside the plane mirror contour observed by the camera when the camera is horizontal to the plane mirror, and the filling part is the picture area composed of the human back contour and the plane mirror contour at this time;

[0051] S3-2: the recognition process of the human back contour is as shown in Figure 2 When recognizing the human contour, the techniques of flood filling, threshold processing and edge detection are adopted, the flood filling algorithm sets the node value of the image information observed by the camera and the key feature nodes of the human back posture to 1, starts from the key feature nodes of the human back posture, compares the adjacent pixel values, sets the points with a difference exceeding the preset threshold to 0, and finally obtains an area binary matrix about the human back area;

[0052] Edge detection is performed on the area binary matrix to obtain the contour of the human back, first, the horizontal gradient image and the vertical gradient image are obtained by taking the derivative of the pixel value of the original image, specifically:

[0053] p ′ [i]=p[i]-p[i-1]

[0054] p ′ [j]=p[j]-p[j-1]

[0055] Wherein, p[i] is the pixel value vector of image p in the ith row, p[j] is the pixel value vector of image p in the jth column, in order to facilitate operation, the middle value of the adjacent two pixel vectors is taken, and the cross-correlation kernel K is extracted Cross-correlation operation is performed on the cross-correlation kernel K and the area binary matrix, so as to obtain the contour matrix;

[0056] The contour matrix is shifted by increasing the horizontal and vertical coordinates by two pixels, to increase the redundancy of human dynamic activity to modify the offset;

[0057] S3-3: After edge detection, the points on the contour boundary are encoded and tracked in the form of chain code, and the specific process is as follows: taking the starting point of the contour matrix as the current point, detecting the nearest pixel point from the current point, if there is also a pixel value detected as a contour edge point in the adjacent pixel point, recording the chain code value corresponding to the relative direction of the current point, that is, the vector of the current point pointing to the next pixel point, recording the vector, and then setting the pointed point as the next contour current point, repeating the operation until no contour point is detected, and finally obtaining a chain code sequence corresponding to the contour, the chain code sequence includes the contour starting point and the next direction chain code value of each contour point, so as to realize chain code tracking of the contour, and a set of human body boundary contour chain code values which can be quickly accessed are generated;

[0058] S3-4: Finally, a pre-stored plane mirror material module matrix is loaded, which is a matrix for displaying the area within the mirror frame, and the difference set is formed by taking the closure of the chain code sequence corresponding to the contour, so as to cover the image information between the human body back contour boundary and the plane mirror, and after the two pictures are overlapped, the 4*4 pixel points around the human body back contour boundary are modified and recalculated to obtain the edge pixel value matrix of the contour, and the specific process is as follows:

[0059]

[0060]

[0061] So as to obtain the output image matrix, that is, the picture information after the processing is completed.

[0062] Correspondingly, the application also provides an image processing system for displaying the back picture in real time, as shown in Figure 3 The image processing system comprises:

[0063] A distance acquisition unit 1 is used for obtaining the position of a user: after the camera is turned on, initial observation is performed, and after the human body is observed, the position information of the user relative to the display mirror is calculated and confirmed, the position information of the user relative to the display mirror is processed, the distance of the processed human body from the screen is obtained, which is used to represent the distance to be extended in the longitudinal depth axis of the camera and is outputted;

[0064] A precise positioning unit 2 is used for adjusting the specific position of the camera: the distance of the processed human body from the screen is used to call the camera to the position of the user and the screen extended by one time, the longitudinal depth coordinate is taken, the feature value of the back of the human body is dynamically extracted, the height of the eyes of the user is fitted, and thus the specific position of the camera is determined.

[0065] The picture processing unit 3 is used for obtaining a human back picture, obtaining user back image data by using a camera, extracting a user back image and data outside a screen area by using a human back recognition algorithm, dividing the data into an available part and a filling part, combining an image RGB matrix of the available part and an image RGB matrix of a preset plane mirror material, and obtaining processed picture information;

[0066] The result output unit 4 is used for outputting image processing results, receiving the processed picture information by using a mirror, and outputting results to a screen interface. The image processing results are finally seen by a user as back information pictures.

[0067] Therefore, based on user pain points and actual user needs, a feasible method is explored by using simple device combination and use, so as to solve the problem that a user cannot obtain a back picture in real time when facing the smart mirror group. This has important production value and application space for device production. The device of the present application is attached with a method for solving the observation conflict, that is, a method of contour boundary extraction and preset image mask, which solves the screen cycle problem in the mirror and guarantees normal use of the device. The image processing method used in the present application quickly finds the human back boundary by using the methods of flooding filling, threshold processing and edge detection, can more quickly respond to human actions, and is more timely and accurate for picture output, so that the user can have a better back observation experience.

[0068] The above describes a real-time display back picture image processing method and system provided by the embodiment of the present application in detail. The principle and implementation mode of the present application are described by using specific examples. The above embodiment description is only used for helping to understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed. In summary, the content of the present application should not be understood as a limitation of the present application.

Claims

1. An image processing method for real-time display of a rear view, characterized in that, The method includes: Obtaining user position: After the camera is turned on, an initial observation is performed. After the user is observed, the position information of the user relative to the display mirror is calculated and confirmed. The position information of the user relative to the display mirror is processed to obtain the processed distance z1 of the user from the screen, which is used to represent the distance that the camera needs to extend along the depth axis and is output. Adjust the specific position of the camera: using the processed distance z1 between the user and the screen, move the camera to a position where the distance between the user and the screen is twice the length of the distance between the user and the screen, take the depth coordinates, dynamically extract the feature values ​​of the user's back, fit the user's eye height, and thus determine the specific position of the camera. Acquire user's back view: Use a camera to acquire image data of the user's back, extract the user's back image and data outside the screen area through a user back recognition algorithm, divide it into usable part and filled part, and combine the RGB matrix of the usable part image with the RGB matrix of the preset display mirror material image to obtain the processed image information; Output image processing results: The display mirror receives the processed image information and outputs the results to the screen interface. The image processing result is the back information image that the user finally sees. Specifically, using the processed distance z1 between the user and the screen, the camera is positioned at a point where the distance between the user and the screen is twice the length of the screen. The depth coordinates are then taken, and the feature values ​​of the user's back are dynamically extracted to fit the user's eye height, thereby determining the specific position of the camera. Specifically: The system employs a "coarse-fine combined" registration strategy for the user image matrix information by inputting the real-time observed user image matrix. First, the user pose point cloud is preprocessed. Then, the NDT algorithm is used to perform coarse registration on the processed point cloud to provide a more ideal initial pose for fine registration. Next, the 3D-Harris feature point detection algorithm is used to extract feature points from the point cloud. Finally, the ICP algorithm is used to perform fine registration on the point cloud set after the feature points are extracted to obtain real-time user point cloud information. Then, using the user pose recognition method, based on the real-time user point cloud information, the key feature nodes of the user's back pose are obtained. For the fitted values ​​of the left and right eyes, their height values ​​are x1 and x2, respectively, and their horizontal offset positions are y1 and y2, respectively. For the processed distance z1 between the user and the screen, the camera is moved to a position where the distance between the user and the screen is extended by one time, and the depth coordinate is z2, which is twice z1. The center positions of the two eyes are x3 and y3. The center position of the camera is determined by the three values ​​of x3, y3 and z2. At this time, the camera is in a state of horizontal orientation towards the display mirror.

2. The image processing method for real-time display of a rear view as described in claim 1, characterized in that, The process of acquiring the user's back image involves using a camera to capture the image data of the user's back, extracting the user's back image and data outside the screen area using a user back recognition algorithm, dividing it into usable and filled portions, and combining the RGB matrix of the usable portion with the RGB matrix of a preset display lens material to obtain the processed image information. Specifically: The user back recognition algorithm is used to process the user back image data, dividing the user back image into a usable part and a filled part, and combining the image RGB matrix of the usable part with the image RGB matrix of the preset display mirror material. The usable part is the information inside the user back outline and outside the display mirror outline observed when the camera is horizontally facing the display mirror, and the filled part is the screen area composed of the user back outline and the display mirror outline at this time. A flooding filling algorithm is used to identify user contours, and finally a binary area matrix is ​​obtained about the area of ​​the user's back. Edge detection is performed on the area binary matrix to obtain the contour of the user's back. First, the horizontal gradient image and the vertical gradient image are obtained by differentiating the pixel values ​​of the original image, specifically: p[i] ′ =p[i]-p[i-1] p[j] ′ =p[j]-p[j-1] Where p[i] is the pixel value vector of image p in the i-th row, and p[j] is the pixel value vector of image p in the j-th column. The cross-correlation kernel is extracted by taking the median value of two adjacent pixel vectors. A cross-correlation operation is performed on the cross-correlation kernel K and the area binary matrix to obtain the contour matrix; For the contour matrix, add a two-pixel displacement to the horizontal and vertical coordinates to increase the redundancy of user dynamic activities and modify the offset; After edge detection, the points on the contour boundary are encoded and tracked using chain code. The specific process is as follows: taking the starting point of the contour matrix as the current point, the nearest pixel to the current point is detected. If a pixel value among the neighboring pixels is also detected as a point on the contour edge, the chain code value corresponding to the relative direction of the current point is recorded, which is the vector pointing from the current point to the next pixel. This vector is recorded, and then the pointed point is set as the current point of the next contour. This operation is repeated until no contour points are detected. Finally, a chain code sequence corresponding to the contour is obtained. The chain code sequence includes the starting point of the contour and the direction chain code value next to each contour point, thereby realizing the chain code tracking of the contour and generating a set of user boundary contour chain code values ​​that can be quickly accessed. Finally, the pre-stored display mirror material module matrix is ​​loaded. This matrix represents the area within the display mirror's border. By taking the difference set of the closure formed by the chain code sequence corresponding to the contour, the image information between the user's back contour boundary and the display mirror is covered. After overlaying the two images, the 4*4 pixels around the user's back contour boundary are refined and recalculated to obtain the edge pixel value matrix of the contour, specifically: This yields the output image matrix, which is the processed image information.

3. An image processing system for real-time display of a rear view, characterized in that, The system includes: The spacing acquisition unit is used to obtain the user's position: after the camera is turned on, an initial observation is performed. After the user is observed, the position information of the user relative to the display mirror is calculated and confirmed. The position information of the user relative to the display mirror is processed to obtain the processed distance z1 of the user from the screen, which is used to represent the distance that the camera needs to extend along the depth axis and is output. A precise positioning unit is used to adjust the specific position of the camera: using the processed distance z1 between the user and the screen, the camera is moved to a position where the distance between the user and the screen is twice the length of the distance between the user and the screen, the depth coordinates are taken, the feature values ​​of the user's back are dynamically extracted, and the user's eye height is fitted to determine the specific position of the camera. The image processing unit is used to acquire the user's back image: it uses a camera to acquire the user's back image data, extracts the user's back image and the data outside the screen area through the user's back recognition algorithm, divides it into the usable part and the filling part, and combines the image RGB matrix of the usable part with the image RGB matrix of the preset display mirror material to obtain the processed image information. The result output unit is used to output the image processing result: the display mirror receives the processed image information and outputs the result to the screen interface. The image processing result is the back information image that the user finally sees. Specifically, in the precise positioning unit, using the processed distance z1 between the user and the screen, the camera is positioned at a point where the distance between the user and the screen is twice the length of the screen. The depth coordinates are then taken, and the feature values ​​of the user's back are dynamically extracted to fit the user's eye height, thereby determining the specific position of the camera. Specifically: The system employs a "coarse-fine combined" registration strategy for the user image matrix information by inputting the real-time observed user image matrix. First, the user pose point cloud is preprocessed. Then, the NDT algorithm is used to perform coarse registration on the processed point cloud to provide a more ideal initial pose for fine registration. Next, the 3D-Harris feature point detection algorithm is used to extract feature points from the point cloud. Finally, the ICP algorithm is used to perform fine registration on the point cloud set after the feature points are extracted to obtain real-time user point cloud information. Then, using the user pose recognition method, based on the real-time user point cloud information, the key feature nodes of the user's back pose are obtained. For the fitted values ​​of the left and right eyes, their height values ​​are x1 and x2, respectively, and their horizontal offset positions are y1 and y2, respectively. For the processed distance z1 between the user and the screen, the camera is moved to a position where the distance between the user and the screen is extended by one time, and the depth coordinate is z2, which is twice z1. The center positions of the two eyes are x3 and y3. The center position of the camera is determined by the three values ​​of x3, y3 and z2. At this time, the camera is in a state of horizontal orientation towards the display mirror.

4. The image processing system for real-time display of a rear view as described in claim 3, characterized in that, In the image processing unit, the user's back image is acquired by using a camera to obtain the image data of the user's back. A user back recognition algorithm is used to extract the image of the user's back and the data outside the screen area, dividing them into usable and filled portions. The RGB matrix of the usable portion is then combined with the RGB matrix of a preset display lens material to obtain the processed image information. Specifically: The user back recognition algorithm is used to process the user back image data, dividing the user back image into a usable part and a filled part, and combining the image RGB matrix of the usable part with the image RGB matrix of the preset display mirror material. The usable part is the information inside the user back outline and outside the display mirror outline observed when the camera is horizontally facing the display mirror, and the filled part is the screen area composed of the user back outline and the display mirror outline at this time. A flooding filling algorithm is used to identify user contours, and finally a binary area matrix is ​​obtained about the area of ​​the user's back. Edge detection is performed on the area binary matrix to obtain the contour of the user's back. First, the horizontal gradient image and the vertical gradient image are obtained by differentiating the pixel values ​​of the original image, specifically: p[i] ′ =p[i]-p[i-1] p[j] ′ =p[j]-p[j-1] Where p[i] is the pixel value vector of image p in the i-th row, and p[j] is the pixel value vector of image p in the j-th column. The cross-correlation kernel is extracted by taking the median value of two adjacent pixel vectors. A cross-correlation operation is performed on the cross-correlation kernel K and the area binary matrix to obtain the contour matrix; For the contour matrix, add a two-pixel displacement to the horizontal and vertical coordinates to increase the redundancy of user dynamic activities and modify the offset; After edge detection, the points on the contour boundary are encoded and tracked using chain code. The specific process is as follows: taking the starting point of the contour matrix as the current point, the nearest pixel to the current point is detected. If a pixel value among the neighboring pixels is also detected as a point on the contour edge, the chain code value corresponding to the relative direction of the current point is recorded, which is the vector pointing from the current point to the next pixel. This vector is recorded, and then the pointed point is set as the current point of the next contour. This operation is repeated until no contour points are detected. Finally, a chain code sequence corresponding to the contour is obtained. The chain code sequence includes the starting point of the contour and the direction chain code value next to each contour point, thereby realizing the chain code tracking of the contour and generating a set of user boundary contour chain code values ​​that can be quickly accessed. Finally, the pre-stored display mirror material module matrix is ​​loaded. This matrix represents the area within the display mirror's border. By taking the difference set of the closure formed by the chain code sequence corresponding to the contour, the image information between the user's back contour boundary and the display mirror is covered. After overlaying the two images, the 4*4 pixels around the user's back contour boundary are refined and recalculated to obtain the edge pixel value matrix of the contour, specifically: This yields the output image matrix, which is the processed image information.

Images