Image compression method, device and equipment

Abstract

The invention discloses an image compression method, device and equipment, and relates to the technical field of image processing, and the method comprises the steps: obtaining a sampling region in a to-be-compressed image; wherein the sampling area comprises a fixation area and an interval sampling area; acquiring a sampling mode corresponding to each sampling area; wherein the sampling rate of the fixation area in the sampling area is maximum; sampling the pixel points in each sampling area according to the sampling mode corresponding to each sampling area to obtain a sampling compression image corresponding to the to-be-compressed image; by utilizing the characteristic that the fixation point of the eyes of the user is not in the whole image but focuses on a certain area, the area outside the fixation area can be compressed, so that the compression transmission of the image is realized on the basis of not influencing the watching effect of the head-mounted display equipment as far as possible, the image transmission and coding and decoding delay is reduced, and the user experience is improved. And the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image compression method, device and equipment. Background Art

[0002] With the development of modern science and technology, head-mounted display devices such as VR (Virtual Reality) devices, AR (Augmented Reality) devices and MR (Mix Reality) devices are becoming more and more widely used. In the actual use of head-mounted display devices, in order to use rich content resources, head-mounted display devices need to use streaming technology to stream content on terminal devices (such as mobile phones or computers) to the head-mounted display devices for display.

[0003] Since the overall delay of the video codec (encoder and decoder) is directly related to the number of bits that need to be encoded, the transmission of high-resolution images from the terminal device to the head-mounted display device will affect the transmission and display rate of the head-mounted display device and the experience of the head-mounted display device. Therefore, how to compress the image to be displayed by the head-mounted display device without affecting the viewing effect of the head-mounted display device as much as possible, reduce the image transmission and encoding and decoding delays, and improve the user experience is an urgent problem that needs to be solved today. Summary of the invention

[0004] The purpose of the present invention is to provide an image compression method, device and equipment to compress the image to be displayed by a head-mounted display device, reduce image transmission and encoding and decoding delays, and improve user experience.

[0005] In order to solve the above technical problems, the present invention provides an image compression method, comprising:

[0006] Acquire a sampling area in the image to be compressed; wherein the sampling area includes a fixation area and an interval sampling area;

[0007] Obtaining a sampling mode corresponding to each sampling area; wherein the sampling rate of the attention area in the sampling area is the largest;

[0008] The pixel points in each sampling area are sampled according to the sampling mode corresponding to each sampling area, so as to obtain a sampled compressed image corresponding to the image to be compressed.

[0009] In some embodiments, obtaining a sampling area in the image to be compressed includes:

[0010] Acquire a fixation point in the image to be compressed;

[0011] The sampling area in the image to be compressed is determined according to the gaze point.

[0012] In some embodiments, obtaining the gaze point in the image to be compressed includes:

[0013] The gaze point in the image to be compressed is located according to the eye feature image collected by the eye tracking camera in the head mounted display device.

[0014] In some embodiments, determining the sampling area in the image to be compressed according to the gaze point includes:

[0015] According to the gaze point, a pair of dividing lines in the image to be compressed is determined; wherein the dividing lines include a horizontal dividing line and a vertical dividing line, the horizontal dividing line is perpendicular to the vertical dividing line and the intersection point is the gaze point;

[0016] Determining a segmentation point on the dividing line according to the gaze point and the dividing line;

[0017] The image to be compressed is segmented using the segmentation lines at the segmentation points to obtain the sampling areas; wherein the segmentation lines include a horizontal segmentation line and a vertical segmentation line that are perpendicular to each other.

[0018] In some embodiments, determining a segmentation point on the dividing line according to the gaze point and the dividing line includes:

[0019] According to the preset initial segmentation length, the gaze point is taken as the origin and the segmentation point is obtained by dividing the four segmentation line segments in a geometric manner; wherein the segmentation line segments include the upward segment and the downward segment obtained by dividing the longitudinal segment from the gaze point and the left segment and the right segment obtained by dividing the transverse segment from the gaze point.

[0020] In some embodiments, the segmentation point is obtained by dividing the four segmentation line segments in a geometric manner according to the preset initial segmentation length and taking the gaze point as the origin, including:

[0021] Taking the gaze point as the origin, through a n =a1*q n-1 , respectively divide on the four dividing segmentation line segments to obtain the dividing points; wherein, a n is the line segment between the segmentation point n-1 and the segmentation point n on the segmentation line segment, n is a positive integer greater than or equal to 1, a1 is the line segment between the gaze point and the segmentation point 1 of the segmentation line segment, the length of a1 is the preset initial segmentation length, and q is a value greater than 0 and less than 1;

[0022] Correspondingly, the attention area in the image area to be compressed corresponding to two adjacent dividing line segments is the area enclosed by the attention point on the two adjacent dividing line segments and the dividing line on the dividing point 1, and the interval sampling area in the image area to be compressed corresponding to two adjacent dividing line segments includes the areas outside the attention area obtained by dividing the image area to be compressed using the dividing line on the dividing point on the two adjacent dividing line segments and the image boundary line on the image area to be compressed.

[0023] In some embodiments, the segmentation point includes the gaze point, the segmentation line includes the dividing line and a vertical line perpendicular to any of the dividing lines at a segmentation point other than the gaze point, and the number of the segmentation lines is the number of the segmentation points plus 1.

[0024] In some embodiments, after sampling the pixels in each sampling area according to the sampling mode corresponding to each sampling area to obtain the sampled compressed image corresponding to the image to be compressed, the method further includes:

[0025] The terminal device transmits the sampled compressed image to a head-mounted display device, so that the head-mounted display device displays the sampled compressed image.

[0026] In some embodiments, the acquiring of the sampling modes corresponding to the respective sampling areas includes:

[0027] According to the positions of the interval sampling areas and the attention area, the sampling modes corresponding to the sampling areas are determined.

[0028] In some embodiments, the sampling areas in the image to be compressed are distributed in a matrix, and the determining of the sampling modes corresponding to the sampling areas according to the positions of the interval sampling areas and the attention area includes:

[0029] The sampling mode of the gaze area is set to sample each pixel once;

[0030] The sampling mode of the interval sampling area of ​​the i-th row directly above or below the fixation area is set to every u in the vertical direction. i Pixel collection v i pixels, i and v are integers greater than or equal to 1, and u is a positive integer greater than v;

[0031] The sampling mode of the interval sampling area of ​​the jth column directly to the left or right of the gaze area is set to every u in the horizontal direction. j Pixel collection v j pixels, j is an integer greater than or equal to 1;

[0032] The sampling mode of the interval sampling area in the i-th row above or below the fixation area and the j-th column to the left or right is set to every u j ×u i Pixel collection v j ×v i Pixels.

[0033] The present invention also provides an image compression device, comprising:

[0034] A region acquisition module, used to acquire a sampling region in the image to be compressed; wherein the sampling region includes a fixation region and an interval sampling region;

[0035] A sampling acquisition module, used to acquire the sampling mode corresponding to each sampling area; wherein the sampling rate of the attention area among the sampling areas is the largest;

[0036] The sampling compression module is used to sample the pixel points in each sampling area according to the sampling mode corresponding to each sampling area, so as to obtain a sampled compressed image corresponding to the image to be compressed.

[0037] In addition, the present invention also provides an image compression device, comprising:

[0038] Memory for storing computer programs;

[0039] A processor is used to implement the steps of the image compression method as described above when executing the computer program.

[0040] An image compression method provided by the present invention comprises: obtaining a sampling area in an image to be compressed; wherein the sampling area comprises a focus area and an interval sampling area; obtaining a sampling mode corresponding to each sampling area; wherein the sampling rate of the focus area among the sampling areas is the largest; sampling the pixel points in each sampling area according to the sampling mode corresponding to each sampling area, and obtaining a sampled compressed image corresponding to the image to be compressed;

[0041] It can be seen that the present invention utilizes the characteristic that the user's eye gaze point is not on the entire image, but focuses on a certain area, and adopts the sampling methods corresponding to the gaze area and the interval sampling area in the image to be compressed, and samples and compresses the pixels in the image to be compressed, and obtains the sampled compressed image corresponding to the image to be compressed, which can focus on compressing the area outside the gaze area, thereby realizing the compression of the image to be displayed by the head-mounted display device without affecting the viewing effect of the head-mounted display device as much as possible, reducing image transmission and encoding and decoding delays, and improving user experience. In addition, the present invention also provides an image compression device and equipment, which also have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0043] Figure 1 A flowchart of an image compression method provided by an embodiment of the present invention;

[0044] Figure 2 A flowchart of another image compression method provided by an embodiment of the present invention;

[0045] Figure 3 A schematic diagram of an image segmentation method provided by an embodiment of the present invention;

[0046] Figure 4 A schematic diagram of another image segmentation method provided by an embodiment of the present invention;

[0047] Figure 5 A schematic diagram of a sampling method of another image compression method provided by an embodiment of the present invention;

[0048] Figure 6 A flowchart of another image compression method provided by an embodiment of the present invention;

[0049] Figure 7 A structural block diagram of an image compression device provided by an embodiment of the present invention;

[0050] Figure 8 A schematic diagram of the structure of an image compression device provided by an embodiment of the present invention;

[0051] Fig. 9 A schematic diagram of the specific structure of an image compression device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Please refer to Figure 1 , Figure 1 A flowchart of an image compression method provided by an embodiment of the present invention. The method may include:

[0054] Step 101: Acquire a sampling area in the image to be compressed; wherein the sampling area includes a fixation area and an interval sampling area.

[0055] It is understandable that the image to be compressed in this embodiment may be an image that needs to be compressed, such as an image that needs to be transmitted to the head-mounted display device after being compressed by the terminal device (such as an image that the head-mounted display device needs to display), or an image that the head-mounted display device needs to compress, etc. The sampling areas in this embodiment may be various areas obtained by dividing the image to be compressed; the sampling areas may be divided into the sampling areas where the user's eyes are looking (i.e., the looking areas) and other sampling areas (i.e., the interval sampling areas).

[0056] Correspondingly, the specific manner in which the processor obtains the sampling areas in the image to be compressed in this step can be set by the designer according to practical scenarios and user needs. For example, when the sampling areas in the image to be compressed are distributed in a matrix, that is, the sampling areas in the image to be compressed are areas obtained by dividing the image to be compressed using mutually perpendicular horizontal dividing lines and vertical dividing lines, the processor can use the horizontal dividing lines and the vertical dividing lines to divide the image to be compressed into a preset number of divided areas; the divided area where the gaze point in the image to be compressed is located is determined as the gaze area, and the divided area outside the gaze area is determined as the interval sampling area; that is, the processor can directly use the mutually perpendicular horizontal dividing lines and the vertical dividing lines to divide the image to be compressed to obtain a preset number of divided sampling areas; for example, the processor can use u vertical horizontal dividing lines and v vertical dividing lines to divide the image to be compressed into (w+1)*(z+1) divided areas, such as 1 central gaze area and (w+1)*(z+1)-1 interval sampling areas; wherein w and z can be positive integers greater than or equal to 2, respectively.

[0057] Correspondingly, the processor can also obtain the gaze point in the image to be compressed; determine the sampling area in the image to be compressed according to the gaze point; that is, the processor can use the gaze point in the image to be compressed to divide the sampling area according to the position of the gaze point; if the sampling area in the image to be compressed is distributed in a matrix, the processor can determine a pair of dividing lines in the image to be compressed according to the gaze point; wherein the dividing line includes a horizontal dividing line and a vertical dividing line, the horizontal dividing line is perpendicular to the vertical dividing line and the intersection is the gaze point; according to the gaze point and the dividing line, determine the segmentation point on the dividing line; use the segmentation line on the segmentation point to segment the image to be compressed to obtain the sampling area; wherein the segmentation line includes a horizontal dividing line and a vertical dividing line that are perpendicular to each other. The sampling area in the image to be compressed can also be distributed in other ways, such as each interval sampling area can be surrounded outside the gaze area in turn, such as the first interval sampling area can directly contact and surround the gaze area, and then each interval sampling area can respectively surround the previous interval sampling area. This embodiment does not impose any restrictions on this.

[0058] Step 102: Obtain the sampling mode corresponding to each sampling area; wherein the sampling rate of the fixation area among the sampling areas is the largest.

[0059] It can be understood that this embodiment achieves sampling compression of the image to be compressed by sampling the pixels in each sampling area in the image to be compressed. In this embodiment, the sampling method corresponding to each sampling area can be a method of sampling the pixels in the corresponding sampling area.

[0060] Correspondingly, the specific settings of the sampling methods corresponding to each sampling area in this embodiment can be set by the designer according to practical scenarios and user needs. For example, since the user's eyes will focus on the gaze area, in this embodiment, among the sampling rates corresponding to each sampling area (i.e., the quotient of the number of pixel points sampled in the sampling area and the number of all pixel points in the sampling area), the sampling rate of the gaze area can be the largest. For example, the sampling rate of the gaze area can be 1, that is, each pixel in the gaze area is sampled once to keep the image content of the gaze area unchanged; in a certain direction starting from the center of the gaze area, the closer the interval sampling area is to the gaze area, the greater the sampling rate, that is, the farther the interval sampling area is from the gaze area, the lower the sampling rate, so as to focus on compressing the edge position of the avatar to be compressed that is far away from the gaze area, and try not to affect the user's viewing effect of the image. For example, when each interval sampling area can be surrounded by the gaze area in sequence, the sampling rate of the gaze area can be 1, and the sampling rate of the i-th interval sampling area can be q i , q is a value greater than 0 and less than 1.

[0061] Similarly, the specific method for the processor to obtain the sampling mode corresponding to each sampling area in this step can be set by the designer. For example, when the number and position of the sampling areas in different images to be compressed are fixed and the position of the attention area is fixed, the processor can obtain the sampling mode corresponding to each sampling area set in advance; when the position of the attention area is not fixed, the processor can also determine the sampling mode corresponding to each sampling area according to the position of each interval sampling area and the attention area. This embodiment does not impose any restrictions on this.

[0062] Step 103: sampling the pixels in each sampling area according to the sampling mode corresponding to each sampling area, and obtaining a sampled compressed image corresponding to the image to be compressed.

[0063] It can be understood that in this step, the processor can sample the pixel points in each sampling area according to the sampling method corresponding to each sampling area, and can sample a compressed image (i.e., a sampled compressed image), thereby achieving sampling compression of the image to be compressed.

[0064] It should be noted that, in this embodiment, after the processor obtains the sampled compressed image corresponding to the image to be compressed, the sampled compressed image can be displayed using the head-mounted display device. For example, when the method provided in this embodiment is applied to a terminal device connected to a head-mounted display device, after the processor of the terminal device obtains the sampled compressed image corresponding to the image to be displayed by the head-mounted display device (i.e., the image to be compressed), the sampled compressed image can be transmitted to the head-mounted display device, so that the head-mounted display device displays the sampled compressed image, thereby avoiding the decompression process of traditional image compression transmission, thereby compressing the image to be displayed by the head-mounted display device on the basis of ensuring the image effect viewed by the user on the head-mounted display device as much as possible, and reducing image transmission and encoding and decoding delays.

[0065] In this embodiment, the embodiment of the present invention utilizes the characteristic that the user's eyes focus not on the entire image but on a certain area, and adopts the sampling methods corresponding to the focus area and the interval sampling area in the image to be compressed, and performs sampling compression on the pixels in the image to be compressed, so as to obtain a sampled compressed image corresponding to the image to be compressed, and can focus on compressing the area outside the focus area, thereby realizing compression of the image to be displayed by the head-mounted display device without affecting the viewing effect of the head-mounted display device as much as possible, reducing image transmission and encoding and decoding delays, and improving user experience

[0066] Based on the above embodiment, the present invention also provides another image compression method. Figure 2 , Figure 2 A flowchart of another image compression method provided by an embodiment of the present invention. The method may include:

[0067] Step 201: Obtain a gaze point in the image to be compressed.

[0068] The gaze point in the image to be compressed in this embodiment may be a position point where the eyes of the user gaze when viewing the image to be compressed.

[0069] Specifically, the specific manner in which the processor in this embodiment obtains the gaze point in the image to be compressed can be set by the designer according to the usage scenario and user needs, such as the processor determines the preset point in the image to be compressed as the gaze point; wherein the preset point can be a pre-set position point that the user will most likely focus on when viewing the image to be compressed, such as a center point or a position point of a certain pattern (such as the crosshair of a sight). The processor can also locate the gaze point in the image to be compressed based on the eye feature image collected by the eye tracking camera in the head-mounted display device; for example, the processor of the terminal device can determine the position point at which the user's glasses of the head-mounted display device are currently looking based on the eye feature image collected by the eye tracking camera in the head-mounted display device, thereby correspondingly determining the gaze point in the image to be compressed to achieve dynamic adjustment of the gaze point. The processor can also locate the gaze point in the image to be compressed based on the gaze point information sent by the head-mounted display device; that is, the head-mounted display device can obtain the gaze point information based on the eye feature image collected by its eye tracking camera, and send the gaze point information to the head-mounted display device.

[0070] Correspondingly, the specific method for the above-mentioned processor to locate the gaze point in the image to be compressed based on the eye feature image collected by the eye tracking camera in the head-mounted display device can be set by the designer, such as using the same or similar method as the gaze point tracking method in the existing eye tracking technology. This embodiment does not impose any restrictions on this.

[0071] Step 202: Determine a sampling area in the image to be compressed according to the gaze point; wherein the sampling area includes a gaze area and an interval sampling area.

[0072] It is understandable that in this step, the processor can segment and divide the image to be compressed according to the gaze point in the image to be compressed to obtain the sampling area in the image to be compressed. Accordingly, the gaze area in this embodiment can include the gaze point.

[0073] Correspondingly, the specific manner in which the processor in this embodiment determines the sampling area in the image to be compressed according to the gaze point can be set by the designer according to practical scenarios and user needs. For example, when each interval sampling area surrounds the gaze area in sequence, the processor can divide the gaze area (i.e., circular area) in the image to be compressed with the gaze point as the center and the preset radius length as the radius; then divide the first interval sampling area (i.e., fan-shaped area) in the image to be compressed with the gaze point as the center, the preset outer circle radius corresponding to the first interval sampling area as the outer circle radius, and the preset radius length as the inner circle radius; thereafter, each interval sampling area is divided into each interval sampling area (i.e., fan-shaped area or partial area in the fan) in the image to be compressed with the gaze point as the center, the preset outer circle radius corresponding to the first interval sampling area as the outer circle radius, and the preset outer circle radius corresponding to the previous interval sampling area as the inner circle radius.

[0074] Correspondingly, when the sampling areas in the image to be compressed are distributed in a matrix, the processor can determine a pair of dividing lines in the image to be compressed according to the gaze point; wherein the dividing lines include a horizontal dividing line and a vertical dividing line, the horizontal dividing line is perpendicular to the vertical dividing line and the intersection is the gaze point; according to the gaze point and the dividing line, the dividing point on the dividing line is determined; the image to be compressed is divided using the dividing line on the dividing point to obtain the sampling area; wherein the dividing lines include a horizontal dividing line and a vertical dividing line that are perpendicular to each other.

[0075] It should be noted that the specific method for the processor to determine the segmentation points on the dividing line according to the gaze point and the dividing line can be set by the designer. For example, the processor can divide the segmentation points on the four segmentation line segments in a geometric manner based on the preset initial segmentation length, with the gaze point as the origin; wherein the segmentation line segments include the longitudinal segmentation line dividing from the gaze point to obtain the upward segment and the downward segment, and the transverse segmentation line dividing from the gaze point to obtain the left segment and the right segment. In other words, the processor can divide the segmentation points on the dividing line in a geometric manner with the gaze point in the image to be compressed as the origin.

[0076] For example, the processor can use the gaze point as the origin and pass a n =a1*q n-1 , and divide the segmentation points on the four segmentation segments respectively; among them, a n is the line segment between the segmentation point n-1 and the segmentation point n on the segmentation line segment, n is a positive integer greater than or equal to 1, a1 is the line segment between the gaze point and the segmentation point 1 that divides the segmentation line segment, and the length of a1 is the preset initial segmentation length. q can be a value greater than 0 and less than 1, that is, the length of the line segment between the segmentation point n-1 and the segmentation point n can be greater than the length of the line segment between the segmentation point n and the segmentation point n+1; q can also be a value of 1 or greater than 1, and this embodiment does not impose any restrictions on this.

[0077] Correspondingly, the four dividing line segments can divide the area of ​​the image to be compressed (i.e., the image area to be compressed) into four areas, the attention area in the image area to be compressed (i.e., one of the four areas) corresponding to two adjacent dividing line segments is the area surrounded by the attention points on the two adjacent dividing line segments and the dividing line on the dividing point 1, and the interval sampling area in the image area to be compressed corresponding to the two adjacent dividing line segments includes the areas outside the attention area obtained by dividing the image area to be compressed using the dividing lines on the dividing points on the two adjacent dividing line segments and the image boundary line on the image area to be compressed.

[0078] Wherein, the segmentation point may include the fixation point, the segmentation line may include the dividing line and the vertical line perpendicular to any dividing line at the segmentation point other than the fixation point, and the number of the segmentation lines is the number of the segmentation points plus 1; Figure 3 As shown, a pair of mutually perpendicular dividing lines (i.e., dividing lines) can be set on the gaze point, and a dividing line perpendicular to the dividing line where the dividing point is located can be set on any dividing point other than the gaze point. Figure 4 As shown, the segmentation point includes the gaze point, and the segmentation line may also include a line segment connecting the segmentation points n in two adjacent segmentation line segments and a pair of mutually perpendicular segmentation lines on the gaze point. The pair of segmentation lines on the gaze point may be parallel to some other segmentation lines respectively.

[0079] Correspondingly, the gaze area may be an area (ie, a cell area) including the initial point and surrounded by the segmentation line on the gaze point and the segmentation line on the segmentation point 1 on each segmentation segment, and the interval sampling area may be other sampling areas.

[0080] Step 203: Determine the sampling mode corresponding to each sampling area according to the positions of each interval sampling area and the attention area.

[0081] It can be understood that in this step, the processor can obtain the sampling mode corresponding to each sampling area according to the positions of each interval sampling area and the attention area.

[0082] Correspondingly, in this embodiment, the processor determines the specific method of the sampling method corresponding to each sampling area according to the position of each interval sampling area and the attention area, which can be set by the designer. For example, when the sampling areas in the image to be compressed are distributed in a matrix, in this step, the processor can set the sampling method of the attention area to sample once for each pixel; set the sampling method of the interval sampling area in the i-th row directly above or directly below the attention area to sample once for each u-th row in the vertical direction. i Pixel collection v i pixel points; set the sampling mode of the interval sampling area in the jth column to the left or right of the gaze area to every u in the horizontal directionj Pixel collection v j pixel points; set the sampling mode of the interval sampling area in the i-th row above or below the gaze area and the j-th column to the left or right of the gaze area to every u j ×u i Pixel collection v j ×v i pixel points; wherein i, j and v are integers greater than or equal to 1, and u is a positive integer greater than v; for example, q=v / u.

[0083] like Figure 5 As shown in FIG. 1 , when u is 2 and v is 1, the sampling method of the gaze area can be to sample each pixel once (1×1), the sampling method of the interval sampling area in the first row directly above or below the gaze area can be to sample one pixel point for every 2 pixels in the vertical direction (1×2), and the sampling method of the interval sampling area in the first row directly to the left or right of the gaze area can be to sample one pixel point for every 2 pixels in the horizontal direction (2×1); the sampling method of the interval sampling area in the first row above or below the gaze area and the first column to the left or right of the gaze area is set to sample one pixel point for every 2 pixels in the horizontal direction (2×1). The sampling mode of the interval sampling area in the second row above or below the gaze area and the first column to the left or right is set to collect 1 pixel point for every 2×4 pixels (2×4); the sampling mode of the interval sampling area in the first row above or below the gaze area and the second column to the left or right is set to collect 1 pixel point for every 4×2 pixels (4×2); the sampling mode of the interval sampling area in the second row above or below the gaze area and the second column to the left or right is set to collect 1 pixel point for every 4×4 pixels (4×4).

[0084] Step 204: Sample the pixels in each sampling area according to the sampling method corresponding to each sampling area to obtain a sampled compressed image corresponding to the image to be compressed.

[0085] It can be understood that in this step, the processor can sample the pixel points in each sampling area according to the sampling method corresponding to each sampling area, and can sample a compressed image (i.e., a sampled compressed image), thereby achieving sampling compression of the image to be compressed.

[0086] It should be noted that when the method provided in this embodiment is applied to a terminal device connected to a head-mounted display device, the processor of the terminal device can transmit the sampled compressed image to the head-mounted display device after obtaining the sampled compressed image corresponding to the image that needs to be transmitted to the head-mounted display device (i.e., the image to be compressed), so that the head-mounted display device displays the sampled compressed image. In this way, the image to be displayed by the head-mounted display device can be compressed and transmitted on the basis of ensuring the image effect viewed by the user on the head-mounted display device as much as possible, thereby reducing image transmission and encoding and decoding delays.

[0087] Based on the above embodiment, the present invention also provides another image compression method. Figure 6 , Figure 6 A flowchart of another image compression method provided by an embodiment of the present invention. The method may include:

[0088] Step 301: using a horizontal dividing line and a vertical dividing line to divide the image to be compressed into a preset number of divided areas; wherein the horizontal dividing line is perpendicular to the vertical dividing line.

[0089] It is understandable that in this embodiment, the processor can use mutually perpendicular horizontal dividing lines and vertical dividing lines to divide the image to be compressed into a preset number of divided areas (ie, sampling areas), that is, the divided areas in the image to be compressed can be distributed in a matrix.

[0090] Correspondingly, the specific value of the preset number of divisions in this embodiment, that is, the specific number of the horizontal division lines and the vertical division lines, can be set by the designer according to practical scenarios and user needs. For example, the number of the horizontal division lines and the vertical division lines can be greater than or equal to 2; for example, the number of the horizontal division lines and the vertical division lines can be the same and both are m, and the preset number of divisions can be (m+1) 2 ; m is a positive integer greater than or equal to 2.

[0091] Step 302: Determine the segmented area where the gaze point in the image to be compressed is located as the gaze area, and determine the segmented area outside the gaze area as the interval sampling area.

[0092] In this step, the processor may determine the segmented area where the gaze point is located as the gaze area, and determine the segmented area outside the gaze area as the interval sampling area, so as to obtain all the sampling areas in the image to be compressed.

[0093] Correspondingly, the specific way in which the processor determines the segmented area where the gaze point in the image to be compressed is located as the gaze area in this step, that is, the specific way of determining the gaze point, can be set by the designer. The processor can directly determine the preset point in the image to be compressed as the gaze point, and determine the segmented area where the preset point is located as the gaze area. The processor can also locate the gaze point in the image to be compressed based on the eye feature image collected by the eye tracking camera in the head-mounted display device; and determine the segmented area where the gaze point is located as the gaze area. This embodiment does not impose any restrictions on this.

[0094] Step 303: Obtain the sampling mode corresponding to each sampling area; wherein the sampling rate of the attention area among the sampling areas is the largest.

[0095] It can be understood that, in this embodiment, sampling compression of the image to be compressed is achieved by sampling pixel points in a sampling area distributed in a matrix in the image to be compressed.

[0096] Correspondingly, the specific method for the processor to obtain the sampling method corresponding to each sampling area in this step can be set by the designer. For example, when the position of the attention area in different images to be compressed is fixed, the processor can obtain the sampling method corresponding to each sampling area set in advance; when the position of the attention area is not fixed, the processor can also determine the sampling method corresponding to each sampling area according to the position of each interval sampling area and the attention area; for example, the processor can set the sampling method of the attention area to sample each pixel once; set the sampling method of the interval sampling area of ​​the i-th row directly above or below the attention area to sample each pixel once in the vertical direction. i Pixel collection v i pixel points; set the sampling mode of the interval sampling area in the jth column to the left or right of the gaze area to every u in the horizontal direction j Pixel collection v j pixel points; set the sampling mode of the interval sampling area in the i-th row above or below the gaze area and the j-th column to the left or right of the gaze area to every u j ×u i Pixel collection v j ×v i pixel points; wherein i, j and v are integers greater than or equal to 1, and u is a positive integer greater than v. This embodiment does not impose any limitation on this.

[0097] Step 304: Sample the pixels in each sampling area according to the sampling method corresponding to each sampling area to obtain a sampled compressed image corresponding to the image to be compressed.

[0098] It can be understood that in this step, the processor can sample the pixel points in each sampling area according to the sampling method corresponding to each sampling area, and can sample a compressed image (i.e., a sampled compressed image), thereby achieving sampling compression of the image to be compressed.

[0099] It should be noted that when the method provided in this embodiment is applied to a terminal device connected to a head-mounted display device, the processor of the terminal device can transmit the sampled compressed image to the head-mounted display device after obtaining the sampled compressed image corresponding to the image that needs to be transmitted to the head-mounted display device (i.e., the image to be compressed), so that the head-mounted display device displays the sampled compressed image. In this way, the image to be displayed by the head-mounted display device can be compressed and transmitted on the basis of ensuring the image effect viewed by the user on the head-mounted display device as much as possible, thereby reducing image transmission and encoding and decoding delays.

[0100] Corresponding to the above method embodiment, the embodiment of the present invention further provides an image compression device. The image compression device described below and the image compression method described above can be referred to in correspondence with each other.

[0101] Please refer to Figure 7 , Figure 7 This is a structural block diagram of an image compression device provided by an embodiment of the present invention. The device may include:

[0102] The region acquisition module 10 is used to acquire a sampling region in the image to be compressed; wherein the sampling region includes a fixation region and an interval sampling region;

[0103] The sampling acquisition module 20 is used to acquire the sampling mode corresponding to each sampling area; wherein the sampling rate of the fixation area in the sampling area is the largest;

[0104] The sampling compression module 30 is used to sample the pixels in each sampling area according to the sampling mode corresponding to each sampling area, so as to obtain a sampled compressed image corresponding to the image to be compressed.

[0105] In some embodiments, the region acquisition module 10 may include:

[0106] A gaze point acquisition submodule, used to acquire the gaze point in the image to be compressed;

[0107] The region acquisition submodule is used to determine the sampling region in the image to be compressed according to the gaze point.

[0108] In some embodiments, the gaze point acquisition submodule can be specifically used to locate the gaze point in the image to be compressed based on the eye feature image collected by the eye tracking camera in the head-mounted display device.

[0109] In some embodiments, the region acquisition submodule may include:

[0110] A dividing line determination unit, used to determine a pair of dividing lines in the image to be compressed according to the gaze point; wherein the dividing lines include a horizontal dividing line and a vertical dividing line, the horizontal dividing line is perpendicular to the vertical dividing line and the intersection point is the gaze point;

[0111] A segmentation point determination unit, used for determining a segmentation point on the segmentation line according to the gaze point and the segmentation line;

[0112] The segmentation unit is used to segment the image to be compressed using the segmentation lines on the segmentation points to obtain sampling areas; wherein the segmentation lines include mutually perpendicular horizontal segmentation lines and vertical segmentation lines.

[0113] In some embodiments, the segmentation point determination unit can be specifically used to obtain segmentation points on four segmentation line segments in a geometric manner based on a preset initial segmentation length and with the gaze point as the origin; wherein the segmentation line segments include a longitudinal dividing line that is divided from the gaze point to obtain an upward segment and a downward segment, and a transverse dividing line that is divided from the gaze point to obtain a left segment and a right segment.

[0114] In some embodiments, the segmentation point determination unit may be specifically configured to use the gaze point as the origin, and n =a1*q n -1 , and divide the segmentation points on the four segmentation segments respectively; among them, a n is the line segment between the segmentation point n-1 and the segmentation point n on the segmentation line segment, n is a positive integer greater than or equal to 1, a1 is the line segment between the gaze point and the segmentation point 1 of the segmentation line segment, the length of a1 is the preset initial segmentation length, and q is a value greater than 0 and less than 1;

[0115] Correspondingly, the attention area in the image area to be compressed corresponding to two adjacent dividing line segments is the area enclosed by the attention point on the two adjacent dividing line segments and the dividing line on the dividing point 1, and the interval sampling area in the image area to be compressed corresponding to two adjacent dividing line segments includes the areas outside the attention area obtained by dividing the image area to be compressed using the dividing line on the dividing point on the two adjacent dividing line segments and the image boundary line on the image area to be compressed.

[0116] In some embodiments, the segmentation point includes a fixation point, the segmentation line includes a dividing line and a vertical line perpendicular to any dividing line at a segmentation point other than the fixation point, and the number of segmentation lines is the number of segmentation points plus 1.

[0117] In some embodiments, the apparatus is applied to a terminal device, and the apparatus may further include:

[0118] The transmission module is used to transmit the sampled compressed image to the head-mounted display device so that the head-mounted display device displays the sampled compressed image.

[0119] In some embodiments, the sampling acquisition module 20 may be specifically configured to determine the sampling mode corresponding to each sampling area according to the positions of each interval sampling area and the attention area.

[0120] In some embodiments, the sampling areas in the image to be compressed are distributed in a matrix, and the sampling acquisition module 20 may include:

[0121] A first sampling setting submodule is used to set the sampling mode of the gaze area to sample each pixel once;

[0122] The second sampling setting submodule is used to set the sampling mode of the interval sampling area of ​​the i-th row directly above or below the fixation area to every u in the vertical direction. i Pixel collection v i pixels, i and v are integers greater than or equal to 1, and u is a positive integer greater than v;

[0123] The third sampling setting submodule is used to set the sampling mode of the interval sampling area of ​​the jth column to the left or right of the fixation area to every u in the horizontal direction. j Pixel collection v j pixels, j is an integer greater than or equal to 1;

[0124] The fourth sampling setting submodule is used to set the sampling mode of the interval sampling area of ​​the i-th row above or below the fixation area and the j-th column to the left or right to every u j ×u i Pixel collection v j ×v i Pixels.

[0125] In this embodiment, the embodiment of the present invention utilizes the characteristic that the user's eyes are not fixated on the entire image, but focus on a certain area. The sampling compression module 30 adopts the sampling methods corresponding to the fixation area and the interval sampling area in the image to be compressed, and samples and compresses the pixels in the image to be compressed to obtain a sampled compressed image corresponding to the image to be compressed. The area outside the fixation area can be compressed, thereby realizing compression of the image to be displayed by the head-mounted display device without affecting the viewing effect of the head-mounted display device as much as possible, reducing image transmission and encoding and decoding delays, and improving user experience.

[0126] Corresponding to the above method embodiment, the embodiment of the present invention further provides an image compression device. The image compression device described below and the image compression method described above can refer to each other.

[0127] Please refer to Figure 8 , Figure 8 A schematic diagram of the structure of an image compression device provided by an embodiment of the present invention. The image compression device may include:

[0128] A memory D1, for storing computer programs;

[0129] The processor D2 is used to implement the steps of the image compression method provided by the above method embodiment when executing a computer program.

[0130] For details, please refer to Fig. 9 , Fig. 9This is a schematic diagram of the specific structure of an image compression device provided by an embodiment of the present invention. The image compression device 410 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 422 (for example, one or more processors) and a memory 432, and one or more storage media 430 (for example, one or more mass storage devices) storing application programs 442 or data 444. Among them, the memory 432 and the storage medium 430 can be short-term storage or permanent storage. The program stored in the storage medium 430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the data processing device. Furthermore, the central processing unit 422 can be configured to communicate with the storage medium 430 to execute a series of instruction operations in the storage medium 430 on the image compression device 410.

[0131] The image compression device 410 may further include one or more power supplies 426, one or more wired or wireless network interfaces 450, one or more input and output interfaces 458, and / or one or more operating systems 441. For example, Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0132] The image compression device provided in this embodiment may be specifically a terminal device, such as a terminal device connected to a head-mounted display device, such as a mobile phone and a computer. The image compression device may also be specifically a head-mounted display device, such as an AR device (such as AR glasses), a VR device, and an MR device.

[0133] The steps in the image compression method described above can be implemented by the structure of the image compression device.

[0134] Corresponding to the above method embodiment, an embodiment of the present invention further provides a computer-readable storage medium. The computer-readable storage medium described below and the image compression method described above can be referred to in correspondence with each other.

[0135] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image compression method provided by the above method embodiment are implemented.

[0136] The computer-readable storage medium may be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or any other readable storage medium that can store program codes.

[0137] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the devices, equipment, and computer-readable storage media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0138] The above is a detailed introduction to an image compression method, device and equipment provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method and core ideas of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, the present invention can also be improved and modified, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. An image compression method, characterized in that: include: Acquire a sampling area in the image to be compressed; wherein the sampling area includes a fixation area and an interval sampling area; Obtaining a sampling mode corresponding to each sampling area; wherein the sampling rate of the attention area in the sampling area is the largest; The pixel points in each sampling area are sampled according to the sampling mode corresponding to each sampling area, so as to obtain a sampled compressed image corresponding to the image to be compressed.

2. The image compression method according to claim 1, characterized in that: The step of obtaining a sampling area in the image to be compressed includes: Acquire a fixation point in the image to be compressed; The sampling area in the image to be compressed is determined according to the gaze point.

3. The image compression method according to claim 2, characterized in that: The obtaining of the fixation point in the image to be compressed comprises: The gaze point in the image to be compressed is located according to the eye feature image collected by the eye tracking camera in the head mounted display device.

4. The image compression method according to claim 2, characterized in that: The step of determining the sampling area in the image to be compressed according to the gaze point includes: According to the gaze point, a pair of dividing lines in the image to be compressed is determined; wherein the dividing lines include a horizontal dividing line and a vertical dividing line, the horizontal dividing line is perpendicular to the vertical dividing line and the intersection point is the gaze point; Determining a segmentation point on the dividing line according to the gaze point and the dividing line; The image to be compressed is segmented using the segmentation lines at the segmentation points to obtain the sampling areas; wherein the segmentation lines include a horizontal segmentation line and a vertical segmentation line that are perpendicular to each other.

5. The image compression method according to claim 4, characterized in that: The step of determining a segmentation point on the dividing line according to the gaze point and the dividing line comprises: According to the preset initial segmentation length, the gaze point is taken as the origin and the segmentation point is obtained by dividing the four segmentation line segments in a geometric manner; wherein the segmentation line segments include the upward segment and the downward segment obtained by dividing the longitudinal segment from the gaze point and the left segment and the right segment obtained by dividing the transverse segment from the gaze point.

6. The image compression method according to claim 5, characterized in that: The method of obtaining the segmentation points by dividing the four segmentation line segments in geometric proportion based on the preset initial segmentation length and taking the gaze point as the origin includes: Taking the gaze point as the origin, through a n =a1*q n-1 , respectively divide on the four dividing line segments to obtain the dividing points; wherein, a n is the line segment between the segmentation point n-1 and the segmentation point n on the segmentation line segment, n is a positive integer greater than or equal to 1, a1 is the line segment between the gaze point and the segmentation point 1 of the segmentation line segment, the length of a1 is the preset initial segmentation length, and q is a value greater than 0 and less than 1; Correspondingly, the attention area in the image area to be compressed corresponding to two adjacent dividing line segments is the area enclosed by the attention point on the two adjacent dividing line segments and the dividing line on the dividing point 1, and the interval sampling area in the image area to be compressed corresponding to two adjacent dividing line segments includes the areas outside the attention area obtained by dividing the image area to be compressed using the dividing line on the dividing point on the two adjacent dividing line segments and the image boundary line on the image area to be compressed.

7. The image compression method according to claim 4, characterized in that: The segmentation points include the gaze points, the segmentation lines include the dividing lines and vertical lines perpendicular to any of the dividing lines at segmentation points other than the gaze points, and the number of the segmentation lines is the number of the segmentation points plus 1.

8. The image compression method according to claim 1, characterized in that: After sampling the pixels in each sampling area according to the sampling mode corresponding to each sampling area to obtain the sampled compressed image corresponding to the image to be compressed, the method further includes: The terminal device transmits the sampled compressed image to a head-mounted display device, so that the head-mounted display device displays the sampled compressed image.

9. The image compression method according to any one of claims 1 to 8, characterized in that: The obtaining of the sampling modes corresponding to the respective sampling areas includes: According to the positions of the interval sampling areas and the attention area, the sampling modes corresponding to the sampling areas are determined.

10. The image compression method according to claim 9, characterized in that: The sampling areas in the image to be compressed are distributed in a matrix, and the sampling modes corresponding to the sampling areas are determined according to the positions of the interval sampling areas and the attention area, including: The sampling mode of the gaze area is set to sample each pixel once; The sampling mode of the interval sampling area of ​​the i-th row directly above or below the fixation area is set to every u in the vertical direction. i Pixel collection v i pixels, i and v are integers greater than or equal to 1, and u is a positive integer greater than v; The sampling mode of the interval sampling area of ​​the jth column to the left or right of the gaze area is set to every u in the horizontal direction. j Pixel collection v j pixels, j is an integer greater than or equal to 1; The sampling mode of the interval sampling area in the i-th row above or below the fixation area and the j-th column to the left or right is set to every u j ×u i Pixel collection v j ×v i Pixels.

11. An image compression device, characterized in that: include: A region acquisition module, used to acquire a sampling region in the image to be compressed; wherein the sampling region includes a fixation region and an interval sampling region; A sampling acquisition module, used to acquire the sampling mode corresponding to each sampling area; wherein the sampling rate of the attention area among the sampling areas is the largest; The sampling compression module is used to sample the pixel points in each sampling area according to the sampling mode corresponding to each sampling area, so as to obtain a sampled compressed image corresponding to the image to be compressed.

12. An image compression device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the image compression method according to any one of claims 1 to 10 when executing the computer program.

Images