The processing method, device and storage medium for media presentation scoring

Abstract

Aspects of the disclosure provide methods, apparatuses, and storage media for processing of media presentation ratings. In some examples, processing circuitry receives a score by a subject rating a media presentation. The subject's score includes a plurality of self-comparison scores rated to a self-comparison test in the media presentation. The processing circuitry applies a first rule and a second rule to the plurality of self-comparison scores. The first rule requires a first subset of the plurality of self-comparison scores to be within a first range. The second rule requires a second subset of the plurality of self-comparison scores to be within a second range to limit at least one outlier to the first rule according to the second range. In response to the first rule and the second rule being satisfied, the processing circuitry determines that the subject's score is eligible for a subjective assessment.

Description

[0001] References merged

[0002] This disclosure claims priority to U.S. Application No. 17 / 752,551, filed May 24, 2022, entitled "Qualification Test in Subject Scoring," and to U.S. Provisional Application No. 63 / 217,439, filed July 1, 2021, entitled "Qualification Test in Subject Scoring," which are incorporated herein by reference in their entirety. Technical Field

[0003] This disclosure describes embodiments that generally involve subjective evaluation. Background Technology

[0004] The background description provided herein is intended to present the overall context of this application. The extent of the work of the currently named inventors described in the background section and various aspects of this specification does not imply that it was prior art at the time of filing of this application, nor is it expressly or implied that it was acknowledged as prior art to this application.

[0005] In virtual reality or augmented reality applications, to give users the feeling of being in the application's virtual world, audio in the application's virtual scene is perceived as real-world audio, where the sounds originate from associated virtual characters within the virtual scene. In some examples, a user's physical movements in the real world are perceived as having matching movements within the application's virtual scene. Furthermore, and importantly, users can interact with the virtual scene using audio that is perceived as realistic and matches their experience in the real world. Summary of the Invention

[0006] This disclosure provides methods and apparatus for processing media presentation ratings. In some examples, the apparatus for subjective evaluation includes processing circuitry. The processing circuitry receives scores from a subject rating a media presentation (e.g., an immersive audio or video media presentation). The subject's scores include multiple self-comparison scores, which are graded as self-comparison tests in the media presentation. The processing circuitry applies a first rule and a second rule to the multiple self-comparison scores. The first rule requires a first subset of the multiple self-comparison scores to be within a first range. The second rule requires a second subset of the multiple self-comparison scores to be within a second range, to restrict at least one anomalous score among the multiple self-comparison scores to the first rule according to the second range. When the first and second rules are satisfied, the processing circuitry determines that the subject's score is qualified for subjective evaluation.

[0007] In some embodiments, the first rule requires that at least N1 self-comparison scores fall within a first range of [-M1, M1], where N1 is a first positive integer less than or equal to N-1, and N is the total number of self-comparison tests. The second rule requires that at least N2 self-comparison scores fall within a second range of [-M2, M2], where N2 is a second positive integer less than or equal to N, and M1 and M2 are each positive numbers.

[0008] In some embodiments, the first rule requires that at least N-1 self-comparison scores be in the first range of [-M1, M1]; and the second rule requires that all N self-comparison scores be in the second range of [-M2, M2].

[0009] In some embodiments, when multiple self-comparison scores do not meet at least one of the first rule and the second rule, the processing circuit excludes the subject's score for subjective evaluation.

[0010] In some examples, the scores consist of four self-comparison scores. The first rule requires that three of the four self-comparison scores be in the first range, and the second rule requires that all four self-comparison scores be in the second range. In the example, the first range is [-0.5, 0.5], and the second range is [-1.5, 1.5].

[0011] In some examples, the first rule requires that a majority (e.g., ≥50%) of multiple self-comparison scores fall within the first range. In some examples, the second rule requires that more than a majority of the multiple self-comparison scores fall within the second range. In the example, the second rule requires that the multiple self-comparison scores fall within the second range.

[0012] In some examples, the first rule requires that at most one outlier among multiple self-comparison scores is not in the first range, and the second rule requires that multiple self-comparison scores be in the second range.

[0013] In some examples, the second range includes the first range and is at least twice the size of the first range.

[0014] In some examples, the first rule requires that a first subset of multiple self-comparison scores be in the first range, and the second rule requires that no self-comparison scores be outside the second range.

[0015] This disclosure also provides a non-volatile computer-readable medium that stores instructions that, when executed by a computer, cause the computer to perform a method of processing for media presentation scoring. Attached Figure Description

[0016] Other features, properties, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings, wherein:

[0017] Figure 1 A schematic diagram of an environment using 6 degrees of freedom (6DoF) is shown in some examples.

[0018] Figure 2 Examples of grading criteria used for comparison tests are shown in some examples.

[0019] Figure 3 A flowchart outlining another process according to an embodiment of this disclosure is shown.

[0020] Figure 4 This is a schematic diagram of a computer device according to an embodiment. Detailed Implementation

[0021] Various aspects of this disclosure provide techniques for subjectively evaluating media quality (e.g., audio processing quality, image processing quality, video processing quality, etc.).

[0022] According to some aspects of this disclosure, some technologies attempt to create or mimic the physical world through digital simulations known as immersive media. Immersive media processing can be implemented according to immersive media standards, such as the Moving Picture Experts Group Immersive (MPEG-I) suite standard, which includes "immersive audio," "immersive video," and "system support." Immersive media standards can support VR or AR presentations, where users can navigate and interact with the environment using six degrees of freedom (6DoF), including spatial navigation (x, y, z) and user head orientation (yaw, pitch, roll).

[0023] Figure 1 The diagram illustrates an environment using 6 degrees of freedom (6DoF) in some examples. 6DoF can be represented by spatial navigation (x, y, z) and user head orientation (yaw, pitch, roll).

[0024] According to one aspect of this disclosure, immersive media can be used to give users the feeling of truly existing in a virtual world. In some examples, the audio of the scene is perceived as audio from the real world, where the sound originates from associated visual figures. For example, in the scene, sound is perceived as having the correct location and distance. The user's physical movement in the real world is perceived as having matching movement in the virtual world scene. Furthermore, the user can interact with the scene, generating sounds that are perceived as real and match the user's experience in the real world.

[0025] Generally, subjective assessments are used to evaluate media quality as a human experience. In some examples, objective assessments such as peak signal-to-noise ratio (PSNR) may not be strongly correlated with human perception. Subjective assessments emphasize how people (also known as subjects) perceive the media and evaluate media quality based on the opinions of these subjects.

[0026] It should be noted that although the subjective evaluation of audio quality is used as an example in the following description of techniques for subjective evaluation, the techniques used for subjective evaluation can be used for the subjective evaluation of media quality of other media types (e.g., images, videos, etc.).

[0027] According to some aspects of this disclosure, in order to perform subjective evaluation, subjects are selected to form a subject panel. Then, media presentations, including tests, can be performed on the subjects, and the subjects can be graded based on scores in the media presentations. Scores are collected, and the scores, along with other suitable information relevant to the subjective evaluation, are processed. Statistical methods can be used to analyze the scores, and media quality can be interpreted based on the statistical analysis.

[0028] During data processing, in order to have reliable and consistent subjective assessments, a qualification test, also known as post-screening, is performed on subject scores. Subjects whose scores are unreliable or inconsistent are excluded from the subjective assessment in some examples.

[0029] The qualification test (also known as post-screening) is based on a self-comparison test in media presentation.

[0030] Typically, comparative tests are used to compare a first medium and a second medium. In this example, the first medium can be processed, such as encoded, rendered by an evaluation technique, and the second medium can be a reference medium. During the comparative test, the first and second media are presented to a subject, who can provide scores to indicate the evaluation of the quality differences.

[0031] Figure 2 An example of a grading standard used for comparative testing is shown. Figure 2In the example, the scores range from -3 to 3. A score of 0 indicates that the quality of the first and second media is roughly the same; a score of -1 indicates that the first media is slightly worse than the second media; a score of 1 indicates that the first media is slightly better than the second media; a score of -2 indicates that the first media is worse than the second media; a score of 2 indicates that the first media is better than the second media; a score of -3 indicates that the first media is much worse than the second media; and a score of 3 indicates that the first media is much better than the second media.

[0032] It should be noted that, although Figure 2 The scores in the examples are integers, but non-integers can be used as scores. In some examples, the smallest unit of change for a score is 0.5. For example, the following scores can be used to grade a comparison test: 3, 2.5, 2, 1.5, 1, 0.5, 0, -0.5, -1, -1.5, -2, -2.5, -3. The scores for comparison tests are also referred to as score differences in some examples.

[0033] According to one aspect of this disclosure, during the presentation of a comparison test to a subject, a self-comparison test (also referred to as a self-comparison experiment) can be mixed with a regular comparison test without informing the subject, and scores collected from the self-comparison test can be used for post-screening. When the self-comparison test is presented, the first and second media are the same media, for example, processed by the same codec, the same rendering, etc. Therefore, a large value (e.g., a large absolute value) in the scores graded by the subject on the self-comparison test can indicate that the subject's score is unreliable or inconsistent with subjective evaluation. In some examples, the scores of the self-comparison test are referred to as self-comparison scores.

[0034] In some examples, among other tests, the files used for audio playback (audio presentation) by the listeners (the subjects used for audio quality assessment) include four self-comparison tests. During audio playback based on the listeners' files, the listeners grade all tests using scores. After audio playback, the subjects' scores for all tests are collected, and the scores for the four self-comparison tests are used for post-screening to determine whether the listeners' scores can be used for further data analysis and media evaluation.

[0035] In relevant examples, rules are applied in post-screening based on the subject's self-comparison test scores. These rules define the subject's eligibility, and the subject's scores can be used for further data analysis and media evaluation. Specifically, a subject's score is retained and used for further media evaluation when at least three of the four self-comparison scores are within a score difference range of -0.5 to 0.5 ([-0.5, 0.5]). In some examples, a subject's score is excluded from further data analysis and media evaluation if two or more scores are outside the score difference range of -0.5 to 0.5 ([-0.5, 0.5]).

[0036] In the relevant examples, post-screening was able to exclude subjects whose two (out of four) or more self-comparison scores fell outside the range of -0.5 to 0.5 ([-0.5, 0.5]). However, some subjects who passed post-screening had three self-comparison scores within the range of -0.5 to 0.5 ([-0.5, 0.5]), but one self-comparison score was as high as 3.0 or as low as -3.0. Due to this large variability, in some examples, the ratings of these subjects were considered unreliable and may need to be excluded from the subjective assessment.

[0037] The various aspects of this disclosure provide multi-rule techniques for post-screening and enhance the reliability of scores used for subjective evaluation.

[0038] According to some aspects of this disclosure, two rules can be applied in post-screening based on the subject's self-comparison test score. The first rule specifies the subject's eligibility based on a majority of the self-comparison scores (e.g., more than 50% of the self-comparison scores), and the second rule specifies eligibility based on outliers in the self-comparison scores (scores outside the first range). When a subject's self-comparison score meets both rules, the subject's score can be used for further data analysis and media evaluation. When a subject's self-comparison score does not meet at least one of the two rules, the subject's score is excluded from further data analysis and media evaluation.

[0039] According to one aspect of this disclosure, the first rule restricts the majority of self-comparison scores of qualified subjects (subjects whose scores are qualified for further data analysis and media evaluation after post-screening) to a narrow range, and the second rule restricts outliers in the self-comparison scores of qualified subjects to not changing significantly compared to the majority of self-comparison scores.

[0040] In the example, the first rule can be expressed as: at least three of the four self-comparison scores fall within a score difference range such as -0.5 to 0.5 ([-0.5, 0.5]); and the second rule can be expressed as: all four self-comparison scores fall within another score difference range such as -1.5 to 1.5 ([-1.5, 1.5]). In this example, the majority of self-comparison scores of qualified subjects (whose scores are qualified for further data analysis and media evaluation after post-screening) are restricted to a narrow range represented by [-Vmajority, Vmajority] (where Vmajority is a positive number, such as 0.5), for example, from -0.5 to 0.5. Outliers in the self-comparison scores of qualified subjects are restricted to not changing significantly compared to the majority of their self-comparison scores, for example, restricted to the range of [-3×Vmajority, 3×Vmajority].

[0041] In the example above, a self-comparison score outside the first range, such as [-0.5, 0.5], is considered an abnormal self-comparison score. When an abnormal self-comparison score varies too much (e.g., outside [-1.5, 1.5]), the subject's score is excluded from further data analysis and media evaluation. Using these two rules in the post-screening, all four self-comparisons are constrained to a given score range, ensuring that there are no high self-comparison difference scores such as 3.0 or low self-comparison scores such as -3.0. Therefore, subjects with high or low self-comparison scores (even if there might only be one high or low self-comparison score) will be excluded from the subjective evaluation.

[0042] In some examples, the post-selection is based on N self-comparison scores, where N is a positive number. The post-selection based on N self-comparison scores uses two rules. The first rule can be expressed as: at least N-1 self-comparison scores fall within a score difference range such as [-M1, M1]; and the second rule can be expressed as: all N self-comparison scores fall within another score difference range such as [-M2, M2], where M1 and M2 are positive numbers, and M2 can be greater than or equal to M1.

[0043] In the example above, most self-comparison scores of qualified subjects (subjects whose scores are qualified for further data analysis and media evaluation after post-screening) are restricted to a narrow range such as [-M1, M1], and outliers in the self-comparison scores of qualified subjects (e.g., self-comparison scores outside [-M1, M1]) are restricted to a range that does not vary much compared to most self-comparison scores, such as being restricted to the range of [-M2, M2].

[0044] In some examples, post-selection is based on N self-comparison scores, where N is a positive number. Post-selection based on N self-comparison scores uses two rules. The first rule can be expressed as at least N1 self-comparison scores falling within a score difference range such as [-M1, M1], where N1 is a positive integer less than or equal to N-1, and M1 is a positive number. The second rule can be expressed as at least N2 self-comparison scores falling within another score difference range such as [-M2, M2], where N2 is a positive integer less than or equal to N, M2 is a positive number, and M2 can be greater than or equal to M1. Note that N2 can be greater than or equal to N1.

[0045] Figure 3 A flowchart of an overview process (300) according to an embodiment of the present disclosure is shown. The process (300) can be used for subjective evaluation. In some embodiments, the process (300) is implemented as software instructions, so that the processing circuit executes the process (300) when the software instructions are executed. The process begins at (S301) and proceeds to (S310).

[0046] At (S310), a score is received that is graded by the subject in response to media presentation (e.g., immersive audio or video media presentation). The subject's score includes multiple self-comparison scores, which are graded as self-comparison tests in the media presentation.

[0047] At (S320), the first rule is applied to multiple self-comparison scores. The first rule requires that a first subset of the multiple self-comparison scores be within a first range.

[0048] At (S330), a second rule is applied to multiple self-comparison scores. The second rule requires a second subset of the multiple self-comparison scores to be within a second range, so as to restrict at least one outlier among the multiple self-comparison scores to the first rule according to the second range.

[0049] In some embodiments, the first rule includes the requirement that at least N1 self-comparison scores fall within a first range of [-M1, M1], where N1 is a first positive integer less than or equal to N-1, and N is the total number of self-comparison scores. The second rule includes the requirement that at least N2 self-comparison scores fall within a second range of [-M2, M2], where N2 is a second positive integer less than or equal to N, and M1 and M2 are each positive numbers.

[0050] In some embodiments, the first rule includes the requirement that at least N-1 self-comparison scores are in a first range of [-M1, M1]; and the second rule includes the requirement that all N self-comparison scores are in a second range of [-M2, M2].

[0051] In some examples, the scores consist of four self-comparison scores. The first rule requires that three of the four self-comparison scores be in the first range, and the second rule requires that all four self-comparison scores be in the second range. In the example, the first range is [-0.5, 0.5], and the second range is [-1.5, 1.5].

[0052] In some examples, the first rule requires that the majority of multiple self-comparison scores be in the first range. In some examples, the second rule requires that more than the majority of multiple self-comparison scores be in the second range. In the example, the second rule requires that multiple self-comparison scores be in the second range.

[0053] In some examples, the first rule requires that at most one outlier among multiple self-comparison scores is not in the first range, and the second rule requires that multiple self-comparison scores be in the second range.

[0054] In some examples, the second range includes the first range and is at least twice the first range (e.g., 2 times, 3 times, etc.).

[0055] In some examples, the first rule requires that a first subset of multiple self-comparison scores be in the first range, and the second rule requires that no self-comparison scores be outside the second range.

[0056] At (S340), when multiple self-comparison scores satisfy the first and second rules, it is determined that the subject's score is qualified for subjective evaluation.

[0057] In some examples, a subject's score is excluded from subjective evaluation when multiple self-comparison scores do not meet at least one of the first and second rules.

[0058] Then, the process proceeds to (S399) and terminates.

[0059] The process (300) can be modified as appropriate. One or more steps in the process (300) can be improved and / or omitted. One or more additional steps can be added. Any suitable order of implementation can be used.

[0060] The techniques described above can be implemented as computer software using computer-readable instructions and physically stored in one or more computer-readable media. For example, Figure 4 A computer device (400) is shown, which is adapted to implement certain embodiments of the disclosed subject matter.

[0061] The computer software can be encoded using any suitable machine code or computer language, and code including instructions can be created through mechanisms such as assembly, compilation, and linking. These instructions can be executed directly by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., or executed through decoding, microcode, etc.

[0062] The instructions can be executed on various types of computers or their components, including, for example, personal computers, tablets, servers, smartphones, gaming devices, Internet of Things devices, etc.

[0063] Figure 4 The components shown for the computer device (400) are exemplary in nature and are not intended to limit the scope or functionality of the computer software used to implement the embodiments of this application. Nor should the configuration of the components be construed as having any dependency or requirement on any component or combination thereof shown in the exemplary embodiments of the computer device (400).

[0064] The computer device (400) may include certain human-machine interface input devices. Such human-machine interface input devices may respond to input from one or more human users through tactile input (e.g., keyboard input, swiping, data glove movement), audio input (e.g., sound, applause), visual input (e.g., gestures), and olfactory input (not shown). The human-machine interface device may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sound), images (e.g., scanned images, photographic images obtained from still cameras), and video (e.g., two-dimensional video, three-dimensional video including stereoscopic video).

[0065] Human-machine interface input devices may include one or more of the following (only one is shown): keyboard (401), mouse (402), touchpad (403), touch screen (410), data glove (not shown), joystick (405), microphone (406), scanner (407), and camera (408).

[0066] The computer device (400) may also include certain human-machine interface output devices. Such human-machine interface output devices may stimulate the senses of one or more human users through, for example, tactile output, sound, light, and smell / taste. Such human-machine interface output devices may include tactile output devices (e.g., tactile feedback via a touch screen (410), data gloves (not shown), or joystick (405), but may also include tactile feedback devices that are not used as input devices), audio output devices (e.g., speakers (409), headphones (not shown)), visual output devices (e.g., screens (410) including cathode ray tube screens, liquid crystal screens, plasma screens, organic light-emitting diode screens, each of which may or may not have touch screen input functionality, each of which may or may not have tactile feedback functionality—some of which may output two-dimensional or more three-dimensional visual outputs by means such as stereoscopic image output; virtual reality glasses (not shown), holographic displays, and smoke boxes (not shown)), and printers (not shown).

[0067] The computer device (400) may also include human-accessible storage devices and related media, such as optical media including high-density read-only / rewritable optical discs (CD / DVD ROM / RW) (420) or similar media (421) with CD / DVD, thumb drives (422), removable hard disk drives or solid-state drives (423), conventional magnetic media such as magnetic tapes and floppy disks (not shown), dedicated devices based on ROM / ASIC / PLD such as security software protectors (not shown), etc.

[0068] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the disclosed subject matter does not include transmission media, carrier waves, or other transient signals.

[0069] The computer device (400) may also include an interface (454) to one or more communication networks (455). For example, the network may be wireless, wired, or optical. The network may also be a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a vehicle network and an industrial network, a real-time network, a latency-tolerant network, and so on. The network also includes LANs such as Ethernet, wireless LANs, cellular networks (GSM, 3G, 4G, 5G, LTE, etc.), cable or wireless wide area digital networks (including cable television, satellite television, and terrestrial broadcast television), vehicle and industrial networks (including CANbus), and so on. Some networks typically require an external network interface adapter for connection to certain general-purpose data ports or peripheral buses (449) (e.g., a USB port on the computer device (400)); other systems are typically integrated into the core of the computer device (400) via a system bus as described below (e.g., an Ethernet interface integrated into a PC computer system or a cellular network interface integrated into a smartphone computer system). By using any of these networks, the computer device (400) can communicate with other entities. The communication can be unidirectional, used only for receiving (e.g., wireless television), unidirectional, used only for sending (e.g., CAN bus to certain CAN bus devices), or bidirectional, such as through a local area or wide area digital network to other computer systems. Each of the above networks and network interfaces can use certain protocols and protocol stacks.

[0070] The aforementioned human-computer interface device, human-accessible storage device, and network interface can be connected to the core (440) of the computer device (400).

[0071] The core (440) may include one or more central processing units (CPU) (441), graphics processing units (GPUs) (442), dedicated programmable processing units in the form of field-programmable gate arrays (FPGAs) (443), task-specific hardware accelerators (444), graphics adapters (450), etc. These devices, along with read-only memory (ROM) (445), random access memory (446), internal mass storage (e.g., internal non-user-accessible hard disk drives, solid-state drives, etc.) (447), etc., may be connected via a system bus (448). In some computer systems, the system bus (448) may be accessed via one or more physical connectors to allow for expansion with additional CPUs, GPUs, etc. Peripheral devices may be directly attached to the core's system bus (448) or connected via a peripheral bus (449). In some examples, a screen (410) may be connected to a graphics adapter (450). Peripheral bus architectures include external controller interfaces (PCI), universal serial buses (USB), etc.

[0072] The CPU (441), GPU (442), FPGA (443), and accelerator (444) can execute certain instructions, which, when combined, constitute the aforementioned computer code. This computer code can be stored in ROM (445) or RAM (446). Transient data can also be stored in RAM (446), while permanent data can be stored, for example, in internal mass storage (447). Fast storage and retrieval of any memory device can be achieved by using a cache memory, which can be closely associated with one or more CPUs (441), GPUs (442), mass storage (447), ROM (445), RAM (446), etc.

[0073] The computer-readable medium may contain computer code for performing various computer-implemented operations. The medium and computer code may be specifically designed and constructed for the purposes of this application, or they may be media and code well-known and usable by those skilled in the art of computer software.

[0074] By way of example and not limitation, a computer system having an architecture, particularly a core (440), can provide functionality as a processor (including a CPU, GPU, FPGA, accelerator, etc.) to execute software contained in one or more tangible computer-readable media. Such computer-readable media can be media associated with the aforementioned user-accessible mass storage, as well as specific memory of the core (440) that is non-volatile, such as internal mass storage (447) or ROM (445). Software implementing various embodiments of this application can be stored in such a device and executed by the core (440). Depending on specific needs, the computer-readable medium may include one or more storage devices or chips. The software can cause the core (440), particularly the processor therein (including a CPU, GPU, FPGA, etc.), to execute specific processes or specific portions of specific processes described herein, including defining data structures stored in RAM (446) and modifying such data structures according to software-defined processes. Alternatively or as an alternative, the computer system may provide logic hardwired or otherwise incorporated into circuitry (e.g., an accelerator (444)) that may replace or operate with the software to perform the specific process or a specific portion of the specific process described herein. References to software may include logic, and vice versa, where appropriate. References to computer-readable media may include, where appropriate, circuitry storing the execution of software (such as an integrated circuit (IC)), circuitry containing the execution logic, or both. This application includes any suitable combination of hardware and software.

[0075] While this application has described several exemplary embodiments, various modifications, arrangements, and equivalent substitutions of the embodiments are all within the scope of this application. Therefore, it should be understood that those skilled in the art can design various systems and methods that, although not explicitly shown or described herein, embody the principles of this application and are thus within its spirit and scope.

Claims

1. A method for scoring media presentation, characterized in that, include: The system receives scores from a subject that classifies immersive audio or video media presentations. These scores include N self-comparison scores, which are the subject's ratings of the quality differences in N self-comparison tests within the immersive audio or video media presentation. In the self-comparison tests, the two media being compared are the same media. The first rule is applied to the N self-comparison scores, wherein the first rule requires that at least N1 of the N self-comparison scores are within the first range [-M1, M1]. The second rule is applied to the N self-comparison scores, requiring at least N² self-comparison scores to be within a second range [-M², M²], so as to restrict at least one anomalous score among the N self-comparison scores to the second rule according to the second range; and In response to the N self-comparison scores satisfying the first rule and the second rule, it is determined that the subject's score is qualified for subjective evaluation; Where N1 is the first positive integer less than or equal to N-1, N2 is the second positive integer less than or equal to N, N2 is greater than or equal to N1, M1 and M2 are each positive numbers, and M2 is greater than or equal to M1.

2. The method according to claim 1, characterized in that, The value of N is 4, N1 and N2 are 3 and 4 respectively, and M1 and M2 are 0.5 and 1.5 respectively.

3. The method according to claim 1, characterized in that, Further includes: In response to the N self-comparison scores not conforming to at least one of the first rule and the second rule, the subject's score is excluded from the subjective assessment.

4. The method according to claim 1, characterized in that, The first rule requires that most of the N self-comparison scores be within the first range, and that the N self-comparison scores be within the second range.

5. The method according to claim 1, characterized in that, The first rule requires that at most one outlier among the N self-comparison scores is not within the first range, and the second rule requires that the N self-comparison scores are within the second range.

6. The method according to claim 1, characterized in that, The second range is at least twice the size of the first range.

7. The method according to claim 1, characterized in that, The second rule requires that no self-comparison score is outside the second range.

8. An apparatus for processing media presentation scoring, characterized in that, Includes processing circuitry, the processing circuitry being configured to: The system receives scores from a subject that classifies immersive audio or video media presentations. These scores include N self-comparison scores, which are the subject's ratings of the quality differences in N self-comparison tests within the immersive audio or video media presentation. In the self-comparison tests, the two media being compared are the same media. The first rule is applied to the N self-comparison scores, wherein the first rule requires that at least N1 of the N self-comparison scores are within the first range [-M1, M1]. The second rule is applied to the N self-comparison scores, requiring at least N² self-comparison scores to be within a second range [-M², M²], so as to restrict at least one anomalous score among the N self-comparison scores to the second rule according to the second range; and In response to satisfying the first rule and the second rule, it is determined that the subject's score is qualified for subjective evaluation; Where N1 is the first positive integer less than or equal to N-1, N2 is the second positive integer less than or equal to N, N2 is greater than or equal to N1, M1 and M2 are each positive numbers, and M2 is greater than or equal to M1.

9. A non-volatile computer-readable storage medium for storing instructions, characterized in that, When executed by at least one processor, the instructions cause the at least one processor to perform the method of any one of claims 1-7.

10. A computer device, characterized in that, It includes a processor and a memory; the memory stores a computer program that, when executed by the processor, causes the processor to perform the method of any one of claims 1-7.

Images