Visual perception test method and device, electronic equipment, product and storage medium

Abstract

The invention provides a visual perception test method and device, electronic equipment, a product and a storage medium, and relates to the technical field of machine vision, and the method comprises the steps: correcting projection pictures of a plurality of projectors, and determining a projection surface; projecting a spectrum test image to the projection surface, and acquiring a to-be-analyzed image on the projection surface acquired by a tested device; the analysis result label of the to-be-analyzed image is compared with preset label data corresponding to the spectrum test image, a visual perception test result of the tested device is obtained, and the analysis result label comprises a category label, a geometric information label and a spectrum information label of each object in the to-be-analyzed image. The method is used for overcoming the defects that in the prior art, visual perception testing of a machine vision system depends on data collection of an actual scene, the collection workload of the actual scene is huge, and data processing and labeling consume a large amount of manpower and time.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, and in particular to a visual perception test method and device, electronic equipment, product and storage medium. BACKGROUND

[0002] With the rapid development of artificial intelligence and image processing technology, machine vision systems have been widely used in industrial detection, autonomous robots, automatic driving and other fields. In practical applications, machine vision systems not only have different spectral response characteristics, but also need to face diversified working environments, such as complex target types, changing backgrounds, metamerism, and significant fluctuations in environmental lighting and target visibility. Such variability puts higher requirements on the intelligent perception ability of machine vision systems. Therefore, how to effectively evaluate and verify the perception performance of machine vision systems in different scenarios has become a key requirement for production enterprises to improve product quality and verify system performance. The traditional test method arranges test scenes in the actual environment according to test requirements to realize visual perception testing of machine vision systems.

[0003] However, this relies on data collection of actual scenes, and the collection of actual scenes has a huge workload, and data processing and labeling consume a lot of manpower and time, making it difficult to cover variable environments, extreme situations and low probability events. Therefore, there is an urgent need for a visual perception test method that can comprehensively and efficiently test the visual perception of machine vision systems. SUMMARY

[0004] The present application provides a visual perception test method, device, electronic equipment, product and storage medium to solve the defect that the prior art visual perception test of machine vision systems relies on data collection of actual scenes, and the collection of actual scenes has a huge workload, and data processing and labeling consume a lot of manpower and time.

[0005] The present application provides a visual perception test method, comprising: correcting the projection pictures of a plurality of projectors to determine a projection surface; projecting a spectral test image onto the projection surface to obtain a to-be-analyzed image and an analysis result label on the projection surface collected by a device under test; comparing the analysis result label of the to-be-analyzed image with preset label data corresponding to the spectral test image to obtain a visual perception test result of the device under test, wherein the analysis result label includes a category label, a geometric information label and a spectral information label of each object in the to-be-analyzed image.

[0006] According to the visual perception test method provided by the present application, the spectral test image is projected onto the projection surface, comprising: determining a spectral calibration table, the spectral calibration table comprising a mapping relationship between gray scale values and spectral information, the spectral information comprising spectral light intensity information; adjusting, based on the spectral information of the spectral test image and the spectral calibration table, the gray scale value of each color channel of each of the plurality of projectors at a corresponding position of each pixel in the spectral test image, to project the spectral test image onto the projection surface.

[0007] According to the visual perception test method provided by the present application, the determination of the spectral calibration table comprises: determining a preset color channel set and a preset gray scale value set based on the color channels and the gray scale adjustment range of the plurality of projectors; determining the spectral calibration table through calibration based on the preset color channel set and the preset gray scale value set, wherein each calibration is as follows: projecting a gray scale test image corresponding to any preset color channel in the preset color channel set and any preset gray scale value in the preset gray scale value set onto the projection surface, the gray scale test image having at least one designated measurement region, all pixels in the designated measurement region having the same gray scale value as the preset gray scale value; and measuring the spectral information corresponding to the designated measurement region of the gray scale test image.

[0008] According to the visual perception test method provided by the present application, the spectral calibration table is corrected based on the following method: measuring the spectral information of a specific gray scale value to obtain corrected spectral information; determining a correction coefficient based on the spectral information corresponding to the specific gray scale value in the spectral calibration table and the corrected spectral information; correcting the spectral information corresponding to all gray scale values except the specific gray scale value in the spectral calibration table based on the correction coefficient.

[0009] According to the visual perception test method provided by the present application, the projection images of the plurality of projectors are corrected to determine a projection surface, comprising: determining a reference projector from the plurality of projectors, and determining the projectors other than the reference projector as superimposed projectors; determining a reference projection region in which the projection image of the reference projector is located; adjusting the regions in which the projection images of the plurality of superimposed projectors are located, so that the regions in which the projection images of the plurality of superimposed projectors are located are fixed in relative geometric relationship with the reference projection region, to determine a target projection region; The images projected by the plurality of superimposed projectors are cropped and deformed to make the images projected by the plurality of superimposed projectors consistent and fused together to determine a projection surface.

[0010] According to the visual perception test method provided by the application, the analysis result label of the to-be-analyzed image is compared with the preset label data corresponding to the spectrum test image to obtain the visual perception test result of the to-be-tested device, which comprises the following steps: If the category label of the object in the analysis result label matches the object label in the preset label data, the object in the analysis result label is determined as an analysis correct object; Based on the proportion of the analysis correct objects in the analysis result label in all objects in the analysis result label and the proportion of the analysis correct objects in the analysis result label in all object labels in the preset label data, the evaluation score of the to-be-tested device is determined; Based on the evaluation score, the visual perception test result of the to-be-tested device is determined.

[0011] The application further provides a visual perception test device, which comprises: A correction module is configured to correct the projection pictures of the plurality of projectors to determine a projection surface; A collection module is configured to project a spectrum test image onto the projection surface to obtain a to-be-analyzed image and an analysis result label collected by a to-be-tested device on the projection surface; A test module is configured to compare the analysis result label of the to-be-analyzed image with preset label data corresponding to the spectrum test image to obtain the visual perception test result of the to-be-tested device, wherein the analysis result label comprises the category label, the geometric information label and the spectrum information label of each object in the to-be-analyzed image.

[0012] The application further provides an electronic device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the visual perception test method according to any one of the above-mentioned methods when executing the computer program.

[0013] The application further provides a non-transitory computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the visual perception test method according to any one of the above-mentioned methods.

[0014] The application further provides a computer program product, which comprises a computer program, wherein the computer program is executed by a processor to implement the visual perception test method according to any one of the above-mentioned methods.

[0015] The visual perception test method, device, electronic device, product and storage medium provided by the present invention calibrate the projection images of multiple projectors to determine the projection surface; project a spectral test image onto the projection surface, and obtain the image to be analyzed and the analysis result label on the projection surface collected by the device under test; compare the analysis result label of the image to be analyzed with the preset label data corresponding to the spectral test image to obtain the visual perception test result of the device under test, wherein the analysis result label includes the category label, geometric information label and spectral information label of each object in the image to be analyzed. The present invention calibrates the projection images of multiple projectors in the test device to ensure that the images projected by each projector can accurately overlap on the projection surface. The calibration process determines the final projection surface for testing by adjusting the geometric position and alignment of the projection images, ensuring the accuracy and consistency of the projected image. Through the projection of spectral test images, the complex spectral characteristics of actual scenes can be efficiently reproduced in a laboratory environment, covering a variety of lighting conditions and target characteristics, thereby more comprehensively evaluating the performance of the machine vision system; a large amount of new test data can be obtained based on public data sets, a small amount of typical data collected, and data augmentation and generation technologies, which greatly reduces the cost and time of scene construction, data collection and manual labeling, and improves test efficiency. The controllable spectral test image projection can easily simulate real scenes and even extreme scenes that are difficult to achieve with traditional testing methods, ensuring more comprehensive test results; in addition, the projected spectral test image can be quickly adjusted, supporting multiple repeated tests and comparative experiments, which helps to quickly discover and solve problems in the machine vision system; by comparing the analysis result label of the image to be analyzed with the preset label data corresponding to the spectral test image, the visual perception ability of the device under test can be accurately evaluated. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 It is a flow chart of the visual perception testing method provided by the present invention.

[0018] Figure 2 It is a schematic diagram of spectral test image conversion of the visual perception test method provided by the present invention.

[0019] Figure 3 It is a schematic diagram of the correction process in the visual perception testing method provided by the present invention.

[0020] Figure 4It is a structural schematic diagram of the visual perception test device provided by the present application.

[0021] Figure 5 It is a structural schematic diagram of the electronic device provided by the present application. DETAILED DESCRIPTION

[0022] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0023] The applicant found in the process of long-term research and development and testing of machine vision systems that the traditional test method mainly relies on data acquisition of actual scenes. This method has significant limitations: the data acquisition workload is huge, and the test scene construction and data labeling require a large amount of manpower and time; at the same time, actual scenes are difficult to simulate extreme conditions or low probability events, such as complex spectral features or special environmental lighting, resulting in test results that cannot fully reflect the real performance of machine vision systems. In addition, human errors in the labeling data process also affect the reliability and efficiency of the test. These problems seriously restrict the research and optimization process of machine vision systems.

[0024] Specifically, most of the prior art only supports RGB three channels, which cannot reproduce the complex spectral features in real scenes, resulting in information loss in the spectral dimension. At the same time, virtual scenes running on computers cannot directly test the performance of real hardware systems, nor can they provide substantial help for the optimization of software and hardware matching. In addition, the prior art can usually only generate uniform light fields that match the spectrum, lack image information, and are difficult to meet the test needs of image recognition, detection and other machine vision tasks; as can be seen, the prior art has weak support for scene flexibility and is difficult to quickly adapt to different test needs. Therefore, the present application proposes an efficient visual perception test method, which aims to provide a better solution for performance evaluation of machine vision systems.

[0025] In view of the above problems, the present application proposes the following embodiments.

[0026] Figure 1 It is a flowchart of the visual perception test method provided by the present application, as shown in Figure 1 The method comprises the following steps: Step 110, correcting the projection pictures of the plurality of projectors to determine the projection surface.

[0027] Here, the projection surface is the final receiving area of the projected image by the projector, which can be a screen, a wall or other smooth surface. The fusion of the consistent projection surface is the basis for the projection of the spectral test image, and its flatness and correction quality directly affect the accuracy of the test results.

[0028] Exemplarily, the projector is generally composed of a projection module and a light source. The projection module provides an independently adjustable pixel array for the projected image, which can be a liquid crystal type such as a transmissive LCD (Liquid Crystal Display) or LCOS (Liquid Crystal on Silicon), or a digital micromirror type such as a DMD (Digital Micromirror Device). The light source can be various bandwidth light sources such as a broadband light source used with a narrowband filter, a narrowband LED light source or a laser light source. Specifically, the light source enters the projection module after passing through the homogenization system to ensure the uniformity of the projection light field on the projection plane.

[0029] Exemplarily, the test device can include multiple projectors to achieve spatial superposition of the projection light field. Each projector can independently play an image or a video to generate a more complex projection light field through spatial superposition. The color channels of each projector can be combined in a desired manner within a waveband range according to the type of light source and filter used, for example, each color channel covers a different narrow sub-waveband range. The projector is connected to the computer through a video controller and is recognized by the computer as an independent external display device. The computer manages multiple external projectors in the form of an extended screen and allocates image (or video) content displayed by each projector. The video controller is a bridge between the test device and the computer. The main function of the video controller is to distribute the image signals generated by the computer to each projector to ensure that each projector operates independently and can flexibly control the content displayed by each projector. The relationship between the test device and the computer is a close cooperative relationship. Multiple projectors in the test device are responsible for projecting the content generated by the computer onto the projection surface to build a test scene.

[0030] It should be noted that combining the waveband ranges of multiple projectors not only expands the working waveband range of the test device, but also improves the flexibility, accuracy and applicability of spectral testing, which can better meet diversified testing needs.

[0031] Further, there are two types of video controllers. The first type is a built-in multi-output graphics card connected to the PC through a PCIe interface. The second type is an external graphics card or video expansion card connected to the PC through a USB interface, a Thunderbolt TM ) interface or a HDMI interface.

[0032] Specifically, the testing device has multiple independent color channels, each channel's projection screen is composed of independently controllable pixels, each color channel corresponds to a gray channel, by adjusting the gray scale value of each color channel, the superposition of the spectrum at each pixel position in the projection screen is realized; on the projection screen, the position of each pixel can be controlled by adjusting the gray scale value of the corresponding gray channel, and the intensity output by the color channel.

[0033] It should be noted that by using multiple projectors and adopting spatial superposition, a more complex and fine projection light field can be generated on the projection surface. This makes the test scene more diverse and flexible, can simulate more realistic environmental changes, and meets the testing needs of machine vision systems for complex scenes. Through the gray scale value adjustment of each color channel, accurate superposition of the spectrum at each pixel position of the projection screen can be achieved. This method can simulate different spectral conditions, help test the perception ability of machine vision systems under various lighting and color environments, and improve the comprehensiveness of the test. By using the extended screen mode of the computer to manage multiple projectors, the control of the testing device is more centralized and efficient. The computer can uniformly manage the work of all projectors, quickly switch different test scenes, and avoid manual intervention, improving the test efficiency.

[0034] Step 120, projecting the spectral test image onto the projection surface to obtain the analyzed image and analysis result label on the projection surface collected by the measured device.

[0035] Here, the spectral test image refers to a specific test image, such as a multispectral or hyperspectral image dataset organized in a certain way, which may include information in various wavelength ranges such as visible light, infrared light, etc. Through the projection of these images, different lighting environments and scene changes can be simulated to test the perception ability of machine vision systems under various spectral conditions.

[0036] Here, the measured device refers to the machine vision system being tested, which includes cameras, sensors, and other devices that can collect images on the projection surface in real time. These image data will be used for subsequent analysis and evaluation.

[0037] Here, the analyzed image is the image data collected by the measured device, and through further processing and analysis of the analyzed image, the visual recognition ability of the machine vision system under different lighting and spectral conditions can be evaluated.

[0038] Exemplarily, in the test device, multiple projectors can project respective images or videos onto the same projection surface. In order to ensure correct fusion of the projected images, the projection pictures of these projectors can be combined in different spatial overlapping manners. Common overlapping manners include maximum overlap, partial overlap, picture adjacency, and no overlap; when spatial overlap is required, correction must be performed to ensure accurate alignment of the respective projection pictures on the projection surface, thereby avoiding image distortion caused by projector position errors or inconsistent projection pictures.

[0039] Exemplarily, the three color channels of each projector can contain a single color or multiple colors, and if the color channels contain multiple colors, these colors will be combined into a color sequence in a specific order. In this sequence, the on-off timing of each color is synchronized with the frame sequence of the projector. In a certain frame or a certain group of specific frames, only part of the colors (such as red, green, or blue) will be turned on, while the other colors will be turned off. In this way, the on-off timing of each color is precisely controlled, so that the spectral information is superimposed at each pixel position in the projection picture. In this way, the spectral information of different color channels is superimposed in the projection picture, ensuring that the output intensity and spectral characteristics of each pixel can be precisely adjusted, thereby achieving a more rich and complex spectral information projection effect.

[0040] Step 130, comparing the analysis result label of the image to be analyzed with the preset label data corresponding to the spectral test image, obtaining the visual perception test result of the device under test, the analysis result label including the category label, geometric information label and spectral information label of each object in the image to be analyzed.

[0041] Here, the preset label data refers to the standard data known before the test, and the preset label data corresponds to each pixel or region in the spectral test image one by one, indicating the object category, attribute or other visual features that should be recognized in each region. For example, a certain region in the image can be labeled as "red object" or "target region", and these label data serve as a reference to help the system evaluate the correctness of the analysis test result of the image to be analyzed. The visual perception test result is obtained by comparing the label data obtained after analyzing the image to be analyzed with the preset label data; the test result can include recognition accuracy, error rate, classification precision and other dimensions.

[0042] Here, the category label refers to the category to which the object (or pixel) in the analysis result label is classified, such as "crack", "vehicle", "pedestrian", etc. The geometric information position label refers to the specific position coordinates or area, size, angle, shape of the object in the analysis result label, such as the side length of the minimum enclosing rectangle of the area occupied by the object, or a certain vertex coordinate. The spectral information label refers to the label related to the spectral wavelength and amplitude size of the object in the analysis result, such as color coordinates, color categories, color temperature, etc.

[0043] Here, the visual perception test result is obtained by comparing the image to be analyzed with the preset label data. The test result can include recognition accuracy, error rate, classification precision and other dimensions, helping to evaluate the perception ability of the machine vision system in this scene.

[0044] The present application corrects the projection images of multiple projectors in the test device, ensures that the images projected by each projector can be accurately overlapped on the projection surface, and determines the final consistent projection surface used for testing by adjusting the geometric position and alignment of the projection image, ensuring the accuracy and consistency of the projection image. Through the diversified projection of spectral test images, complex spectral characteristics in actual scenes can be efficiently reproduced in a laboratory environment, covering various lighting conditions and target characteristics, thereby more comprehensively evaluating the performance of the machine vision system. A large amount of new test data can be obtained based on public data sets, a small amount of typical data collected, data augmentation and generation technology, greatly reducing the cost and time of scene construction, data collection and manual labeling, improving test efficiency, and easily simulating extreme scenes that are difficult to achieve with traditional test methods, ensuring more comprehensive test results. In addition, the projected spectral test images can be quickly adjusted to support multiple repeated tests and comparative experiments, which helps to quickly find and solve problems of the machine vision system. By comparing the label data obtained by analyzing the image to be analyzed with the preset label data corresponding to the spectral test image, the visual perception ability of the device under test can be accurately evaluated.

[0045] Based on any of the above embodiments, in the method, the projecting the spectral test image onto the projection surface comprises: determining a spectral calibration table, the spectral calibration table comprising a mapping relationship between gray scale values and spectral information; the spectral information comprising light intensity information of a sub-spectrum; based on the spectral information of the spectral test image and the spectral calibration table, adjusting the gray scale value of each color channel of each of the plurality of projectors at the corresponding position of each pixel in the spectral test image to project the spectral test image onto the projection surface.

[0046] Here, the spectral calibration table is a mapping table that defines the relationship between the gray scale value and the spectral information. Specifically, the gray scale value is the adjustable level of the output light intensity of the projection channel, and the spectral information relates to the actual physical intensity of light of a specific color or wavelength (usually represented by physical quantities such as spectral radiance, spectral irradiance, etc.). The mapping relationship can generally be determined by using a spectral radiance meter or a spectral irradiance meter.

[0047] Among them, the light intensity information of the sub-spectrum covers multiple physical quantities such as power, radiance, irradiance, illumination, and brightness, which can comprehensively describe the light radiation characteristics, support multi-scene application requirements, and improve the accuracy and flexibility of the test.

[0048] In an embodiment, during the projection process, the gray scale value of each color channel (such as red, green, and blue) of each projector at each pixel position needs to be adjusted according to the spectral information of the spectral test image and the spectral calibration table, that is, the gray scale value of each color channel is adjusted to a value corresponding to the spectral test image, to ensure that the required spectral information can be accurately generated at each pixel position, and the spectral test image can be accurately projected onto the projection surface.

[0049] The method for determining the spectral calibration table can refer to the following embodiments, which will not be described in detail here.

[0050] The visual perception test method provided by the embodiment of the present application can map the spectral information of each pixel in the spectral test image to the corresponding gray scale value based on the spectral calibration table, adjust the gray scale value of each color channel of each projector at each pixel position, and thus ensure that the spectral information output by each projector is accurate. The adjustment process is multi-channel coordinated, meaning that the gray scale value of each color channel is adjusted simultaneously or in a certain order to ensure that the spectral information of multiple projectors can be accurately superimposed on the projection surface. The flexible adjustment of the spectral calibration table and the gray scale value enables the system to adapt to various test requirements, whether it is the output of different color channels or the generation of complex spectral images. The system can accurately control the output by adjusting the spectral calibration table and the gray scale value, reducing the need for manual intervention and improving the degree of automation of the system. It is especially suitable for large-scale test applications and reduces the possibility of human error.

[0051] Based on any of the above embodiments, the method comprises: determining a preset color channel set and a preset gray scale value set based on the color channels and the gray scale adjustment range of the multiple projectors; performing a calibration operation based on the preset color channel set and the preset gray scale value set to determine the spectral calibration table; and projecting a gray scale test image corresponding to any preset color channel in the preset color channel set and any preset gray scale value in the preset gray scale value set onto a projection surface, the gray scale test image having at least one designated measurement region, all pixels in the designated measurement region having the same gray scale value as the preset gray scale value; measuring spectral information corresponding to the designated measurement region of the gray scale test image.

[0052] It should be noted that the designated measurement region is able to meet the requirements of normal and accurate measurement of the spectral radiation calibration device (e.g., filling the field of view of the calibration device), and the region outside the designated measurement region is set to 0 gray scale value to avoid the influence of pixel crosstalk. At the same time during the calibration process, other channels of the same projector are closed or set to 0 gray scale, and other projectors are shielded or closed to avoid unnecessary stray light interference. For the selected color channel and the selected preset gray scale value, the calibration operation is repeated multiple times, and the spectral calibration table is determined.

[0053] Here, the multiple projectors in the test device project a specific gray scale test image onto the projection surface, and the test image is designed to have given gray scale values covering the range of light output intensity adjustment of the color channels. The pixels with the given gray scale values constitute a test region on the test image.

[0054] Here, the preset color channel set is a set of color channels selected from the color channels supported by the projector for spectral calibration testing. Each color channel usually corresponds to a single light source color controlled by the projector. The preset gray scale value set is a set of gray scale values determined from the gray scale adjustment range supported by the projector, each gray scale value corresponding to the light output intensity of the projector light source under a single color channel. It should be noted that the gray scale value range supported by the projector (e.g., 0-255) is not necessarily consistent with the gray scale value range corresponding to the minimum and maximum light output (e.g., 0-190), and the selected preset gray scale values and the light output should be monotonous as much as possible to facilitate the operation of the measurement device.

[0055] In an embodiment, by analyzing the gray scale values of the pixels in the designated test region of the gray scale test image and their corresponding spectral information, i.e., by analyzing the gray scale values of the pixels in the test region of the gray scale test image and the spectral radiance or spectral irradiance of the corresponding pixels, a mapping relationship between the gray scale values and the spectral information is established, which is the spectral calibration table, providing a basis for subsequent projection and analysis.

[0056] Figure 2The following is a diagram of the conversion from the original spectrum test image to the spectrum test image that matches the color channel of the test device. Since the number of wavelength channels and the bandwidth of the acquisition of the original spectrum test image are generally inconsistent with the color channel of the test device, it is necessary to Figure 2 For example, the light intensity of each independent color channel of the test device can be independently controlled by the grayscale value of each pixel in the image or video. The adjustment range of the grayscale value is determined by the bit depth of the projector. For example, each color channel of an 8-bit projector corresponds to 256 grayscale levels from 0 to 255, where 0 indicates that the channel is at minimum output and 255 indicates that the channel is at full output. In order to achieve precise control of the spectral characteristics, it is necessary to adjust the grayscale values ​​of the independent color channels (marked as k) of the test device at different grayscale values. The spectral information under the calibration is shown as ,in Represents the grayscale value of independent color channel k, Represents the wavelength of the spectrum, {} represents the set, that is, Take the sum of all spectral information at different grayscale values ​​within the preset grayscale value set of the color channel. The spectral calibration table is determined based on the measurement results. The spectral calibration table records the mapping relationship between grayscale values ​​and spectral information to ensure that the projector can accurately control the output spectral information according to the grayscale values ​​when outputting the test image.

[0057] For example, when using multiple projectors to project a spectral test image, the original spectral test image is input into the testing device. To ensure that the spectral information corresponding to the pixel (i, j) on the projection surface is as close as possible to the spectral information corresponding to the original spectral test image, the grayscale value of each pixel in the target area of ​​each projector's color channel needs to be adjusted according to the spectral calibration table. Specifically, the spectral information S corresponding to the pixel (i, j) on the projection surface is expressed as follows: = ; in, Indicates the wavelength of the spectrum. Compared with the original spectral test image in pixels ( i, j ) Spectral information corresponding to the position The residual between the two should be as small as possible. The residual can be expressed using metrics such as mean squared error (MSE) and root mean square error (RMSE). To achieve this fitting process, numerical algorithms such as nonlinear fitting can be used.

[0058] For example, if the spectral shapes of different grayscales for each color channel k are considered to be the same with sufficient accuracy, the spectral calibration table can be simplified using the following parameters: (1) coefficients representing the ratio of channel output intensities ; (2) Spectrum when the channel output intensity is maximum and the corresponding eigenvalues L k,max (e.g. brightness, radiance, spectral peak, or spectral radiance / irradiance value at a fixed wavelength, etc.); (3) Grayscale G k and eigenvalues L k,max The corresponding relationship G k - L k,max (4) The background spectrum information on the projector measured when the grayscale of all color channels is set to 0. For a given spectrum to be fitted, that is, the pixels in the original spectrum test image ( i, j ) corresponding to the spectrum , the optimization objective function of nonlinear fitting can be formally written as: ; here is the total background spectrum, is the given ambient background spectrum superimposed on the projection screen, is the amplitude adjustment coefficient so that the fitted Not greater than 1. The Levenberg-Marquardt (LM) algorithm can be used to perform nonlinear fitting on the above optimization objective function to obtain the coefficient Then we calculate and through G k -L k，max Determination of calibration curve and curve interpolation method G k , that is, the color channel matches the spectrum test image corresponding to ( i, j )The grayscale value of the pixel at position .

[0059] The visual perception test method provided by the embodiment of the present application projects test images onto a projection surface through a plurality of projectors in a test device, generates gray scale test images, and determines a spectral calibration table based on the gray scale value of each pixel and the corresponding spectral information. By gradually determining each gray scale value and collecting the corresponding spectral information, the mapping relationship between the gray scale value and the spectral information can be accurately established, thereby realizing accurate calibration. This accurate spectral calibration table provides a reliable basis for spectral control in the subsequent projection process, so that the projection device can accurately adjust the spectral output of each pixel according to the requirements, thereby meeting the high-precision requirements of complex spectral test and analysis. This method not only improves the accuracy of spectral test, but also enhances the flexibility and controllability of the test device in multi-channel spectral regulation.

[0060] Based on any of the above embodiments, in the method, the spectral calibration table is corrected based on the following manner: Measuring the spectral information of a specific gray scale value to obtain corrected spectral information; Determining a correction coefficient based on the spectral information corresponding to the specific gray scale value in the spectral calibration table and the corrected spectral information; Correcting the spectral information corresponding to all gray scale values except the specific gray scale value in the spectral calibration table based on the correction coefficient.

[0061] In an embodiment, the spectral information generated after projecting a plurality of specific gray scale values is measured by an optical calibration device such as a spectral radiometer in the test device to obtain actual corrected spectral information. This process can obtain the real spectral output characteristics of the projection system at this gray scale value. Further, by comparing the measured corrected spectral information with the initial calibration spectral information corresponding to this gray scale value in the spectral calibration table, a correction coefficient is calculated. The correction coefficient reflects the deviation between the actual spectral output of the test device and the initial calibration value, and can be used to adjust the spectral information of other gray scale values.

[0062] Exemplarily, since the absolute value of the light source output may fluctuate due to the change of the number of device start-ups, use time or environmental conditions, causing the change of the spectral information in the spectral calibration table, when recalibration is required, a number of specific gray scale values can be selected for spectral measurement, for example, at least two gray scale values such as 0 gray scale and the gray scale when the channel light output is maximum, and if a four-point interpolation algorithm is required, two additional gray scale values can be selected; four-point interpolation can obtain better interpolation effect than two-point interpolation. The actual spectral information of these gray scale values is obtained, the measured actual spectral information is compared with the initial calibration spectral information of the corresponding gray scale value in the spectral calibration table, a correction coefficient is calculated to describe the deviation between the original value and the current actual value, and the correction coefficient is applied to the correction of the spectral information of other gray scale values in the spectral calibration table, so as to efficiently correct the output accuracy of the entire spectral calibration table.

[0063] The visual perception test method provided by the embodiments of the present application improves the calibration efficiency and data accuracy of the test device under the light source output fluctuation condition by dynamically correcting the spectral calibration table. First, the actual spectral information of a number of specific gray scale values is measured to obtain corrected spectral information; then, the correction coefficient is calculated by comparing the initial calibration theoretical spectral information of the specific gray scale with the corrected spectral information, and the correction coefficient is used to describe the influence of the light source output fluctuation. Next, the spectral information of other gray scale values is batch corrected based on the correction coefficient, thereby avoiding the complex operation of measuring all gray scales one by one. This method not only maintains the relative stability of the spectral information, but also effectively compensates for the deviation caused by the change of the light source output, significantly improves the efficiency and reliability of the spectral calibration, and ensures the accuracy of subsequent spectral test and analysis.

[0064] Based on any of the above embodiments, the method further includes: determining a reference projector from the plurality of projectors, and determining the projectors other than the reference projector as superimposed projectors; determining a reference projection area in which the projection picture of the reference projector is located; adjusting the areas in which the projection pictures of the superimposed projectors are located, so that the areas in which the projection pictures of the superimposed projectors are located and the reference projection area have a fixed geometric relationship, and determining a target projection area; cropping and deforming the images projected by the superimposed projectors, so that the images projected by the superimposed projectors are coordinated and consistent and fused together, and determining a projection surface.

[0065] Here, the reference projector refers to the projector used as a reference standard during the calibration process. The position and area of ​​its projected image do not need to be adjusted, but serve as the basis for calibration of other projectors. The superimposed projector refers to other projectors other than the reference projector. Its projection image needs to be adjusted and calibrated to overlap or merge with the reference projection area. The reference projection area refers to the area where the reference projector's projection image is located, which serves as the target area for adjustment and calibration of other projectors. The target projection area is a unified area formed by overlapping and splicing the projection images of multiple projectors after calibration. The projection surface refers to the final aligned and fused projection image.

[0066] For example, Figure 3 As shown, if the reference projection area is a quadrilateral, the areas where the projection images of the remaining multiple superimposed projectors are located are adjusted so that the areas where the projection images of the multiple superimposed projectors are located overlap with the reference projection area to determine the target projection area; Furthermore, each stacked projector projects a calibration image A in sequence, and a digital camera at a fixed position captures the calibration image A of each projector in sequence. The calibration image A can be a regular grid line or a dot matrix. Based on the captured images, a four-point calibration method is proposed: First, the pixel coordinates of the four vertices of the quadrilateral area of ​​the reference projection screen are determined (denoted as ), the pixel coordinates of the four vertices can be determined using a calibration image such as the center of the intersection of the grid lines or the center of the midpoint of the dot matrix; for each of the four vertices, the pixel coordinates of the four calibration control points surrounding the vertex pixel coordinates in the image projected by the stacking projector are determined; these four calibration control points are generally the intersections of the orthogonal grid lines closest to the vertex pixel coordinates, or the points in the dot matrix; each stacking projector has four sets of calibration control points, i.e., 16 calibration control points, and the pixel coordinates of the four sets of calibration control points corresponding to the kth stacking projector are denoted as , the kth superposition projector projects the calibration image A onto the target projection area, and then the image formed after being photographed and recorded by the digital camera is , then the vertex pixel coordinates With image 4 sets of calibration control points The relative position relationship between them is mapped to the coordinates of the calibration image A through geometric transformation (such as projection transformation), so that the four vertex pixel coordinates corresponding to the k-th projector can be obtained and the pre-deformation area enclosed.

[0067] Furthermore, for the k-th projector, the image or video to be projected is transformed into The determined pre-deformation region determines a projection surface after each superimposed projector sequentially performs the above light field overlap calibration step; the reference projector and each superimposed projector can realize consistent overlap when playing the same picture in the target projection region.

[0068] The visual perception test method provided by the embodiment of the application provides a clear correction basis for picture adjustment of other projectors by determining the reference projector and its projection region as a reference standard. Then, by adjusting the picture region of the superimposed projector and combining image deformation processing, the geometric distortion and alignment problem between pictures are solved, and it is ensured that all projection pictures are coordinated and consistent in the target projection region. Finally, this method effectively constructs a unified projection surface, significantly improves the spatial consistency and light field uniformity of the projection system, and provides a larger picture range and higher accuracy for complex projection tests, simplifies subsequent test operations and enhances the flexibility and adaptability of the system.

[0069] Based on any of the above embodiments, in the method, the comparison of the analysis result label of the to-be-analyzed image with the preset label data corresponding to the spectrum test image to obtain the visual perception test result of the measured device comprises: If the category label of the object in the analysis result label matches the object label in the preset label data, the object in the analysis result label is determined as an analysis correct object; Based on the proportion of the analysis correct object in the analysis result label to all objects in the analysis result label and the proportion of the analysis correct object in the analysis result label to all object labels in the preset label data, an evaluation score of the measured device is determined; Based on the evaluation score, a visual perception test result of the measured device is determined.

[0070] It should be noted that by changing the spectrum information, geometric information (position, size, angle, etc.), adding interference recognition object factors (such as noise, shielding), image generation algorithm, etc. of the labeled objects and the background in the spectrum test image to obtain a new data set and projection, the probability of correct recognition of the object label by the measured machine vision system is investigated by multiple repeated tests, and the relationship between the recognition probability and the change of the visual perception performance with respect to the controlled parameters can be given.

[0071] Here, the analysis correct object refers to an object in the analysis result label that completely matches the preset label data, indicating that the recognition of the measured device on this object is accurate.

[0072] In one embodiment, the objects in the analysis result label are matched one by one with the object labels in the preset label data. If the object in the analysis result label matches the object in the preset label, the object is determined to be the correct object for analysis; through precise comparison, the accuracy of the device under test in identifying the objects in the scene is determined.

[0073] In one embodiment, the ratio of correct objects to all objects in the analysis result labels is analyzed to measure the accuracy of the device under test in identifying objects; the ratio of correct objects to all object labels in the preset labels is analyzed to measure the coverage of the preset targets by the device under test; the evaluation score of the device under test is calculated by combining the accuracy and coverage to quantify the overall performance of the device in the recognition task.

[0074] In one embodiment, a new dataset is generated and projected by varying the spectral information and geometric information (position, size, angle, etc.) of the labeled object and background in the spectral test image, adding factors that interfere with object recognition (such as noise and occlusion), and employing an image generation algorithm. Repeated testing is then used to examine the probability of the machine vision system correctly identifying the object label. This provides a relationship between the recognition probability and the change in visual perception performance as the controlled parameters change. This can simulate test scenarios where objects are exposed to varying ambient lighting conditions, quantifying the robustness of the device to changes in the test environment during recognition tasks.

[0075] The visual perception testing method provided by the embodiments of this invention comprehensively measures the visual perception capabilities of the device under test using the dual metrics of accuracy and coverage. It focuses not only on recognition accuracy but also on target coverage. By calculating an evaluation score, the visual perception capabilities of the device under test are quantified, providing a clear basis for subsequent performance optimization and improvement. By analyzing the proportion of correct objects, the strengths and weaknesses of the device under test in specific tasks can be clearly identified, providing guidance for subsequent adjustments and optimizations.

[0076] In another embodiment, a control program of a testing device is provided, the control program includes a plurality of modules, each responsible for different functions, ensuring the smooth progress of the testing process, and being able to accurately evaluate the test results. Specifically, it includes a human-computer interaction interface, a projection picture overlap correction module, an image processing module, a spectral calibration module, a communication control module and a scoring module. Among them, the human-computer interaction interface provides an interactive interface between the user and the system, allowing the user to control the playback process of the image or video, including automatic operation, image or video selection, playback control, etc. The user can set the projection content and the system through the interface. The projection picture overlap correction module is responsible for adjusting the overlapping area between multiple projector pictures, ensuring that the image scene in the overlapping area can be accurately aligned when multiple projectors play the same image or video at the same time. The image processing module is used to process each frame of image or video played in the overlapping area, to ensure the spectral superposition and spectral intensity adjustment of each pixel point, so as to realize the light field projection conforming to the target spectral distribution in the projection process. The spectral calibration module is used to save and manage the spectral information corresponding to different gray scale values of each independent color channel (such as red, green, blue, etc.). The communication control module is used for data interaction with the measured machine vision system, responsible for obtaining the collected images and their analysis result labels from the measured device. The scoring module is used to statistically analyze the accuracy indicators of all analysis result labels, and generate the final score.

[0077] The visual perception testing device provided by the application is described below. The visual perception testing device described below can be referred to in correspondence with the visual perception testing method described above.

[0078] FIG. 4 is a structural schematic diagram of a visual perception testing device provided by the application. As shown in FIG. 4, the visual perception testing device, The correction module 410 is used for correcting the projection pictures of multiple projectors and determining the projection surface. The acquisition module 420 is used for projecting the spectral test image onto the projection surface, and obtaining the to-be-analyzed image on the projection surface collected by the measured device and the analysis result label. The test module 430 is used for comparing the analysis result label of the to-be-analyzed image with the preset label data corresponding to the spectral test image, obtaining the visual perception test result of the measured device, and the analysis result label includes the category label, the geometric information label and the spectral information label of each object in the to-be-analyzed image.

[0079] Figure 5 An example of an electronic device is shown in the structural schematic diagram, Figure 5As shown, the electronic device can include a processor 510, a communications interface 520, a memory 530, and a communications bus 540, wherein the processor 510, the communications interface 520, and the memory 530 communicate with each other through the communications bus 540. The processor 510 can invoke the logic instructions in the memory 530 to execute the visual perception test method, which includes correcting the projection pictures of multiple projectors to determine a projection surface, projecting a spectral test image onto the projection surface, acquiring a to-be-analyzed image on the projection surface collected by a measured device and an analysis result label, comparing the analysis result label of the to-be-analyzed image with preset label data corresponding to the spectral test image to obtain a visual perception test result of the measured device, and the analysis result label includes a category label, geometric information label, and spectral information label of each object in the to-be-analyzed image.

[0080] In addition, the logic instructions in the memory 530 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0081] On the other hand, the present application also provides a computer program product, which includes a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program is executed by a processor, so that the computer can execute the visual perception test method provided by the above-mentioned methods, which includes: correcting the projection pictures of multiple projectors to determine a projection surface; projecting a spectral test image onto the projection surface, acquiring a to-be-analyzed image on the projection surface collected by a measured device and an analysis result label; comparing the analysis result label of the to-be-analyzed image with preset label data corresponding to the spectral test image to obtain a visual perception test result of the measured device, and the analysis result label includes a category label, geometric information label, and spectral information label of each object in the to-be-analyzed image.

[0082] In yet another aspect, the present application also provides a non-transitory computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the visual perception test method provided by any of the above methods, and the method comprises: correcting projection pictures of a plurality of projectors to determine a projection surface; projecting a spectral test image onto the projection surface to obtain a to-be-analyzed image on the projection surface collected by a device under test and an analysis result label; comparing the analysis result label of the to-be-analyzed image with preset label data corresponding to the spectral test image to obtain a visual perception test result of the device under test, wherein the analysis result label comprises a category label, geometric information label and spectral information label of each object in the to-be-analyzed image.

[0083] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0084] From the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software and necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a plurality of instructions to make a computer device (which can be a personal computer, server, or network device, etc.) execute the method described in each embodiment or some parts of the embodiment.

[0085] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A visual perception test method, characterized in that: include: Calibrate the projection images of multiple projectors to determine the projection surface; Projecting the spectral test image onto the projection surface, and obtaining the image to be analyzed and the analysis result label on the projection surface collected by the device under test; The analysis result label of the image to be analyzed is compared with the preset label data corresponding to the spectral test image to obtain the visual perception test result of the device under test, wherein the analysis result label includes the category label, geometric information label and spectral information label of each object in the image to be analyzed.

2. The visual perception test method according to claim 1, characterized in that: The projecting of the spectrum test image onto the projection surface comprises: Determining a spectrum calibration table, wherein the spectrum calibration table includes a mapping relationship between grayscale values ​​and spectrum information; the spectrum information includes light intensity information of the sub-spectrum; Based on the spectral information of the spectral test image and the spectral calibration table, the grayscale value of each pixel corresponding to each color channel of the plurality of projectors in the spectral test image is adjusted to project the spectral test image onto the projection surface.

3. The visual perception testing method according to claim 2, wherein: Determining the spectrum calibration table includes: Determining a preset color channel set and a preset grayscale value set based on the color channels and grayscale adjustment ranges of the plurality of projectors; Based on the preset color channel set and the preset grayscale value set, a calibration operation is performed to determine a spectral calibration table; any calibration operation is as follows: Projecting a grayscale test image corresponding to any preset color channel in the preset color channel set and any preset grayscale value in the preset grayscale value set onto a projection surface, wherein the grayscale test image has at least one designated measurement area, and the grayscale values ​​of all pixels in the designated measurement area are the same as the preset grayscale value; Spectral information corresponding to the designated measurement area of ​​the grayscale test image is measured.

4. The visual perception testing method according to claim 2, wherein: The spectrum calibration table is modified based on the following method: Measuring spectral information of a specific grayscale value to obtain corrected spectral information; Determining a correction coefficient based on the spectral information corresponding to the specific grayscale in the spectral calibration table and the corrected spectral information; The spectral information corresponding to all grayscale values ​​except the specific grayscale value in the spectral calibration table is corrected based on the correction coefficient.

5. The visual perception testing method according to claim 1, wherein: The method of calibrating the projection images of the plurality of projectors to determine the projection surface includes: Determining a reference projector from among the plurality of projectors, and determining projectors from among the plurality of projectors except the reference projector as superposition projectors; Determine the area where the projection image of the reference projector is located as the reference projection area; Adjusting the areas where the projection images of the plurality of superimposed projectors are located so that the geometric relationship between the areas where the projection images of the plurality of superimposed projectors are located and the reference projection area is fixed, thereby determining a target projection area; The images projected by the multiple superimposed projectors are cropped and deformed so that the images projected by the multiple superimposed projectors are coordinated and integrated together to determine a projection surface.

6. The visual perception testing method according to claim 1, wherein: The step of comparing the analysis result label of the image to be analyzed with the preset label data corresponding to the spectral test image to obtain the visual perception test result of the device under test includes: If the category label of the object in the analysis result label matches the object label in the preset label data, the object in the analysis result label is determined to be a correct object; Determining an evaluation score for the device under test based on a ratio of correctly analyzed objects in the analysis result labels to all objects in the analysis result labels, and a ratio of correctly analyzed objects in the analysis result labels to all object labels in the preset label data; Based on the evaluation score, a visual perception test result of the device under test is determined.

7. A visual perception testing device, characterized in that: include: A correction module, which is used to correct the projection images of multiple projectors and determine the projection surface; an acquisition module, configured to project the spectral test image onto the projection surface and obtain the image to be analyzed and the analysis result label on the projection surface acquired by the device under test; A testing module is used to compare the analysis result label of the image to be analyzed with the preset label data corresponding to the spectral test image to obtain the visual perception test result of the device under test. The analysis result label includes the category label, geometric information label and spectral information label of each object in the image to be analyzed.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the visual perception testing method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the visual perception testing method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the visual perception testing method according to any one of claims 1 to 6 is implemented.

Images