A single-pixel task-aware method, apparatus, medium, and device

Abstract

The application discloses a single-pixel task perception method and device, a medium and equipment, and relates to the technical field of machine vision. The application models the sequence dependence of a one-dimensional measurement sequence, fully utilizes the timing prior information determined by the modulation pattern loading sequence, maps and rearranges the extracted timing features into spatial representation, fully utilizes the sequence order information, establishes a stable correspondence relationship between the intermediate representation and the spatial geometry of the target scene, and overcomes the problem of unstable intermediate representation caused by the task evidence distribution under the condition of undersampling. In combination with multi-scale spatial feature extraction and fusion, the key spatial structure information related to the task can be more accurately captured, so that the accuracy and robustness of single-pixel task perception are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a single-pixel task perception method, apparatus, medium, and device. Background Technology

[0002] Currently, single-pixel sensing typically modulates and projects a target scene sequentially under the control of a preset modulation pattern sequence, and a barrel detector outputs an intensity scalar for each modulation pattern, thus forming a one-dimensional measurement sequence corresponding to the loading order of the modulation patterns. On platforms with limited bandwidth, storage, or computing resources, this method can be used to reduce the amount of data acquired and transmitted.

[0003] Task inference schemes related to single-pixel measurement typically include post-reconstruction inference processes and end-to-end reconstruction inference processes without intermediate steps. The end-to-end reconstruction inference process first involves a single-pixel imaging system acquiring a one-dimensional measurement sequence containing target scene information, which is then directly fed into an end-to-end trained neural network model. This model skips the "explicit reconstruction" step in traditional processes that restores measurement data to a two-dimensional pixel image. Instead, it uses deep learning algorithms to directly mine the highly nonlinear mapping relationship between the one-dimensional measurement sequence and a specific task (such as image classification, target recognition, or anomaly detection) in the measurement domain. Finally, the model directly outputs task decision results (such as category labels or judgment conclusions) that meet business requirements, thus achieving efficient inference from raw data to high-level semantic information while completely avoiding time-consuming image reconstruction computations.

[0004] However, single-pixel measurement sequences have a temporal structure generated by the loading order of modulation patterns. Under undersampling conditions, task-related evidence often appears in a distributed manner. Existing single-pixel task perception methods do not make sufficient use of sequence order information, which easily leads to the problem of unstable correspondence between intermediate representations and spatial geometric relationships, resulting in poor accuracy of single-pixel task perception. Summary of the Invention

[0005] Therefore, it is necessary to provide a single-pixel task perception method, apparatus, medium, and device to address the aforementioned technical problems.

[0006] The present invention adopts the following technical solution: This invention provides a single-pixel task perception method, comprising: Acquire a one-dimensional measurement sequence after it is projected onto the target scene under the control of a preset modulation pattern sequence in a single pixel sensor; The sequence dependencies between one-dimensional measurement sequences are modeled, and the temporal features of the one-dimensional measurement sequences are extracted. The temporal features of the one-dimensional measurement sequences are then mapped and rearranged to transform them into spatial representations. Multi-scale spatial feature extraction and fusion are performed on the spatial representation of a one-dimensional measurement sequence to obtain the spatial features of the one-dimensional measurement sequence. Based on the spatial characteristics of the one-dimensional measurement sequence, the single-pixel task perception result is determined.

[0007] Optionally, the temporal features of the one-dimensional measurement sequence are extracted through the temporal feature extraction backbone and the temporal features are converted into spatial representations; The spatial features of the one-dimensional measurement sequence are obtained by extracting the backbone from the spatial features. The single-pixel task perception result is determined by the task decoder; Training the temporal feature extraction backbone, spatial feature extraction backbone, and task decoder specifically includes: Obtain the one-dimensional measurement sequence as training samples, and the target scene map and the real perception result of the target scene map corresponding to the one-dimensional measurement sequence as sample annotation; The training samples are input into the temporal feature extraction backbone to extract the temporal features of the training samples and convert the temporal features into spatial representations. The spatial representations of the training samples are input into the spatial feature extraction backbone to extract the spatial features of the training samples. The task decoder determines the predicted perception result based on the spatial features of the training samples. The task perception loss is determined based on the deviation between the actual perception result and the predicted perception result. The spatial representation of the training samples is mapped to an auxiliary feature map in the image domain; the image domain composite loss is determined based on the deviation between the auxiliary feature map and the target scene map. The image domain composite loss is weighted by auxiliary loss weights, and the task perception loss is weighted by task loss weights. The sum of the two is determined as the total loss. The temporal feature extraction backbone, spatial feature extraction backbone, and task decoder are trained with minimizing the total loss as the optimization objective.

[0008] Optionally, determining the task loss weight and the auxiliary loss weight specifically includes: When the number of training rounds is less than or equal to the threshold of the first training round, the auxiliary loss weight is made greater than the task loss weight; When the number of training rounds is greater than the first training round threshold and less than or equal to the second training round threshold, the auxiliary loss weight is made equal to the task loss weight. When the number of training rounds is greater than the second training round threshold, the auxiliary loss weight is made less than the task loss weight; The task loss weight and the auxiliary loss weight are both greater than 0 and their sum is 1; the first training round threshold is less than the second training round threshold.

[0009] Optionally, determining the image domain composite loss based on the deviation between the auxiliary feature map and the target scene map specifically includes: The image domain composite loss is determined by the weighted sum of the pixel-level absolute error loss, pixel-level squared error loss, and structural similarity loss between the auxiliary feature map and the target scene map.

[0010] Optionally, the time-series feature extraction backbone includes a cyclic sequence modeling module and a ConvNeXt module; the cyclic sequence modeling module is a GRU network or an LSTM network; the cyclic sequence modeling module extracts the time-series features of the one-dimensional measurement sequence, and the ConvNeXt module converts the time-series features into spatial representations.

[0011] Optionally, the spatial feature extraction backbone is a Unet++ network.

[0012] Optionally, determining the single-pixel task perception result based on the spatial features of the one-dimensional measurement sequence specifically includes: When the task is classification, the category probability vector of objects in the target scene is determined based on the spatial features of the one-dimensional measurement sequence. When the task is object detection, the bounding boxes and class confidence of candidate objects in the target scene are determined based on the spatial features of the one-dimensional measurement sequence. When the task is semantic segmentation, a pixel-level semantic probability map of the target scene is output based on the spatial features of the one-dimensional measurement sequence.

[0013] This invention provides a single-pixel task sensing device, comprising: The acquisition module is used to acquire a one-dimensional measurement sequence after it is projected onto the target scene under the control of a preset modulation pattern sequence in a single-pixel sensor. The time series processing module is used to model the sequence dependencies between one-dimensional measurement sequences, extract the time series features of the one-dimensional measurement sequences, map and rearrange the time series features of the one-dimensional measurement sequences, and convert the time series features into spatial representations. The spatial processing module is used to extract and fuse multi-scale spatial features from the spatial representation of a one-dimensional measurement sequence to obtain the spatial features of the one-dimensional measurement sequence. The perception module is used to determine the perception result of a single pixel task based on the spatial characteristics of a one-dimensional measurement sequence.

[0014] The present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described single-pixel task perception method.

[0015] The present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described single-pixel task perception method.

[0016] The above-mentioned at least one technical solution adopted in this invention can achieve the following beneficial effects: This invention fully utilizes the temporal prior information determined by the loading order of modulation patterns by modeling the sequence dependencies of one-dimensional measurement sequences. By mapping and rearranging the extracted temporal features into spatial representations, it fully utilizes the sequence order information and establishes a stable correspondence between intermediate representations and the spatial geometry of the target scene. This overcomes the problem of unstable intermediate representations caused by the distributed occurrence of task evidence under undersampling conditions. Combined with multi-scale spatial feature extraction and fusion, it can more accurately capture key spatial structure information related to the task, thereby significantly improving the accuracy and robustness of single-pixel task perception. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0018] Figure 1 This is a schematic diagram of a single-pixel task perception method provided by the present invention; Figure 2 A flowchart illustrating a single-pixel task perception method provided by the present invention; Figure 3 This invention provides a schematic diagram of the relationship between task priority loss scheduling and spatiotemporal fuzzy fusion. Figure 4 A schematic diagram of a single-pixel task sensing device provided by the present invention; Figure 5 A schematic diagram of a computer device for implementing a single-pixel task perception method provided by the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0020] Currently, inference schemes for tasks related to single-pixel measurement typically include post-reconstruction inference workflows and end-to-end intermediate-free reconstruction inference workflows. For post-reconstruction inference workflows, a one-dimensional measurement sequence is usually reconstructed into a two-dimensional image based on a physical model or inversion algorithm of the single-pixel measurement. The reconstructed image is then input into a classification, detection, or segmentation network to obtain the task output. This workflow includes explicit image reconstruction steps. Under low sampling rate conditions, the reconstruction results may introduce artifacts or smoothing distortions, and the reconstruction and intermediate data incur additional computational and storage overhead, thus affecting the consistency of downstream task outputs.

[0021] For end-to-end reconstruction inference processes without intermediate steps, one-dimensional measurement sequences are directly mapped to task outputs to reduce or avoid explicit reconstruction steps. Single-pixel measurement sequences have a temporal structure generated by the loading order of modulation patterns. Under undersampling conditions, task-related evidence often appears in a distributed manner. When the model does not fully utilize the sequence order information, the correspondence between intermediate representations and spatial geometric relationships may become unstable. Furthermore, to improve training stability under undersampling conditions, existing end-to-end methods may introduce auxiliary supervision objectives. When the auxiliary objective and the task objective use fixed loss weights, the optimization requirements at different training stages are difficult to match, easily leading to multi-objective gradient interference, which affects the convergence process and the quality of task output.

[0022] This invention provides a single-pixel task perception method for classification, object detection and semantic segmentation tasks, in order to improve the problems of insufficient utilization of measurement sequence order information in end-to-end task inference under undersampling conditions and the instability of training caused by using fixed loss weights in multi-objective training.

[0023] The technical solutions provided by the various embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0024] Figure 1 This is a schematic diagram of a single-pixel task perception method according to the present invention, which specifically includes the following steps: S101: Obtain a one-dimensional measurement sequence after it is projected onto the target scene under the control of a preset modulation pattern sequence in a single pixel sensor.

[0025] S102: Model the sequence dependencies between one-dimensional measurement sequences and extract the temporal features of the one-dimensional measurement sequences; map and rearrange the temporal features of the one-dimensional measurement sequences to convert the temporal features into spatial representations.

[0026] S103: Multi-scale spatial feature extraction and fusion are performed on the spatial representation of the one-dimensional measurement sequence to obtain the spatial features of the one-dimensional measurement sequence.

[0027] S104: Determine the single-pixel task perception result based on the spatial characteristics of the one-dimensional measurement sequence.

[0028] For ease of explanation, the following description focuses solely on the server as the executing entity. The server mentioned in this invention can be a server set up on a business platform, or a device such as a desktop computer or laptop computer capable of executing the solution of this invention.

[0029] Figure 2 This is a schematic flowchart of a single-pixel task perception method according to the present invention. Figure 2 In this paper, t represents time, and I represents the total light intensity corresponding to each measurement in the one-dimensional measurement sequence. The invention acquires a one-dimensional measurement sequence under the control of a preset modulation pattern sequence, and inputs the one-dimensional measurement sequence into a task-aware network to output task results. The task-aware network includes a temporal feature extraction backbone and a spatial feature extraction backbone. Temporal features are obtained by sequentially modeling the one-dimensional measurement sequence, and these features are converted into spatial representations within the temporal feature extraction backbone. Then, the spatial feature extraction backbone extracts spatial features from the spatial representations and generates task results.

[0030] In one or more embodiments of the present invention, the temporal feature extraction backbone includes a cyclic sequence modeling module and a ConvNeXt module; the cyclic sequence modeling module can be a GRU network or an LSTM network; the cyclic sequence modeling module extracts the temporal features of the one-dimensional measurement sequence, and the ConvNeXt module converts the temporal features into spatial representations. The spatial feature extraction backbone can be a Unet++ network.

[0031] When training the task-aware network, in one or more embodiments of the present invention, the server can extract the backbone of the training sample input temporal features, extract the temporal features of the training sample, and convert the temporal features into spatial representations; input the spatial representations of the training sample into the backbone of the spatial features extraction, and extract the spatial features of the training sample; determine the predicted perception result based on the spatial features of the training sample through the task decoder; and determine the task perception loss based on the deviation between the actual perception result and the predicted perception result.

[0032] Meanwhile, the server can also map the spatial representation of the training samples into an auxiliary feature map in the image domain; and determine the image domain composite loss based on the deviation between the auxiliary feature map and the target scene map.

[0033] Finally, the server can weight the image domain composite loss using auxiliary loss weights and the task perception loss using task loss weights, and determine the sum of the two as the total loss; the temporal feature extraction backbone, spatial feature extraction backbone, and task decoder are trained with minimizing the total loss as the optimization objective.

[0034] Regarding the auxiliary loss weight and the task loss weight, in one or more embodiments of the present invention, when the training round is less than or equal to a first training round threshold, the auxiliary loss weight is made greater than the task loss weight; when the training round is greater than the first training round threshold and less than or equal to a second training round threshold, the auxiliary loss weight is made equal to the task loss weight; when the training round is greater than the second training round threshold, the auxiliary loss weight is made less than the task loss weight; both the task loss weight and the auxiliary loss weight are greater than 0 and their sum is 1; the first training round threshold is less than the second training round threshold.

[0035] Figure 3 This is a schematic diagram illustrating the relationship between task priority loss scheduling and spatiotemporal fuzzy fusion in this invention. Figure 3 For example, during the training phase, the task-aware network further outputs auxiliary feature maps and calculates the image domain composite loss. Simultaneously calculate task perception loss According to training rounds Set auxiliary loss weights With task loss weight .

[0036] in , and Constructing total loss To update network parameters; the training process is divided into at least three phases, making the early training phase... Mid-training phase and Equal or close, later stages of training This allows for task-priority loss scheduling. During the inference phase, no auxiliary feature maps are output, and intermediate image reconstruction is not performed; only the task results are output based on the one-dimensional measurement sequence.

[0037] This invention uses task-priority loss scheduling to adjust the ratio of auxiliary supervision to task supervision at different training stages, enabling the temporal feature extraction backbone and the spatial feature extraction backbone to receive staged constraints and optimization emphases during training, thereby forming a feature representation that takes into account both temporal structure and spatial geometry. Compared with existing technologies, this invention eliminates the need for intermediate image reconstruction during the inference stage, which helps reduce computational and intermediate data overhead. Furthermore, by configuring auxiliary loss and task loss in stages at different training stages through task-priority loss scheduling, this invention helps reduce multi-objective gradient interference and improve convergence stability under undersampling conditions, thus enhancing the consistency of task results.

[0038] Based on the above Figure 2 and Figure 3The following is a detailed description of the invention. First, the one-dimensional measurement sequence initially obtained by the server of the business platform can be denoted as... Their serial numbers correspond to the order in which the modulation patterns are loaded. The true value of the target scene image can be denoted as... Its vectorized form is The perception matrix (corresponding to the preset modulation pattern sequence) is denoted as... The noise term is denoted as The linear forward model for single-pixel measurement can be expressed as:

[0039] (1) The sampling rate is defined as: (2) In the formula, To measure length, This represents the number of pixels in the scene image.

[0040] The preset modulation pattern sequence can be obtained by using a structured modulation pattern sequence, such as a Hadamard modulation pattern sequence, a Fourier modulation pattern sequence, or a combination thereof. When performing single-pixel sensing, the acquisition order information can be retained; this order can be correlated with the sequence order of the one-dimensional measurement sequence itself.

[0041] After that, the server can... Normalization or standardization processing can be performed; preprocessing methods may include mean and variance normalization, maximum and minimum scaling, or scaling according to the sensor's dynamic range; when outliers exist, quantile clipping or robust normalization can be used to improve numerical stability.

[0042] Then, the server can process the one-dimensional measurement sequence. Perform temporal feature extraction and spatial representation transformation. Specifically, this can be achieved by... Input temporal feature extraction backbone and perform sequential modeling to obtain temporal features ; and within the main framework of temporal feature extraction, Transformation into spatial representation Spatial representation It can be in the form of a two-dimensional spatial feature map, and its dimensions are represented as follows:

[0043] (3) In the formula, For the number of channels, and The spatial dimensions are specified. The temporal feature extraction backbone may include a cyclic sequence modeling module; the cyclic sequence modeling module may employ a GRU structure or an LSTM structure to model the order dependencies of the measured sequences.

[0044] The server can then extract spatial features from the spatial representation of the one-dimensional measurement sequence. Specifically, the spatial representation can be... The input spatial feature extraction backbone performs spatial feature extraction and fusion to obtain the feature representation used for task output. The spatial feature extraction backbone may include multi-scale feature extraction structures and feature fusion structures; the feature fusion structure may include upsampling processing, downsampling processing, and cross-scale skip connections to achieve the fusion of spatial features at different scales.

[0045] Finally, the server can perform task awareness based on the spatial features of the one-dimensional measurement sequence. The feature representation of the extracted spatial features is input into the task decoder to output the task results. For classification, the output class logarithmic values ​​are processed by Softmax to obtain the class probability vector; for object detection, the bounding box parameters and class confidence of candidate objects are output, and the object detection result set is obtained through thresholding and non-maximum suppression; for semantic segmentation, the output pixel-level logarithmic map is processed by Sigmoid or Softmax to obtain the pixel-level probability map, and then the segmentation result is obtained through thresholding.

[0046] During the training of the temporal feature extraction backbone, the spatial feature extraction backbone, and the task decoder, the server outputs auxiliary feature maps in addition to the task results. The auxiliary feature map Spatial representation Obtained through mapping. The auxiliary feature map is then compared with the ground truth in the image domain. Constructing auxiliary loss after alignment In one or more embodiments of the present invention, the image domain composite loss is determined based on the weighted sum of the pixel-level absolute error loss, the pixel-level squared error loss, and the structural similarity loss between the auxiliary feature map and the target scene map.

[0047] (4) In the formula, For image domain composite loss, For pixel-level absolute error loss, For pixel-level squared error loss, For structural similarity loss, and satisfying , , and .when and When the resolutions are inconsistent, you can... Interpolation or learnable upsampling can be performed to achieve size matching.

[0048] Regarding losses, the server can also determine task-aware losses, specifically, task losses. The appropriate loss method should be selected based on the task type. For classification tasks, cross-entropy loss can be used; for object detection tasks, a weighted sum of classification loss and bounding box regression loss can be used; for semantic segmentation tasks, a weighted sum of Dice loss and cross-entropy loss or binary cross-entropy loss can be used.

[0049] Based on the determined total training loss, the server can update the network parameters. Specifically, this can be done according to the training epochs. Set auxiliary loss weights With task loss weight ,in , and The total loss can be constructed and the network parameters updated using the following formula:

[0050] (5) The training process is divided into at least three phases, so that the early stages of training meet the requirements. During the training period, the requirements were met. and Equal or close, satisfied in the later stages of training In one implementation, let the total number of training rounds be... The stage boundary is and ,satisfy:

[0051] (6) And it adopts segmented constant weight scheduling: (7) in, , and Equal or close to And for All meet The task-priority loss scheduling is used to: impose structural constraints on the mapping from measurement sequences to spatial representations in the early stages of training by using higher auxiliary loss weights; balance auxiliary supervision and task supervision in the middle stages of training to promote co-adaptation; and enhance task discrimination ability in the later stages of training by using higher task loss weights. Under these phased adjustments, it is beneficial to impose progressive constraints on the feature representations generated by the temporal feature extraction backbone and the spatial feature extraction backbone, thereby improving training stability and task output consistency under undersampling conditions.

[0052] The input for the inference phase is a one-dimensional measurement sequence. It does not output auxiliary feature maps and does not perform any intermediate image reconstruction steps, only outputs the task results; in one or more embodiments of the present invention, when the task is classification, the category probability vector of objects in the target scene is determined based on the spatial features of the one-dimensional measurement sequence; when the task is object detection, the bounding boxes and category confidence of candidate objects in the target scene are determined based on the spatial features of the one-dimensional measurement sequence; when the task is semantic segmentation, the pixel-level semantic probability map of the target scene is output based on the spatial features of the one-dimensional measurement sequence.

[0053] based on Figure 1 The single-pixel task perception method shown in this invention fully utilizes the temporal prior information determined by the loading order of the modulation pattern by modeling the sequence dependency of the one-dimensional measurement sequence. By mapping and rearranging the extracted temporal features into spatial representations, it fully utilizes the sequence order information and establishes a stable correspondence between intermediate representations and the spatial geometry of the target scene, overcoming the problem of unstable intermediate representations caused by the distributed occurrence of task evidence under undersampling conditions. Combined with multi-scale spatial feature extraction and fusion, it can more accurately capture key spatial structure information related to the task, thereby significantly improving the accuracy and robustness of single-pixel task perception.

[0054] When applying the single-pixel task perception method provided by this invention, it is not necessary to consider... Figure 1 The steps shown are executed in sequence. The specific execution order of each step can be determined as needed, and this invention does not impose any restrictions on it.

[0055] The above describes a single-pixel task perception method provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding single-pixel task perception device, such as... Figure 4 As shown.

[0056] Figure 4 A schematic diagram of a single-pixel task sensing device provided by the present invention includes: The acquisition module 201 is used to acquire a one-dimensional measurement sequence after it is projected onto the target scene under the control of a preset modulation pattern sequence in a single pixel sensor; The time series processing module 202 is used to model the sequence dependency relationship between one-dimensional measurement sequences, extract the time series features of the one-dimensional measurement sequences, map and rearrange the time series features of the one-dimensional measurement sequences, and convert the time series features into spatial representations. Spatial processing module 203 is used to extract and fuse multi-scale spatial features of the spatial representation of a one-dimensional measurement sequence to obtain the spatial features of the one-dimensional measurement sequence. The perception module 204 is used to determine the single-pixel task perception result based on the spatial characteristics of the one-dimensional measurement sequence.

[0057] Specific limitations regarding the single-pixel task sensing device can be found in the limitations of the single-pixel task sensing method described above, and will not be repeated here. Each module in the aforementioned single-pixel task sensing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0058] The present invention also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 The provided single-pixel task awareness method.

[0059] The present invention also provides Figure 5 The schematic diagram of the computer device shown is as follows: Figure 5 As shown, at the hardware level, this computer device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then executes it to achieve the above. Figure 1 The provided single-pixel task awareness method.

[0060] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0061] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this invention.

Claims

1. A single-pixel task perception method, characterized in that, include: Acquire a one-dimensional measurement sequence after it is projected onto the target scene under the control of a preset modulation pattern sequence in a single pixel sensor; Model the sequence dependencies between one-dimensional measurement sequences and extract the temporal features of the one-dimensional measurement sequences; The temporal features of a one-dimensional measurement sequence are mapped and rearranged to convert the temporal features into spatial representations. Multi-scale spatial feature extraction and fusion are performed on the spatial representation of a one-dimensional measurement sequence to obtain the spatial features of the one-dimensional measurement sequence. Based on the spatial characteristics of the one-dimensional measurement sequence, the single-pixel task perception result is determined.

2. The single-pixel task perception method as described in claim 1, characterized in that, Temporal features are extracted from the backbone of a one-dimensional measurement sequence and then converted into spatial representations. The spatial features of the one-dimensional measurement sequence are obtained by extracting the backbone from the spatial features. The single-pixel task perception result is determined by the task decoder; Training the temporal feature extraction backbone, spatial feature extraction backbone, and task decoder specifically includes: Obtain the one-dimensional measurement sequence as training samples, and the target scene map and the real perception result of the target scene map corresponding to the one-dimensional measurement sequence as sample annotation; The training samples are input into the temporal feature extraction backbone to extract the temporal features of the training samples and convert the temporal features into spatial representations. The spatial representations of the training samples are input into the spatial feature extraction backbone to extract the spatial features of the training samples. The task decoder determines the predicted perception result based on the spatial features of the training samples. The task perception loss is determined based on the deviation between the actual perception result and the predicted perception result. The spatial representation of the training samples is mapped to an auxiliary feature map in the image domain; the image domain composite loss is determined based on the deviation between the auxiliary feature map and the target scene map. The image domain composite loss is weighted by auxiliary loss weights, and the task perception loss is weighted by task loss weights. The sum of the two is determined as the total loss. The temporal feature extraction backbone, spatial feature extraction backbone, and task decoder are trained with minimizing the total loss as the optimization objective.

3. The single-pixel task perception method as described in claim 2, characterized in that, Determining the task loss weight and the auxiliary loss weight specifically includes: When the number of training rounds is less than or equal to the threshold of the first training round, the auxiliary loss weight is made greater than the task loss weight; When the number of training rounds is greater than the first training round threshold and less than or equal to the second training round threshold, the auxiliary loss weight is made equal to the task loss weight. When the number of training rounds is greater than the second training round threshold, the auxiliary loss weight is made less than the task loss weight; The task loss weight and the auxiliary loss weight are both greater than 0 and their sum is 1; the first training round threshold is less than the second training round threshold.

4. The single-pixel task perception method as described in claim 2, characterized in that, The step of determining the image domain composite loss based on the deviation between the auxiliary feature map and the target scene map specifically includes: The image domain composite loss is determined by the weighted sum of the pixel-level absolute error loss, pixel-level squared error loss, and structural similarity loss between the auxiliary feature map and the target scene map.

5. The single-pixel task perception method as described in claim 2, characterized in that, The temporal feature extraction backbone includes a cyclic sequence modeling module and a ConvNeXt module; The cyclic sequence modeling module is a GRU network or an LSTM network; the temporal features of the one-dimensional measurement sequence are extracted through the cyclic sequence modeling module, and the temporal features are converted into spatial representations through the ConvNeXt module.

6. The single-pixel task perception method as described in claim 2, characterized in that, The spatial feature extraction backbone is the Unet++ network.

7. The single-pixel task perception method as described in claim 1, characterized in that, The step of determining the single-pixel task perception result based on the spatial features of the one-dimensional measurement sequence specifically includes: When the task is classification, the category probability vector of objects in the target scene is determined based on the spatial features of the one-dimensional measurement sequence. When the task is object detection, the bounding boxes and class confidence of candidate objects in the target scene are determined based on the spatial features of the one-dimensional measurement sequence. When the task is semantic segmentation, a pixel-level semantic probability map of the target scene is output based on the spatial features of the one-dimensional measurement sequence.

8. A single-pixel task sensing device, characterized in that, include: The acquisition module is used to acquire a one-dimensional measurement sequence after it is projected onto the target scene under the control of a preset modulation pattern sequence in a single-pixel sensor. The time series processing module is used to model the sequence dependencies between one-dimensional measurement sequences and extract the time series features of the one-dimensional measurement sequences. The temporal features of a one-dimensional measurement sequence are mapped and rearranged to convert the temporal features into spatial representations. The spatial processing module is used to extract and fuse multi-scale spatial features from the spatial representation of a one-dimensional measurement sequence to obtain the spatial features of the one-dimensional measurement sequence. The perception module is used to determine the perception result of a single pixel task based on the spatial characteristics of a one-dimensional measurement sequence.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 7.

10. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in any one of claims 1 to 7.

Images