Low-power-consumption intelligent lighting control method based on RISC-V expansion acceleration calculation

Abstract

The invention relates to a low-power-consumption intelligent lighting control method based on RISC-V expansion acceleration calculation, which comprises the following steps that: an ambient light sensor monitors ambient light intensity in real time, a human body motion sensor detects a human body activity state, and if the current ambient light intensity is lower than a set threshold value and human body activity is detected, a controller (RISC-V MCU) controls a switch (MOSFET) to turn on an LED lamp; after the LED lamp is turned on, the monitoring equipment collects video frames and analyzes the motion condition; whether a moving object exists or not is judged according to the analysis result, if the monitoring equipment detects that no person moves, the controller turns off the LED lamp, and if not, illumination continues to be kept till the moving object cannot be detected. According to the method, the RISC-VV extension vectorization instruction is utilized, gray value calculation of the multiple pixel points is processed in parallel, the multiple pixel points can be calculated in a single instruction period through frame difference calculation, and the calculation efficiency is greatly improved.

Description

TECHNICAL FIELD

[0001] The application relates to a low-power intelligent lighting control method based on RISC-V V extension acceleration calculation, and belongs to the technical field of intelligent lighting. BACKGROUND

[0002] In a traditional daily lighting system, such as a corridor induction lamp, there is a lack of sensing capability for environmental parameters such as light intensity and personnel presence; after triggering lighting, the judgment logic for extinguishing is unreasonable, there is a situation that the light is extinguished when the person has not walked far, and there is a situation that the light is turned on after the person passes by, so that the lighting system has high energy consumption and poor flexibility.

[0003] The existing intelligent lighting system usually adopts the following ways for light control: based on ambient light sensing: detecting external brightness through an ambient light sensor, and turning on the lighting when the light is insufficient. However, this method cannot distinguish whether there is a person present, and may turn on the light when it is not needed, resulting in increased energy consumption. Based on human body sensing: using millimeter wave radar to detect human body movement to trigger lighting, but there is a probability of misjudgment, which will be disturbed by other light objects in the environment. Moreover, the initial installation requires relatively complex configuration. Visual detection based on camera: some schemes use a camera for video analysis, such as frame difference method to detect moving targets, thereby deciding the light switch. However, this kind of method usually requires a large amount of computing resources, and it is difficult to run efficiently in a low-power embedded system.

[0004] A single sensing method often cannot achieve ideal energy saving and comfort in different environments and application scenarios. Therefore, there is an urgent need for a solution that combines multi-sensor fusion, low-power computing and intelligent control to optimize lighting management. SUMMARY

[0005] In view of the deficiencies of the prior art, the application provides a low-power intelligent lighting control method based on RISC-V V extension acceleration calculation, which uses a single-chip microcomputer (MCU) based on RISC-V V extension as a core processing unit, integrates an ambient light sensor, a radar human body sensing sensor, a camera and an LED lighting unit, realizes multi-modal fusion detection and intelligent light control; the traditional frame difference method uses a pixel-by-pixel traversal method for difference calculation, which has large amount of calculation and low processing efficiency, the application uses RISC-V V extension vectorization instructions to process the gray scale values of multiple pixel points in parallel, so that the frame difference calculation can operate on multiple pixel points in a single instruction cycle, greatly improving the calculation efficiency; the vector threshold comparison instruction is used to directly perform binaryzation processing, reducing the overhead of multiple memory accesses and improving the detection real-time performance; the vector morphological filtering instruction is combined to suppress noise, so that the motion detection is more accurate.

[0006] Term explanation: 1、RISC-V MCU: Microcontroller Unit (MCU) based on RISC-V Instruction Set Architecture (ISA), a small computer that integrates CPU, memory (SRAM, Flash) and peripherals (such as GPIO, UART, SPI, ADC, etc.), commonly used in embedded systems such as smart home, IoT devices, home appliance control, sensor systems, etc.

[0007] 2、MOSFET: Metal Oxide Semiconductor Field Effect Transistor, an electronic switch device that can be controlled by voltage to turn on and off.

[0008] 3、SIMD (Single Instruction, Multiple Data): a parallel computing architecture that performs the same operation on multiple data simultaneously in one clock cycle.

[0009] 4、RISC-V V extension instruction (RISC-V vector extension instruction): RVV instruction, a set of hardware instructions in the RISC-V instruction set architecture for implementing SIMD (Single Instruction Multiple Data) parallel operation. It includes a large number of instructions such as vadd, vsub, vle8, vse8, vmin, vmax, vslideup, etc., which can perform the same operation on multiple data in parallel, used to accelerate image processing, matrix operation, machine learning and other workloads. These instructions are machine instructions decoded and executed directly by the CPU, existing at the assembly level, just like add, lw and other normal instructions.

[0010] 5、RVV intrinsic (built-in function): a special function provided in programming languages (such as C) to directly call the function of the underlying RVV instruction in the source code without writing assembly code. These functions are usually provided by the compiler, such as the GCC for RISC-V which provides a large number of RVV intrinsics defined in <riscv_vector.h>. The compiler will compile the instructions mentioned above at compile time.

[0011] The technical solutions of the present application are as follows: The first aspect of the present application provides a low-power intelligent lighting control method based on RISC-V V extension accelerated computing, comprising: The ambient light sensor monitors the ambient light intensity in real time, and the human motion sensor detects the human activity state, If the current ambient light intensity is lower than the set threshold and human body activity is detected, the controller (RISC-V MCU) controls the switch (MOSFET) to turn on the LED lamp; After turning on the LED lamp, the monitoring device collects video frames and analyzes the motion situation, including: S1, using RISC-V V extension instructions to preprocess the video frame, converting the video frame into a grayscale image; S2, using RISC-V V extension instructions to calculate the difference image of the grayscale image; S3, using RISC-V V extension instructions to binarize the difference image to distinguish static background and moving foreground; S4, using RISC-V V extension instructions to perform morphological processing on the binary image to obtain a morphologically processed binary image; S5, judging whether there is object motion; According to the analysis result, it is judged whether there is moving object, if the monitoring device detects no human motion, the controller (RISC-V MCU) turns off the LED lamp to reduce the power consumption caused by long-time lighting, otherwise the lighting is maintained until no moving object is detected.

[0012] According to the application, the video frame is preprocessed using RISC-V V extension instructions to convert the video frame into a grayscale image, including: After the monitoring device is turned on, video frames are collected at a fixed frequency, and the collected video frames are RGB color images, and the collection frequency should not be too high, and should not exceed the processing range of the single-chip microcomputer (MCU); Color image (RGB): Each pixel has three channels: red (R), green (G), and blue (B), each channel is generally 8 bits, and the total is 24 bits; Grayscale image: Each pixel has only one value representing brightness, which is usually 8 bits (0-255), 0 is black, and 255 is white; First, the vlse8_v_u8m1 instruction is used to load the RGB three-channel vector data in byte mode at a fixed interval; since the image pixel value is represented by an 8-bit unsigned integer, to avoid overflow in multiplication, the vwaddu_vx_u16m2 instruction is used to expand the vector data into a 16-bit vector; Then, the vmul_vx_u16m2 instruction is used to multiply the expanded R, G, and B three components by the weight coefficient (77, 150, 29) to approximately realize weighted grayscale conversion; and the vadd_vv_u16m2 instruction is used to perform element-by-element summation to obtain the weighted grayscale value; Finally, the 16-bit result is right-shifted by the vnsrl_wx_u8m1_u16m2 instruction and truncated to 8-bit grayscale image data; The vlse8_v_u8m1 instruction represents loading an 8-bit unsigned integer vector by intervals; a plurality of non-continuous 8-bit unsigned data is loaded by setting a stride, which is used to load a plurality of rows of data in a two-dimensional image; the vlse8_v_u8m1 instruction extracts pixel values of R, G and B channels in original RGB image data respectively, so as to realize vectorized reading by channel unpacking; The vwaddu_vx_u16m2 instruction expands the loaded 8-bit unsigned integer vector into a 16-bit unsigned integer vector; this operation avoids numerical overflow that may occur in subsequent multiplication operations, and at the same time provides higher calculation accuracy, and is suitable for preprocessing of values of various channels in a grayscale weighting process; The vmul_vx_u16m2 instruction realizes multiplication between a 16-bit unsigned integer vector and a constant; the expanded R, G and B vectors are respectively multiplied by integer weights (77, 150 and 29) to approximately replace floating-point weights (0.299, 0.587 and 0.114) for weighted grayscale calculation; The vadd_vv_u16m2 instruction realizes element-wise addition between two 16-bit unsigned integer vectors; the weighted R, G and B channel results are summed to obtain an intermediate grayscale value of each pixel point; The vnsrl_wx_u8m1_u16m2 instruction right-shifts each element in the 16-bit vector by a plurality of bits (for example, 8 bits) and truncates the 8-bit unsigned integer data output; the weighted sum result is normalized to a standard 8-bit grayscale image format, so as to be stored and processed subsequently.

[0013] Further preferably, the entire conversion process of converting into a grayscale image processes a plurality of pixels by a vectorized instruction at a time, which greatly improves data channel utilization compared with a scalar implementation, and is suitable for image preprocessing acceleration in a resource-limited scenario; because integer calculation is more convenient, a grayscale image is obtained, as follows: Y = ( 77 * R + 150 * G + 29 * B )>>8; Wherein, Y represents a value corresponding to the grayscale image, and R, G and B represent pixel data of the RGB image. According to the present application, a difference image of a grayscale image is calculated using a RISC-V V extension instruction; including: The vsub_vv_u8m1 instruction is used to calculate the difference vectors of gray1-gray2 and gray2-gray1 in two directions, wherein gray1 and gray2 are values of two adjacent frames of gray-scale images; then, the vmax_vv_u8m1 instruction is used to perform element-by-element maximum value operation on the two difference vectors, so as to obtain the absolute value of the difference between the corresponding pixel points of the two frames of images; finally, the vse8_v_u8m1 instruction is used to write the difference vector back to the memory as the input of the subsequent image processing step of the frame difference method, as follows: Diff(x,y)=∣gray1(x,y)−gray2(x,y)∣; wherein Diff(x,y) represents the value of the obtained difference image, and gray1(x,y) and gray2(x,y) represent the values of the horizontal coordinate x and the vertical coordinate y of the two gray-scale images; The vsub_vv_u8m1 instruction represents an element-by-element subtraction instruction between 8-bit unsigned vectors, and the vsub_vv_u8m1 is one of the basic arithmetic operations in the RISC-V vector extension, which subtracts corresponding elements in two 8-bit unsigned integer vectors and saves the result as a vector of the same type; the vsub_vv_u8m1 instruction calculates the pixel-by-pixel difference of the gray-scale values of two frames of images, that is, respectively performs gray1-gray2 and gray2-gray1, to prepare for subsequent absolute value calculation of the pixel difference; The vmax_vv_u8m1 instruction represents an element-by-element maximum value instruction of two 8-bit unsigned vectors, which compares each pair of elements of two same type unsigned vectors and returns the larger one; the vmax_vv_u8m1 instruction calculates a-b and b-a respectively, and then uses vmax_vv_u8m1 to take the maximum value, which equivalently implements the pixel-by-pixel abs(a-b) operation, thereby completing the core part of the image frame difference calculation, and Abs is the absolute value; The vse8_v_u8m1 instruction represents an instruction for storing an 8-bit unsigned vector into the memory, which writes the 8-bit unsigned integer data in the vector register into the specified memory address in sequence, and is the most basic one in the vector storage instruction; that is, the finally obtained frame difference vector is written into the output buffer to provide data support for subsequent binarization processing and motion detection judgment.

[0014] According to the application, the difference image is binarized using the RISC-V V extension instruction to distinguish the static background and the moving foreground; including: The vle8_v_u8m1 instruction is used to read the gray-scale values of each continuous pixel from the difference image; The gray value of each pixel is compared with a set threshold (for example, 45) by the vmsgtu_vx_u8m1_b8 instruction, if greater than the set threshold, it is considered that the pixel here has obvious change, and the pixel here is marked as having change; otherwise, it is considered as background unchanged; Then, a vector with all elements being 255 is generated by using the vmv_v_x_u8m1 instruction, representing a white region, and then the vmerge_vvm_u8m1 instruction is used to set the pixel position marked as having change to 255 and set other positions to 0, so as to obtain a binary image with only black and white colors; finally, the vse8_v_u8m1 instruction is used to save the binary image to the memory for subsequent use. The vector-based binarization method adopted by the application avoids the branch jump operation of traditional pixel judgment and conditional assignment, has advantages of regular logical structure, strong instruction parallelism, and suitability for SIMD optimization, and is especially suitable for large-scale real-time processing of images under the RISC-V vector extension architecture. The vle8_v_u8m1 instruction represents loading an 8-bit unsigned vector, sequentially loading a segment of 8-bit unsigned integer data from the memory into the vector register, that is, loading the gray value of each pixel in the difference image for subsequent comparison. The vmsgtu_vx_u8m1_b8 instruction represents element-by-element comparison to determine whether it is greater than (unsigned), and compares each vector element with a constant value, if the element is greater than the constant, it returns true, otherwise false, and the result is a mask vector (mask) of a boolean type; the gray value of each pixel in the difference image is compared with a set threshold (for example, 45) to determine which pixels belong to the obvious change region, and a boolean flag is generated. The vmv_v_x_u8m1 instruction represents copying a constant as a vector, and an 8-bit unsigned constant (for example, 255) is copied to all positions in the entire vector to construct a constant value vector; a vector with all elements being 255 is generated and used as the brightness value (white color) of the change region in the binary image. The vmerge_vvm_u8m1 instruction represents selecting and merging results according to a boolean vector, and according to the boolean mask (mask), the values at corresponding positions in the two vectors are selected and merged, the mask is true, the value of the first vector is used, otherwise the value of the second vector is used; the pixel position marked as having change in the mask is set to 255, and other positions are set to 0, to realize the black and white binarization of the image.

[0015] According to the application, the RISC-V V extension instruction is used to perform morphological processing on the binary image to obtain a binary image after morphological processing. The morphological processing includes an expansion operation and an erosion operation; In order to further remove some small white spots or edges in the image and improve the definition and contour continuity of the overall image, the application adopts an image erosion method based on neighborhood judgment; the processing rules of the erosion operation include: at each pixel point, a small area (for example, a 3*3 pixel range with the pixel point as the center) around the pixel point is checked, if all the pixels in this area are white, that is, the pixel value is 255, the center pixel is kept white; if there is any pixel in this area that is black, that is, the pixel value is 0, the center pixel is changed to black; in this way, isolated white points, small white areas and intermittent edges in the image can be effectively removed, so that the main area in the image is more compact and the edge is smoother; erosion is usually an important step after binary image processing, which can provide a more clean image basis for subsequent target extraction and judgment; In the application, in order to realize the erosion processing of the binary image, a sliding window method is adopted to combine RISC-V vector extension instructions for parallel acceleration; the expansion operation includes: using the vlse8_v_u8m1 instruction, reading the pixel data of multiple adjacent positions in the binary image by row, loading the data of the current row and the upper and lower two rows in the binary image respectively, for constructing a 3*3 neighborhood window; using the vslideup_vx_u8m1 and vslidedown_vx_u8m1 instructions, sliding the pixel vector in the same row to obtain the values of the left and right adjacent positions of each pixel in the row; by combining the sliding results of the row direction (up and down) and the column direction (left and right), 9 pixel values in the 3*3 neighborhood corresponding to each pixel are constructed; Using the vmin_vv_u8m1 instruction, the minimum value of the 9 pixel values is calculated one by one, that is, whether there is a pixel value of 0 in the neighborhood is judged; if there is, the minimum value is 0, and the center pixel value will be set to 0, indicating that there is a black pixel around it; when the 9 pixel values are all 255, the minimum value is 255, and the center pixel remains white; After each pixel is processed, the image after the erosion operation is obtained, and the vse8_v_u8m1 instruction is used to store the image result after the erosion operation; After the erosion operation is completed, in order to restore the white areas that are excessively reduced in the image and enhance the continuity of the object edge, the application further performs an expansion operation on the binary image; the rule of the expansion operation is: taking each pixel as the center, a small area (for example, a 3*3 neighborhood) around it is checked; if at least one pixel in this area is white (value 255), the center pixel is also set to white; only when all the surrounding pixels are black (value 0), the center pixel is kept black; Through this processing mode, the white area in the image becomes thicker and more coherent, and small gaps and discontinuities can be filled, playing a role of "expanding" the white area and connecting the structure, and is often used for strengthening or shape recovery of the target area; The processing is performed in a sliding neighborhood plus maximum value comparison mode, including: using a vlse8_v_u8m1 instruction to load multiple pixel data from the current row and the two rows above and below the binary image as input areas of the dilation operation; then, using vslideup_vx_u8m1 and vslidedown_vx_u8m1 instructions to obtain the left and right adjacent positions of each pixel in the same row, thereby constructing a complete 3*3 neighborhood window; Using a vmax_vv_u8m1 instruction, element-wise maximum value operations are performed on the 9 pixel values in sequence, that is, it is judged whether there is a position with a pixel value of 255 in the neighborhood; if there is, the center position pixel value is set to 255; if the entire neighborhood is black, that is, the pixel value is 0, the center position pixel value remains black; After the processing of each pixel is completed, the image after the dilation operation is obtained, and a vse8_v_u8m1 instruction is used to store the image result after the dilation operation, thereby completing the image structure recovery process. Through the cooperation of the above instructions, the vectorization acceleration of image dilation is realized without introducing any branch statements, the processing efficiency is significantly improved, and the method is especially suitable for structure recovery and region connection operations in the image preprocessing stage. The vlse8_v_u8m1 instruction loads the data of the row where the current pixel is located and the two rows above and below it from the image at one time, and constructs the 3*3 neighborhood row direction data required for the erosion operation. The vslideup_vx_u8m1 instruction represents a vector element sliding up, sliding the elements in the vector to the high bit direction (left shift), and filling the vacated positions with 0; the pixels are aligned to the left to obtain the value of the left neighbor of each pixel, helping to build the neighborhood window of the current row or adjacent row.

[0016] The vslidedown_vx_u8m1 instruction represents a vector element sliding down, sliding the elements in the vector to the low bit direction (right shift), and filling the vacated positions with 0; the pixels are aligned to the right to obtain the value of the right neighbor of each pixel, which is also used to build the horizontal neighborhood of the current row. The vmin_vv_u8m1 instruction represents an element-wise minimum value operation of two 8-bit unsigned vectors, compares the values at corresponding positions in the two vectors, and selects the smaller one; the minimum value comparison is sequentially performed on 9 pixel values constituting a 3*3 neighborhood, and finally the minimum pixel value in the entire neighborhood is obtained, realizing the erosion judgment logic: as long as there is a black point (value 0) in the neighborhood, the result is 0; The vmax_vv_u8m1 instruction compares each pixel in the 3*3 neighborhood one by one, selects the maximum value; as long as one pixel is white (255), the center pixel will be set to white, realizing the dilation operation logic; The above instructions cooperate under the RVV architecture to build and process the local neighborhood of each pixel in the image, thereby realizing a parallel erosion processing flow without using complex branch judgment, having the advantages of high efficiency, simple structure, and being suitable for embedded systems; According to the application, it is judged whether there is object motion; comprising: By counting the number of white pixels in the image, i.e., the pixel value is 255, it is judged whether there is a significant motion area; all pixels in the morphologically processed binary image are traversed, the vmseq_vx_u8m1_b8 instruction is used to judge whether each pixel value is equal to 255, and a corresponding Boolean vector mask is generated, if the pixel value is equal to 255, it is considered that the position of the pixel has changed, then the vpopc_m_b8 instruction is used to count the number of pixels whose value is 255, and the cumulative count is accumulated; if the pixel value is 0, it is skipped; by continuously accumulating the count result, the total number of pixels with a value of 255 in the morphologically processed binary image is obtained; If the total number of pixels with a value of 255 exceeds a set threshold (such as 100 points), it is determined that there is a moving target in the current video frame; otherwise, it is determined that the image is in a static state. This method has the advantages of simple logic, easy implementation, and strong adaptability, and is suitable for deployment in resource-limited embedded devices for real-time detection; The vmseq_vx_u8m1_b8 instruction represents an element-wise judgment of whether it is equal to a certain value, judges whether each pixel is equal to 255, and generates a Boolean mask for subsequent statistics; The vpopc_m_b8 instruction represents counting the number of true elements in a Boolean vector, counting how many pixel values equal to 255 in the current block; Through the above judgment method, the application realizes efficient and controllable motion detection function without relying on an image recognition model, and by virtue of the parallelism of the RISC-V vector extension, realizes the acceleration of the whole flow from reading to judging in image analysis, and is suitable for low-power and low-latency visual front-end devices.

[0017] The application has the advantages that: 1. Reduce power consumption: The present application only turns on the light when necessary, and improves energy efficiency through RISC-V V extension low-power computing. The frame difference method calculation uses vectorized operation to reduce calculation time and reduce MCU power consumption.

[0018] 2. Improve detection accuracy: The present application combines radar sensing and camera data to improve the accuracy of human body detection, avoid false triggering or false closing of the light. Vector morphological filtering improves detection stability and reduces false detection.

[0019] 3. Reduce hardware cost: The present application uses a low-power MCU with RISC-V V extension to replace expensive high-performance processors, reducing cost and improving system performance. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 The sensor of the present application is connected to the MCU schematic diagram; Figure 2 The light-on logic flowchart of the present application; Figure 3 The calculation flowchart of the frame difference method of the present application. DETAILED DESCRIPTION

[0021] The present application will be further described below by way of examples and in conjunction with the accompanying drawings, but is not limited thereto.

[0022] Example 1 A low-power intelligent lighting control method based on RISC-V V extension accelerated calculation, as shown in Figures 1-2 includes: The ambient light sensor monitors the ambient light intensity in real time, and the human motion sensor detects the human activity state, If the current ambient light intensity is lower than the set threshold and human activity is detected, the controller (RISC-V MCU) controls the switch (MOSFET) to turn on the LED lamp; After turning on the LED lamp, the monitoring device collects video frames and analyzes the motion; as Figure 3 includes: S1, using RISC-V V extension instructions to preprocess the video frame, converting the video frame to a grayscale image; S2, using RISC-V V extension instructions to calculate the difference image of the grayscale image; S3, using RISC-V V extension instructions to binarize the difference image, distinguishing static background and moving foreground; S4, using RISC-V V extension instructions to perform morphological processing on the binary image to obtain the morphologically processed binary image; S5, judging whether there is an object moving; According to the analysis result, it is judged whether there is a moving object. If the monitoring device detects no human movement, the controller (RISC-V MCU) turns off the LED light, reducing power consumption caused by long-time lighting. Otherwise, the lighting is continuously maintained until no moving object is detected.

[0023] Embodiment 2 The low-power intelligent lighting control method based on RISC-V V extension accelerated computing according to Embodiment 1 is different in that: The RISC-V V extension instruction is used to pre-process the video frame and convert the video frame into a grayscale image; including: After the monitoring device is turned on, video frames are collected at a fixed frequency. The collected video frames are RGB color images, and the collection frequency should not be too high, which should not exceed the processing range of the MCU (MCU); Color image (RGB): Each pixel has three channels: red (R), green (G), and blue (B), each channel is generally 8 bits, and the total is 24 bits; Grayscale image: Each pixel has only one value, representing brightness, which is usually 8 bits (0-255), 0 is black, and 255 is white; First, the vlse8_v_u8m1 instruction is used to load the RGB three-channel vector data in byte mode at a fixed interval; since the image pixel value is represented by an 8-bit unsigned integer, to avoid overflow in multiplication, the vwaddu_vx_u16m2 instruction is used to expand the vector data to a 16-bit vector; Then, the vmul_vx_u16m2 instruction is used to multiply the expanded R, G, and B three components by the weight coefficient (77, 150, 29) to approximately realize weighted grayscale conversion; and the vadd_vv_u16m2 instruction is used to perform element-wise summation to obtain the weighted grayscale value; Finally, the vnsrl_wx_u8m1_u16m2 instruction is used to right shift the 16-bit result by bits and truncate it to 8-bit grayscale image data; The vlse8_v_u8m1 instruction represents loading an 8-bit unsigned integer vector by interval; by setting the stride, multiple non-continuous 8-bit unsigned data are loaded, which is used to load multiple rows of data in a two-dimensional image; the vlse8_v_u8m1 instruction extracts the pixel values of the R, G, and B three channels in the original RGB image data, realizing vectorized reading by channel unpacking; The vwaddu_vx_u16m2 instruction extends the loaded 8-bit unsigned integer vector into a 16-bit unsigned integer vector; the operation avoids numerical overflow that may occur in subsequent multiplication operations, while providing higher calculation accuracy, and is suitable for preprocessing of channel values in a gray weighted process; The vmul_vx_u16m2 instruction is used to implement multiplication between a 16-bit unsigned integer vector and a constant; by multiplying the extended R, G, and B vectors with integer weights (77, 150, and 29) respectively, the weighted gray calculation is approximated by using floating-point weights (0.299, 0.587, and 0.114); The vadd_vv_u16m2 instruction implements element-wise addition between two 16-bit unsigned integer vectors; the weighted R, G, and B channel results are summed to obtain the intermediate gray value of each pixel point; The vnsrl_wx_u8m1_u16m2 instruction right shifts each element in the 16-bit vector by a number of bits (for example, 8 bits) and truncates it to 8-bit unsigned integer data output; the weighted sum result is normalized to the standard 8-bit gray image format for subsequent storage and processing.

[0024] The entire conversion process of converting to a gray image processes multiple pixels at a time through vectorization instructions, which greatly improves the data path utilization compared to scalar implementation, and is suitable for image preprocessing acceleration in resource-constrained scenarios; because integer calculation is more convenient, the gray image is obtained as follows: Y = ( 77 * R + 150 * G + 29 * B )>>8; where Y represents the value corresponding to the gray image, and R, G, and B represent the pixel data of the RGB image; According to the present application, the difference image of the gray image is calculated using the RISC-V V extension instruction; including: The vsub_vv_u8m1 instruction is used to calculate gray1-gray2 and gray2-gray1, and the difference value vectors in two directions are obtained, where gray1 and gray2 are the values of the adjacent two frames of gray images; then, the vmax_vv_u8m1 instruction is used to perform element-wise maximum value operation on the two difference value vectors, so as to obtain the absolute value of the difference between the pixel points corresponding to the two images; finally, the vse8_v_u8m1 instruction is used to write the difference value vector back to the memory as the input of the subsequent image processing step of the frame difference method, as follows: Diff(x,y)=∣gray1(x,y)−gray2(x,y)∣; wherein Diff(x, y) represents the value of the obtained difference image, and gray1(x, y) and gray2(x, y) represent the values at the horizontal coordinate x and the vertical coordinate y of the two gray-scale images.

[0025] The vsub_vv_u8m1 instruction represents an element-wise subtraction instruction between 8-bit unsigned vectors, and the vsub_vv_u8m1 is one of the basic arithmetic operations in the RISC-V vector extension, which subtracts the corresponding elements in the two 8-bit unsigned integer vectors and saves the result as a vector of the same type; the vsub_vv_u8m1 instruction calculates the pixel-by-pixel difference value of the gray-scale values of two image frames, i.e., respectively performs gray1-gray2 and gray2-gray1, to prepare for the subsequent calculation of the absolute value of the pixel difference value; The vsub_vv_u8m1 instruction represents an element-wise subtraction instruction between 8-bit unsigned vectors, and the vsub_vv_u8m1 is one of the basic arithmetic operations in the RISC-V vector extension, which subtracts the corresponding elements in the two 8-bit unsigned integer vectors and saves the result as a vector of the same type; the vsub_vv_u8m1 instruction calculates the pixel-by-pixel difference value of the gray-scale values of two image frames, i.e., respectively performs gray1-gray2 and gray2-gray1, to prepare for the subsequent calculation of the absolute value of the pixel difference value; The vse8_v_u8m1 instruction represents an instruction for storing an 8-bit unsigned vector into memory, which writes the 8-bit unsigned integer data in the vector register back to the specified memory address in sequence, and is the most basic one among the vector storage instructions; that is, the finally obtained frame difference vector is written into the output buffer to provide data support for subsequent binarization processing and motion detection judgment.

[0026] The difference image is binarized using the RISC-V V extension instruction to distinguish between static background and moving foreground; including: The vle8_v_u8m1 instruction is used to read the gray-scale values of each pixel in sequence from the difference image; The vmsgtu_vx_u8m1_b8 instruction is used to compare the gray-scale value of each pixel with a set threshold (for example, 45), and if it is greater than the set threshold, it is considered that the pixel at this position has obvious changes and is marked as having changes; otherwise, it is considered as background unchanged; Then, the vmv_v_x_u8m1 instruction is used to generate a vector with all elements being 255, representing a white area, and the vmerge_vvm_u8m1 instruction is used to assign 255 to the pixel positions marked as having changes and 0 to other positions, to obtain a binary image with only black and white colors; finally, the vse8_v_u8m1 instruction is used to save the binary image to the memory for subsequent use; The vector quantization and binarization method adopted by the present application avoids the branch jump operation of traditional pixel judgment and conditional assignment, has advantages such as regular logical structure, strong instruction parallelism, and suitability for SIMD optimization, and is especially suitable for large-scale real-time processing of images under the RISC-V vector expansion architecture; The vle8_v_u8m1 instruction represents loading an 8-bit unsigned vector, sequentially loading a segment of 8-bit unsigned integer data from memory into a vector register, that is, loading the gray value of each pixel in the difference image, preparing for subsequent comparison; The vmsgtu_vx_u8m1_b8 instruction represents element-by-element comparison to determine whether it is greater than (unsigned), compares each vector element with a constant value, and returns true if the element is greater than the constant, otherwise false, and the result is a mask vector (mask) of boolean type; compare the gray value of each pixel in the difference image with the set threshold (such as 45) to determine which pixels belong to the obvious change area and generate a boolean label; The vmv_v_x_u8m1 instruction represents copying a constant as a vector, copying an 8-bit unsigned constant (such as 255) to all positions in the entire vector to construct a constant value vector; generate a vector of all 255, which is used as the brightness value (white) of the change area in the binary image; The vmerge_vvm_u8m1 instruction represents selecting and merging results according to the boolean vector, selecting and merging values at corresponding positions between two vectors according to the boolean mask (mask), using the value of the first vector at the position marked as true in the mask, otherwise using the value of the second vector; set the pixel position marked as changed in the mask to 255, and set other positions to 0, to realize the black and white binarization of the image.

[0027] The RISC-V V expansion instruction is used for morphological processing of the binary image to obtain a morphologically processed binary image; including: The morphological processing includes dilation operation and erosion operation; In order to further remove some small white spots or burrs in the image and improve the clarity and contour continuity of the overall image, the application adopts an image erosion method based on neighborhood judgment; the processing rules of the erosion operation include: at each pixel point, a small area (for example, a 3*3 pixel range with the pixel point as the center) around it is checked, if all the pixels in this area are white, that is, the pixel value is 255, the center point pixel is kept white; if there is any pixel in this area is black, that is, the pixel value is 0, the center pixel point is changed to black; in this way, isolated white points, small white areas and intermittent edges in the image can be effectively removed, so that the main area in the image is more compact and the edge is smoother; erosion is usually an important step after binary image processing, which can provide a more clean image basis for subsequent target extraction and judgment; In the application, in order to realize the erosion processing of the binary image, a sliding window mode is adopted to combine RISC-V vector extension instructions for parallel acceleration; the expansion operation includes: using the vlse8_v_u8m1 instruction, reading the pixel data of multiple adjacent positions in the binary image by row, loading the data of the current row and the upper and lower two rows in the binary image respectively, for constructing a 3*3 neighborhood window; using the vslideup_vx_u8m1 and vslidedown_vx_u8m1 instructions, sliding the pixel vector in the same row to obtain the values of the left and right adjacent positions of each pixel in the row; by combining the sliding results of the row direction (up and down) and the column direction (left and right), 9 pixel values in the 3*3 neighborhood corresponding to each pixel are constructed; Using the vmin_vv_u8m1 instruction to find the minimum value of the 9 pixel values one by one, that is, to judge whether there is a pixel value of 0 in the neighborhood; if there is, the minimum value is 0, and the center point pixel value will be set to 0, indicating that there is a black pixel around it; when the 9 pixel values are all 255, the minimum value is 255, and the center point pixel remains white; After each pixel is processed, the image after the erosion operation is obtained, and the vse8_v_u8m1 instruction is used to store the image result after the erosion operation; After the erosion operation is completed, in order to restore the white area that is excessively reduced in the image and enhance the continuity of the object edge, the application further performs an expansion operation on the binary image, and the rule of the expansion operation is: taking each pixel as the center, checking a small area (for example, a 3*3 neighborhood) around it; if at least one pixel in this area is white (value 255), the center pixel is also set to white; only when all the surrounding pixels are black (value 0), the center pixel is kept black; Through the processing mode, the white area in the image becomes thicker and more coherent, and small gaps and discontinuities can be filled, which plays a role of "expanding" the white area and connecting the structure, and is often used for strengthening or shape recovery of the target area; The processing is performed in a sliding neighborhood plus maximum value comparison mode, including: using a vlse8_v_u8m1 instruction to load a plurality of pixel data from a current row and two rows above and below the binary image as input areas of the dilation operation; then, using vslideup_vx_u8m1 and vslidedown_vx_u8m1 instructions to obtain the left and right adjacent positions of each pixel in the same row, thereby constructing a complete 3*3 neighborhood window; Using a vmax_vv_u8m1 instruction, element-wise maximum value operations are sequentially performed on the 9 pixel values, that is, whether there is a position with a pixel value of 255 in the neighborhood is judged; if there is, the center position pixel value is set to 255; if the entire neighborhood is black, that is, the pixel value is 0, the center position pixel value remains black; After the processing of each pixel is completed, the image after the dilation operation is obtained, and a vse8_v_u8m1 instruction is used to store the image result after the dilation operation, thereby completing the image structure recovery process. Through the cooperation of the above instructions, the vectorization acceleration of image dilation is realized without introducing any branch statements, the processing efficiency is significantly improved, and the method is especially suitable for structure recovery and region connection operations in the image preprocessing stage. The vlse8_v_u8m1 instruction loads the data of the row where the current pixel is located and the two rows above and below it from the image at one time, and constructs the 3*3 neighborhood row direction data required for the erosion operation. The vslideup_vx_u8m1 instruction represents a vector element sliding up, sliding the elements in the vector to the high bit direction (left shift), and filling the positions vacated by 0; the pixels are aligned to the left to obtain the values of the left neighbors of each pixel, helping to build the neighborhood window of the current row or adjacent row.

[0028] The vslidedown_vx_u8m1 instruction represents a vector element sliding down, sliding the elements in the vector to the low bit direction (right shift), and filling the positions vacated by 0; the pixels are aligned to the right to obtain the values of the right neighbors of each pixel, which is also used to build the horizontal neighborhood of the current row. The vmin_vv_u8m1 instruction represents an element-wise minimum value operation of two 8-bit unsigned vectors, compares the values at corresponding positions in the two vectors, and selects the smaller one; the minimum value comparison is sequentially performed on the 9 pixel values constituting a 3x3 neighborhood, and finally the minimum pixel value in the entire neighborhood is obtained, realizing the erosion judgment logic: as long as there is a black point (value 0) in the neighborhood, the result is 0; The vmax_vv_u8m1 instruction compares each pixel in the 3x3 neighborhood one by one, selects the maximum value; as long as one pixel is white (255), the center pixel will be set to white, realizing the dilation operation logic; The above instructions cooperate under the RVV architecture to build and process the local neighborhood of each pixel in the image, thereby realizing a parallel erosion processing flow without using complex branch judgment, having the advantages of high efficiency, simple structure, and being suitable for embedded systems; Judging whether there is object motion; including: By counting the number of white pixels in the image, i.e., the pixel value is 255, it is judged whether there is a significant motion area; all pixels in the morphological processed binary image are traversed, the vmseq_vx_u8m1_b8 instruction is used to judge whether each pixel value is equal to 255, and a corresponding Boolean vector mask is generated, if the pixel value is equal to 255, it is considered that the position of the pixel has changed, then the vpopc_m_b8 instruction is used to count the number of pixels whose value is 255, and the cumulative count is accumulated; if the pixel value is 0, it is skipped; by continuously accumulating the count result, the total number of pixels with value 255 in the morphologically processed binary image is obtained; If the total number of pixels with value 255 exceeds a certain threshold (such as 100 points), it is determined that there is a moving target in the current video frame; otherwise, it is determined that the image is in a static state. This method has the advantages of simple logic, easy implementation, and strong adaptability, and is suitable for deployment in resource-limited embedded devices for real-time detection; The vmseq_vx_u8m1_b8 instruction represents an element-wise judgment of whether it is equal to a certain value, judges whether each pixel is equal to 255, and generates a Boolean mask for subsequent statistics; The vpopc_m_b8 instruction represents counting the number of true elements in the Boolean vector, counting how many pixel values equal to 255 in the current block; Through the above judgment method, the present application realizes efficient and controllable motion detection function without relying on image recognition model, and by virtue of the parallelism of RISC-V vector extension, realizes the whole process acceleration of image analysis from reading to judgment, and is suitable for low-power and low-latency visual front-end devices.

Claims

1. A low-power intelligent lighting control method based on RISC-V V extension accelerated computing, characterized in that, The application relates to an intelligent LED light control method based on RISC-V V extension instructions. The environmental light sensor monitors the environmental light intensity in real time, and the human motion sensor detects the human activity state, If the current environmental light intensity is lower than the set threshold value and human activity is detected, the controller controls the switch to turn on the LED lamp; After the LED lamp is turned on, the monitoring device collects video frames and analyzes the motion situation, which comprises the following steps: S1, using the RISC-V V extension instruction to preprocess the video frame, converting the video frame into a grayscale image; S2, using the RISC-V V extension instruction to calculate the difference image of the grayscale image; S3, using the RISC-V V extension instruction to binarize the difference image; S4, using the RISC-V V extension instruction to perform morphological processing on the binary image to obtain a morphologically processed binary image; S5, judging whether there is object motion; According to the analysis result, it is judged whether there is moving object, if the monitoring device detects no human motion, the controller turns off the LED lamp, otherwise the lighting is continuously maintained until no moving object is detected.

2. The low-power intelligent lighting control method based on RISC-V V extension accelerated computing according to claim 1, characterized in that, The RISC-V V extension instruction is used to preprocess the video frame, and the video frame is converted into a grayscale image; The application relates to an intelligent LED light control method based on RISC-V V extension instructions. After the monitoring device is turned on, video frames are collected at a fixed frequency, and the collected video frames are RGB color images; Firstly, the vlse8_v_u8m1 instruction is used to load the RGB three-channel vector data in a byte mode at a fixed interval; and the vwaddu_vx_u16m2 instruction is used to expand the vector data into a 16-bit vector; Then, the vmul_vx_u16m2 instruction is used to multiply the expanded R, G and B three components by a weight coefficient to approximately realize weighted grayscale conversion; and the vadd_vv_u16m2 instruction is used for element-by-element summation to obtain the weighted grayscale value; Finally, the vnsrl_wx_u8m1_u16m2 instruction is used to right shift the 16-bit result by bits to truncate it into 8-bit grayscale image data; The vlse8_v_u8m1 instruction represents loading an 8-bit unsigned integer vector at an interval; a plurality of non-continuous 8-bit unsigned data are loaded by setting a step length, which is used for loading multiple rows of data in a two-dimensional image; the vlse8_v_u8m1 instruction extracts the pixel values of the R, G and B three channels in the original RGB image data, realizes vectorized reading according to channel unpacking, and realizes vectorized reading according to channel unpacking. The vwaddu_vx_u16m2 instruction expands the loaded 8-bit unsigned integer vector into a 16-bit unsigned integer vector; The vmul_vx_u16m2 instruction realizes the multiplication operation between the 16-bit unsigned integer vector and a constant; the expanded R, G and B vectors are multiplied by an integer weight respectively to approximately replace the floating-point weight for weighted grayscale calculation; The vadd_vv_u16m2 instruction realizes the element-by-element addition between two 16-bit unsigned integer vectors; the weighted R, G and B channel results are summed to obtain the intermediate grayscale value of each pixel point. The vnsrl_wx_u8m1_u16m2 instruction right shifts each element in a 16-bit vector by a number of bits and truncates to 8-bit unsigned integer data output; the weighted sum result is normalized to the standard 8-bit grayscale image format.

3. The low-power intelligent lighting control method based on RISC-V V extension accelerated computing according to claim 2, characterized in that, The entire conversion process of converting to a grayscale image processes multiple pixels at a time through vectorization instructions to obtain a grayscale image, as follows: Y = ( 77 * R + 150 * G + 29 * B ) >> 8; Wherein, Y represents the value corresponding to the grayscale image, R, G, B represent the pixel data of the RGB image.

4. The low-power intelligent lighting control method based on RISC-V V extension accelerated computing according to claim 3, characterized in that, Using RISC-V V extension instructions to calculate the difference image of the grayscale image; including: Using the vsub_vv_u8m1 instruction to calculate gray1-gray2 and gray2-gray1, the difference value vectors in two directions are obtained, wherein gray1 and gray2 are the values of the adjacent two frames of grayscale images; then through the vmax_vv_u8m1 instruction, the element-by-element maximum value operation is performed on the two difference value vectors, so as to obtain the absolute value of the difference between the corresponding pixel points of the two images; finally, using the vse8_v_u8m1 instruction, the difference value vector is written back to the memory, as follows: Diff(x,y)=∣gray1(x,y)−gray2(x,y)∣; Wherein, Diff(x,y) represents the value of the obtained difference image, gray1(x,y) and gray2(x,y) represent the values at the horizontal coordinate x and the vertical coordinate y of the two grayscale images; The vsub_vv_u8m1 instruction represents an element-by-element subtraction instruction between 8-bit unsigned vectors, subtracts the corresponding elements in the two 8-bit unsigned integer vectors, and saves the result as a vector of the same type; the vsub_vv_u8m1 instruction calculates the pixel-by-pixel difference of the grayscale values of the two images, that is, respectively performs gray1-gray2 and gray2-gray1; The vmax_vv_u8m1 instruction represents an element-by-element maximum value instruction of two 8-bit unsigned vectors, compares each pair of elements of the two same type unsigned vectors, and returns the larger one; The vse8_v_u8m1 instruction represents an instruction for storing an 8-bit unsigned vector into memory, which writes the 8-bit unsigned integer data in the vector register back to the specified memory address in sequence, that is, writes the finally obtained frame difference vector into the output buffer.

5. The low-power intelligent lighting control method based on RISC-V V extension accelerated computing according to claim 4, characterized in that, Using RISC-V V extension instructions to binarize the difference image; including: Using the vle8_v_u8m1 instruction to read the grayscale value of each pixel in sequence from the difference image; Through the vmsgtu_vx_u8m1_b8 instruction, the grayscale value of each pixel is compared with the set threshold value, if it is greater than the set threshold value, it is considered that the pixel at this position has obvious change, and this position is marked as changed; otherwise, it is considered as background unchanged; Then, a vector with all elements being 255, representing the white region, is generated using the vmv_v_x_u8m1 instruction, and the pixel positions marked as changed are assigned a value of 255 and the other positions are assigned a value of 0 using the vmerge_vvm_u8m1 instruction, to obtain a binary image with only black and white colors; finally, the binary image is saved to the memory using the vse8_v_u8m1 instruction; The vle8_v_u8m1 instruction represents loading an 8-bit unsigned vector, sequentially loading a segment of 8-bit unsigned integer data from the memory into the vector register, that is, loading the gray value of each pixel from the difference image; The vmsgtu_vx_u8m1_b8 instruction represents element-wise comparison to determine whether it is greater than, comparing each vector element with a constant value, and returning true if the element is greater than the constant, otherwise false, and the result is a boolean mask vector; compare the gray value of each pixel in the difference image with the set threshold to determine which pixels belong to the obvious change region and generate a boolean marker; The vmv_v_x_u8m1 instruction represents copying a constant to a vector, copying an 8-bit unsigned constant to all positions in the vector to construct a constant value vector; generate a vector with all elements being 255, which is used as the brightness value of the change region in the binary image; The vmerge_vvm_u8m1 instruction represents selecting and merging results according to the boolean vector, selecting and merging the values at corresponding positions between two vectors according to the boolean mask, using the value of the first vector at the position where the mask is true, and using the value of the second vector otherwise; set the pixel positions marked as changed in the mask to 255 and set the other positions to 0.

6. The low-power intelligent lighting control method based on RISC-V V extension accelerated computing according to claim 5, characterized in that, Morphological processing is performed on the binary image using RISC-V V extension instructions to obtain a morphologically processed binary image; The morphological processing includes dilation and erosion operations. The dilation operation includes: using the vlse8_v_u8m1 instruction to read pixel data of multiple adjacent positions in the binary image by row, loading data of the current row and the two rows above and below it respectively to construct a 3x3 neighborhood window; using the vslideup_vx_u8m1 and vslidedown_vx_u8m1 instructions to slide the pixel vector within the same row to obtain the values of the left and right adjacent positions of each pixel in the row; and combining the sliding results in the row direction and the column direction to construct 9 pixel values in the 3x3 neighborhood corresponding to each pixel; The vmin_vv_u8m1 instruction is used to find the minimum value of the 9 pixel values one by one, that is, to determine whether there is a pixel value of 0 in the neighborhood; if there is, the minimum value is 0 and the center pixel value will be set to 0, indicating that there is a black pixel around it; when all 9 pixel values are 255, the minimum value is 255 and the center pixel remains white. ​ After each pixel processing is completed, the image after the erosion operation is obtained, and the vse8_v_u8m1 instruction is used to store the image result after the erosion operation; After the erosion operation is completed, a dilation operation is performed on the binary image, including: using the vlse8_v_u8m1 instruction to load multiple pixel data from the current row and the two rows above and below the binary image as the input region of the dilation operation; then, using the vslideup_vx_u8m1 and vslidedown_vx_u8m1 instructions to obtain the left and right adjacent position values of each pixel in the same row, thereby constructing a complete 3*3 neighborhood window; The vmax_vv_u8m1 instruction is used to perform element-wise maximum value operation on the 9 pixel values in turn, that is, to determine whether there is a position with a pixel value of 255 in the neighborhood; if there is, the center position pixel value is set to 255; if the entire neighborhood is black, that is, the pixel value is 0, the center position pixel value remains black; After each pixel processing is completed, the image after the dilation operation is obtained, and the vse8_v_u8m1 instruction is used to store the image result after the dilation operation, completing the image structure recovery process; The vlse8_v_u8m1 instruction loads the data of the row where the current pixel is located and the two rows above and below it from the image at a time, constructing the 3*3 neighborhood row direction data required for the erosion operation; The vslideup_vx_u8m1 instruction represents a vector element sliding up, sliding the elements in the vector to the high bit direction, and filling 0 in the vacant position; the pixels are aligned to the left to obtain the value of the left neighbor of each pixel. The vslidedown_vx_u8m1 instruction represents a vector element sliding down, sliding the elements in the vector to the low bit direction, and filling 0 in the vacant position; the pixels are aligned to the right to obtain the value of the right neighbor of each pixel. The vmin_vv_u8m1 instruction represents element-wise minimum value operation of two 8-bit unsigned vectors, comparing the values at the corresponding positions in the two vectors and selecting the smaller one; the minimum value comparison is performed on the 9 pixel values constituting the 3*3 neighborhood in turn, and finally the minimum pixel value in the entire neighborhood is obtained, realizing the erosion judgment logic: as long as there is a black point in the neighborhood, the result is 0. The vmax_vv_u8m1 instruction compares each pixel in the 3*3 neighborhood one by one and selects the maximum value; as long as one pixel is white, the center pixel will be set to white, realizing the dilation operation logic.

7. The low-power intelligent lighting control method based on RISC-V V extension accelerated computing according to claim 6, characterized in that, Judging whether there is object motion; including: Traverse all pixels in the morphologically processed binary image, use the vmseq_vx_u8m1_b8 instruction to judge whether each pixel value is equal to 255, and generate the corresponding Boolean vector mask, if the pixel value is equal to 255, it is considered that the position of the pixel here has changed, then use the vpopc_m_b8 instruction to count the number of pixels whose value is equal to 255, and accumulate the count; if the pixel value is 0, skip; by continuously accumulating the count result, the total number of pixels with value 255 in the morphologically processed binary image is obtained; If the total number of pixels with value 255 exceeds the set threshold, it is determined that there is a moving target in the current video frame; otherwise, it is determined that the image is in a static state; The vmseq_vx_u8m1_b8 instruction means element-by-element judgment of whether it is equal to a certain value, judges whether each pixel is equal to 255, and generates a Boolean mask; The vpopc_m_b8 instruction means counting the number of true elements in the Boolean vector, counting how many pixel values in the current block are equal to 255.

Images