Image processing method for vehicle, vehicle and medium

Abstract

The invention discloses an image processing method for a vehicle, the vehicle and a computer readable storage medium, and the method comprises the steps: obtaining a vehicle cabin image, determining a plurality of target feature maps corresponding to the vehicle cabin image according to the vehicle cabin image, and obtaining a plurality of target feature maps corresponding to the vehicle cabin image according to the plurality of target feature maps. And determining a target image and attribute features of the target image. Thus, the object of the target category can be identified, and compared with a mode of marking the object of the target category in the vehicle cabin image through a candidate frame, the object of the target category is represented through the target image instead of a mode of frame selection of the candidate frame, so that the effectiveness and reliability of the target image and the attribute characteristics can be ensured, and the identification accuracy of the object of the target category is improved. And the matching of the target image and the attribute characteristics of the target image with the vehicle cabin image can be ensured, so that the recognition precision of the target image and the attribute characteristics can be further ensured.

Description

Technical Field

[0001] The present application relates to the field of vehicle technology, and in particular to an image processing method for a vehicle, a vehicle, and a computer-readable storage medium. Background Art

[0002] In related technologies, when identifying the driver and occupants in a vehicle cabin, the recognition results are often represented in the form of candidate boxes (Bounding Boxes) based on target detection. However, because the candidate boxes may contain other elements besides the driver and occupants, such as seats, when performing operations based on the recognition results, such as detecting the user's fatigue status, interference from other elements outside the driver and occupants may lead to incorrect detection of the user's fatigue status, which to some extent affects the user experience of the in-vehicle smart cockpit. Summary of the Invention

[0003] The present application provides an image processing method for a vehicle, a vehicle, and a computer-readable storage medium.

[0004] An embodiment of the present application provides an image processing method for a vehicle, comprising:

[0005] Acquire vehicle cabin images;

[0006] determining, based on the vehicle cockpit image, a plurality of target feature maps corresponding to the vehicle cockpit image, wherein the plurality of target feature maps have different dimensions;

[0007] A target image and attribute features of the target image are determined according to the multiple target feature maps, wherein the target image is used to indicate an object of a target category in the vehicle cabin image.

[0008] Thus, in an embodiment of the present application, multiple target feature maps corresponding to the vehicle cabin image are determined based on the vehicle cabin image, and based on the multiple target feature maps, a target image for indicating an object of a target category in the vehicle cabin image and the attribute features of the target image are determined, thereby achieving recognition of objects of the target category. Compared to the method of labeling objects of the target category in the vehicle cabin image with candidate boxes, since the objects of the target category are represented by the target image rather than by selection with candidate boxes, the effectiveness and reliability of the target image and the attribute features can be guaranteed. In addition, the target image and the attribute features of the target image can both be determined by multiple target feature maps of different dimensions corresponding to the vehicle cabin image, thereby ensuring that the target image and the attribute features of the target image match the vehicle cabin image, thereby further ensuring the recognition accuracy of the target image and the attribute features.

[0009] In certain embodiments of the present application, the multiple target feature maps include a first target feature map, a second target feature map, and a third target feature map, the dimensions of which decrease in sequence. Determining, based on the vehicle cockpit image, the multiple target feature maps corresponding to the vehicle cockpit image includes:

[0010] Performing residual convolution processing on the vehicle cabin image to determine the first target feature map;

[0011] Performing pooling and residual convolution processing on the first target feature map to determine the second target feature map;

[0012] Perform residual convolution processing on the second target feature map to determine the third target feature map.

[0013] In this way, in an embodiment of the present application, residual convolution processing can be performed on the vehicle cabin image to determine the first target feature map, and pooling and residual convolution processing can be performed on the first target feature map to determine the second target feature map, and residual convolution processing can be performed on the second target feature map to determine the third target feature map, thereby achieving the acquisition of the first target feature map, the second target feature map and the third target feature map.

[0014] In some embodiments of the present application, performing residual convolution processing on the second target feature map to determine the third target feature map includes:

[0015] Performing multiple residual convolution processes on the second target feature map to determine a first feature map;

[0016] Determining target attention data corresponding to the first feature map according to the first feature map;

[0017] Determine the third target feature map based on the first feature map and the target attention data.

[0018] In this way, in an embodiment of the present application, the second target feature map can be subjected to multiple residual convolution processes to determine the first feature map, and based on the first feature map, the target attention data corresponding to the first feature map can be determined, and based on the first feature map and the target attention data, the third target feature map can be determined, so that the third target feature map is determined based on the attention mechanism, thereby ensuring the validity of the third target feature map, target image and attribute features.

[0019] In certain embodiments of the present application, determining target attention data corresponding to the first feature map according to the first feature map includes:

[0020] Determining, according to the first feature map, position attention data and channel attention data corresponding to the first feature map;

[0021] The target attention data is determined based on the position attention data and the channel attention.

[0022] In this way, in an embodiment of the present application, channel attention data and position attention data can be determined based on the first feature map, and target attention data can be determined based on the channel attention data and position attention data, so that the determination of the third target feature map can be achieved based on channel attention and position attention, thereby further ensuring the accuracy of the third target feature map, target image and attribute features.

[0023] In certain embodiments of the present application, determining the third target feature map based on the first feature map and the target attention data includes:

[0024] Performing a first convolution process and a pooling process on the first feature map to determine a second feature map and a third feature map;

[0025] performing a second convolution process on the first feature map according to a predetermined expansion rate parameter to determine a fourth feature map;

[0026] Determine the third target feature map based on the second feature map, the third feature map, the fourth feature map and the target attention data.

[0027] In this way, in an embodiment of the present application, the first feature map can be subjected to a first convolution process and a pooling process respectively to determine a second feature map and a third feature map, and the first feature map can be subjected to a second convolution process according to a predetermined expansion rate parameter to determine a fourth feature map, and the third target feature map can be determined according to the second feature map, the third feature map, the fourth feature map and the target attention data, thereby expanding the receptive field and reducing the number of parameters and computational complexity to a certain extent.

[0028] In certain embodiments of the present application, the plurality of target feature maps include a first target feature map, a second target feature map, and a third target feature map of decreasing dimensions, and determining the target image and the attribute features of the target image based on the plurality of target feature maps includes:

[0029] Performing upsampling processing on the third target feature map to determine a fifth feature map;

[0030] Determining a sixth characteristic map based on the second target characteristic map and the fifth characteristic map;

[0031] The target image and the attribute features of the target image are determined according to the first target feature map and the sixth feature map.

[0032] In this way, in an embodiment of the present application, the third target feature map can be upsampled to determine the fifth feature map, and the sixth feature map can be determined based on the second target feature map and the fifth feature map, and the target image and the attribute characteristics of the target image can be determined based on the first target feature map and the sixth feature map, thereby achieving the determination of the target image and the attribute characteristics of the target image.

[0033] In certain embodiments of the present application, determining the sixth feature map based on the second target feature map and the fifth feature map includes:

[0034] Performing depthwise convolution and pointwise convolution on the second target feature map to determine a seventh feature map;

[0035] performing splicing processing on the fifth feature map and the seventh feature map to determine an eighth feature map;

[0036] Performing depthwise convolution, point-by-point convolution, and upsampling on the eighth feature map to determine the sixth feature map.

[0037] In this way, in an embodiment of the present application, the second target feature map can be subjected to depth convolution and point-by-point convolution processing to determine the seventh feature map, and the fifth feature map and the seventh feature map can be spliced ​​to determine the eighth feature map, and the eighth feature map can be subjected to depth convolution, point-by-point convolution processing and upsampling processing to determine the sixth feature map, thereby completing the acquisition of the sixth feature map.

[0038] In certain embodiments of the present application, determining the target image and the attribute features of the target image based on the first target feature map and the sixth feature map includes:

[0039] Performing depthwise convolution and pointwise convolution on the first target feature map to determine a ninth feature map;

[0040] performing splicing processing on the sixth feature map and the ninth feature map to determine a tenth feature map;

[0041] Performing depth convolution, point-by-point convolution processing, and upsampling processing on the tenth feature map to determine the target image and the attribute features of the target image.

[0042] In this way, in the embodiment of the present application, the first target feature map can be subjected to depth convolution and point-by-point convolution processing to determine the ninth feature map, and the sixth feature map and the ninth feature map can be spliced ​​to determine the tenth feature map, and the tenth feature map can be subjected to depth convolution, point-by-point convolution processing and upsampling processing to determine the target image and the attribute characteristics of the target image, thereby completing the determination of the target image and the attribute characteristics of the target image.

[0043] An embodiment of the present application provides a vehicle, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned image processing method for the vehicle is implemented.

[0044] An embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program is executed by one or more processors, the above-mentioned image processing method for a vehicle is implemented.

[0045] The vehicle and computer-readable storage medium provided in the embodiments of the present application can achieve recognition of objects of a target category by determining multiple target feature maps corresponding to the vehicle cabin image based on the vehicle cabin image, and determining a target image for indicating an object of a target category in the vehicle cabin image based on the multiple target feature maps, and determining the attribute features of the target image. Compared to the method of annotating objects of a target category in the vehicle cabin image with candidate boxes, since the objects of the target category are represented by the target image rather than by selection with candidate boxes, the effectiveness and reliability of the target image and attribute features can be guaranteed. In addition, the target image and the attribute features of the target image can both be determined by multiple target feature maps of different dimensions corresponding to the vehicle cabin image, thereby ensuring that the target image and the attribute features of the target image match the vehicle cabin image, thereby further ensuring the recognition accuracy of the target image and attribute features.

[0046] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become obvious from the description below, or will be learned through practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0048] Figure 1 A flowchart of an image processing method for a vehicle in certain embodiments of the present application is provided;

[0049] Figure 2 This is a schematic diagram of an application scenario in some embodiments of the present application;

[0050] Figure 3 A flowchart of an image processing method for a vehicle in certain embodiments of the present application is provided;

[0051] Figure 4 A flowchart of an image processing method for a vehicle in certain embodiments of the present application is provided;

[0052] Figure 5A flowchart of an image processing method for a vehicle in certain embodiments of the present application is provided;

[0053] Figure 6 This is a schematic diagram of an application scenario in some embodiments of the present application;

[0054] Figure 7 This is a schematic diagram of an application scenario in some embodiments of the present application;

[0055] Figure 8 A flowchart of an image processing method for a vehicle in certain embodiments of the present application is provided;

[0056] Figure 9 A flowchart of an image processing method for a vehicle in certain embodiments of the present application is provided;

[0057] Figure 10 A flowchart of an image processing method for a vehicle in certain embodiments of the present application is provided;

[0058] Figure 11 A flowchart of an image processing method for a vehicle in certain embodiments of the present application is provided;

[0059] Figure 12 A flowchart of an image processing method for a vehicle in certain embodiments of the present application is provided;

[0060] Figure 13 Schematic diagram of a flow chart of an image processing method for a vehicle in certain embodiments of the present application. DETAILED DESCRIPTION

[0061] The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and should not be understood as limiting the embodiments of the present application.

[0062] Understandably, in a vehicle cockpit environment, the position and posture of the driver and passengers are in a dynamic process of change. Consequently, when designing a human body segmentation scheme for vehicle cockpit environment images, it is generally necessary to consider the fact that the human body presents multiple scales at different distances and postures. However, semantic segmentation schemes used for human body segmentation in related technologies often rely on feature extraction methods at a single scale. Therefore, when faced with this dynamically changing human body scale segmentation, it is difficult to effectively fuse features of different scales, resulting in a significant decrease in the accuracy of edge and detail segmentation for human bodies of different scales when processing vehicle cockpit images.

[0063] For example, in a vehicle's in-cabin driver fatigue monitoring system, if the driver's facial details cannot be accurately segmented, it will be difficult to accurately identify key fatigue characteristics such as the driver's eye closure state and head tilt angle. This will not only seriously affect the accuracy of fatigue monitoring, resulting in the system being unable to provide timely safety warnings to the driver, but will also further increase safety risks during driving.

[0064] Understandably, the vehicle cabin environment is complex, consisting of multiple elements besides the driver and passengers, including seats, instrument panels, and center consoles. Furthermore, interior designs vary across different vehicle models, with a rich variety of colors and textures. Furthermore, when extracting multi-scale features from vehicle cabin images, without a reliable attention enhancement module, it becomes difficult to highlight key human features during the cockpit image processing process, nor is it possible to effectively suppress interference from unimportant features such as the background.

[0065] For example, when seats are made of leather that matches human skin tone, or when the center console's texture resembles clothing, if the body segmentation solution lacks a reliable attention enhancement module, the body will easily be confused with the background, resulting in blurred details and unclear edges in the segmented body image, failing to meet the requirements for accurate body segmentation in the vehicle cockpit. This flaw is particularly evident in the passenger behavior analysis system of the in-vehicle intelligent cockpit, which not only interferes with the accurate judgment of the status of the vehicle occupants, but also greatly reduces the intelligent experience of the in-vehicle intelligent cockpit, making it difficult for various applications based on body segmentation to operate stably and efficiently.

[0066] In addition, it is also understandable that the vehicle cabin faces complex and changeable lighting conditions and background textures. Specifically, during the day, sunlight shining through the car windows may form strong light and shadows, and at night, the lights inside the car or the street lights outside the car will produce different degrees of reflection and refraction. At the same time, the textures of seat fabrics and interior decorations are also different. When the background and the human body have similar colors or textures, the semantic segmentation scheme in the relevant technology is difficult to accurately distinguish between the human body and the background. It is understandable that in the vehicle-mounted security monitoring system, this inaccurate segmentation may lead to missed detections or false detections of abnormal behaviors in the car, such as the inability to identify dangerous actions of passengers, thereby failing to ensure the safety of people in the car, and seriously hindering the practical application of security functions related to the vehicle-mounted smart cockpit.

[0067] Based on the above problems you may encounter, please refer to Figure 1 , an embodiment of the present application provides an image processing method for a vehicle, comprising:

[0068] 01: Get the vehicle cabin image;

[0069] 02: Determine, based on the vehicle cabin image, a plurality of target feature maps corresponding to the vehicle cabin image, wherein the plurality of target feature maps have different dimensions;

[0070] 03: Determine a target image and attribute features of the target image based on the multiple target feature maps, wherein the target image is used to indicate an object of a target category in the vehicle cabin image.

[0071] Embodiments of the present application also provide a vehicle comprising a memory and a processor. The image processing method of the embodiments of the present application can be implemented by the vehicle of the embodiments of the present application. Specifically, the memory stores a computer program, and the processor is configured to obtain a vehicle cabin image, determine multiple target feature maps corresponding to the vehicle cabin image based on the vehicle cabin image, and determine a target image and attribute features of the target image based on the multiple target feature maps, wherein the multiple target feature maps have different dimensions, and the target image is configured to indicate an object of a target category in the vehicle cabin image.

[0072] Specifically, in this embodiment of the present application, a vehicle acquires a vehicle cabin image—that is, an image obtained by capturing the vehicle cabin space. The vehicle then processes the vehicle cabin image, such as through feature extraction, to determine multi-scale features corresponding to the vehicle cabin image, namely, multiple target feature maps of varying dimensions. Finally, based on these multiple target feature maps, the vehicle reconstructs a target image and determines the attributes of the target image. Specifically, the vehicle reconstructs an image indicating "objects of the target category in the vehicle cabin image" and predicts the attributes of the "objects of the target category" in the image, such as the current behavior type or the thickness of clothing currently being worn.

[0073] Thus, in an embodiment of the present application, multiple target feature maps corresponding to the vehicle cabin image are determined based on the vehicle cabin image, and based on the multiple target feature maps, a target image for indicating an object of a target category in the vehicle cabin image and the attribute features of the target image are determined, thereby achieving recognition of objects of the target category. Compared to the method of labeling objects of the target category in the vehicle cabin image with candidate boxes, since the objects of the target category are represented by the target image rather than by selection with candidate boxes, the effectiveness and reliability of the target image and the attribute features can be guaranteed. In addition, the target image and the attribute features of the target image can both be determined by multiple target feature maps of different dimensions corresponding to the vehicle cabin image, thereby ensuring that the target image and the attribute features of the target image match the vehicle cabin image, thereby further ensuring the recognition accuracy of the target image and the attribute features.

[0074] In one example, a vehicle may capture the cabin space environment by using components such as cameras installed inside the vehicle, thereby obtaining a vehicle cabin image.

[0075] In one example, multiple target feature maps of different dimensions can be understood as feature maps of different image dimensions. It is understood that in the embodiments of this application, the dimensionality of the target feature map is related to the vehicle cabin image and the processing of the vehicle cabin image. In other words, the dimensionality of the target feature map can be set or adjusted based on actual conditions. For example, the dimensions of one target feature map may be 56×56×64, while the dimensions of another target feature map may be 28×28×128.

[0076] In one example, multiple target feature maps with different dimensions can be understood as multi-scale features.

[0077] In one example, the vehicle can perform layer-by-layer feature extraction on the vehicle cabin image based on a pre-trained image processing module that can be deployed locally on the vehicle and contains multiple feature extraction layers, thereby obtaining high-dimensional features, low-dimensional features, etc. corresponding to the vehicle cabin image, that is, multiple target feature maps of different dimensions as mentioned above.

[0078] In one example, a vehicle may fuse multiple target feature maps of varying dimensions, thereby utilizing information from each target feature map to determine a target image and its attribute features, indicating an "object of a target category in the vehicle cabin image." It is understood that low-dimensional (or higher-dimensional) target feature maps contain richer detailed information, while high-dimensional (or lower-dimensional) target feature maps contain more abstract semantic information. Consequently, based on each target feature map of varying dimensions, the target image and its attribute features can be effectively determined to a certain extent.

[0079] In one example, a vehicle may process a vehicle cabin image based on an image processing model that has been pre-trained and can be deployed locally in the vehicle to determine multiple target feature maps corresponding to the vehicle cabin image, and determine the target image and attribute features of the target image based on the multiple target feature maps.

[0080] Further, see Figure 2 , Figure 2 This is a schematic diagram of an application scenario in some embodiments of the present application. In one example, a vehicle can Figure 2 The image processing model shown in FIG. 1 processes the vehicle cabin image 110 to determine a plurality of target feature maps corresponding to the vehicle cabin image, and determines the target image and the attribute features of the target image based on the plurality of target feature maps. Specifically, in FIG. Figure 2In the image processing model shown, the encoder 120 is used to process the vehicle cabin image to determine multiple target feature maps corresponding to the vehicle cabin image, and the decoder 130 is used to determine the output result 130, that is, the target image and the attribute features of the target image, based on the multiple target feature maps.

[0081] In one example, the target image can be understood as an image reconstructed based on multiple target feature maps corresponding to the vehicle cabin image and used to characterize the driver and passengers in the vehicle cabin image. In other words, the target image is used to indicate the "object of category 'person'" in the vehicle cabin image.

[0082] In one example, the attribute features of the target image can be understood as the attributes of the "target class object." For example, if the target image is used to indicate a "person class object" in a vehicle cabin image, the attribute features of the target image can be understood as the current behavior of the "person class object" or the type of clothing (e.g., thickness) currently worn by the "person class object." The current behavior can be at least one of sleeping, drinking, eating, making a phone call, and smoking.

[0083] In one example, a target image is used to indicate an "object of category 'person'" in a vehicle cockpit image. The attribute characteristics of the target image are the current behavior of the "object of category 'person'," which can be sleeping, drinking water, eating, or making phone calls. Furthermore, the vehicle can determine the current behavior of the people in the cockpit based on the target image, and adjust the volume of each sound zone in the cockpit accordingly, thereby realizing an intelligent cockpit.

[0084] In one example, a target image is used to indicate an "object of category 'person'" in a vehicle cabin image. The attribute feature of the target image is the thickness type of clothing currently worn by the "object of category 'person'." Furthermore, the vehicle can determine the different clothing thicknesses of people in the cabin based on the target image, and accordingly adjust the air conditioning temperature or airbag pressure in the cabin, thereby realizing a smart cabin.

[0085] In addition, it can also be understood that the image processing method provided in the embodiments of the present application can be applied not only to vehicles, but also to servers communicating with vehicles, or in other words, the vehicle can send the vehicle cockpit image to the server when the vehicle cockpit image is obtained.

[0086] Then, after receiving the vehicle cockpit image, the server can call pre-designed programs, codes, functions, models, etc. to process the vehicle cockpit image, thereby determining multiple target feature maps corresponding to the vehicle cockpit image based on the vehicle cockpit image, and determining the target image and the attribute characteristics of the target image based on the multiple target feature maps.

[0087] Finally, the server can send the target image and the attribute characteristics of the target image to the vehicle, or the server can generate corresponding control instructions based on the target image and the attribute characteristics of the target image. For example, when the target image is used to indicate the "object of category 'person'" in the vehicle cabin image, and the attribute characteristics of the target image are the thickness type of the clothing currently worn by the "object of category 'person'", the server can generate corresponding cabin air conditioning temperature adjustment instructions or airbag pressure adjustment instructions based on the different clothing thicknesses of the people in the cabin, and send the generated instructions to the vehicle for execution, thereby realizing a smart cockpit.

[0088] See also Figure 3 In some embodiments of the present application, the plurality of target feature maps include a first target feature map, a second target feature map, and a third target feature map, the dimensions of which decrease in sequence. Furthermore, step 02 includes:

[0089] 020: Perform residual convolution processing on the vehicle cabin image to determine the first target feature map;

[0090] 021: Perform pooling and residual convolution on the first target feature map to determine the second target feature map;

[0091] 022: Perform residual convolution processing on the second target feature map to determine the third target feature map.

[0092] The processor of the embodiment of the present application is also used to perform residual convolution processing on the vehicle cabin image to determine a first target feature map, and to perform pooling and residual convolution processing on the first target feature map to determine a second target feature map, and to perform residual convolution processing on the second target feature map to determine a third target feature map.

[0093] Specifically, in an embodiment of the present application, residual convolution processing and pooling processing can be performed on the vehicle cabin image based on the residual convolution module in the pre-trained image processing model to obtain the first target feature map, the second target feature map and the third target feature map.

[0094] To more clearly illustrate the implementation of this application, please refer to Figure 2 , that is, Figure 2 As shown, in the embodiment of the present application, the encoder 120 includes a feature extraction module 121. The feature extraction module 121 uses a residual network as a backbone architecture and performs deep extraction of image features with the help of hierarchical convolution and pooling operations.

[0095] Specifically, the feature extraction module 121 includes four convolution groups (ie, Layer 0, Layer 1, Layer 2, Layer 3, and Layer 4) and a maximum pooling layer (ie, Maxpool).

[0096] After the vehicle cabin image 110 is input into the feature extraction module 121 , the first convolution group (ie, Layer 0 ) in the feature extraction module 121 performs residual convolution processing on the vehicle cabin image 110 , thereby generating a first target feature map.

[0097] Next, the maximum pooling layer (ie, Maxpool) and the second convolution group (ie, Layer1) in the feature extraction module 121 sequentially perform pooling and residual convolution processing on the first target feature map to generate a second target feature map.

[0098] Finally, the third convolution group (i.e., Layer 2), the fourth convolution group (i.e., Layer 3), and the fifth convolution group (i.e., Layer 4) in the feature extraction module 121 sequentially extract the second target feature. Figure 3 The secondary residual convolution processing is performed to output the corresponding processing result, and then the network model can generate a third target feature map 122 based on the processing result.

[0099] In one example, each convolution group in the feature extraction module 121 is composed of multiple residual blocks. The residual blocks can be used to alleviate the gradient vanishing problem in deep network training. The residual blocks can be expressed as follows:

[0100] F(x)=H(x)-x

[0101] Among them, x is the input feature map, H(x) represents the output after the convolution operation, and F(x) represents the residual output.

[0102] It is understandable that by adding the residual output to the input (i.e., F(x)+x), the model is able to learn the identity mapping more easily, allowing for training deeper networks.

[0103] In one example, the residual network 111 is implemented based on ResNet-101. It is understandable that compared to shallow networks such as ResNet-34 and ResNet-50, ResNet-101, with its increased number of network layers (101 layers) and convolution kernels, is able to capture complex semantic information such as human posture and background texture in vehicle cabin images, thereby ensuring the improvement of the generalization ability of the image processing model under different lighting and angles.

[0104] In one example, to preserve more image details, Figure 2 The stride of the convolution layer in the fourth convolution group (i.e., Layer 4) is 1. Furthermore, in the convolution operation corresponding to the fourth convolution group, the size calculation formula of the output feature map can be expressed as:

[0105]

[0106]

[0107] Among them, H in and W in is the height and width of the input feature map, padding is the padding size, dilation is the expansion rate, kernelSize is the convolution kernel size, and stride is the step size.

[0108] It can be understood that when the step size is 1, the size of the output feature map is relatively larger, which can retain more detail information. Finally, the encoder outputs a feature map with a resolution of 1 / 16 of the original image, which can provide rich detail information for subsequent processing.

[0109] In this way, in the embodiment of the present application, residual convolution processing can be performed on the vehicle cabin image to determine the first target feature map, and pooling and residual convolution processing can be performed on the first target feature map to determine the second target feature map, and residual convolution processing can be performed on the second target feature map to determine the third target feature map, thereby achieving the acquisition of the first target feature map, the second target feature map and the third target feature map.

[0110] See also Figure 4 In certain embodiments of the present application, step 022 includes:

[0111] 0220: Perform multiple residual convolution processes on the second target feature map to determine the first feature map;

[0112] 0221: Determine target attention data corresponding to the first feature map according to the first feature map;

[0113] 0222: Determine a third target feature map based on the first feature map and the target attention data.

[0114] The processor of the embodiment of the present application is also used to perform multiple residual convolution processes on the second target feature map to determine the first feature map, and determine the target attention data corresponding to the first feature map based on the first feature map, and determine the third target feature map based on the first feature map and the target attention data.

[0115] Specifically, in order to suppress the interference of background factors in the vehicle cockpit image and ensure the accurate identification of foreground factors, the embodiment of the present application introduces an attention mechanism to ensure the accurate capture of foreground feature information (or feature information of the target object), thereby ensuring the accuracy of the target image and attribute features.

[0116] In an example, please refer to Figure 2 , that is, Figure 2As shown, in order to strengthen the targeted extraction of human features and thus enhance the feature expression capability of the image processing model, the embodiment of the present application introduces an attention module 124.

[0117] Specifically, the third convolution group (i.e., Layer 2), the fourth convolution group (i.e., Layer 3), and the fifth convolution group (i.e., Layer 4) in the residual network 111 sequentially perform the second target feature Figure 3 The residual convolution is performed again to generate a first feature map 123.

[0118] The first feature map 123 is input to the attention module 124. The attention module 124 calculates corresponding attention data based on the received first feature map 123. Furthermore, the encoder 120 can determine the third target feature map 122 based on the attention data and the first feature map 123.

[0119] In this way, in an embodiment of the present application, the second target feature map can be subjected to multiple residual convolution processes to determine the first feature map, and based on the first feature map, the target attention data corresponding to the first feature map can be determined, and based on the first feature map and the target attention data, the third target feature map can be determined, so that the third target feature map is determined based on the attention mechanism, thereby ensuring the validity of the third target feature map, target image and attribute features.

[0120] See also Figure 5 In certain embodiments of the present application, step 0221 includes:

[0121] 02210: Determine, based on the first feature map, position attention data and channel attention data corresponding to the first feature map;

[0122] 02211: Determine target attention data based on position attention data and channel attention.

[0123] The processor of the embodiment of the present application is also used to determine the position attention data and channel attention corresponding to the first feature map based on the first feature map, and to determine the target attention data based on the position attention data and channel attention.

[0124] Specifically, in the embodiment of the present application, in order to strengthen the targeted extraction of human features and significantly enhance the feature expression ability of the network, a dual attention mechanism (Dual Attention Mechanism Module, DAMM) consisting of position attention (Position Attention) and channel attention (Channel Attention) can be introduced. It can be understood that based on the introduction of position attention and channel attention, the model can more accurately capture target information during image processing, thereby effectively coping with segmentation tasks in complex scenes, thereby improving segmentation accuracy and the ability to process complex details.

[0125] In an example, see Figure 2 . Specifically, Figure 2 As shown, in an embodiment of the present application, after the first feature map 123 is input into the attention module 124, the channel attention unit 125 in the attention module 124 calculates the channel attention data based on the first feature map 123. At the same time, the position attention unit 126 in the attention module 124 calculates the position attention data based on the first feature map 123. Then, the attention module 124 can process the channel attention data and the position attention data to obtain the target attention data.

[0126] In one example, the channel attention data and the position attention data can both be expressed in the form of matrices. Then, the attention module 124 can fuse the channel attention data and the position attention data based on element-by-element summation to obtain target attention data.

[0127] It is understandable that the position attention unit 125 can focus on the spatial position relationship of the image, improve the network's attention to the target area, and thus accurately capture the spatial information of the foreground target. In one example, the position attention unit 125 can be as follows: Figure 6 As shown, Figure 6 This is a schematic diagram of an application scenario in certain embodiments of the present application.

[0128] It can also be understood that when processing complex backgrounds or targets with rich details, the position attention unit 125 can improve the edge clarity and segmentation accuracy of the "target category object" to a certain extent. Specifically, the position attention unit 125 can selectively enhance the association between any two points in the image based on the semantic correlation relationship. More specifically, the feature extraction module 121 outputs the feature map A (A∈R C×H×W , that is, the first feature map 123), three 1×1 convolution operations are performed on the feature map A to obtain a B vector with a channel of C / 8, a C vector with a channel of C / 8, and a D vector with a channel of C.

[0129] Then, convert the dimension of the B vector to B ′ , B ′ ∈R C / 8×N , N=H×W. At the same time, the matrix B ′ Transpose to B T Similarly, convert the dimension of the C vector into C ′ , where C ′ ∈R C / 8×N Then, C ′ With B ′T Perform matrix multiplication to obtain a matrix of size (H×W)×(H×W).

[0130] Then, call the SoftMax function to calculate the position attention matrix S∈R N×N , the element S in S ji It can be obtained by the following formula:

[0131]

[0132] Among them, N represents the number of elements in the current channel, S ji Represents the influence factor of position i on position j.

[0133] After that, the D vector is multiplied by the position attention matrix S through dimension conversion and then multiplied by the first feature map 123 (i.e. Figure 6 A) in the sum of elements to obtain the position attention data E, E∈R C×H×W , the process can be expressed by the following formula:

[0134]

[0135] Where D i Denotes the matrix D∈R N×C The i-th element in S ji is the i-th element of the matrix S; α is a learnable parameter, which is initially 0.

[0136] It can be understood that the channel attention unit 126 can enhance the network's ability to respond to important features by weighting the feature channels, highlighting the key feature channels, and suppressing the interference of unimportant channels. In addition, in the process of multi-scale feature fusion, the channel attention unit 126 can better capture and enhance key features.

[0137] In one example, the channel attention unit 126 is as follows Figure 7 As shown, Figure 7 Schematic diagram of application scenarios in some embodiments of the present application. Specifically, the channel attention unit 126 can be based on the feature map A (A∈R C×H×W , that is, the first feature map 123) calculates the channel attention matrix X(X∈RC×C ), the process may include: first, transforming the dimension of the feature map A to obtain A′, A ′ ∈R C×N , N = H × W. Then, for A ′ With A ′T Perform matrix multiplication, and calculate the channel attention matrix X based on the result of matrix multiplication and the SoftMax function.

[0138] In one example, the similarity value of each row in the channel attention matrix X is subtracted by the maximum value to increase the attention to other similarities.

[0139] In one example, after obtaining the channel attention matrix X, the channel attention matrix X and A can also be calculated. ′ The matrix multiplication result of , and then multiply it by a learnable coefficient β, that is, the element summation with the feature map A to obtain the final result E, which is the channel attention data. Among them, E∈R C×H×W .

[0140] In one example, the element X in the channel attention matrix X ji The calculation process can be shown as follows:

[0141]

[0142] Where, X ji represents the impact factor of the i-th channel, A i Represents the value of the i-th element in the feature map A, Represents the j-th element value of the transposed matrix.

[0143] In an example, the final result E can be calculated by the following formula:

[0144]

[0145] It can be understood that the final feature of each channel can be understood as the weighted sum of all channel features and the original channel features. Therefore, the channel attention unit 126 can effectively enhance the feature information of different channels, allowing the network processing model to function more efficiently, highlighting key feature channels, and suppressing redundant information, thereby improving the network's sensitivity and responsiveness to important features.

[0146] In this way, in an embodiment of the present application, channel attention data and position attention data can be determined based on the first feature map, and target attention data can be determined based on the channel attention data and position attention data, so that the determination of the third target feature map can be achieved based on channel attention and position attention, thereby further ensuring the accuracy of the third target feature map, target image and attribute features.

[0147] Moreover, the embodiment of the present application fuses position attention and channel attention, thereby taking into account both spatial and channel information during image processing. At the same time, the position attention data and channel attention data can be added together to obtain a feature map enhanced by the dual attention mechanism (i.e., a third target feature map), thereby enabling the model to adaptively focus on the spatial position and key feature channels of the "target category object", thereby avoiding problems such as blurred edges of the detected target and missed detection of large-scale targets, thereby improving the feature extraction accuracy and model robustness in complex backgrounds.

[0148] In addition, the position attention mechanism can accurately locate the target area and enhance the perception of the target from the spatial dimension. The channel attention mechanism can optimize the expression of features and highlight key features from the channel dimension. Based on the collaborative operation of this dual attention mechanism, the model can perform excellently in tasks such as instance segmentation and background replacement. Accordingly, when faced with complex scenes and fine detail processing, the collaborative operation based on this dual attention mechanism can enable the model to more accurately identify and segment target objects, thereby improving overall segmentation accuracy. At the same time, in the background replacement task, the replacement effect can be made more natural, effectively avoiding problems such as discordant edges or feature loss, thereby adapting to diverse task requirements and improving performance in complex visual tasks.

[0149] In addition, the introduction of the dual attention mechanism can, to a certain extent, solve the problems of slow fitting speed, inaccurate edge target segmentation, inconsistent large-scale target segmentation, and omissions when the model generates target images. It can help the entire model to obtain better robustness, better retain spatial details, and make feature extraction richer.

[0150] See also Figure 8 In certain embodiments of the present application, step 0222 includes:

[0151] 02220: Perform a first convolution process and a pooling process on the first feature map to determine a second feature map and a third feature map;

[0152] 02221: Perform a second convolution process on the first feature map according to a predetermined expansion rate parameter to determine a fourth feature map;

[0153] 02222: Determine the third target feature map based on the second feature map, the third feature map, the fourth feature map and the target attention data.

[0154] The processor of the embodiment of the present application is also used to perform a first convolution process and a pooling process on the first feature map, respectively, to determine a second feature map and a third feature map, and to perform a second convolution process on the first feature map according to a predetermined expansion rate parameter to determine a fourth feature map, and to determine a third target feature map according to the second feature map, the third feature map, the fourth feature map and the target attention data.

[0155] Specifically, in the process of image processing, in order to expand the receptive field while reducing the number of parameters and computational complexity to avoid gradient explosion, the implementation method of the present application introduces a depth-separable-atrous spatial pyramid pooling mechanism to reduce the number of parameters and computational complexity while maintaining the receptive field.

[0156] For details, please refer again to Figure 2 , that is, Figure 2 As shown, in the embodiment of the present application, the end of the feature extraction module 121 is also connected to the depthwise separable-atrous spatial pyramid pooling module 127. The depthwise separable-atrous spatial pyramid pooling module 127 includes 1 1×1 convolution layer, 3 depthwise separable atrous convolutions (i.e., DSAConv) and 1 pooling layer. Among them, the 1×1 convolution layer performs a first convolution process on the input feature map (i.e., the first feature map 123) to determine the second feature map, the pooling layer can perform pooling processing (such as maximum pooling or average pooling) on ​​the input feature map (i.e., the first feature map 123) to determine the third feature map, and the 3 depthwise separable atrous convolutions can perform depthwise convolution and pointwise convolution on the input feature map (i.e., the first feature map 123) to determine three fourth feature maps.

[0157] In one example, the dilation rate parameters of the three depthwise separable atrous convolutions in the depthwise separable-atrous spatial pyramid pooling module 127 are 6, 12, and 18, respectively.

[0158] It is understandable that although the traditional void convolution can expand the receptive field, there is a problem of a surge in computational complexity. In the implementation of this application, by decomposing the standard convolution into depth convolution and point-by-point convolution, the number of parameters and computational complexity can be reduced while maintaining the receptive field.

[0159] It can also be understood that for the input channel C in , output channel is C out , the convolution kernel size is K × K standard convolution, the parameter amount corresponding to the standard convolution is C in ×C out ×K×K. In contrast, in the depthwise separable dilated convolution of the embodiment of the present application, the parameter amount of the depthwise convolution is C in ×K×K, the number of parameters of point-by-point convolution is C in ×C out , the total number of parameters is Cin ×(K×K+C out ), the number of parameters and computational complexity are relatively low.

[0160] In one example, the depthwise convolution in the depthwise separable atrous convolution (DSAConv) can be expressed by the following formula:

[0161]

[0162] Where x is the input feature map, y is the depthwise convolution output feature map, w is the convolution kernel of the depthwise convolution, c represents the channel, and (i, j) represents the pixel position in the i-th row and j-th column of the feature map.

[0163] In one example, the point-by-point convolution operation in depthwise separable atrous convolution (DSAConv) can be expressed by the following formula:

[0164]

[0165] Among them, z is the point-by-point convolution output feature map, v is the convolution kernel of the point-by-point convolution, and c represents the output channel.

[0166] It can be understood that since the expansion rates of the three depth-wise separable dilated convolutions are 6, 12, and 18 respectively, the model can capture small-scale details (such as hand details and facial details) and large-scale contours (such as human body contours), thereby avoiding the information loss caused by a single receptive field.

[0167] In one example, the pooling operation of the pooling layer in the depthwise separable-atrous spatial pyramid pooling module 127 can be expressed by the following formula:

[0168]

[0169] Where g c is the value of the cth channel after pooling, H and W are the height and width of the input feature map.

[0170] In one example, when the depthwise separable-atrous spatial pyramid pooling module 127 receives an input feature map (i.e., the first feature map 123), the 1×1 convolution layer performs a first convolution on the input feature map (i.e., the first feature map 123) to determine a second feature map, and the pooling layer performs a pooling process (such as maximum pooling or average pooling) on ​​the input feature map (i.e., the first feature map 123) to determine a third feature map. After three depthwise separable atrous convolutions perform depthwise convolution and pointwise convolution on the input feature map (i.e., the first feature map 123) to determine three fourth feature maps, the depthwise separable-atrous spatial pyramid pooling module 127 can upsample the third feature map, and then splice and fuse the second feature map, the fourth feature map, and the upsampled third feature map, and then sum the spliced ​​and fused feature map with the target attention data element-wise to obtain the third target feature map 122.

[0171] In this way, in the implementation mode of the present application, the first feature map can be subjected to the first convolution processing and the pooling processing respectively to determine the second feature map and the third feature map, and the first feature map can be subjected to the second convolution processing according to the predetermined expansion rate parameter to determine the fourth feature map, and the third target feature map can be determined according to the second feature map, the third feature map, the fourth feature map and the target attention data, thereby expanding the receptive field and reducing the number of parameters and the computational complexity to a certain extent.

[0172] See also Figure 9 In some embodiments of the present application, the plurality of target feature maps include a first target feature map, a second target feature map, and a third target feature map with successively decreasing dimensions. Furthermore, step 03 includes:

[0173] 030: performing upsampling processing on the third target feature map to determine a fifth feature map;

[0174] 031: Determine a sixth characteristic map according to the second target characteristic map and the fifth characteristic map;

[0175] 032: Determine a target image and attribute features of the target image based on the first target feature map and the sixth feature map.

[0176] The processor of the embodiment of the present application is also used to upsample the third target feature map to determine the fifth feature map, and determine the sixth feature map based on the second target feature map and the fifth feature map, and determine the target image and the attribute features of the target image based on the first target feature map and the sixth feature map.

[0177] Specifically, in the embodiment of the present application, for the first target feature map, the second target feature map, and the third target feature map whose dimensions decrease in sequence, the vehicle may perform upsampling processing on the third target feature map so that the dimension of the third target feature map is the same as that of the second target feature map. Figure 1Thus, the fifth feature map is obtained.

[0178] Then, the vehicle may generate a sixth feature map based on the second target feature map and the fifth feature map having the same dimension, such as by fusing the second target feature map and the fifth feature map to obtain the sixth feature map.

[0179] Finally, the vehicle can determine the target image and the attribute characteristics of the target image based on the sixth feature map and the first target feature map. For example, after performing dimension adjustment processing on the sixth feature map so that the sixth feature map has the same dimension as the first target feature map, the sixth feature map and the first target feature map are fused to generate the target image and identify the attribute characteristics of the target image.

[0180] In one example, the third target feature map may be upsampled by four times bilinear interpolation to obtain the fifth feature map.

[0181] In this way, in the embodiment of the present application, the third target feature map can be upsampled to determine the fifth feature map, and the sixth feature map can be determined based on the second target feature map and the fifth feature map, and the target image and the attribute characteristics of the target image can be determined based on the first target feature map and the sixth feature map, thereby achieving the determination of the target image and the attribute characteristics of the target image.

[0182] See also Figure 10 In certain embodiments of the present application, step 031 includes:

[0183] 0310: Perform depth convolution and point-by-point convolution on the second target feature map to determine the seventh feature map;

[0184] 0311: Splicing the fifth feature map and the seventh feature map to determine the eighth feature map;

[0185] 0312: Perform depth convolution, point-by-point convolution, and upsampling on the eighth feature map to determine the sixth feature map.

[0186] The processor of the embodiment of the present application is also used to perform depth convolution and point-by-point convolution processing on the second target feature map to determine the seventh feature map, and to perform splicing processing on the fifth feature map and the seventh feature map to determine the eighth feature map, and to perform depth convolution, point-by-point convolution processing and upsampling processing on the eighth feature map to determine the sixth feature map.

[0187] Specifically, in an embodiment of the present application, a depthwise convolution and a pointwise convolution process may be performed on the fifth target feature map to obtain a seventh feature map. Next, a splicing process may be performed on the fifth and seventh feature maps to obtain an eighth feature map. Finally, in an embodiment of the present application, a depthwise convolution, a pointwise convolution process, and an upsampling process may be performed on the eighth feature map to obtain a sixth feature map.

[0188] To more clearly illustrate the implementation of this application, please refer to Figure 2 , that is, Figure 2 As shown, the encoder 130 of the image processing model first performs four times bilinear interpolation upsampling (i.e., "upsampling × 4") on the third target feature map 122, thereby outputting the upsampled third target feature map 122, which is also the fifth feature map.

[0189] At the same time, the encoder 130 performs depthwise convolution and pointwise convolution on the second target feature map output by the second convolution group (i.e., Layer1) in the feature extraction module 121, using a depthwise separable convolution (i.e., 1×1 DSConv) with a convolution kernel of 1×1 to obtain the seventh feature map.

[0190] Subsequently, the encoder 130 concatenates and fuses the fifth feature map and the seventh feature map to obtain an eighth feature map.

[0191] Then, the encoder 130 performs depthwise convolution and pointwise convolution on the eighth feature map using a depthwise separable convolution (i.e., 3×3 DSConv) with a convolution kernel of 3×3 size, thereby obtaining a processed eighth feature map.

[0192] Finally, the encoder 130 performs two-fold bilinear interpolation upsampling (ie, “upsampling × 2”) on the processed eighth feature map, thereby obtaining a sixth feature map.

[0193] In this way, in an embodiment of the present application, the second target feature map can be subjected to depth convolution and point-by-point convolution processing to determine the seventh feature map, and the fifth feature map and the seventh feature map can be spliced ​​to determine the eighth feature map, and the eighth feature map can be subjected to depth convolution, point-by-point convolution processing and upsampling processing to determine the sixth feature map, thereby completing the acquisition of the sixth feature map.

[0194] See also Figure 11 In certain embodiments of the present application, step 032 includes:

[0195] 0320: Perform depthwise convolution and pointwise convolution on the first target feature map to determine a ninth feature map;

[0196] 0321: Splice the sixth feature map and the ninth feature map to determine the tenth feature map;

[0197] 0322: Perform depth convolution, point-by-point convolution, and upsampling on the tenth feature map to determine the target image and attribute features of the target image.

[0198] The processor of the embodiment of the present application is also used to perform deep convolution and point-by-point convolution processing on the first target feature map, determine the ninth feature map, and perform splicing processing on the sixth feature map and the ninth feature map, determine the tenth feature map, and perform deep convolution, point-by-point convolution processing and upsampling processing on the tenth feature map to determine the target image and the attribute features of the target image.

[0199] Specifically, in an embodiment of the present application, the vehicle can perform deep convolution and point-by-point convolution processing on the first target feature map to determine a ninth feature map, then perform splicing processing on the sixth feature map and the eighth feature map for fusion, thereby determining the tenth feature map, and finally perform deep convolution, point-by-point convolution processing and upsampling processing on the tenth feature map to determine the target image and the attribute features of the target image.

[0200] To more clearly illustrate the implementation of this application, please refer to Figure 2 , that is, Figure 2 As shown, the encoder 130 performs depthwise convolution and pointwise convolution on the "first target feature map output by the first convolution group (i.e., Layer0) in the feature extraction module 121" using a depthwise separable convolution (i.e., 1×1 DSConv) with a convolution kernel of 1×1 size, thereby obtaining the ninth feature map.

[0201] Subsequently, the encoder 130 concatenates and fuses the sixth feature map and the ninth feature map to obtain a tenth feature map.

[0202] Then, the encoder 130 performs depthwise convolution and pointwise convolution on the tenth feature map using a depthwise separable convolution (i.e., 3×3 DSConv) with a convolution kernel of 3×3 size, thereby obtaining a processed tenth feature map.

[0203] Finally, the encoder 130 performs two-fold bilinear interpolation upsampling (ie, "upsampling × 2") on the processed tenth feature map, thereby obtaining the target image and the attribute features of the target image.

[0204] In this way, in the embodiment of the present application, the first target feature map can be subjected to depth convolution and point-by-point convolution processing to determine the ninth feature map, and the sixth feature map and the ninth feature map can be spliced ​​to determine the tenth feature map, and the tenth feature map can be subjected to depth convolution, point-by-point convolution processing and upsampling processing to determine the target image and the attribute characteristics of the target image, thereby completing the determination of the target image and the attribute characteristics of the target image.

[0205] Please also refer to Figure 2 and Figure 12 , Figure 12This is a flowchart of an image processing method for a vehicle in certain embodiments of the present application. Assuming that the target image is used to indicate a "person" in a vehicle cabin image, the attribute features of the target image can be understood as the current behavior of the "person" or the type of clothing (e.g., thickness) currently worn by the "person". Therefore, the following are true:

[0206] like Figure 2 As shown, the image processing model proposed in some embodiments of the present application includes an encoder 120 and a decoder 130. The encoder 120 includes a feature extraction model 121, an attention module 125, and a depthwise separable-atrous spatial pyramid pooling module 127.

[0207] Feature extraction model 121 employs the ResNet-101 deep residual network structure to extract features of human figures at various scales within the vehicle cabin. ResNet-101 possesses powerful multi-scale information capture capabilities, extracting rich and accurate high-level semantic features from vehicle cabin images. This provides comprehensive feature information for the subsequent decoder, effectively enhancing the model's ability to perceive and integrate human figures at various scales. This improves the accuracy of human edge and detail segmentation, meeting the requirements for segmenting human figures at various scales within the vehicle cabin.

[0208] Furthermore, in the processing link for the "target feature map output by the feature extraction model 121", the embodiment of the present application also introduces a depthwise separable-atrous spatial pyramid pooling module 127. It can be understood that the depthwise separable-atrous spatial pyramid pooling module 127 can reduce the number of parameters and computational complexity while expanding the receptive field, thereby effectively avoiding the gradient explosion problem and significantly improving training efficiency. In addition, the depthwise separable-atrous spatial pyramid pooling module 127 can achieve efficient capture of multi-scale contextual information and reduce feature loss by sampling features of different scales in parallel.

[0209] Furthermore, to enhance the targeted nature of feature extraction, the attention module 125 implemented based on the position attention unit 126 and the channel attention unit 125 can be connected in parallel with the degree-separable-atrous spatial pyramid pooling module 127. Specifically, the position attention unit 126 strengthens the position information of the human target from the spatial dimension, while the channel attention unit 125 enhances the expression of key features from the channel dimension. The position attention unit 126 and the channel attention unit 125 can work together to increase fitting speed, ensure edge segmentation clarity, and avoid problems such as missed detection of large-scale targets. This can improve the model's accuracy and robustness in extracting features of human bodies in vehicles under complex backgrounds.

[0210] It is understandable that in actual applications, whether it is the obvious change in body scale after the driver makes a large adjustment to the seat, or the subtle scale difference caused by the passenger's small posture adjustment in the car, the model can perceive and integrate different scale features based on the attention module 125.

[0211] It is also understood that, based on the design of the position attention unit 126 and the channel attention unit 125, the attention module 125 can precisely focus on the key features of a person, enhancing attention to the important features of the person, while effectively suppressing interference from irrelevant features such as the background. For example, in a vehicle cabin, when a passenger wears clothing that is similar in color to the seat, the attention module 125 can accurately distinguish the person from the background through precise control of the position and channel details, resulting in a segmented person image with clear details and sharp edges.

[0212] It is worth noting that in the processing of the target feature map output by feature extraction model 121, in addition to attention module 125, the present embodiment also designs a depthwise separable-atrous spatial pyramid pooling module 127 with different dilation rate parameters to specifically address the multi-scale feature extraction challenge in vehicle cabin imagery. Specifically, depthwise separable-atrous spatial pyramid pooling module 127 can effectively compensate for the insufficient receptive field when processing larger-scale human targets in vehicle cabin imagery, ensuring the continuity and integrity of feature extraction.

[0213] After receiving the output of encoder 120, decoder 130 restores the spatial resolution and position information of the image based on the received feature map input to obtain the final human segmentation result. Specifically, the decoder fuses the second target feature map with the "high-level feature map obtained by upsampling the third target feature map by a factor of 4." This is then fused with the first target feature map output by encoder 120 through a 3×3 convolution and a factor of 2 upsampling. Finally, the target image and its attribute features are obtained through a 3×3 convolution and a factor of 2 upsampling.

[0214] As can be understood, decoder 130 fully captures contextual information and integrates shallow and intermediate features from the backbone network. This design specifically addresses the complex background environment of vehicle cabins. Under the interference of complex lighting and background textures, such as the complex light and shadow created by interior lighting reflections at night and sunlight projections during the day, as well as various unique interior decorative textures, MS-FF helps the model deeply understand image content and accurately distinguish between people and background. It preserves human edge details, enhances the ability to extract people from vehicle cabin images in complex scenes, ensures the accuracy and stability of human segmentation results, and provides reliable data support for various human segmentation-based applications in vehicle intelligent cabins.

[0215] See also Figure 13 , Figure 13 This is a flow chart of an image processing method for a vehicle in certain embodiments of the present application, that is, in one example, the training process of the image processing model may include a data collection step, a data labeling step, a data set division step, a model training step, a model evaluation step, and a model testing step.

[0216] The data collection process specifically involves using onboard cameras to collect a large amount of image data covering various in-vehicle cabin scenes, under different vehicle models, lighting conditions (such as strong daylight, low nightlight, and alternating light and dark in tunnels), time periods (morning, noon, evening, etc.), and weather conditions (sunny, cloudy, rainy, snowy, etc.). In one example, the captured cabin space images show the driver and passengers in different postures (such as sitting upright, hunched over, with hands extended) and in different positions (front row, back row, near the window, or center console), fully covering the various situations that may arise in real-world applications.

[0217] The data annotation process specifically involves using professional image annotation tools and organizing annotators to carefully annotate the collected image data. The annotation content includes the precise outline of the human body (such as using polygon annotation tools to outline the edges of the body), the location information of various body parts (such as the head, arms, torso, legs, etc.), and other attributes related to the body segmentation task (such as the current behavior and the type of clothing currently being worn).

[0218] The dataset partitioning process specifically involves dividing the labeled image data into training, validation, and test sets according to a certain ratio (e.g., 70% for training, 15% for validation, and 15% for testing). It is important to note that during the partitioning process, each subset is ensured to have similar distribution characteristics to the original dataset, including images under various lighting, pose, and position conditions, to ensure the model's generalization ability across different scenarios.

[0219] The model training process specifically includes:

[0220] First, reasonably set the model's hyperparameters, including but not limited to the learning rate (e.g., the initial learning rate is set to 0.001, and a learning rate decay strategy is adopted to gradually reduce the learning rate as the number of training rounds increases to balance the model's convergence speed and accuracy), batch size (based on hardware resources and data scale, set an appropriate batch size, such as 32 or 64), and training rounds (e.g., set it to 100 rounds, and decide whether to terminate early or continue training based on the performance of the validation set). At the same time, select a suitable optimizer, such as the Adam optimizer, to improve training stability and efficiency. Select a suitable loss function, such as a combination of the cross-entropy loss function and the Dice loss function, to better handle the class imbalance problem in the segmentation task and improve segmentation accuracy.

[0221] Next, the training set data is input into the model, and iterative training is performed according to the set hyperparameters and training strategy. In each training batch, the model performs forward propagation based on the input image data, calculates the loss value between the predicted result and the labeled true value, and then calculates the gradient through the backpropagation algorithm, and updates the model parameters according to the optimizer's rules. During the training process, the loss value on the training set and the validation set, as well as the changes in evaluation indicators such as the mean intersection over (mIoU) and pixel accuracy, are monitored in real time. Training curves such as loss curves and mIoU curves are plotted to intuitively analyze the training status of the model. If the performance on the validation set no longer improves or shows a downward trend within a certain round, the early stopping strategy (Early Stopping) can be used to terminate the training early to prevent the model from overfitting. At the same time, a model checkpoint mechanism can be used to regularly save the model weights with the best performance during training for subsequent use.

[0222] The model evaluation process specifically includes: using the validation set to evaluate the trained model, calculating evaluation indicators such as mean intersection over union (mIoU) and pixel accuracy, and adjusting and optimizing the model based on the evaluation results. Result analysis and adjustment: Based on the calculated evaluation indicator results, conduct an in-depth analysis of the model's performance. If it is found that the model performs poorly on certain indicators, such as a low mIoU, it may mean that the model's ability to distinguish between different categories (such as human bodies and backgrounds) is insufficient, and further adjustments to the model's structure are required, such as adding or adjusting the parameters of certain modules, or optimizing the training strategy, such as adjusting hyperparameters and replacing loss functions. By comparing the evaluation results of different model versions or different training settings, the key factors affecting model performance can be identified, and the model can be gradually optimized until satisfactory performance is achieved on the validation set.

[0223] The model testing phase specifically includes:

[0224] First, the optimized model is tested using the test set. While the testing process is similar to the evaluation process, the model parameters are fixed and no longer adjusted. The test set images are fed into the model sequentially to obtain the model's predicted segmentation results. To ensure the accuracy and reliability of the test results, the test environment must be as similar as possible to the actual application environment to avoid deviations in the test results due to environmental differences.

[0225] In addition, the evaluation indicators on the test set are calculated and compared with the results on the validation set to verify the generalization ability and stability of the model. If the performance on the test set is similar to that on the validation set, it means that the model has good generalization ability and can maintain stable performance on unseen data. At the same time, some representative images in the test set are visualized and analyzed to intuitively view the segmentation effect of the model and check whether there are obvious segmentation errors or unreasonable situations, such as discontinuous segmentation of human body edges, mis-segmentation, etc., so as to further evaluate the usability of the model in practical applications. Through rigorous verification in the testing phase, it is ensured that the model can meet the actual needs of the vehicle cockpit human body segmentation task, providing reliable technical support for subsequent application deployment.

[0226] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by one or more processors, the above-mentioned image processing method for a vehicle is implemented.

[0227] An embodiment of the present application further provides a computer program product, comprising a computer program / instruction, which implements the above-mentioned image processing method for a vehicle when executed by a processor.

[0228] In the description of this specification, the descriptions with reference to the terms "particularly", "further", "particularly", "understandably", etc. are intended to mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic expressions of the above terms are not intended to refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0229] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present application belong.

[0230] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A vehicle image processing method, characterized in that: include: Acquire vehicle cabin images; determining, based on the vehicle cockpit image, a plurality of target feature maps corresponding to the vehicle cockpit image, wherein the plurality of target feature maps have different dimensions; A target image and attribute features of the target image are determined according to the multiple target feature maps, wherein the target image is used to indicate an object of a target category in the vehicle cabin image.

2. The method according to claim 1, characterized in that The multiple target feature maps include a first target feature map, a second target feature map, and a third target feature map, the dimensions of which decrease in sequence. The determining, based on the vehicle cockpit image, the multiple target feature maps corresponding to the vehicle cockpit image includes: Performing residual convolution processing on the vehicle cabin image to determine the first target feature map; Performing pooling and residual convolution processing on the first target feature map to determine the second target feature map; Perform residual convolution processing on the second target feature map to determine the third target feature map.

3. The method according to claim 2, characterized in that The performing residual convolution processing on the second target feature map to determine the third target feature map includes: Performing multiple residual convolution processes on the second target feature map to determine a first feature map; Determining target attention data corresponding to the first feature map according to the first feature map; Determine the third target feature map based on the first feature map and the target attention data.

4. The method according to claim 3, characterized in that The determining, based on the first feature map, target attention data corresponding to the first feature map includes: Determining, according to the first feature map, position attention data and channel attention data corresponding to the first feature map; The target attention data is determined based on the position attention data and the channel attention.

5. The method according to claim 3, characterized in that The determining, according to the first feature map and the target attention data, the third target feature map comprises: Performing a first convolution process and a pooling process on the first feature map to determine a second feature map and a third feature map; performing a second convolution process on the first feature map according to a predetermined expansion rate parameter to determine a fourth feature map; Determine the third target feature map based on the second feature map, the third feature map, the fourth feature map and the target attention data.

6. The method according to claim 1, characterized in that The multiple target feature maps include a first target feature map, a second target feature map, and a third target feature map, the dimensions of which decrease in sequence. Determining the target image and the attribute features of the target image based on the multiple target feature maps includes: Performing upsampling processing on the third target feature map to determine a fifth feature map; Determining a sixth characteristic map based on the second target characteristic map and the fifth characteristic map; The target image and the attribute features of the target image are determined according to the first target feature map and the sixth feature map.

7. The method according to claim 6, characterized in that The determining of the sixth characteristic map according to the second target characteristic map and the fifth characteristic map includes: Performing depthwise convolution and pointwise convolution on the second target feature map to determine a seventh feature map; performing splicing processing on the fifth feature map and the seventh feature map to determine an eighth feature map; Performing depthwise convolution, point-by-point convolution, and upsampling on the eighth feature map to determine the sixth feature map.

8. The method according to claim 6, characterized in that The determining the target image and the attribute features of the target image according to the first target feature map and the sixth feature map includes: Performing depthwise convolution and pointwise convolution on the first target feature map to determine a ninth feature map; performing splicing processing on the sixth feature map and the ninth feature map to determine a tenth feature map; Performing depth convolution, point-by-point convolution processing, and upsampling processing on the tenth feature map to determine the target image and the attribute features of the target image.

9. A vehicle, characterized in that: The method comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the method according to any one of claims 1 to 8 is implemented.

Images