Image processing method, device, equipment and storage medium

Abstract

The present disclosure provides an image processing method, device, equipment and storage medium, which are applied to a programmable chip in a vehicle-mounted terminal. The image processing method includes: obtaining camera images of multiple actual channels, one actual channel corresponds to one vehicle-mounted camera; merging the camera images of multiple actual channels into camera images of one or multiple virtual channels, one virtual channel corresponds to at least two actual channels; through a target detection model, performing target detection on sensitive objects contained in the camera images of the virtual channels to obtain detection results; based on the detection results, mosaic processing is performed on the camera images of the multiple actual channels to obtain desensitized images corresponding to the camera images of the multiple actual channels. Thus, the CPU / GPU resources of the vehicle are not occupied, the vehicle performance is avoided from being reduced, and resource sharing of target detection of multi-channel camera images is realized by merging multi-channel camera images, the target detection efficiency is improved, and then the image desensitization efficiency is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, specifically to technical fields such as image processing, computer vision, deep learning, intelligent vehicles and intelligent driving, and in particular to an image processing method, device, equipment and storage medium. Background Art

[0002] In intelligent driving, private content such as faces and license plates may appear in the driving environment images collected by the on-board camera. In order to protect the privacy content, the driving environment images need to be desensitized.

[0003] In the related art, a processor in a vehicle, such as a central processing unit (CPU) and / or a graphics processing unit (GPU), may be used to perform desensitization processing on the driving environment images captured by the vehicle-mounted camera.

[0004] However, the computing resources of the vehicle are limited. When multiple on-board cameras are deployed on the vehicle, the image desensitization efficiency of the above method is low and leads to reduced vehicle performance, posing a safety hazard. Summary of the invention

[0005] The present disclosure provides an image processing method, device, equipment and storage medium for improving image desensitization efficiency and saving vehicle performance.

[0006] According to a first aspect of the present disclosure, there is provided an image processing method, which is applied to a programmable chip located in a vehicle-mounted terminal, wherein a target detection model is deployed in the programmable chip, and the image processing method comprises: acquiring camera images of multiple actual channels, wherein one actual channel corresponds to one vehicle-mounted camera; merging the camera images of the multiple actual channels into camera images of one or multiple virtual channels, wherein one virtual channel corresponds to at least two actual channels; performing target detection on sensitive objects contained in the camera images of the virtual channels through the target detection model to obtain detection results; and performing mosaic processing on the camera images of the multiple actual channels based on the detection results to obtain desensitized images corresponding to the camera images of the multiple actual channels.

[0007] According to a second aspect of the present disclosure, there is provided an image processing device, which is applied to a programmable chip located in a vehicle-mounted terminal, wherein a target detection model is deployed in the programmable chip, and the image processing device comprises: an acquisition unit, which is used to acquire camera images of multiple actual channels, where one actual channel corresponds to one vehicle-mounted camera; a merging unit, which is used to merge the camera images of the multiple actual channels into camera images of one or multiple virtual channels, where one virtual channel corresponds to at least two actual channels; a detection unit, which is used to perform target detection on sensitive objects contained in the camera images of the virtual channels through the target detection model to obtain detection results; and a mosaic unit, which is used to perform mosaic processing on the camera images of the multiple actual channels based on the detection results to obtain desensitized images corresponding to the camera images of the multiple actual channels.

[0008] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image processing method described in the first aspect.

[0009] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the image processing method described in the first aspect.

[0010] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising: a computer program, wherein the computer program is stored in a readable storage medium, and at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the electronic device executes the image processing method described in the first aspect.

[0011] According to the sixth aspect of the present disclosure, a programmable chip is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image processing method described in the first aspect.

[0012] According to a seventh aspect of the present disclosure, there is provided a vehicle-mounted terminal, comprising a memory and a programmable chip according to the sixth aspect.

[0013] According to the technical solution provided by the present invention, image desensitization is implemented on the programmable chip of the vehicle terminal without occupying the CPU / GPU in the vehicle, thereby avoiding degradation of vehicle performance due to image desensitization; in the image desensitization process, camera images of multiple actual channels are merged into camera images of one or more virtual channels, and target detection is performed on sensitive objects contained in the camera images of the virtual channels through a target detection model to obtain detection results, and mosaic processing is performed on the camera images of the multiple actual channels based on the detection results to obtain desensitized images corresponding to the camera images of the multiple actual channels, so that multiple actual channels can share resources during target detection, thereby improving target detection efficiency and further improving image desensitization efficiency.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0016] Figure 1 is a schematic diagram of an application scenario to which the present disclosure is applicable;

[0017] Figure 2 is a schematic diagram according to a first embodiment of the present disclosure;

[0018] Figure 3 This is an example diagram of multi-channel grouping and merging;

[0019] Figure 4 is a schematic diagram according to a second embodiment of the present disclosure;

[0020] Figure 5 This is an example diagram of the timing of image processing in a programmable chip;

[0021] Figure 6 is a schematic diagram of the structure of a mosaic unit 600 in a programmable chip;

[0022] Figure 7 is a diagram showing an example structure of a programmable chip 700;

[0023] Figure 8 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0024] Fig. 9 FIG. 7 is a schematic block diagram of an example electronic device 700 that may be used to implement embodiments of the present disclosure. DETAILED DESCRIPTION

[0025] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0026] Image desensitization is performed on the vehicle's processor. Image desensitization will occupy the vehicle's limited system resources, reduce vehicle performance, increase the latency fluctuation of the vehicle's data processing link, and bring safety risks. Especially when there are multiple on-board cameras deployed on the vehicle, real-time image desensitization of camera images taken by multiple on-board cameras will occupy more system resources of the vehicle, and the image desensitization efficiency is low.

[0027] In order to solve the above problems, the present invention performs desensitization of camera images on a programmable chip of a vehicle-mounted terminal without occupying the computing resources of a processor in the vehicle-mounted terminal. By merging the camera images of multiple actual channels in the programmable chip, the camera images of the actual channels involved in the merging can share target detection resources, and then perform image desensitization based on the detection results. By merging multi-channel camera images into one or more virtual channels, the camera images of the same virtual channel are time-division multiplexed during the target detection process, realizing resource sharing during the target detection process, improving the target detection efficiency and reducing resource occupancy, thereby improving the image desensitization efficiency.

[0028] Among them, the programmable chip is a programmable device, on which corresponding schemes and functions can be implemented through programming.

[0029] Figure 1 It is a schematic diagram of an application scenario applicable to the present disclosure. The application scenario includes a vehicle 100, on which multiple vehicle-mounted cameras 110 and a vehicle-mounted terminal 120 are deployed. The vehicle-mounted camera 110 is used to capture the driving environment around the vehicle 100 to obtain a camera image. The vehicle-mounted camera 110 transmits the camera image to the vehicle-mounted terminal 120, and a programmable chip 121 is deployed in the vehicle-mounted terminal 120. The programmable chip 121 is used for image desensitization of the camera image. A target detection model 1211 is deployed in the programmable chip 121. After receiving the multi-channel camera image, the multi-channel camera image is merged. After the target detection model 1211 performs target detection on the sensitive objects contained in the merged camera image, the camera image is mosaic processed based on the detection result to obtain a desensitized image.

[0030] Among them, the vehicle-mounted terminal 120 is a computing terminal deployed on the vehicle, such as a vehicle computer terminal.

[0031] The following specific embodiments are used to describe in detail the technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0032] The following specific embodiments are applied to the programmable chip in the vehicle terminal. The structure of the vehicle terminal and the structure of the programmable chip can be referred to Figure 1 As shown, no further details are given.

[0033] Figure 2 Schematic diagram of the first embodiment of the present disclosure. Figure 2 As shown, the image processing method provided by the first embodiment of the present disclosure includes:

[0034] S201, acquiring camera images of multiple actual channels, where one actual channel corresponds to one vehicle-mounted camera.

[0035] In order to collect the surrounding conditions from all directions, multiple on-board cameras can be deployed on the vehicle. One on-board camera can be understood as one on-board camera. Each on-board camera can correspond to one camera channel. In order to distinguish it from the merged virtual channel, the camera channel is called the actual channel.

[0036] In this embodiment, the programmable chip can obtain camera images captured by multiple on-board cameras in real time to obtain camera images of multiple actual channels. As the shooting time increases, one actual channel can include multiple frames of camera images captured by the on-board camera corresponding to the actual channel at different times.

[0037] S202: merging camera images of multiple actual channels into camera images of one or more virtual channels, where one virtual channel corresponds to at least two actual channels.

[0038] In this embodiment, the camera images of multiple actual channels can be merged into a camera image of a virtual channel, or the camera images of multiple actual channels can be merged into camera images of multiple virtual channels. In the merging process, for each virtual channel, a camera image can be selected from at least two actual channels corresponding to the virtual channel to obtain a camera image of the virtual channel.

[0039] S203, performing target detection on sensitive objects contained in the camera image of the virtual channel through the target detection model to obtain a detection result.

[0040] The target detection model is a deep learning model that can identify sensitive objects appearing in camera images and detect the image position of sensitive objects on camera images. The deep learning model can be pre-trained and deployed to a programmable chip. The model structure and training process of the deep learning model are not limited here.

[0041] Among them, sensitive objects include face images and license plate number images that appear on camera images.

[0042] In this embodiment, the camera image of the virtual channel can be input into the target detection model, or the camera image of the virtual channel can be preprocessed, and the preprocessed camera image can be input into the target detection model, and the sensitive objects contained in the input image can be detected in the target detection model to obtain a detection result. The camera image of the virtual channel can be one frame or multiple frames, and 0, 1 or more sensitive objects may appear in one frame of camera image. One frame of camera image can correspond to one detection result, and one detection result can reflect the object information of one or more sensitive objects. If no sensitive object appears in the camera image, the detection result corresponding to the camera image can indicate that no sensitive object appears in the camera image.

[0043] S204: Based on the detection result, mosaic processing is performed on the camera images of the multiple actual channels to obtain desensitized images corresponding to the camera images of the multiple actual channels.

[0044] In this embodiment, the detection result can reflect the object information of the sensitive object on the camera image of the virtual channel. For each virtual channel, the camera image of the virtual channel is selected from the camera images of at least two actual channels. Therefore, based on the detection result identified from the camera image of the virtual channel, the camera images on at least two actual channels can be mosaic processed to obtain the corresponding desensitized image. In this way, the desensitized image corresponding to the camera images of multiple actual channels can be obtained.

[0045] In the disclosed embodiment, image desensitization is performed on a programmable chip on a vehicle-mounted terminal without occupying computing resources of a processor on the vehicle; for camera images of multiple actual channels taken by multiple vehicle-mounted cameras, the camera images of the multiple actual channels are merged into camera images of one or more virtual channels, and sensitive objects contained in the camera images of the virtual channels are detected by a target detection model on the programmable chip. Based on the detection results, the camera images of the multiple actual channels are desensitized to achieve time-division multiplexing of camera images of at least two actual channels merged into the same virtual channel during the target detection process, thereby improving target detection efficiency through time-division multiplexing and thereby improving image desensitization efficiency.

[0046] In a possible implementation, the programmable chip may synchronously acquire camera images of multiple actual channels to improve the accuracy and orderliness of the programmable chip in merging the camera images of multiple actual channels into camera images of one or more virtual channels.

[0047] Among them, the programmable chip can synchronously obtain camera images of multiple actual channels, which means that the programmable chip synchronously receives the first frame image, the second frame image, etc. of multiple actual channels. For example, when the kth frame image of a certain actual channel is received, the kth frame image of the remaining actual channels is also received.

[0048] Furthermore, the programmable chip triggers multiple on-board cameras on the vehicle at the same frame rate and synchronously receives camera images of multiple actual channels. The programmable chip triggers multiple on-board cameras on the vehicle at the same frame rate, so that multiple on-board cameras can synchronously collect images.

[0049] With respect to the camera image merging of multiple actual channels in the aforementioned embodiment, some implementation methods are provided:

[0050] In a possible implementation, merging camera images of multiple actual channels into camera images of one or more virtual channels includes: in multiple actual channels, merging camera images of every N actual channels into camera images of one virtual channel, where N is greater than or equal to 2. For example, when N is equal to 2, camera images of every two actual channels are merged into camera images of one virtual channel, so that camera images of every two actual channels share target detection resources.

[0051] In another possible implementation, merging camera images of multiple actual channels into camera images of one or more virtual channels may include: randomly merging multiple actual channels to form camera images of one or more virtual channels. For example, in the random merging method, camera images of two actual channels may be merged into a camera image of one virtual channel, or camera images of three actual channels may be merged into a camera image of one virtual channel, so that camera images of two, three, or other actual channels can share target detection resources.

[0052] In another possible implementation, merging camera images of multiple actual channels into camera images of one or multiple virtual channels may include: merging camera images of actual channels in the same group into camera images of one virtual channel according to the grouping of multiple actual channels. The grouping of multiple actual channels is equivalent to the grouping of multiple vehicle-mounted cameras. In the grouping of multiple actual channels, each group may include at least two actual channels, and one actual channel may appear in one group. By merging camera images according to the grouping of multiple actual channels, the rationality of camera image merging is improved.

[0053] In this embodiment, the grouping of multiple actual channels may include one or more groups. For each group, the camera image of the virtual channel may be selected from the camera images of the actual channels in the group, so that the camera image of the virtual channel corresponding to each group may be obtained.

[0054] Furthermore, the grouping of multiple actual channels is determined according to the position distribution of multiple on-board cameras on the vehicle or according to the shooting directions of multiple on-board cameras, so as to improve the rationality of the grouping of multiple actual channels. For example, the actual channels corresponding to the on-board cameras with symmetrical positions on the vehicle are grouped into one group; for another example, the actual channels corresponding to the on-board cameras with shooting directions all facing forward of the vehicle are grouped into one group, and the actual channels corresponding to the on-board cameras with shooting ranges all facing backward of the vehicle are grouped into one group.

[0055] Furthermore, the grouping of multiple actual channels can be pre-configured. For example, professionals can pre-configure the grouping of multiple actual channels based on their own knowledge, experience and experimental conditions.

[0056] In a possible implementation, in the grouping of multiple actual channels, one group may include two actual channels. According to the grouping of multiple actual channels, the camera images of the actual channels in the same group are merged into the camera images of one virtual channel, including: combining the odd-numbered frame images in the camera image of the first channel with the even-numbered frame images in the camera image of the second channel in sequence to obtain the camera image of one virtual channel; wherein the first channel and the second channel are two actual channels in the same group. Thus, by combining the odd-numbered frames and the even-numbered frames of the two actual channels in sequence, the fairness and rationality of selecting the camera images combined into the virtual channel from the two actual channels is improved, and the consistency of the image desensitization effect of the camera images of different actual channels is ensured.

[0057] In this embodiment, as the vehicle-mounted camera collects images, the programmable chip can receive multiple frames of images in the actual channel corresponding to the vehicle-mounted camera, for example, the first frame of the image, the second frame of the image, the third frame of the image, etc. of the image channel are received in sequence. As the vehicle-mounted camera receives the camera image of the first channel and the camera image of the second channel, the odd frame images of the first channel (the first frame of the image, the third frame of the image, the fifth frame of the image, ...) can be extracted from the camera image of the first channel, and the even frame images of the second channel (the second frame of the image, the fourth frame of the image, the sixth frame of the image, ...) can be extracted from the camera image of the second channel, and the odd frame images of the first channel and the even frame images of the second channel are sequentially combined according to the image reception timing to obtain the camera image of the virtual channel. In the case where the vehicle-mounted camera synchronously triggers multiple vehicle-mounted cameras and synchronously receives the camera images of multiple actual channels, the camera image of the virtual channel obtained by merging the first channel and the second channel includes: the first frame of the first channel, the second frame of the second channel, the third frame of the first channel, the fourth frame of the second channel, the fifth frame of the first channel, ..., and so on.

[0058] As an example, Figure 3 This is an example diagram of multi-channel grouping and merging, such as Figure 3 As shown, in the programmable chip, vehicle-mounted camera 1 and vehicle-mounted camera 2 are in the same group, vehicle-mounted camera 3 and vehicle-mounted camera 4 are in the same group, and vehicle-mounted camera 5 and vehicle-mounted camera 6 are in the same group, that is, the actual channel corresponding to vehicle-mounted camera 1 and the actual channel corresponding to vehicle-mounted camera 2 are in the same group, the actual channel corresponding to vehicle-mounted camera 3 and the actual channel corresponding to vehicle-mounted camera 4 are in the same group, and the actual channel corresponding to vehicle-mounted camera 5 and the actual channel corresponding to vehicle-mounted camera 6 are in the same group. According to this grouping method, the actual channel corresponding to vehicle-mounted camera 1 and the actual channel corresponding to vehicle-mounted camera 2 are merged into the same virtual channel, the actual channel corresponding to vehicle-mounted camera 3 and the actual channel corresponding to vehicle-mounted camera 4 are merged into the same virtual channel, and the actual channel corresponding to vehicle-mounted camera 5 and the actual channel corresponding to vehicle-mounted camera 6 are merged into the same virtual channel. After target detection and desensitization processing, the desensitized images of each actual channel can still be obtained, and then the video data of each vehicle-mounted camera is obtained and output.

[0059] The above provides a scheme for combining the camera images of two actual channels into an odd-even combination when the camera images of two actual channels are combined into a camera channel of a virtual channel. By analogy, if the cameras of three actual channels are combined into a camera channel of a virtual channel, the first frame image can be selected from the first actual channel, the second frame image can be selected from the second actual channel, the third frame image can be selected from the third actual channel, the fourth frame image can be selected from the first actual channel, the fifth frame image can be selected from the second actual channel, and so on. For the case where the number of actual channels in a group is greater than 2, this combination method can be referred to and will not be described one by one.

[0060] Based on any of the foregoing embodiments and any of the implementations:

[0061] In a possible implementation, when the programmable chip merges camera images of multiple actual channels into camera images of one or more virtual channels, for each virtual channel, camera images can be selected from at least two actual channels corresponding to the virtual channel, and placed in the image queue corresponding to the virtual channel in chronological order; when performing target detection on the camera images of the virtual channel through the target detection model, camera images can be read in sequence from the image queue corresponding to the virtual channel, and target detection can be performed on the read camera images through the target detection model. Thus, the orderliness of the camera images in the virtual channel is maintained based on the image queue.

[0062] Furthermore, based on the fact that in the grouping of multiple actual channels, a group may include two actual channels, and the first channel and the second channel are two actual channels in the same group, after receiving the odd-numbered image of the first channel, the programmable chip puts the odd-numbered image into the image queue of the virtual channel, and after receiving the even-numbered image of the second channel, puts the even-numbered image into the image queue of the same virtual channel, and combines the odd-numbered frame image received in the first channel and the even-numbered frame image received in the second channel by means of the image queue, so as to ensure the order in which the camera images of multiple actual channels are combined. Afterwards, in the target detection process, the camera images are read sequentially from the image queue, and the target detection is performed on the read camera images by the target detection model.

[0063] With respect to the mosaic processing of the camera images of multiple actual channels based on the detection results in the aforementioned embodiment, some implementation methods are provided:

[0064] In one possible implementation, based on the detection results, mosaic processing is performed on camera images of multiple actual channels to obtain desensitized images corresponding to the camera images of the multiple actual channels, including: based on the detection results, mosaic processing is performed on camera images corresponding to the detection results in multiple actual channels to obtain desensitized images corresponding to the camera images, thereby applying the detection results to desensitization of the corresponding camera images based on the correspondence between the detection results and the camera images, thereby improving the accuracy of desensitization.

[0065] In another possible implementation, based on the detection results, mosaic processing is performed on the camera images of multiple actual channels to obtain desensitized images corresponding to the camera images of the multiple actual channels, including: based on the detection results, mosaic processing is performed on the next frame of adjacent images of the camera images corresponding to the detection results in the multiple actual channels to obtain the desensitized images corresponding to the next frame of adjacent images. Therefore, considering that sensitive object detection takes a period of time, if the detection result is applied to the camera image corresponding to the detection result, there is a time difference between the camera image and the detection result corresponding to the camera image, applying the detection result to the next frame of adjacent images of the camera image corresponding to the detection result is conducive to shortening the time difference between the detection result and the camera image to which the detection result is applied, thereby improving the real-time performance of image desensitization.

[0066] Figure 4 is a schematic diagram according to the second embodiment of the present disclosure. Figure 4 As shown, the image processing method provided by the second embodiment of the present disclosure includes:

[0067] S401, acquiring camera images of multiple actual channels, where one actual channel corresponds to one vehicle-mounted camera.

[0068] S402: merging camera images of multiple actual channels into camera images of one or more virtual channels, where one virtual channel corresponds to at least two actual channels.

[0069] S403, performing target detection on sensitive objects contained in the camera image of the virtual channel through the target detection model to obtain a detection result.

[0070] The implementation principles and technical effects of S401 to S403 may refer to the aforementioned embodiments and will not be described in detail.

[0071] S404, determining the target actual channel to which the detection result belongs.

[0072] In this embodiment, in the process of merging the camera images of multiple actual channels into the camera images of one or more virtual channels, the camera images of the multiple actual channels can be combined in a certain order to obtain the camera images of one or more virtual channels. Therefore, for each virtual channel, the target actual channel to which the inspection result belongs can be determined in the at least two actual channels based on the combination order of the camera images of at least two actual channels corresponding to the virtual channel. Therefore, there is no need to record the correspondence between the camera image and the actual image, and the target actual channel to which the inspection result belongs can be directly derived, reducing the amount of data that needs to be recorded during the image desensitization process.

[0073] S405 , according to the detection result, mosaic processing is performed on the camera image of the actual channel of the target to obtain a desensitized image corresponding to the camera image of the actual channel of the target.

[0074] In this embodiment, according to the detection result, mosaic processing can be performed on the camera image corresponding to the detection result in the actual channel of the target to obtain a desensitized image of the camera image corresponding to the detection result; or, according to the detection result, mosaic processing can be performed on the next frame of adjacent images of the camera image corresponding to the detection result in the actual channel of the target to obtain a desensitized image of the next frame of adjacent images, so as to improve the real-time performance of image desensitization; or, according to the detection result, mosaic processing can be performed on the camera image currently received by the programmable chip in the actual channel of the target to obtain a desensitized image of the camera image, so as to improve the real-time performance of image desensitization and improve the efficiency of image desensitization.

[0075] In the disclosed embodiment, image desensitization is performed on a programmable chip on a vehicle terminal, without occupying the computing resources of the processor on the vehicle; by merging camera images of multiple actual channels into camera images of one or multiple virtual channels, the camera images of at least two actual channels merged into the same virtual channel share target detection resources, thereby improving target detection efficiency; after target detection, the target actual channel to which the detection result belongs is determined, and the detection result is applied to the image desensitization of the camera image in the target actual channel, thereby improving the image desensitization effect. Thus, the efficiency and effect of image desensitization are improved.

[0076] In a possible implementation, determining the target actual channel to which the detection result belongs may include: for the odd-numbered detection result among multiple detection results, determining the target actual channel to be the third channel; for the even-numbered detection result among multiple detection results, determining the target actual channel to be the fourth channel. Among them, the camera images of a group of virtual channels are obtained by sequentially combining the odd-numbered frame images in the camera image of the third channel and the even-numbered frame images in the camera image of the fourth channel, and the third channel and the fourth channel are two actual channels corresponding to the same virtual channel. Thus, the detection results are accurately divided into corresponding actual channels based on the odd-even combination of camera images, thereby improving the accuracy of image desensitization.

[0077] Among them, when the camera channels of the actual channels are combined according to the groups of multiple actual channels, each group may include two actual channels, the third channel is equivalent to the first channel in the aforementioned embodiment, and the fourth channel is equivalent to the second channel in the aforementioned embodiment. The detection results of the camera images of the virtual channels after each group is merged after target detection can be divided with reference to this implementation method.

[0078] In the present implementation, for the virtual channel obtained by combining the third channel and the fourth channel, the odd-numbered frame images come from the third channel, and the even-numbered frame images come from the fourth channel. The odd-numbered detection result among the multiple detection results is obtained based on the odd-numbered frame images in the virtual channel, and the even-numbered detection result among the multiple detection results is obtained based on the even-numbered frame images in the virtual channel. Therefore, for the odd-numbered detection result among the multiple detection results, it can be determined that the target actual channel to which the detection result belongs is the third channel, and for the even-numbered detection result among the multiple detection results, it can be determined that the target actual channel to which the detection result belongs is the fourth channel.

[0079] In a possible implementation, mosaic processing is performed on the camera image of the actual channel of the target according to the detection result to obtain a desensitized image corresponding to the camera image of the actual channel of the target, including: mosaic processing is performed on the next even-numbered frame image adjacent to the third channel of the camera image corresponding to the odd-numbered detection result according to the odd-numbered detection result to obtain a desensitized image corresponding to the next even-numbered frame image; mosaic processing is performed on the next odd-numbered frame image adjacent to the fourth channel of the camera image corresponding to the even-numbered detection result according to the even-numbered detection result to obtain a desensitized image corresponding to the next odd-numbered frame image. Thus, considering the time difference between target detection and camera image reception, the real-time performance of image desensitization is improved by applying the detection result to the next adjacent image of the actual channel of the target.

[0080] The third channel and the fourth channel may refer to the description of the aforementioned embodiment and will not be described in detail.

[0081] In this embodiment, among the multiple detection results, the odd-numbered detection results are obtained based on the detection of the odd-numbered images of the third channel, and the even-numbered detection results are obtained based on the detection of the even-numbered images of the fourth channel. After receiving the odd-numbered images, the programmable chip puts the odd-numbered images into the image queue of the virtual channel, and after receiving the even-numbered images, puts the even-numbered images into the image queue of the same virtual channel, and then reads the camera images from the image queue in sequence, performs target detection on the read camera images, and obtains the multiple detection results. There is a time difference between receiving the odd-numbered images and detecting the odd-numbered images, and there is a time difference between receiving the even-numbered images and detecting the even-numbered images. If the detection results are to be applied to the camera images corresponding to the detection results, the camera images corresponding to the detection results need to wait for a long time before desensitization. In order to improve the efficiency and real-time performance of image desensitization, the similarity of adjacent images is utilized and the odd-numbered detection result is used for mosaic processing of the next even-numbered frame image adjacent to the camera image corresponding to the odd-numbered detection result in the third channel; the even-numbered detection result is used for mosaic processing of the next odd-numbered frame image adjacent to the camera image corresponding to the even-numbered detection result in the fourth channel.

[0082] As an example, Figure 5 This is an example of the timing diagram of image processing in a programmable chip. Figure 5 As shown, the programmable chip can send camera trigger signals at a certain frequency to trigger multiple on-board cameras to collect images. Figure 5 Taking multiple vehicle-mounted cameras including camera 1 and camera 2 as an example, camera 1 and camera 2 collect camera images, and the programmable chip can receive the camera images collected by camera 1 according to the timing of camera 1, and receive the camera images collected by camera 2 according to the timing of camera 2. The odd-numbered frame images in the camera images collected by camera 1 and the even-numbered frame images in the camera images collected by camera 2 are merged into camera images in the same virtual channel; the corresponding actual channels are merged into the camera images of a virtual channel, and the formed image sequence is: the first frame image of camera 1, the second frame image of camera 2, the third frame image of camera 1, the fourth frame image of camera 2, the fifth frame image of camera 1, the sixth frame image of camera 2, and so on; the target detection model can perform target detection on the camera images in this image sequence in sequence according to the model processing timing to obtain multiple detection results, such as Figure 5Obj-info 1, Obj-info 2, Obj-info 3, Obj-info 4, and Obj-info 5 in the figure are the first detection result, the second detection result, and the fifth detection result in turn. Obj-info 1 is obtained by performing target detection on the first frame of camera 1, Obj-info 2 is obtained by performing target detection on the second frame of camera 2, Obj-info 3 is obtained by performing target detection on the third frame of camera 1, Obj-info 4 is obtained by performing target detection on the fourth frame of camera 2, and Obj-info 5 is obtained by performing target detection on the fifth frame of camera 1. It can be seen that the camera images of camera 1 and camera 2 are time-division multiplexed in the target detection process.

[0083] The camera trigger frame rate can be derived when the frame rates of video 1 and video 2 are known, so that time division multiplexing can be used to improve efficiency during target detection. The frame rate of video 1 is related to the output timing of video 1, the frame rate of video 2 is related to the output timing of video 2, and the camera trigger frame rate is related to the camera trigger signal. For example, when the frame rates of video 1 and video 2 are known, the camera trigger frame rate can be set to twice the frame rate of video 1 or video 2.

[0084] like Figure 5 As shown in the figure, in the image desensitization process, Obj-info 1 is used for mosaic processing of the second frame image of camera 1, and the second frame image of camera 1 is the next even frame image adjacent to the first frame image of camera 1 corresponding to Obj-info 1. Obj-info 2 is used for mosaic processing of the third frame image of camera 2, and the third frame image of camera 2 is the next odd frame image adjacent to the second frame image of camera 2 corresponding to Obj-info 2; Obj-info 3 is used for mosaic processing of the fourth frame image of camera 1, and the fourth frame image of camera 1 is the next even frame image adjacent to the third frame image of camera 1 corresponding to Obj-info 3; Obj-info 4 is used for mosaic processing of the fifth frame image of camera 2, and the fifth frame image of camera 2 is the next odd frame image adjacent to the fourth frame image of camera 2 corresponding to Obj-info 4; Obj-info 5 is used for mosaic processing of the sixth frame image of camera 1, and the sixth frame image of camera 1 is Obj-info 5 corresponds to the next even-numbered frame image adjacent to the 5th frame image of camera 1. The desensitized image corresponding to the camera image of camera 1 can be output as a video frame in video 1 according to the output timing of video 1, and the desensitized image corresponding to the camera image of camera 2 can be output as a video frame in video 2 according to the output timing of video 2.

[0085] like Figure 5As shown in the figure, the image inference time from receiving the first frame image of camera 1 to detecting Obj-info 1 based on the first frame image of camera 1 is t1, and the time margin from detecting Obj-info 1 to receiving the second frame image of camera 1 is t2. t1 is significantly greater than t2. Obj-info 1 is applied to the mosaic processing of the second frame image of camera 1. The second frame image of camera 1 only needs to wait for t2 time, which effectively improves the efficiency and real-time performance of image desensitization.

[0086] Based on any of the foregoing embodiments and any of the implementation modes, the following expansion scheme is provided:

[0087] In a possible implementation, the detection result includes the pixel coordinate range of the sensitive object, and based on the detection result, the camera images of multiple actual channels are mosaic processed to obtain the desensitized images corresponding to the camera images of the multiple actual channels, and the method further includes: adjusting the pixel coordinate range to expand the pixel coordinate range. Thus, the pixel coordinate range is used to increase the probability that the sensitive object on the camera image is completely mosaic processed, especially when the detection result is applied to adjacent frame images of the camera image corresponding to the detection result, the position of the sensitive object between the adjacent images changes very little, and by expanding the pixel coordinate range in the detection result a little, the desensitization effect of the detection result applied to the adjacent frame images of the camera image corresponding to the detection result can be effectively improved.

[0088] Furthermore, the pixel coordinate range includes the upper left corner coordinate and the lower right corner coordinate, and the pixel coordinate range is adjusted, including: according to the adjustment coefficient, within the image resolution range, adjusting the upper left corner coordinate to the upper left corner of the image, and adjusting the lower right corner coordinate to the lower right corner of the image. Among them, the adjustment coefficient can be adjusted dynamically, and can be flexibly adjusted according to actual conditions to improve the desensitization effect.

[0089] In this implementation, within the image resolution range, the upper left corner coordinates within the pixel coordinate range are adjusted toward the upper left corner of the image, and the lower right corner coordinates within the pixel coordinate range are adjusted toward the lower right corner of the image. The pixel coordinate range can be expanded to the upper left and lower right. The upper left corner coordinates and the lower right corner coordinates are adjusted according to the same adjustment coefficient. This can improve the balance of expanding the pixel coordinate range in different directions. When the detection result is applied to the adjacent frame image of the camera image corresponding to the detection result, it is ensured that the expanded pixel coordinate range can cover the image range of the sensitive object in the adjacent frame image.

[0090] Furthermore, the formula for expanding the pixel coordinate range can be expressed as:

[0091] x1'=x1-(K1 / 2)*(x2-x1), x2'=x2+(K1 / 2)*(x2-x1), y1'=y1-(K1 / 2)*(y2-y1), y2'=y2+(K1 / 2)*(y2-y1).

[0092] Among them, K1 is a dynamically adjustable adjustment coefficient, (x1, y1) and (x2, y2) are respectively the upper left corner coordinates and lower right corner coordinates in the pixel coordinate range before adjustment, (x1', y1') and (x2', y2') are respectively the upper left corner coordinates and lower right corner coordinates in the pixel coordinate range after adjustment. (x1', y1') and (x2', y2') are within the camera image resolution range.

[0093] Based on any of the foregoing embodiments, the following implementation is provided for the mosaic processing process:

[0094] In a possible implementation, the detection result includes the pixel coordinate range of the sensitive object. For the camera image to which the detection result is applied, the pixel coordinates of the pixel points on the camera image can be matched with the pixel coordinate range in the detection result. If the pixel coordinates of the pixel points in the camera image are within the pixel coordinate range in the detection result, the pixel points in the camera image are mosaic processed. Thus, the pixel points in the camera image that are within the pixel coordinate range can be accurately found by matching the pixel coordinates with the pixel coordinate range, thereby improving the accuracy of mosaic processing for sensitive objects in the camera image.

[0095] In this implementation, if the pixel coordinates of a pixel point in the camera image are within the pixel coordinate range in the detection result, it means that the pixel point is within the pixel coordinate range in the detection result, that is, the pixel point is within the image area where the sensitive object is located, so the pixel point can be mosaic processed, otherwise the pixel point can be not mosaic processed. After the pixel coordinates of each pixel point in the camera image are matched with the pixel coordinate range in the detection result, after determining whether each pixel point needs to be mosaic processed and mosaic processing is performed on the pixel points that need to be mosaic processed, the mosaic processed camera image is obtained, that is, the desensitized image corresponding to the camera image.

[0096] In one possible implementation, the pixel coordinates of the pixel points in the camera image can be matched with the pixel coordinate range in the detection result according to the pixel row and column order of the pixel points in the camera image and the pixel row and column order of the pixel coordinate range in the detection result. In this way, the pixel points in the camera image and the pixel coordinate range in the detection result can be compared in sequence according to the pixel row and column order. After comparing one pixel point, the next pixel point can be read in sequence. After comparing one pixel coordinate range, the next pixel coordinate range can be read in sequence without starting comparison from the first pixel point and the first pixel coordinate range each time, thereby improving the matching efficiency of pixel points and pixel coordinate ranges and improving the image desensitization efficiency.

[0097] In a possible implementation, in the process of matching the pixel coordinates of the pixel points in the camera image with the pixel coordinate range in the detection result according to the pixel row and column order of the pixel points in the camera image and the pixel row and column order of the pixel coordinate range in the detection result, the indication information of the pixel points in the camera image can be obtained according to the pixel row and column order of the pixel points in the camera image; if the indication information of the pixel points in the camera image indicates that the pixel points in the camera image are valid pixel points, the pixel coordinates of the pixel points in the camera image are matched with the pixel coordinate range in the detection result. Thus, the next step of matching is performed while ensuring that the pixel points are valid, saving resources and improving efficiency.

[0098] The indication information of the pixel point is used to indicate whether the pixel point is a valid pixel point or an invalid pixel point. One pixel point may correspond to one indication information.

[0099] In this implementation, when determining whether a pixel point falls within the pixel coordinate range in the detection result, it is not necessary to care about the pixel value of the pixel point, but whether the pixel point is valid and the pixel coordinate of the pixel point. Therefore, the indication information of the pixel point in the camera image can be obtained according to the pixel row and column order of the pixel point in the camera image. If the indication information indicates that the pixel point is a valid pixel point, the pixel coordinate of the pixel point is matched with the pixel coordinate range in the detection result. Otherwise, the pixel coordinate of the pixel point does not need to be matched with the pixel coordinate range in the detection result, which saves resources and improves efficiency.

[0100] In a possible implementation, the detection result also includes the object type of the sensitive object. The programmable chip is configured with a correspondence between the object type and the mosaic scheme. In the process of mosaic processing the camera image based on the detection result to obtain a desensitized image corresponding to the camera image, the mosaic scheme corresponding to the object type of the sensitive object can be used to mosaic process the sensitive object contained in the camera image to obtain a desensitized image corresponding to the camera image. Thus, different mosaic schemes can be used for different object types to distinguish different sensitive objects on the desensitized image.

[0101] Furthermore, the pixel points in the camera image are mosaic processed, including: in the correspondence between the object type and the mosaic scheme, determining the target mosaic scheme corresponding to the object type in the detection result; and masking the pixel points in the camera image according to the target mosaic scheme. Therefore, when masking each pixel point, the corresponding mosaic scheme can be flexibly adopted based on the object type to distinguish different sensitive objects on the desensitized image and improve the desensitization effect.

[0102] Among them, the mosaic scheme can be a color block scheme, such as a color block color, or a random color block. When the mosaic scheme is a certain color block color, the color block color can be used to mask the pixels in the camera image. When the mosaic scheme is a random color block, the random color block can be used to mask the pixels in the camera image.

[0103] In a possible implementation, a mosaic unit may be deployed in a programmable chip, and the mosaic unit implements the above-mentioned solutions related to mosaic processing.

[0104] Figure 6 is a schematic diagram of the structure of a mosaic unit 600 in a programmable chip, such as Figure 6 As shown, the mosaic unit 600 includes a coordinate generation module 601, a coordinate matching module 602, a first-in-first-out module 603, a first selection module 604, and a second selection module 605. The coordinate generation module 601 is used to obtain the pixel coordinates of the pixel points in the camera image, including obtaining the row and column of the pixel points; the coordinate matching module 602 is used to match the pixel coordinates of the pixel points with the pixel coordinate range in the detection result to determine whether the pixel coordinates of the pixel points are within the pixel coordinate range; the first-in-first-out module 603 is used to manage the detection results in a first-in-first-out queue manner; the first selection module 604 is used to select a target mosaic scheme from multiple mosaic schemes based on the object type in the detection result; and the second selection module 605 is used to determine whether to mask the pixel points on the camera image according to the target mosaic scheme based on the coordinate matching result.

[0105] In this implementation, the detection result is read out from the first-in-first-out queue, and the camera image to which the detection result applies is determined; the camera image_indication information is used to indicate whether the pixel point on the camera image is a valid pixel point. If the pixel point is a valid pixel point, the pixel coordinates of the pixel point are obtained in the coordinate generation module 601; the pixel coordinates of the pixel point are matched with the pixel coordinate range in the detection result in the coordinate matching module 602. In the first selection module 604, according to the object type in the detection result, a plurality of mosaic schemes ( Figure 4Taking mosaic scheme 1, ..., mosaic scheme n (a total of n mosaic schemes) as an example, determine the target mosaic scheme corresponding to the pixel point; in the second selection module 605, if the coordinate matching result output by the coordinate matching module is that the pixel coordinates of the pixel point are within the pixel coordinate range in the detection result, then determine to mask the pixel point according to the target mosaic scheme (that is, set the pixel value of the pixel point to the pixel value in the target mosaic scheme), otherwise the pixel value of the pixel point can be directly output (that is, the pixel point is not masked).

[0106] Therefore, based on the above mosaic unit, mosaic processing of each pixel on the target image is achieved, and the mosaic scheme can be flexibly selected to improve the desensitization effect.

[0107] Based on any of the foregoing embodiments, the following implementation is provided for a programmable chip:

[0108] In a possible implementation, after obtaining the desensitized image, the programmable chip can perform video encoding on the desensitized image to obtain the video shot by the vehicle-mounted camera; and store the shot video in an external memory. Thus, after the camera image is desensitized, the camera image is stored as a video on the disk.

[0109] Based on the above implementation method of video encoding, in a possible implementation method, the programmable chip may also include video coding units (VCU). After obtaining the desensitized image, the desensitized image may be input into the VCU for video encoding to obtain the video shot by the vehicle-mounted camera, and the shot video may be stored in an external memory.

[0110] In one possible implementation, the programmable chip may also include a deep learning process unit (DPU) and a central processing unit (CPU). The target detection model is deployed in the DPU, and the output data of the target detection model can be input into the CPU for further calculation to obtain the detection result of the sensitive object. Among them, the output data of the target detection model is an intermediate result, and the detection result of the sensitive object can be calculated in the CPU based on the intermediate result. Thus, the DPU and CPU are used to improve the ability of the programmable chip to perform target detection.

[0111] For example, the intermediate results may include the pixel coordinate range of the sensitive object and the probability that the sensitive object belongs to each object type. Based on the probability that the sensitive object belongs to each object type, the CPU may make some trade-offs and calculations among each object type to ultimately determine the object type of the sensitive object.

[0112] In one possible implementation, the programmable chip may be a field programmable gate array (FPGA) chip. FPGA has the advantages of rich wiring resources, repeatable programming, and high integration. Image desensitization is performed on the FPGA chip, and the FPGA chip can be used to effectively ensure the efficiency and effect of image desensitization.

[0113] Figure 7 is a diagram showing an example structure of a programmable chip 700. Figure 7 As shown, the programmable chip 700 may include a DPU 701 , a CPU 702 , a mosaic unit 703 and a selection unit 704 .

[0114] like Figure 7 As shown, taking camera 1 and camera 2 as examples, in demultiplexing, the camera image of camera 1 is divided into two paths: odd frames (1st, 3rd, 5th...frames) are located in one path, and even frames (2nd, 4th, 6th...frames) are located in another path; in demultiplexing, the camera image of camera 2 is divided into two paths: odd frames (1st, 3rd, 5th...frames) are located in one path, and even frames (2nd, 4th, 6th...frames) are located in another path; the odd frames of camera 1 and the even frames of camera 2 are merged into a virtual channel to obtain an image queue of the virtual channel; through DPU 701 and CPU 702, target detection is performed on the camera images in the image queue to obtain detection results, and the detection results form a result queue (first-in-first-out queue); the detection results in the result queue are divided into corresponding image channels (i.e., actual channels), for example, the first detection result is divided into the image channel corresponding to camera 1, and the second detection result is divided into the image channel corresponding to camera 2...; in the mosaic unit 703, based on the detection results divided into the image channel corresponding to camera 1, mosaic processing is performed on the even frames of camera 1 to obtain desensitized images corresponding to the even frames, and based on the detection results divided into the image channel corresponding to camera 2, mosaic processing is performed on the odd frames of camera 2. , and obtain the desensitized image corresponding to the odd frame; in the selection unit 704 corresponding to camera 1, if the desensitization indication is to indicate the output of the desensitized video, the desensitized image corresponding to the even frame of camera 1 can be output as the image frame in the video data, and if the desensitization indication is to indicate the output of the non-desensitized video, the even frame of camera 1 can be output as the image frame in the video data; in the selection unit 703 corresponding to camera 2, if the desensitization indication is to indicate the output of the desensitized video, the desensitized image corresponding to the odd frame of camera 2 can be output as the image frame in the video data, and if the desensitization indication is to indicate the output of the non-desensitized video, the odd frame of camera 2 can be output as the image frame in the video data.

[0115] Figure 8 Schematic diagram of the fourth embodiment of the present disclosure. Figure 8As shown, the image processing device 800 provided in the fourth embodiment of the present disclosure includes:

[0116] An acquisition unit 801 is used to acquire camera images of multiple actual channels, where one actual channel corresponds to one vehicle-mounted camera;

[0117] A merging unit 802 is used to merge the camera images of multiple actual channels into the camera images of one or more virtual channels, where one virtual channel corresponds to at least two actual channels;

[0118] The detection unit 803 is used to perform target detection on the sensitive objects contained in the camera image of the virtual channel through the target detection model to obtain a detection result;

[0119] The mosaic unit 804 is used to perform mosaic processing on the camera images of multiple actual channels based on the detection results to obtain desensitized images corresponding to the camera images of the multiple actual channels.

[0120] In some embodiments, the merging unit 802 includes: a group merging module (not shown in the figure) for merging the camera images of the actual channels in the same group into a camera image of a virtual channel according to the grouping of multiple actual channels.

[0121] In some embodiments, a group includes two actual channels, and the group merging module includes: an odd-even combination submodule (not shown in the figure), which is used to sequentially combine the odd-numbered frame images in the camera image of the first channel with the even-numbered frame images in the camera image of the second channel to obtain a camera image of a virtual channel; wherein the first channel and the second channel are two actual channels in the same group.

[0122] In some embodiments, the mosaic unit 804 includes: a channel determination module (not shown in the figure), which is used to determine the actual channel of the target to which the detection result belongs; a mosaic module (not shown in the figure), which is used to perform mosaic processing on the camera image of the actual channel of the target according to the detection result to obtain a desensitized image corresponding to the camera image of the actual channel of the target.

[0123] In some embodiments, a group of camera images of a virtual channel are obtained by sequentially combining odd-numbered frame images in the camera image of the third channel and even-numbered frame images in the camera image of the fourth channel, the third channel and the fourth channel are two actual channels corresponding to the same virtual channel, there are multiple detection results, one detection result corresponds to one camera image, and the channel determination module includes: an odd-numbered result channel determination submodule (not shown in the figure), which is used to determine that the target actual channel is the third channel for the odd-numbered detection result among the multiple detection results; and an even-numbered result channel determination submodule (not shown in the figure), which is used to determine that the target actual channel is the fourth channel for the even-numbered detection result among the multiple detection results.

[0124] In some embodiments, the mosaic unit 804 includes: an adjacent even frame mosaic module (not shown in the figure), which is used to perform mosaic processing on the next even frame image adjacent to the third channel of the camera image corresponding to the odd detection result according to the odd detection result, to obtain a desensitized image corresponding to the next even frame image; an adjacent odd frame mosaic module (not shown in the figure), which is used to perform mosaic processing on the next odd frame image adjacent to the fourth channel of the camera image corresponding to the even detection result according to the even detection result, to obtain a desensitized image corresponding to the next odd frame image.

[0125] In some embodiments, the detection result includes the pixel coordinate range of the sensitive object, and the image processing device 800 further includes: a coordinate range expansion unit (not shown in the figure) for adjusting the pixel coordinate range to expand the pixel coordinate range.

[0126] In some embodiments, the pixel coordinate range includes the upper left corner coordinates and the lower right corner coordinates, and the coordinate range expansion unit includes: a coordinate adjustment module (not shown in the figure), which is used to adjust the upper left corner coordinates toward the upper left corner of the image and the lower right corner coordinates toward the lower right corner of the image according to the adjustment coefficient within the image resolution range.

[0127] Figure 8 The provided image processing device can execute the steps of the image processing method involved in the programmable chip in the above-mentioned corresponding method embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0128] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the solution provided by any of the above embodiments.

[0129] According to an embodiment of the present disclosure, the present disclosure further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute a solution provided by any of the above embodiments.

[0130] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program, the computer program is stored in a readable storage medium, at least one processor of an electronic device can read the computer program from the readable storage medium, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the above embodiments.

[0131] According to an embodiment of the present disclosure, the present disclosure also provides a programmable chip, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the solution provided by any of the above embodiments.

[0132] According to an embodiment of the present disclosure, the present disclosure further provides a vehicle-mounted terminal, which includes a memory and the programmable chip provided by the above embodiment.

[0133] Fig. 9 Schematic block diagram of an example electronic device 900 that can be used to implement an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0134] like Fig. 9 As shown, the electronic device 900 includes a computing unit 901, which can store data stored in a read-only memory (ROM) ( Fig. 9 A computer program in ROM 902 is loaded from storage unit 908 to random access memory (RAM) ( Fig. 9 The computer program in the RAM 903 (for example) can be used to perform various appropriate actions and processes. The RAM 903 can also store various programs and data required for the operation of the electronic device 900. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. The input / output (I / O) interface ( Fig. 9 The I / O interface 905 (for example) is also connected to the bus 904 .

[0135] Multiple components in the electronic device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the electronic device 900 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0136] The computing unit 901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSP), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 901 performs the various methods and processes described above, such as image processing methods. For example, in some embodiments, the image processing method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to execute the image processing method in any other appropriate manner (eg, by means of firmware).

[0137] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard parts (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0138] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0139] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0140] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0141] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

[0142] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0143] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0144] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An image processing method is applied to a programmable chip located in a vehicle terminal, wherein a target detection model is deployed in the programmable chip, and the image processing method comprises: Obtain camera images of multiple actual channels, where one actual channel corresponds to one vehicle-mounted camera; Merging the camera images of the multiple actual channels into camera images of one or more virtual channels, where one virtual channel corresponds to at least two actual channels; By using the target detection model, target detection is performed on sensitive objects contained in the camera image of the virtual channel to obtain a detection result; Based on the detection results, mosaic processing is performed on the camera images of the multiple actual channels to obtain desensitized images corresponding to the camera images of the multiple actual channels; The method of performing mosaic processing on the camera images of the multiple actual channels based on the detection results to obtain desensitized images corresponding to the camera images of the multiple actual channels includes: Determine the target actual channel to which the detection result belongs; According to the detection result, mosaic processing is performed on the camera image of the actual channel of the target to obtain a desensitized image corresponding to the camera image of the actual channel of the target; The method of performing mosaic processing on the camera image of the actual channel of the target according to the detection result to obtain a desensitized image corresponding to the camera image of the actual channel of the target comprises: performing mosaic processing on the next frame adjacent image of the camera image corresponding to the detection result in the actual channel of the target according to the detection result to obtain a desensitized image of the next frame adjacent image.

2. The image processing method according to claim 1, wherein: The step of merging the camera images of the multiple actual channels into the camera images of one or more virtual channels includes: According to the grouping of the multiple actual channels, the camera images of the actual channels in the same group are merged into a camera image of a virtual channel.

3. The image processing method according to claim 2, wherein: One group includes two actual channels, and the step of merging the camera images of the actual channels in the same group into a camera image of a virtual channel according to the grouping of the multiple actual channels includes: The odd-numbered frame images in the camera image of the first channel and the even-numbered frame images in the camera image of the second channel are sequentially combined to obtain a camera image of a virtual channel; The first channel and the second channel are two actual channels in the same group.

4. The image processing method according to any one of claims 1 to 3, wherein: A group of camera images of a virtual channel is obtained by sequentially combining odd-numbered frame images in the camera image of the third channel and even-numbered frame images in the camera image of the fourth channel, the third channel and the fourth channel are two actual channels corresponding to the same virtual channel, there are multiple detection results, one detection result corresponds to one camera image, and determining the target actual channel to which the detection result belongs includes: For an odd-numbered detection result among the multiple detection results, determining that the target actual channel is the third channel; For an even-numbered detection result among the multiple detection results, it is determined that the target actual channel is the fourth channel.

5. The image processing method according to claim 4, wherein: According to the detection result, mosaic processing is performed on the next frame adjacent image of the camera image corresponding to the detection result in the actual channel of the target to obtain a desensitized image of the next frame adjacent image, including: According to the odd-numbered detection result, mosaic processing is performed on the next even-numbered frame image of the camera image corresponding to the odd-numbered detection result adjacent to the third channel to obtain a desensitized image corresponding to the next even-numbered frame image; According to the even-numbered detection result, mosaic processing is performed on the next odd-numbered frame image adjacent to the fourth channel of the camera image corresponding to the even-numbered detection result to obtain a desensitized image corresponding to the next odd-numbered frame image.

6. The image processing method according to any one of claims 1 to 3, wherein: The detection result includes the pixel coordinate range of the sensitive object. Based on the detection result, the camera images of the multiple actual channels are mosaic processed to obtain the desensitized images corresponding to the camera images of the multiple actual channels, and the method further includes: The pixel coordinate range is adjusted to expand the pixel coordinate range.

7. The image processing method according to claim 6, wherein: The pixel coordinate range includes the upper left corner coordinate and the lower right corner coordinate, and the adjusting the pixel coordinate range includes: According to the adjustment coefficient, within the image resolution range, the upper left corner coordinate is adjusted toward the upper left corner of the image, and the lower right corner coordinate is adjusted toward the lower right corner of the image.

8. An image processing device, applied to a programmable chip in a vehicle-mounted terminal, wherein a target detection model is deployed in the programmable chip, and the image processing device comprises: An acquisition unit, used to acquire camera images of multiple actual channels, where one actual channel corresponds to one vehicle-mounted camera; A merging unit, used to merge the camera images of the multiple actual channels into camera images of one or more virtual channels, where one virtual channel corresponds to at least two actual channels; A detection unit, configured to perform target detection on sensitive objects contained in the camera image of the virtual channel by using the target detection model to obtain a detection result; A mosaic unit, configured to perform mosaic processing on the camera images of the multiple actual channels based on the detection result, so as to obtain desensitized images corresponding to the camera images of the multiple actual channels; Wherein, the mosaic unit comprises: A channel determination module, used to determine the actual channel of the target to which the detection result belongs; A mosaic module, used for performing mosaic processing on the camera image of the actual channel of the target according to the detection result, to obtain a desensitized image corresponding to the camera image of the actual channel of the target; When mosaic processing is performed on the camera image of the actual channel of the target according to the detection result to obtain a desensitized image corresponding to the camera image of the actual channel of the target, the mosaic module is used to: perform mosaic processing on the next frame adjacent image of the camera image corresponding to the detection result in the actual channel of the target according to the detection result to obtain a desensitized image of the next frame adjacent image.

9. The image processing apparatus according to claim 8, wherein: The merging unit comprises: The group merging module is used to merge the camera images of the actual channels in the same group into a camera image of a virtual channel according to the grouping of the multiple actual channels.

10. The image processing device according to claim 9, wherein: A group includes two actual channels, and the group merging module includes: An odd-even combination submodule, used to sequentially combine the odd-numbered frame images in the camera image of the first channel with the even-numbered frame images in the camera image of the second channel to obtain a camera image of one virtual channel; The first channel and the second channel are two actual channels in the same group.

11. The image processing device according to any one of claims 8 to 10, wherein: A group of camera images of a virtual channel is obtained by sequentially combining odd-numbered frame images in the camera image of the third channel and even-numbered frame images in the camera image of the fourth channel, wherein the third channel and the fourth channel are two actual channels corresponding to the same virtual channel, and the detection results are multiple, and one detection result corresponds to one camera image. The channel determination module includes: An odd-numbered result channel determination submodule, configured to determine, for an odd-numbered detection result among the plurality of detection results, that the target actual channel is the third channel; The even-numbered result channel determination submodule is used to determine, for an even-numbered detection result among the multiple detection results, that the target actual channel is the fourth channel.

12. The image processing device according to claim 11, wherein: The mosaic module includes: An adjacent even-numbered frame mosaic module is used to perform mosaic processing on the next even-numbered frame image adjacent to the third channel of the camera image corresponding to the odd-numbered detection result according to the odd-numbered detection result, so as to obtain a desensitized image corresponding to the next even-numbered frame image; The adjacent odd-numbered frame mosaic module performs mosaic processing on the next odd-numbered frame image adjacent to the fourth channel of the camera image corresponding to the even-numbered detection result according to the even-numbered detection result, so as to obtain a desensitized image corresponding to the next odd-numbered frame image.

13. The image processing device according to any one of claims 8 to 10, wherein: The detection result includes the pixel coordinate range of the sensitive object, and the image processing device further includes: The coordinate range expansion unit is used to adjust the pixel coordinate range to expand the pixel coordinate range.

14. The image processing apparatus according to claim 13, wherein: The pixel coordinate range includes the upper left corner coordinate and the lower right corner coordinate, and the coordinate range expansion unit includes: The coordinate adjustment module is used to adjust the coordinates of the upper left corner toward the upper left corner of the image and adjust the coordinates of the lower right corner toward the lower right corner of the image within the image resolution range according to the adjustment coefficient.

15. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the image processing method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the image processing method according to any one of claims 1 to 7.

17. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 7 are implemented.

18. A programmable chip, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the image processing method according to any one of claims 1 to 7.

19. An in-vehicle terminal comprising a memory and the programmable chip according to claim 18.

Images